Thinking AND Tools (Ollama) Local AI Voice Assistant - Intent error

Configuration Voice Assistant
1

Hey Guys,
I got a bunch of problems with Ollama.
I’m using it in a docker-container with a RTX4000 ADA, so I can play with some models.
However, I can also add it into the Ollama HA integration, but it seems to be completely broken:

  1. When I chose a downloaded model - any is “fine”, and ask it with Assist enabled, I will either get ollama._types.ResponseError: registry.ollama.ai/library/llama3.2:3b does not support thinking (status code: 400) (In this case I did NOT switch-on thinking in the dialog at all!) or ollama._types.ResponseError: registry.ollama.ai/library/llama3.2:3b does not support tools (status code: 400).

But - this model does tools! And so does GPT-OSS:20b. Still Ollama will give this error with eiter “tools” or “thinking”. I tried with Phi4mini, with DeepseekR1 etc. - basically a lot of models that are listed with tools (and some thinking) from Ollama.
To me it seems, that this is not reported correctly.

  1. When I add another agent within Ollama, with another model, click on chat - lets say I chose now GPT-OSS:20b, the error is (literally) ollama._types.ResponseError: registry.ollama.ai/library/llama3.2:3b does not support thinking (status code: 400).
    Only after a restart of HA it will put the error with the correct model. (Still the error as per #1!)
    This also happens when just changing the model in an existing agent.

So what else can I do to get at least far enough to try any model.
I tried HomeLLM (LocalLLMs), but it also does not work - there is usually always and error with the intent recognition.

2

If you take ha out if the oicycan you get an ollama command line to infer or maybe something like openwebui so we can determine of its HA or ollama giving you an issue

3

Good idea!
Seems to work in OpenWebUI (I downloaded the “weather”-tool).

image
image1497×1193 97.6 KB

I also tried the OpenWebUI to HA (it is called home_assistant) tool in OpenWebUI. It was able to see my whole HA, which means it used the tool. Same as the weather, which is correctly displayed and searched online.

I also installed “extended openai conversation”, which gives this error:

TypeError: string indices must be integers, not ‘str’

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/assist_pipeline/pipeline.py", line 1297, in recognize_intent
    conversation_result = await conversation.async_converse(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/usr/src/homeassistant/homeassistant/components/conversation/agent_manager.py", line 126, in async_converse
    result = await method(conversation_input)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/extended_openai_conversation/__init__.py", line 198, in async_process
    query_response = await self.query(user_input, messages, exposed_entities, 0)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/extended_openai_conversation/__init__.py", line 386, in query
    return await self.execute_tool_calls(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        user_input, messages, message, exposed_entities, n_requests + 1
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/config/custom_components/extended_openai_conversation/__init__.py", line 463, in execute_tool_calls
    result = await self.execute_tool_function(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/config/custom_components/extended_openai_conversation/__init__.py", line 496, in execute_tool_function
    result = await function_executor.execute(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        self.hass, function["function"], arguments, user_input, exposed_entities
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/config/custom_components/extended_openai_conversation/helpers.py", line 209, in execute
    return await self.execute_service(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        hass, function, arguments, user_input, exposed_entities
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/config/custom_components/extended_openai_conversation/helpers.py", line 288, in execute_service
    await self.execute_service_single(
        hass, function, service_argument, user_input, exposed_entities
    )
  File "/config/custom_components/extended_openai_conversation/helpers.py", line 247, in execute_service_single
    domain = service_argument["domain"]
             ~~~~~~~~~~~~~~~~^^^^^^^^^^
TypeError: string indices must be integers, not 'str'