this part is where im confused - I'VE NEVER had the assist agent 'silently' fail - its noisy af tbh - and why i started guarding errors religiously - that one where it doesnt call properly - that's weird.
Rudd - you ahve a copy of the full script and the call you're making somewhere?
He posted the script above. It looks good, although a bit verbose.
im interested in the call... And verbose with LLM is good.
Look bro, let me show you the tool call, is that okay with you?
- type: intent-start
data:
engine: conversation.qwen3_coder_next
language: en
intent_input: >-
have the vacuum vacuum and mop the dining room, the kitchen, the
entranceway, the foyer, and the master bedroom
conversation_id: 01KR3PQYA3YV6JP3X74Y9CNYZC
device_id: null
satellite_id: null
prefer_local_intents: true
timestamp: "2026-05-08T11:56:18.696600+00:00"
- type: intent-progress
data:
chat_log_delta:
role: assistant
tool_calls:
- tool_name: clean_specific_areas_of_the_home
tool_args:
area:
- Dining room
- Kitchen
- Entranceway
- Foyer
- Master bedroom
mode: vacuum and mop
iterations: 1
id: 01KR3Q47D9SKH4TKRWBVEH5NF2
external: false
content: ""
timestamp: "2026-05-08T11:56:19.753715+00:00"
- type: intent-progress
data:
chat_log_delta:
content: ""
timestamp: "2026-05-08T11:56:19.760249+00:00"
- type: intent-progress
data:
chat_log_delta:
role: tool_result
agent_id: conversation.qwen3_coder_next
tool_call_id: 01KR3Q47D9SKH4TKRWBVEH5NF2
tool_name: clean_specific_areas_of_the_home
tool_result:
success: true
result: {}
created: "2026-05-08T11:56:19.761966+00:00"
- [a lot of junk omitted]
- type: intent-end
data:
processed_locally: false
intent_output:
response:
speech:
plain:
speech: >-
The vacuum will now vacuum and mop the dining room, kitchen,
entranceway, foyer, and master bedroom.
extra_data: null
card: {}
language: en
response_type: action_done
data:
success: []
failed: []
conversation_id: 01KR3PQYA3YV6JP3X74Y9CNYZC
continue_conversation: false
timestamp: "2026-05-08T11:56:20.411625+00:00"
- type: run-end
data: null
timestamp: "2026-05-08T11:56:20.411665+00:00"
See? The LLM is calling the tool correctly, with five items, and then it receives a response action_done from Home Assistant (despite the fact that the script failed here, because at this point in time, the script had stop: error = True for testing purposes). Therefore the LLM concludes that everything is hunky dory and the vacuum is indeed cleaning, which is exactly what it reports back.
When I say verbose, I mean if then statements. The script can be whittled down using templates, but that's just my preference.
I'm going to make a script that fails and then I'm gonna paste the script here. Then I'm gonna paste the assist debugging output. Please stand by 2 minutes.
I'll run it through some of my setups - which convo agent are you using? Im finding there are some minor deltas, just in case...
qwen3-coder-next 35b via Ollama
WHICH Ollama local conversation Agent - the specific integration.
Yes, but that's not a failing one. LLMs are not consistent. Show a failing one, you said it works 90% of the time. That sounds like a typical LLM. Show the 10%
Nah - this part is deterministic - im sure theres a reason - although now i know its not the default OAI convo agent... Cause that one's NOISY and throws error intent unexpectedd at the drop of a hat and stops the thread.
Here is the script:
sequence:
- stop: The gong is broken
error: true
alias: Play the gong
description: ""
Interestingly, when I hit the Play button in the More Info dialog, no error shows up on screen, despite the fact that the script clearly did fail according to the script trace:
Stop because: The gong is broken
Executed: May 8, 2026 at 1:07:25 PM
Error: The gong is broken
Result:
stop: The gong is broken
error: true
So, no error on screen when a script stops with an error. That's worse than I previously thought.
Now, here comes the Assist debug output complete with tool call and result:
stage: done
run:
pipeline: 01jdnkavjpgqs7jn4973t1mefm
language: en
conversation_id: 01KR3V7N4375EK3PEYS1X5FV0J
runner_data:
stt_binary_handler_id: null
timeout: 300
events:
- type: run-start
data:
pipeline: 01jdnkavjpgqs7jn4973t1mefm
language: en
conversation_id: 01KR3V7N4375EK3PEYS1X5FV0J
runner_data:
stt_binary_handler_id: null
timeout: 300
timestamp: "2026-05-08T13:08:06.403786+00:00"
- type: intent-start
data:
engine: conversation.qwen3_coder_next
language: en
intent_input: please play the gong once
conversation_id: 01KR3V7N4375EK3PEYS1X5FV0J
device_id: null
satellite_id: null
prefer_local_intents: true
timestamp: "2026-05-08T13:08:06.403800+00:00"
- type: intent-progress
data:
chat_log_delta:
role: assistant
tool_calls:
- tool_name: play_the_gong
tool_args: {}
id: 01KR3V7Q9EWQFSGWTMEXSC7Z11
external: false
content: ""
timestamp: "2026-05-08T13:08:08.622266+00:00"
- type: intent-progress
data:
chat_log_delta:
content: ""
timestamp: "2026-05-08T13:08:08.628687+00:00"
- type: intent-progress
data:
chat_log_delta:
role: tool_result
agent_id: conversation.qwen3_coder_next
tool_call_id: 01KR3V7Q9EWQFSGWTMEXSC7Z11
tool_name: play_the_gong
tool_result:
success: true
result: {}
created: "2026-05-08T13:08:08.629794+00:00"
timestamp: "2026-05-08T13:08:08.629844+00:00"
- type: intent-progress (many of these, snipped)
- type: intent-end
data:
processed_locally: false
intent_output:
response:
speech:
plain:
speech: The gong has been played.
extra_data: null
card: {}
language: en
response_type: action_done
data:
success: []
failed: []
conversation_id: 01KR3V7N4375EK3PEYS1X5FV0J
continue_conversation: false
timestamp: "2026-05-08T13:08:09.075593+00:00"
- type: run-end
data: null
timestamp: "2026-05-08T13:08:09.075614+00:00"
started: 2026-05-08T13:08:06.403Z
intent:
engine: conversation.qwen3_coder_next
language: en
intent_input: please play the gong once
conversation_id: 01KR3V7N4375EK3PEYS1X5FV0J
device_id: null
satellite_id: null
prefer_local_intents: true
done: true
processed_locally: false
intent_output:
response:
speech:
plain:
speech: The gong has been played.
extra_data: null
card: {}
language: en
response_type: action_done
data:
success: []
failed: []
conversation_id: 01KR3V7N4375EK3PEYS1X5FV0J
continue_conversation: false
finished: 2026-05-08T13:08:09.075Z
As you can see, "the gong has been played" is what the LLM thinks has happened. Of course no such thing happened — the script errored out. Note the tool_result: success: True! This is a lie!
This has never worked and will always continue. Well, stop with no message.
If oyu trap the error and set a response var and stuff it if error is true - will return the content of the response var this is how i do every bit of errhandling in my scripts. BECAUSE this.
I don't know what other specifics you need. I am using the Ollama integration, connected to a beefy server somewhere in my home, running Ollama with qwen3-coder-next loaded in the GPU. Doesn't really matter very much tho, look at the tool call debug output, it's performing as expected.
'THE' Core builtin Ollama (there are at least 6 others, not core)
You can't set a response variable on a stop action if error = True. HA doesn't even let you save the script when you do that. It would be fucking cool if you could do this — that would effectively solve my problem!
Yes yes of course — I have not specified I'm using anything from hacs, this is the builtin one we're talking about.
Alright, I see what you're saying in regards to using the error option on a stop action. You want LLMs to use that status. From what I understand, that's only used for traces at the moment in the UI, and continue on error.
you have to do this:
from:
zenos-ai/packages/zenos_ai/dojotools/dojotools_scribe.yaml at main · nathan-curtis/zenos-ai
default:
- variables:
result:
ok: false
tool: zen_dojotools_scribe
mode: "{{ _mode }}"
caller_token: "{{ caller_token }}"
error: unsupported_mode
message: Unsupported mode. Call with no mode or mode=help for full contract.
- stop: unsupported_mode
response_variable: result
YOU MUST PLAY DEFENSE
Yep we're on the same page.
Additionally I just realized I also would like any stop: error in the script to show on the UI — I was under the impression that it did (perhaps it did in previous versions of HA) but clearly nothing shows onscreen, not a popup, not a toast, not a message in the More Info dialog. I wanna know if my script failed in the same interaction as me dispatching the script.