Prefer handling commands locally does not seem to work

I have configured assist to use qwen-2.5 as the conversation agent, while also ticking the “Prefer handling commands locally” toggle, which to my understanding, would mean that qwen is used only for things that home assistant itself can’t do.

However, it seems to use qwen for everything, even a simple “what is the weather” or “turn of the kitchen lights”, which also work with the home assistant agent.

Any ideas why this happens?

What does your voice debug log for that assistant say:

Under Voice Assistants >
Assist >
Pick your agent

Select the three dot vertical ellipsis menu to the right >
Click it for the dropdown >
Select Debug…)

Ok, my bad I found it now. I was looking at the conversation agent but didn’t see that under the conversation agent there was a field saying “Processed locally: true”.

So apparently they were processed locally, I just saw the conversation agent as qwen and got confused.