Conversation agent ignoring "Prefer handling locally"

Configuration Voice Assistant
1

I am asking my LLM-powered conversation agent questions that should be handled locally (“What is the attic temperature?”). In fact if I ask it using an Agent not connected to an LLM, it answers correctly instantaneously.

IMG_9730
IMG_97301125×1548 131 KB

However, if I ask the same exact question to my LLM-powered Agent with “Prefer Handling Locally” turned on, it will NOT process locally, take 30 seconds, and gives a long-winded and wrong response.

AFAIK “Prefer locally” means regardless of the LLM’s capacity and capability, as long as the original command can be handled by vanilla HA Assist, it will execute instantly.

IMG_9731
IMG_97311125×2090 238 KB

2

Enter your phrase What is the attic temperature into “Developer Tools - Assist.” And check the output.

3 
intent:
  name: HassGetState
slots:
  name: attic temperature
details:
  name:
    name: name
    value: attic temperature
    text: attic temperature
targets:
  sensor.quietcool_attic_temperature_adjusted:
    matched: true
match: true
sentence_template: >-
  (do you know|tell me|<what_is>) [the [current] (state|value) of] <name>
  [<in_area_floor>]
unmatched_slots: {}
fuzzy_match: false

From what I might be able to see, the agent is unable to call HassGetState?

4

In fact, everything is correct, and the priority of local commands should work. But this naming isn’t good practice.
Try experimenting with the entity name.

There’s no intent to get the room temperature in HA, but it’s very easy to create your own automation.
In the trigger, you request the areaas a variable. And in the response, using several filters (labels and the areafrom the trigger are sufficient), you get the result. This is the most sensible approach.

1 Like
5

I did recently go through my different device and area aliases. However, this happens on all of my devices, when asking for the state.

“Is the garage open?”
“Is the gate open or closed?”
“What’s the temperature in the Office?”
“Is the office fan off?”

Local Assist is able to handle them all flawlessly, instantaneously.

Asking any of those questions via an LLM-powered Assistant bypasses local processing and relies on the LLM, which is much slower, and sometimes incorrect.

It does seem like LLM agents are unable to call HassGetState, and it seems to be chugging on GetLiveContext (fwiw I have 54 entities exposed).

Been experimenting with this now a few days and now I’m back to just using a non-LLM conversation agent and just making sure my aliases and sentences are clean.

6

There’s no HassGetState tool. GetLiveContext performs this role for LLM.
Check your setup again to understand why local intent priority isn’t working. Start by checking basic things, like turning on the lights in the room.

Indeed, these intents are filtered when selecting an LLM agent.

Creating your own automations remains the only option (besides editing pipeline.py) for obtaining local status reports when using LLM.