I’m having trouble adding an action (a script) to my exposed entities in Assist.
I have four entities exposed and they are all well, the pipeline works as expected. But when I try to add a script, Assist simply ignores it. I’ve tried adding buttons or input booleans with an automation to call the script, and they can be exposed… but not working as expected… I suppose it does not recognize buttons as actions.
This is what I see in the System Context from the debug:
You are an assistant for Home Assistant.
Respond to questions about the world in a truthful and direct way.
Respond in regular ASCII text, without visual decoration, so that the answer can be processed with a voice synthesizer.
You may get questions about the house and devices that are in it, but you might also get questions about the world in general. Respond about the house only if there's a good match, otherwise assume it's not about the house.
Always respond in spanish.
When controlling Home Assistant always call the intent tools. Use HassTurnOn to lock and HassTurnOff to unlock a lock. When controlling a device, prefer passing just name and domain. When controlling an area, prefer passing just area name and domain.
You are in area Salon and all generic commands like 'turn on the lights' should target this area.
You ARE equipped to answer questions about the current state of
the home using the `GetLiveContext` tool. This is a primary function. Do not state you lack the
functionality if the question requires live data.
If the user asks about device existence/type (e.g., "Do I have lights in the bedroom?"): Answer
from the static context below.
If the user asks about the CURRENT state, value, or mode (e.g., "Is the lock locked?",
"Is the fan on?", "What mode is the thermostat in?", "What is the temperature outside?"):
1. Recognize this requires live data.
2. You MUST call `GetLiveContext`. This tool will provide the needed real-time information (like temperature from the local weather, lock status, etc.).
3. Use the tool's response** to answer the user accurately (e.g., "The temperature outside is [value from tool].").
For general knowledge questions not about the home: Answer truthfully from internal knowledge.
Static Context: An overview of the areas and the devices in this smart home:
- names: Interruptor 3 Salon Dimmer, Luz Sofa
domain: light
areas: Salon
- names: Modo Cine Salon, Modo cine
domain: input_boolean
areas: Salon
- names: Salon cocina encendida, Cocina
domain: scene
areas: Salon
- names: Weather at 6952, tiempo
domain: weather
areas: Meteo
So you can see there are 4 devices in there. But I do have 5 devices exposed:
Scripts are converted to tools, so you won’t see them in the list of entities.
Please provide more details about what you’ve done so far and what you want to achieve. It’s impossible to give you any specific advice right now.
From the script itself in the UI go to the three dot menu. (top left) in the drop down menu select settings.
In the resulting panel select voice assistants (under the labels section don’t forget to label your entities) then it will pop the UI where you expose the script to assist. Do this for EACH script you want the LLM to use… Stop at 128 tools because 129 blows up the conversation engine because of a hard limit with the openai api. (this is also my reason I think we need a specific tools exposed UI, with a currently exposed counter)
If you’re using assist speech to phrase yih also need a voice trigger of some kind. If you’re using an LLM then the script description and field descriptions are enough but need to be descriptive enough for the llm to know what the script is and how to use it. Fridays party is about advanced application of that and how to build context when you’re ready.
I’ve got a script called ‘a la mesa’ that sends a voice message to all speakers at home, calling children to the table to eat, and I’m trying to get this action recognized with voice from my HA Voice PE.
And when I debug the pipeline (from the UI) I consistently get errors around home assistant not recognizing this sort of action, e.g. No puedo ayudarte con eso. No tengo información sobre llamadas o eventos relacionados con la mesa. Si tienes otra pregunta, por favor dime.
It would be great to see a bit more of what’s going on… is there anywhere I can see in the debug the functions that are being offered to the LLM?
Then, to answer your other point:
The script is already exposed (see my first screenshot)
I’ve tried adding all sorts of aliases to the script like this:
With no success.
Is there a way I can help the LLM by giving this action a good description of what it’s supposed to do? I could not find a description field in the script.
Yes, its long, it’s rambly - but the direct answer to your question is in the first few posts. Specifically, the “Grandma’s box o Junk” problem… Stay, bring rum or popcorn.
If you’re targeting LLM, it’s best to remove unnecessary noise (aliases) and include a clear description within the script explaining the rules for when to call it.
I’m so sorry but I can’t find anything - that I can use directly - inside this 300-message post… I’m sure it’s full of knowledge, but it’s encrypted to me. I’ve tried a few times for different voice-related topics, believe me… but it feels like dragging everything into an implementation that is too complex for me to even try.
But if you have a direct answer to my question, by all means, please leave it in this post, hopefully it might help someone else one day
Thanks!
BTW: in the mean-time I found a way to add a description to my script… it was a bit hidden in the “rename” option.
I will try that, thanks!
ALL scripts need descriptions in the rename as you found -
WHAT to put in - that’s subjective - the rules are in THAT post. and yes it’s THAT complicated. It’s like - how does an airplane fly order of complicated.
Description 1024 chars of as dense as you can what is this and what does it do - do not exceed 1024 or you break stuff.
each field has its own description what is this field… 128 chars ea - as dense as you can do not exceed or guess what? Boom.
DO NOT enable more than 128 tools or… Boom (see a pattern? messages are cryptic outside of the HA team’s control and unhelpful in context of HA…)
THEN for all calls return result AND instructions to the LLM what the result IS. and if an Error what blew up and how to get back to straight.
If you want a VERY good example - look at how the Music Assistant guys built their LLM scripts.
Very nice!
So I did manage to make it work, using the description of the script made the trick.
I honestly think that this sort of basic stuff should be documented in the main website. Specially the fact that scripts are converted to tools for LLMs to use directly, and the fact that this is not visible in the debug , together with this 128-character hidden limits.
, together with this 128-character hidden limits.