I correcty installed and configured Gemini AI to work with Assist, and it works correctly when reading or setting single entites, but im not able to execute scripts (and it looks like i cant expose scripts to voice assistants)
also, i have a (for example) this callable action “evohome.set_system_mode” or “evohome.set_zone_override” that looks like cant be accessed by Assist
any suggestion on how to access script and other things with assist?
thank you
Open your script.
Three for menu top rtt
Then the gear icon for settings
Then voice assistant
Then expose it…
Thank you, i am configured correctly, the scrpt is exposed that way, but still assist says (when i as to run the script or if assist can access script) “i can’t access scripts”
is it a Google Generative AI limit?
It’s definitely something unique to Google if it’s exposed properly. I don’t use gemini in my LLM work
i improved my prompt to “You can also access scripts”, it worked for a couple of time then it tried to execute “set evohome on away” and “set evohome to follow program”, but then the “set evohome on away” give me a “i did not found any device or zone called evohome”!
Yeah it’s either a tools use issue where the model just doesn’t understand or a context issue then.
This isn’t the first time I’ve heard issues with it.
Some models use tools better than others. For.cloud models openapi mini models are great with tool use. I haven’t tried the more recent gemini models. Local you’ll need one explicitly built for ‘tool use’
You could combat it by trying to explain to the model hey this is a tool you can use scripts to do these things. Here’s the list and how it works… But honestly I’d rather fight with it over whether it can automatically save my preferences or not whether it can actually even use my system.
Care to share your settings in the integration. An idea of what your prompt looks like?
that’s a quite simple gemini ai configuration, Gemini AI configured as “Conversational Agent” with the following “instructions”:
Act as a voice assistant for Home Assistant
you can access devices, entities and scripts.
Answer in the language the command is sent (if i talk to you in italian, reply in italian)
Keep track of name of devices and aliases every time you ask for a clarification or a correction on it
also kept the reccomanding settings
Most of commands works, it sometimes struggle with scripta and aliases but im refining it
You have two different problems then
First honestly if your prompt is that simple and it’s having problems with scripts it tells me this model either a has tool use issues or b it’s core prompt (not you, a level below) isn’t explaining the tools properly. (I think the goog and oai integration are very similar under the hood so I find it hard to believe the info isn’t being passed to Google (I’ll check when I have time.). But you won’t be able to fix that if it’s the case.
The second issue you absolutely can fix. Context.
Read top to bottom and ignore my crappy typos. Context matters.
I will read it, and also i will focus more on HA + LLM, despite its about 3 years i use AI in my jobs, its less than 2 hours i use it in HA 
1 Like
You missing a trigger.
Create a button to run the script. Expose the button. I make an alias such as “Pause the 3D printer function” for the button.
Gemini 2.0 flash