I am having the same problem…
There’s both a language AND country list there. Even though one’s language may be on the list of supported languages, not all regions are supported. I’m Dutch and thus live in The Netherlands. Dutch is supported, but th region Netherlands is not 
1 Like
I’ve looked around, but I’m finding trouble getting tips on configuring the prompt template:
This smart home is controlled by Home Assistant.
An overview of the areas and the devices in this smart home:
{%- for area in areas() %}
{%- set area_info = namespace(printed=false) %}
{%- for device in area_devices(area) -%}
{%- if not device_attr(device, "disabled_by") and not device_attr(device, "entry_type") and device_attr(device, "name") %}
{%- if not area_info.printed %}
{{ area_name(area) }}:
{%- set area_info.printed = true %}
{%- endif %}
- {{ device_attr(device, "name") }}{% if device_attr(device, "model") and (device_attr(device, "model") | string) not in (device_attr(device, "name") | string) %} ({{ device_attr(device, "model") }}){% endif %}
{%- endif %}
{%- endfor %}
{%- endfor %}
Answer the user's questions about the world truthfully.
If the user wants to control a device, reject the request and suggest using the Home Assistant app.
What is the syntax for supplying the integration with more than one area, entity, device, device_attr? If someone would be willing to share an example of their template, that would help me lots!
TIA
I’ve used this very successfully to send informative, time-saving notifications. The next big thing would be to be able to use the responses as conditiones.
One example would be to disarm the alarm system when no cars are in the garage, and arm it when at least one car arrives.
Has anyone figured out this part?
You may need to create a helper to store the garage car count. I wouldn’t be terribly surprised if you would be able to count the number of cars with frigate alone - not using Gemini. If you did want to use Gemini, then you would run the automation with a time frequency trigger and store it as a helper value for use as a condition of another automation.
1 Like
Erro:Error generating content: 404 Gemini 1.0 Pro Vision has been deprecated on July 12, 2024. Consider switching to different model, for example gemini-1.5-flash.
IM Getting this error, someone can help me?
I don’t know if you solved this, but according to this, we have upgrade (HA Core, I think):
Hey @tronikos, i saw that you already replied that you don’t have plans to add function calling, but i think that lots of us will be very pleased about it. Maybe you have reconsidered it ? 
I am getting a little confused with these different models.
Gemini Pro 1.5 is the latest “Advanced” model that would benefit from a Gemini Advanced subscription, right?
The Gemini 2.0 Models are all “flash” and thus the dumbed down “free” models, correct?
I think I might need an AI to help me set up my AI…
It’s supported since 2024.6. See 2024.6: Dipping our toes in the world of AI using LLMs 🤖 - Home Assistant and AI agents for the smart home - Home Assistant
Thank you. What i meant is something like the extended open ai custom integration, where you can define custom ‘spec’ to ‘teach’ the agent to do things (like check attributes, history, change correclty loghts colour and so on). Is it possible somehow with the gemini integration? Maybe i’m dumb (probably 
) but i can’t find something related in the links you provided
1 Like
You can do many of these (except history) with scripts exposed to assist. Gemini will try to call the script with the right parameter. It will help if you add a description to the script to help Gemini what the script does, when and how to call it. The script can take action (e.g. send you a mobile notification) or return things to Gemini (e.g. weather, calendar, etc.). Gemini will also be able to chain the scripts, e.g. “send to my partner my schedule for tomorrow”.
I just found something interesting…
I switched the Gemini Agent to the new “1206” in the Integration and asked it one of my usual test questions (how much power is my Solar system currently giving me).
It usually just tells me the value for my carport system and not the rooftop one unless I specify.
Now I told it to remember that my solar system consists of two inverters and then asked it again, and it gave me the correct answer, summing up both inverters.
I have to wait a while and test again, but if I can actually explain things to the Gemini Agent and it remembers it, then this could be a huge step forward and might actually make AI in Home Assistant useful.
Edit:
Sadly the AI Agent forgot after a minute or so and had to be reminded again.
Also, the experimental 1206 Model stopped working after a few prompts. Probably some rate limit although I have the subscription.
Do you know what I would really love to see?
Seperate, small AI agents that I can train for a specific task. Something like the Gemini Gems or Gemma (forgot the exact details).
An example:
I could tell one of those little “Agents” to take control of my AC. For that purpose I would give it some entities to play with like the inside and outside temperature, current energy consumption, energy price etc, then tell it to keep my house in a certain temperature range as efficiently as possible and then let it work out the best solution. This would (theoretically) replace static automations and algorithms with an adaptive system. As long as there is an easy on/off switch for each agent the risk is minimal and the reward could be huge.
Same goes for the Ventilation system, heating etc.
Currently I don’t see any way of implementing this in HA, but I really hope we’ll see it eventually. And since these models would be very specialised it might even be possible to run them locally on relatively weak Hardware.
you can do this with extended open ai conversation. I’ve tried but it’s pretty slow since you have to sum the latency for every agent involved. Take a look here Multi Agents in Home Assistant using Extended OpenAI Conversation - Share your Projects! - Home Assistant Community
1 Like
Thank you @tronikos, maybe you can help me a little more. I need gemini to be able to read some attributes form an entity, how can i make a script that is dynamic? I mean, i have something like 20 sensor with the epg of tv channels. The upcoming shows are stored in two attributes, today and tomorrow. Can i make a script that gemini can use for every entity or i need to make 20 script, one for every channel?
Nevermind, i solved with a template in the prompt. I still miss history of entities and correct use of ligth. Gemini can turn on/off correctly but seems to have problem with lights colour and temperature… I had same problem with open ai solved with a specific function
Thanks. This does look interesting, but it’s not what I am after.
Most things I have found so far are essentially “voice control” implementations, but I don’t want to use voice commands. I think what I would need is an integration that autonomously takes control of a certain task, like controlling the heating or cooling. The AI Agent would get an initial instruction and then teach itself how to best execute it’s function. Something like that is probably still a few years away.
I have some strange behaviour here. I’ve both gemini and extended open ai configured. I saw in the log that gemini is calling the open ai conversation agent, why?
Here the debug
2025-01-26 21:20:41.668 DEBUG (MainThread) [homeassistant.components.google_generative_ai_conversation] Response: [text: “Certo, signore. Metto su Italia Uno nel salotto. Spero che la programmazione sia più interessante di quanto mi aspetti.\n”
, function_call {
name: “MassPlayMediaOnMediaPlayer”
args {
fields {
key: “radio_mode”
value {
string_value: “normal”
}
}
fields {
key: “query”
value {
string_value: “Italia Uno”
}
}
fields {
key: “area”
value {
string_value: “salotto”
}
}
}
}
, text: “\n”
]
2025-01-26 21:20:41.670 DEBUG (MainThread) [homeassistant.components.google_generative_ai_conversation] Tool call: MassPlayMediaOnMediaPlayer({‘query’: ‘Italia Uno’, ‘radio_mode’: ‘normal’, ‘area’: ‘salotto’})
2025-01-26 21:20:41.906 INFO (MainThread) [custom_components.extended_openai_conversation] Prompt for gpt-4o: [{“role”: “system”, “content”:…
2025-01-26 21:20:44.052 DEBUG (MainThread) [homeassistant.components.google_generative_ai_conversation] Tool response: {‘speech’: {‘plain’: {‘speech’:
What am i missing here?
Looks like this only happens if i use it via voice assistant pe, if i use assist via lovelace work as expected, only gemini call
Hey @tronikos i’ve made a script to change source on my sky q set top box. Looks like gemini calls the script just the first time, looking at the logs he didn’t make a tool call at all.
2025-02-06 18:11:12.862 DEBUG (MainThread) [homeassistant.components.google_generative_ai_conversation] Response: [text: "Certamente. Quale dei due canali Sky Cinema Uno desidera, signore? \"Sky Cinema Uno\" o \"Sky Cinema Uno +24\"?\n"
]
2025-02-06 18:11:22.561 DEBUG (MainThread) [homeassistant.components.google_generative_ai_conversation] Response: [function_call {
name: "seleziona_il_canale_tv"
args {
fields {
key: "source"
value {
string_value: "Sky Cinema Uno (301)"
}
}
fields {
key: "media_player_entity_id"
value {
string_value: "media_player.sky_q_mansarda"
}
}
}
}
]
This is the first time, correctly call the script. Then i just get an asnwer that tells me that he’s gonna change the source but he don’t. No function call in the logs.
It’s just the model that doesn’t understand correctly?