Make "what is the weather" smarter?

Configuration Voice Assistant
1

The baseline here is asking about the weather and forecasts with Alexa. That allows all manner of queries about the weather, the forecast and different areas. I’m wondering what it would take to get the assistant to be as versatile.

Alexa allows for all kinds of queries. Some assume current location. But will lookup another if you specific “in location-name”. What’s the forecast? What’s the weather tonight|tomorrow|day-of-week. It’s handy to use this for upcoming travel. As in, you live in one area and are traveling to another. Alexa will readily handle “What’s the forecast for tomorrow in other-location?” Our kid loved to ask it for temps in oddball locations, like apparently the coldest place in Russia: “What’s temperature in Oymyako?”

There are indeed all kinds of other concerns/problems/issues with using an Alexa, that’s not what I’m discussing here. I mention it as a baseline for comparison.

Can we get there with HA?

2

With a LLM that has web query capability. Yes. Easily.

3

Examples of which being? I mean, sure, as a concept it’s certainly possible, but a bit of concrete steps would be illuminating.

4

Any llm which has the tool use capability which has an appropriate tool that can call out an search the web can do this.

To date using the default api calls and chatgpt this was not possible. But now that you see them adding tool use capability to the voice pipe (that’s what MCP is that was added in 2025.2)

Sonwhat happens if you setup the right tools. You say hay llm get x. It searches the elweb and gets the answer and…

1000005230
10000052301080×2340 175 KB

You can also do something like this pointed at a custom ollama installation with tools or anything that allows tool use to be exposed to the ai on its end. MCP allows the tools to ve connected direct to HA.

The beauty is once your llm can use an external to provide capability… it just does

Id pay a lot of attention to the year of voice chapter 9 stuff coming out this week. I strongly suspect it’s a lot of what you’re looking for. It was probably what was supposed to be the ‘star’ of February before backup blew up like it did in January.

2 Likes
5

Thanks, I’ll keep an eye out for it.

6

@NathanCu I’ve been trying to figure out the MCP stuff on my own, but I just can’t make heads or tails of it.

Is there an ELI5 on how I can get MCP connected to HA to achieve what you provided in your example here? I have HA connected to Google Generative AI (Gemini) temporarly and I’ve looked at the MCP (not server) integration, but I need to run my own MCP server somehow to get the URL. This is where it falls down for me. Any guidance for a LLM / MCP noob?

1 Like
7

try this

Not a particularly elegant solution to the problem, but using search allows you to retrieve any information you want

2 Likes
8

Functionally this is EXACTLY what I am looking for, thank you! It would be nice to understand behind the scenes and / or build the pipeline myself so I can switch to a local LLM or other, non-Google in future. But for now this is great, thank you!

I’m just using Gemini right now since it is free and I can figure out how this stuff works. My goal is to replace my current Alexa devices with HA devices.

1 Like