My voice assistant can't browse the web, help!

I have home assistant cloud configured with my home assistant Nabu Casa using OpenAI Conversation as conversation agent, speech-to-text home assistant cloud and text-to-speech home assistant cloud.

It currently can’t acces up to date information from the internet (can’t browse internet)

I have looked everywhere for the setting to enable internet browsing, but can’t find it

Go into your OAI integration you’ll see your conversation agents listed.

For the one you want to edit, lick the gear icon to edit it’s config.
In the resulting dialog scroll tk the bottom and here I unchecked use recommended settings. Then Submit.
Next pick your model, options, etc, submit
Aaaand there it is.

Buried deeeep.

But be aware, that websearch e.g. with 4o-mini can increase costs way faster than normal conversations.

See pricing under built-in tools:
https://platform.openai.com/docs/pricing

Therefore I kept this toggle off and added Gemini Flash 2.5 Lite Preview (which has a lot free websearch calls in the free tier) instead:

I wrote a script like this and gave the LLM assist access to it:

(Just used a normal script, not an intent script instead.)

Also added a comment to the LLMs prompt to tell it that it can search the web using this tool, but has to sum up the response to fit in its 150 token response.

This works amazingly well. :slightly_smiling_face:

Edit: to be honest OpenAIs pricing list makes it look like it’s cheap. But my bill raised faster than I excepted when I used it a lot. Not sure if the fact, that tokens for search aren’t included on the small models (like 4o-mini) is the problem and the search eats a lot?

Edit 2:
I looked again at the usage summary in my OpenAI account.
It really seems to be more expensive than it looks by the figures.
Maybe it uses a lot token while doing this searches.

For example one day I tested the web search:
It lists 4o-mini costs for input/output tokens of 0.04$
But 4o-mini websearch costs for the same day are 0.35$, which is more than 8x …

And I bet that not even all conversations with assist at that day used web searches.

I only get this option

Im using OpenAI Conversation, is that the correct one to use?

This is the OAI integration Im using:

Thanks for the info, I will look into it

Is your installation up to date?
There have been a lot improvements to Assist and LLMs in the past months/year.

Access to scripts might also not work otherwise.

Yes it should be, there are no updates available

Ah, why do they name all their models so similar. :face_with_monocle:

Missed it on your screenshot.
You selected o4-mini wich is a more expensive reasoning model and has no websearch.

The default model is gpt-4o-mini and has the option for web search.

2 Likes

You’re correct and was about to say the same model matersy… I believe GPT4.1-mini and maybe nano support it as well.

In fact I am having HUGE success with both 4.1-mini and 4.1-nano (and price/perf better than GPT4.0)

Op. - don’t run a full reasoning model like o.1, o.3, o.3 pro etc. unless you REALLY like burning cash. I’ve not found anything that o.whatever can do as far as HA is concerned where you need it. Mini or nano are your best bets.

Edit 2: your llm will search for answers if it does not have enough information to satisfy its request. Because the web is now a tool guess what… If you don’t want that you need to EXPLICITLY describe how you want it to act re: web search.

I have a line in my ‘directives’ part of my prompt (thou shalt nots) that describes the expectation for finding data and how to handle fallback to web search. It does it pretty reliably. If you don’t tell it and it has ambiguity… Off it searches like a happy little llm.

1 Like

Thank you, I will try that

Thank you for the information

1 Like