Voice assistant setup help - VPE, Custom wake word, Google

Configuration Voice Assistant
1

Hello Team,

I’m trying to set up now the voice control for my home.
Finally I more or less finished adding the devices, switches and so on, so now I want to finish the voice control.

I have 2 voice preview edition speakers as well.
I’ve set up the voice assistant, registered to the HA cloud service which is now live, and added TTS and STT to work with it.

I also trained a custom wake word, which is set in the assistant’s wake word.

What I would want to achieve:
1 - To make the speaker react to what I’m saying. Turn on, or off stuff.
It’s still very wonky. For example, I have a light called “sexy baby”. Anytime I want to control it, it can’t find it. When I ask ‘hey google’ in the google home integration, it turns it on.
If I say “OK Nabu, turn on the device called sexy baby.” → It replies: "Sorry, I can’t find any device called “device called sexy baby.”

2 - To be able to have a conversation with the speaker/assistant.
For example if I ask what’s the weather, will it be able to answer?
Or add things to the shopping list?
I mean, I started all of HA to switch from Google Home, which is buggy as hell, but what’s 2 click in Google, it’s 2 days of workaround in HA (and probably expensive) Getting really crazy.

3 - To be able to use my custom wake word instead of ‘OK Nabu’.

4 - Google Home control/integration
I would like to have them fully connected, so the VPE speakers would be connect to Google Home/Google Assistant.
It means that when I say ‘Hey Google’, it should be also heard by the VPE, and not just my phone.

Any advice would help. :slight_smile:
We can also have a chat for easier communication on Discord, I don’t mind.

Thanks,
Szabi

2

image
image806×830 43.3 KB

image
image818×764 16.3 KB

image
image507×1079 35.1 KB

image
image1230×548 42.7 KB

3

I assume you have read this and all its related links, especially the best practises bit?

To have decent conversations with assist you will need some kind of AI.

Custom wake words is whole extra learning curve and the are plenty of threads here about that if you use the search.

1 Like
4

Slow down and pick one problem.

You just listed 4-6 major surface areas each with its own major hurdles.

Which do you want first.

For instance

Voice control - wouldn’t even bother until you have control. (was the problem voice or just a problem)
Custom wake word. After all these other things (you’ll be fighting false positives and no calls)
Google - have you worked out how you do this yet?

You do NOT want to be solving these simultaneously because they all overlap and impact each other.

First get ha working with yojr chosen stuff.
Next make that Google link work. Meanwhile learn about what’s required for HA voice an d your options THEN implement your choice… And finally the custom wake word.

Try and do them all at once, you will lose sleep and hair as you rip it out.

2 Likes
5

Thanks both for response. I’m reading all kinds of threads and reddit and all for weeks now, my eyes are bleeding soon. :smiley:
Most of the times, people have slightly different “problems”, which makes all those readings a bit different.

The priority would be 1-4, and one by one, not all the same time.

For now, I have all my devices connected to their places, automations etc. set, switches switching, and so on. Everything has names, areas, set properly. Controlling my HA works fine from the UI. This is why I start Voice control now.
I said “more or less”, because I still have some buggy devices which I have to add later to the system, but it shouldn’t matter.

Google connected, all devices (supported) are visible from google home.
I have a new pixel 10, and I just finished setting up the “Hey Google” properly on this phone.
It works now with Gemini, which can learn whatever I want pretty fast.
For example, “Sexybaby” device was against it’s guidelines, only worked when I added “light at google home”.
Now I had a convo with it, and it learned that sexybaby is a light device, and not something agaist the guidelines, so now it works simply: “turn on sexybaby” → turns on.

So 2 separate things are working together on their own.
My next goal would be, to have easy and simple control with my VPEs.
1st step would be “Ok Nabu” to work properly.
It was very slow before, but now it’s pretty nice, like 1s to response.
Although, it won’t understands almost anything.
Some turn-on turn-off commends are working, but that’s also failing like 2/5 times, not sure why.

After simply turn on and off would work easily, I would like to have a conversation with it, if possible.
This could go 2 ways - either make it use some AI (for example my google phone?), or make Google Home/Gemini to be able to use the speaker.
If I would go “Hey Google, what’s the weather today?” and the speaker would react and tell me (through google home/gemini), that would also be perfect.
If not, then the same with “Ok Nabu, blabla”.

And at the end of this all, I’d like to call my assistant Consuela. :smiley:

6

The biggest issues you will be having is reading threads and reddit stuff that is way out of date combined with some that is new. This will at best lead you round in circles at worst just give you the wrong answers. Voice has come a long way over the last year or 2 and I would say anything over year old will be of no use at all.

As suggested start with basics work your way through that link posted above, step by step, understand each step and get it working then move to the next.

A lot will depend on the speech to text engine and its settings amongst other things, so one step at a time.

3 Likes
7

I’m trying to add Google Gemini integration, and use my free API key.
Let’s see how that works.
Google says a light HA setup will use very little of it. Even if paying-by-use, it’s like dollars a month.

8

No point setting up AI if it you can’t get it to turn a light on, it won’t help.

9

Yes it did.

So basically, when talking to Assist in voice, it always STT-> “Sexy Baby”, but the device called “Sexybaby” without space, therefore it won’t find it…

Now that Gemini API is added, when chatting with Assist, writing “Sexybaby”, it will turn on and off the light instantly, and locally without calling Gemini.
If I type it with space “Sexy Baby”, instead of giving me negative feedback, it will think a bit more (calling Gemini) and turn the light on and off properly.

Although the VPE now gives “I can’t get response” for literally anything after “OK Nabu”.

10

Read the docs first (link above), then do what they say.

1 Like
11

As far as naming, you can create alias’s in the entities settings.

Find your device, click on the entity you want to control (the light on/off switch), click the gear at the top right, look for the voice assistant menu, add alias button at the bottom.

image
image771×615 30.7 KB

add an alias for ‘sexy baby’ and see if that works

1 Like
12

Added the aliases, but it’s even worse now.
The Asisst got stuck in the chat, and the light won’t even turn on at all (not even from the AI API). :frowning:

Also, since I set up the Gemini API for the voice assistant, the VPE will not work at all. Sometimes it says can’t get a response, most times it loads white for a couple of seconds, and turn off without saying or beeping anything…

Should I try Open AI API?
If yes, which model would be the best for simple conversation like random questions and weather (besides control). I don’t want to generate any music or picture nor code through my HA. :smiley:

13

Changed to OpenAI API, and chaching, the VPE also works and answering simple questions like “What’s 2+5?”.

Next up, custom wake word and running conversation instead of always saying OK Nabu. :slight_smile:

1 Like