Setting up a 100% local smart speaker on an Android tablet using Tasker and Snowboy to handle wake word detection

I like your “Hey Computer” reccomendation. That seems to be working much better. “Hey Google” was also working well.
So, I’m using Automate and trying to figure out how to tell when Assist is done “talking”. I ended up using AutoVoice’s ability to listen for noise levels with some extra delay nodes around it to time out when it should be listening. When Assist stops talking it will fire the “back” interact node clearing the dialog window and allowing me to start over with a new command.
Does anyone else have a better implementation yet? I tried checking if media is playing, but I couldn’t seem to get that to work.
I should note I’m not normally an android user. This is all pretty fresh for me. Hoping someone here that is way smarter than I has figured some of this out.

This integration from AlexxIT is an absolute game-changer for me to use Assist !

Now I can have every camera and every phone/tablet to act as voice assistant.

I already configured it and works great, the only 2 small issues I have are how to increase the sensivity of camera’s michrphones and that if I select player entity my alexa media players, they do not work. If I select my wall tablet, it works.

1 Like

Hi folks, Running A11 on the thinksmart lenovo ex Teams device.

What is the preferred method for local wakeword detection?

Tasker/Automate + Wakeword plugin → ?
Do you need to hand off to StreamAssist or run HA app and hand off to the assistant.

My goal would be to use local wakeword, then capture the audio stream to send to my Home Server (with a GPU) for the Piper/Wyoming and even LocalAI stuff.

Does anyone have a good write up of the steps?
TIA

You can either do local wakeword detection with tasker/automate & hotword plugin to call the HA assist pipeline through the HA app. Or you can stream continuous audio to openwakeword on your HA server through streamassist.

The first method is described in the top post.

The second method can be found in more detail here: Home · dinki/View-Assist Wiki · GitHub

1 Like

Hello guys,
i’m trying to use this method Home · dinki/View-Assist Wiki · GitHub with an android phone as remote mic but it doesn’t work with openwakeword. If i open up the assist interface (only trough remote connection, not local because of https) and click on the microphone i can use the phone correctly. No way to get it work with openwakeword. Any ideas?

You may want to put questions about Dinki’s method in his thread: View Assist - Visual feedback for Assist voice assistant on an Android tablet (Install info provided on Wiki)

Or on his github page: dinki/View-Assist · Discussions · GitHub

But it sounds like you’re trying to mix his method with the HA companion app for android, which isn’t possible. His method bypasses the companion app and streams all audio continuously to the HA server for wakeword detection (using openwakeword) and STT processing.

If you want to use the HA companion app “assist” function, you’ll have to stick to the method described in the top post, which is currently incompatible with openwakeword, and uses snowboy for local wakeword detection instead.

1 Like

As @Endlessvoid rightly answered, you will not be able to use the companion app or Assist via web browser without manual interaction. You should be able to set up the device as a Wyoming satellite device with always listening microphone and wake word detect on the HA server. I have full instructions for that on the wiki as well as Youtube videos for all of those steps.

The View Assist system and controls is currently in beta testing and I hope to have full instructions for that out soon.

Thank you. I’ve just used rtpmic on my android phone, installed and configured openwakeword and streamassist on ha server but checking the services of streamassist, my voice is not recognized. I don’t know if it just doesn’t understand the wakeword or there’s something else not working. The STT sensor is in idle.

I don’t know if it’s relevant, but i’m using azure TTS and STT instead of the local piper.

This may be obvious, but make sure the mic is not muted. I have used whisper and HA Cloud for STT both successfully. I’m not sure about Azure STT. You may want to install whisper and piper and test to see if it is a problem with Azure.

Also try turning up the gain on rtpmic.

1 Like

The microphone is not muted and i have tried all the gain settings in rtpmic (the level is correctly moving over 3/4th).

This is what i’ve in the services

Maybe there’s something wrong with the ip of rtpmic? How can i check if HA is receiving the stream?

Tried with whisper, no luck. Same as Azure/Microsoft

Perhaps. Or some sort of firewall block. I don’t know how to test mic in HA. I THINK you can probably use VLC on another computer and see if you can connecto to RTPMic and that it is streaming the audio. If that works then you can eliminate RTPMic issue I guess.

I’ve set this up as described in the original post. I’m having a problem though where the second time I trigger assist with the wake word, it triggers, but doesn’t seem to be able to hear me. I have “always start a new copy” checked in the Tasker task.

I’ve checked the logs in Whisper, and it looks like it’s just getting silence.

If I tap to close the assist popup and then use the wake word again, it seems to work well most of the time. Something about the assist popup already being open when I use the wake word causes it to break.

@janstadt, I see you are using an automation to close the popup. Is this why? Did you find any other ways around this?

Sounds like an android audio handling issue, what version of Android are you using?

Android 13 on a Pixel 4.

I think I’m abandoning this idea for now. Might come back in 6 months or so and see what the landscape is like for HA native voice assist.

I was using StreamAssist successfully for a couple days but it seems to be broken and I have no idea why :smiling_face_with_tear:

Using the debug option in the browser I’m able to confirm that there’s nothing wrong with the pipeline. It detects the wakeword and processes everything just fine. I also tested the RTP stream in VLC (you have to use rtp://0.0.0.0:{port number in RTPMic} to get it to work!) and it’s working perfectly, but it just doesn’t respond anymore. Running an up to date HASS OS install. I wonder if one of the more recent updates caused some issues with the integration?

Has anyone seen an Android Tab with speakers (decent speakers)? Im trying to find something to replace ALexa as its almost dead.

Thanks guys :slight_smile:

I assume you don’t want to use a thinksmart view? That’s the cheapest option by any measure, they’re $27 on woot right now.

I have an Ulefone Armor Pad 2 which wasn’t too pricey and the speakers on it are pretty kickass (for a tablet)

1 Like

Thanks mate :slight_smile:

But I mean good speakers lol I have found POS tablets on Ali that look pretty good. But I want something with integrated speakers. Essential something that’s an Echo Show 10, but with Android so I can use it with the Android wake word.