Setting up a 100% local smart speaker on an Android tablet using Tasker and Snowboy to handle wake word detection

As @Endlessvoid rightly answered, you will not be able to use the companion app or Assist via web browser without manual interaction. You should be able to set up the device as a Wyoming satellite device with always listening microphone and wake word detect on the HA server. I have full instructions for that on the wiki as well as Youtube videos for all of those steps.

The View Assist system and controls is currently in beta testing and I hope to have full instructions for that out soon.

Thank you. I’ve just used rtpmic on my android phone, installed and configured openwakeword and streamassist on ha server but checking the services of streamassist, my voice is not recognized. I don’t know if it just doesn’t understand the wakeword or there’s something else not working. The STT sensor is in idle.

I don’t know if it’s relevant, but i’m using azure TTS and STT instead of the local piper.

This may be obvious, but make sure the mic is not muted. I have used whisper and HA Cloud for STT both successfully. I’m not sure about Azure STT. You may want to install whisper and piper and test to see if it is a problem with Azure.

Also try turning up the gain on rtpmic.

1 Like

The microphone is not muted and i have tried all the gain settings in rtpmic (the level is correctly moving over 3/4th).

This is what i’ve in the services

Maybe there’s something wrong with the ip of rtpmic? How can i check if HA is receiving the stream?

Tried with whisper, no luck. Same as Azure/Microsoft

Perhaps. Or some sort of firewall block. I don’t know how to test mic in HA. I THINK you can probably use VLC on another computer and see if you can connecto to RTPMic and that it is streaming the audio. If that works then you can eliminate RTPMic issue I guess.

I’ve set this up as described in the original post. I’m having a problem though where the second time I trigger assist with the wake word, it triggers, but doesn’t seem to be able to hear me. I have “always start a new copy” checked in the Tasker task.

I’ve checked the logs in Whisper, and it looks like it’s just getting silence.

If I tap to close the assist popup and then use the wake word again, it seems to work well most of the time. Something about the assist popup already being open when I use the wake word causes it to break.

@janstadt, I see you are using an automation to close the popup. Is this why? Did you find any other ways around this?

Sounds like an android audio handling issue, what version of Android are you using?

Android 13 on a Pixel 4.

I think I’m abandoning this idea for now. Might come back in 6 months or so and see what the landscape is like for HA native voice assist.

I was using StreamAssist successfully for a couple days but it seems to be broken and I have no idea why :smiling_face_with_tear:

Using the debug option in the browser I’m able to confirm that there’s nothing wrong with the pipeline. It detects the wakeword and processes everything just fine. I also tested the RTP stream in VLC (you have to use rtp://0.0.0.0:{port number in RTPMic} to get it to work!) and it’s working perfectly, but it just doesn’t respond anymore. Running an up to date HASS OS install. I wonder if one of the more recent updates caused some issues with the integration?

Has anyone seen an Android Tab with speakers (decent speakers)? Im trying to find something to replace ALexa as its almost dead.

Thanks guys :slight_smile:

I assume you don’t want to use a thinksmart view? That’s the cheapest option by any measure, they’re $27 on woot right now.

I have an Ulefone Armor Pad 2 which wasn’t too pricey and the speakers on it are pretty kickass (for a tablet)

1 Like

Thanks mate :slight_smile:

But I mean good speakers lol I have found POS tablets on Ali that look pretty good. But I want something with integrated speakers. Essential something that’s an Echo Show 10, but with Android so I can use it with the Android wake word.

Did some investigations…
So we have to use the companion app for this to work and android 11.
Ok, what about turning the screen on/off though? Or increase/decrease brightness? Is that possible?
These can work with wallpanel and fully kiosk, but then we are not using the companion app and we cant use the voice assistant…

So i figured the above out, you can set a timeout and wake the device using home assistant itself, that’s epic. Is there a way to use a chime from home assistant when it’s ready to listen to commands? The hotword plugin has a ding sound, but it plays immediately and not before home assistant is ready to listen to commands.

Thank you for your guide!

I now configured Tasker, but the voice assistant dialog stays opened and until I close it manually, there is no voice input at all.

Is there any possibility to close the voice assistant windows automatically after one command?

I don’t want to use a wait for X seconds. Is there any event to grab or can I make an automation inside of Home-Assistant?

Not that I’ve found, but if you find a way please share it, I currently just use a timer.

Hi! I’m trying to recreate the OP method, but I’m having trouble. After wakeword detection, tasker opens up assist, but how then do do you get spoken text into the ‘enter your request here’ field? I cant seem to do it without button pressing the mic icon on the keyboard, and I’ve tried using autotools to press it for me, but apparently soft keyboards don’t like that? I’m complicating things by using futo for STT, but it does function as expected after hitting the mic button.
Any thoughts?

vote here to have ViewAssist and Hotword-detection build into the companion app