Wow, an ESP speaker for $13 seems like such a steal! Out of curiosity, does anyone know if there’s a way to make something like the M5 work using a keyword wake up instead of pressing the button? These are small enough to hide them around the place easily as long as we can wake them using words instead of pressing a button!
Wow so much text and options for “The Voice” (let’s hope it’s more positive than the Dutch variant :-).
It seems the [ATOM Echo Smart Speaker Development Kit | m5stack-store]
Is sold out most places.
Any good alternative?
1 reply
Can the voip integration use the touchto e buttons of a phone to trigger automations?
Will this work for a PCI or USB audio device connected to HAOS? I’ve been frustrated with Genie and trying to get the Google SDK to work (can’t find device registration anymore), so it’d be rad if I could use the USB speakerphone I have connected for that purpose, especially if it can be made to listen for a hot phrase of some kind.
Awesome - thank you.
When you have already started to talk about hardware I have to ask what’s are your plans? Because after 1,5 year of playing with and using Rhasspy I can say that ESP-32 with a simple microphone (like the Atom Echo) works … but you need to be in the environment without much ambient noise, talk in the direction of the microphone and not be too far. Anything outside of this norm degrades output very quickly. So it’s good for development, but IMHO makes it a poor replacement for something like Amazon Alexa. There are reasons why Alexa is using a 4 microphone array (I think) and probably DSP for audio processing, and IMHO ESP-32 isn’t up to task (performance wise). Maybe ESP-32-S3, but last time (almost a year ago) I checked there were no open source libraries available for necessary audio processing (far-field voice recognition, beamforming, noise cancelation). So, back to my original question, what are your plans for audio processing before it is sent to STT?
1 reply
We’re looking into 2+ mic hardware solutions that can do beamforming, noise suppression, auto gain control, etc. There are plenty of boards that do this, but we also need to run a good open source wake word model on it and have it be trainable with open source, local tooling.
2 replies
Phenomenal and really exciting work! Do you all see the possibility of supporting RTSP streams as an input method for Assist? A lot of people already have cameras around that could function quite well for this.
What about voice between humans?
We currently lack two-way audio for things like doorbell cameras.
I watched the video then immediately bought 2 m5 atom echoes, a Grandstream and an old-timey phone.
any idea on how to pass model/language in docker-compose. the below is not working
## wyoming
whisper:
container_name: whisper
image: rhasspy/wyoming-whisper
volumes:
- /root/docker/whisper:/data
environment:
- TZ=Australia/Adelaide
- model=medium-int8
- language=en
restart: unless-stopped
network_mode: cust_bridge
ports:
- 10300:10300
piper:
container_name: piper
image: rhasspy/wyoming-piper
volumes:
- /root/docker/piper:/data
environment:
- TZ=Australia/Adelaide
- voice=en-us-lessac-medium
restart: unless-stopped
network_mode: cust_bridge
ports:
- 10200:10200
Awesome development!
Will I be able to connect HA to my telefone system through SIP, so I can call my home assistant from anywhere to have a chat?
This looks awesome! Can’t wait to play around with it! Though I’ll admit my main wish is that someone will make the voice of JARVIS available for it!
1 reply
Instead of environment variables you have to put in the command line options for voice, model,… under the command: tag
version: "3"
services:
## wyoming
whisper:
container_name: whisper
command: --model medium --language nl
image: rhasspy/wyoming-whisper
volumes:
- ./whisper:/data
environment:
- TZ=Europe/Brussels
restart: unless-stopped
ports:
- 10300:10300
piper:
container_name: piper
image: rhasspy/wyoming-piper
command: --voice nl-nathalie-x-low
volumes:
- ./piper:/data
environment:
- TZ=Europe/Brussels
restart: unless-stopped
ports:
- 10200:10200
+1 for support for two way audio using VoIP!! ;
I love it!
Include a BLE tracker for room presence detection as well as cabled (magnetic?) recharging and you’re golden.
Is there any chance of hooking up other existing devices with a voice assistant interface? (Such as Sonos, which are not owned by one of the big players in the voice assistant market)
This would mean quality hardware people might already have, and the privacy of Home Assistant.
2 replies
I like the way forward. 
On the other hand, until even typed commands (here German language) is not doing, what I’m expecting (or what is described or at least how I’m understanding it) in terms of “rules” of sentence spoken, etc. it’s hard to go this way further.
Is there a dedicated thread/entry/… for such problems/questions regarding the commands, etc, where they are not overlokked if hidden in release-threads, etc?
3 replies
I second this with the variation of wanting to use Denon Home speakers. However they are Google assistant and Amazon compatible as far as I know, so maybe they are already locked up?
I have similar problems… Can’t even ask assist what the temperature is in my rooms despite me having sensors everywhere. As far as I understand the command was designed for thermostats, but my sensors also know the temperature, so why am I not able to use these?
Wow … a lot to unpack there.
So for those of us who are doing this all locally without the HA cloud subscription (and I’m possibly a bit slow on the uptake) do we need all of these components … ie do we need the Piper and Whisper add ons and the Wyoming integration, plus whatever voice input/output device such as the M5Stack Echo or VOIP integration ?
I am thinking the same about my alexas… kill them and bring them alive with an esp… reuse speaker and mic…
It would be great if it was possible to connect bluetooth headsets/speakers to Home Assistant through ESP32 or other bluetooth bridge. This would open up multiple hardware solutons for TTS and/or STT.
Unfortunately I can’t get my hands on an m5stack echo in the short term. I was wondering if it would be possible to use an Ai thinker Esp32-A1s audio kit, which I have lying around.
There are two microphones, buttons and an audio out on it. Does anybody have experience with this?
Is it possible to exclude the automation domain from things that assist can control? I have some automations with similar names to lights. Assist has a habit of turning off my automations!
1 reply
This was discussed in the livestream; not part of current developments but they are looking into it.
JARVIS ? we need the ultimate computer ORAC !
With all this native support, it would be cool if the Home Assistant Android app would let you set it as the system voice assistant. I don’t have Google Assistant enabled for privacy reasons, but this I would use!
Just use the core beta and setup the wyoming integration pointed at port 10200
This so much this…
Have you looked at the wake word engine from espressif (WakeNet)?
https://docs.espressif.com/projects/esp-sr/en/latest/esp32/wake_word_engine/README.html
1 reply
We have. It works well, but you have to pay them for any customization: Espressif Speech Wake-up Solution Customization Process - ESP32-S3 - — ESP-SR latest documentation
@synesthesiam, is piper planned to support other arch in the future? I noticed the only image currently is for arm64.
Thanks!
1 reply
Excellent progress, and such a wide set of options from ESP32 speakers to VoIP!
Are there any plans to create an Android solution so that those using tablets or Sonoff NSPanel Pros (or Tuya T6E screens) can interact using voice? Something like a background app that listens for a wake word and feeds your words into HA Assist? My guess is that there may already be a Tasker-based solution that I’m unaware of, though.
Piper supports both amd64 (x86_64) and arm64. The images should support both architectures as well. Where are you seeing otherwise?
look also many years for such a badge
Sorry, maybe I need to be more specific. I run a NAS server with arm7 architecture.
I have published some interesting content that I believe will be useful for going forward with Voice in OpenAI powered assistant. But don’t care about the name.
You see what will be cool for a voice assistant to do.
Let’s say you’re cooking dinner and you run out of a key ingredient. If your Home Assistant is integrated with Grocy, no problem! Just ask your to suggest a substitute, or even offer up a recipe that uses the ingredients you have on hand. And if you’re feeling extra lazy, your it can even order takeout for you.
Great updates.
Is the Grandstream GS-HT802 (2 lines) also supported on the new VoIP integration? Would both lines be connected direct to home assistant or could one be free for regular VoIP?
Why can’t the large model be used for whisper? Unfortunately, for languages other than English, smaller models are very inaccurate and make it unusable.
There is a Rhasspy Android App (obviously without active development)
This works great with rhasspy, so i guess sooner or later something like this will be native in home assistant companion.
Would love it! Also as media Player.
But i’m also fine with the MPD App
ESPHome als MediaPlayer / Assist Satellites. That’s great as well!
So, I can call Home Assistant on a phone…can it ring the receiver for notifications?
I think I missed something from the video. Could someone give me an explanation considering that I could have done just the stupidest mistake ever?
I installed Piper and Whisper. They both triggered the integration “configure” button and clicking on it went fine as expected. I went on enabling both for the default assistant and the “try voice” command form the bottom of the TTS box let me get the nice playback of what I wrote in the box.
I still cannot make the assistant answer me with a TTS response along with the text one I obviously got until now when I asked “turn on the living room main light”. Even if the light does indeed get on.
What didn’t I understand? What is the setting that enables the voice response that the video shows in the 13:00 demo?
Why is this when I want to install piper? Is above only available next month?
You are running Home Assistant 2023.4.6, to update to this version of the add-on you need at least version 2023.5.0.dev20230418 of Home Assistant
I flashed an M5 and its in HA:
But I get a red light and following logging:
Using 'COM12' as serial port.
Showing logs:
[21:13:30][D][binary_sensor:036]: 'Button': Sending state ON
[21:13:30][D][voice_assistant:065]: Requesting start...
[21:13:30][W][voice_assistant:067]: Could not request start.
[21:13:30][D][light:035]: 'M5Stack Atom Echo a63938' Setting:
[21:13:30][D][light:046]: State: ON
[21:13:30][D][light:058]: Red: 100%, Green: 0%, Blue: 0%
[21:13:30][D][binary_sensor:036]: 'Button': Sending state OFF
[21:13:30][D][voice_assistant:073]: Signaling stop...
[21:13:31][D][light:035]: 'M5Stack Atom Echo a63938' Setting:
[21:13:31][D][light:046]: State: OFF
[21:13:31][D][api:102]: Accepted 192.168.1.142
[21:13:31][D][api.connection:961]: Home Assistant 2023.4.6 (192.168.1.142): Connected successfully
[21:13:45][D][binary_sensor:036]: 'Button': Sending state ON
[21:13:45][D][voice_assistant:065]: Requesting start...
[21:13:45][W][voice_assistant:067]: Could not request start.
[21:13:45][D][light:035]: 'M5Stack Atom Echo a63938' Setting:
[21:13:45][D][light:046]: State: ON
[21:13:45][D][light:058]: Red: 100%, Green: 0%, Blue: 0%
[21:13:45][D][binary_sensor:036]: 'Button': Sending state OFF
[21:13:45][D][voice_assistant:073]: Signaling stop...
[21:13:46][D][light:035]: 'M5Stack Atom Echo a63938' Setting:
[21:13:46][D][light:046]: State: OFF
Is it not working because of the above?
Please can you also answer above?
1 reply
Indeed, with Android you can set the app for (voice) assistance (idk for Apple). If we could set this to the HA companion app then it would require no additional hardware 
Thx, managed to buy 2 at another shop
Thanks for showing me this, i just installed. It does work pretty well though sometimes it fails to handle intent, and i have to hit the microphone even to get wakeword to still work. But damn if its not the easiest sattellite ive setup yet!
You joke, but I could seriously see someone doing this, even right now with the M5, it would really just need a battery to power it and the thing is super low energy already, it wouldn’t be that uninmaginable, and would personally love something like this no matter how much fun I get made of lol!
2 replies
Sorry if I missed it, but where can I find the ESPHome sources of the M5Stack Echo " $13 voice remote for Home Assistant" project?
I have a MAX98357A amp and a INMP441 from a similar project I did for Rhasspy.
I’m not really sure if you can do both input and output I2S audio so easily
I think it is here GitHub - esphome/media-players: This repo hosts known, tested devices that can server as media players to Home Assistant. (but why are we having to guess).
1 reply
Really interested to see if there will be support for SIP-based “paging” equipment like speakers and microphones.
The phone trick mentioned here is cool, but that Grandstream device speaks SIP, and it definitely would be easier to set up, more extensible, and a million times more awesome if Home Assistant could speak SIP as well
Can the docs be updated to include that the audio pipeline needs a HTTPS connection (even with the companion app)? Spent most of the morning messaging around with my setup & the companion app, including even trying some android sip clients to see if they’d work with the new VOIP setup, and was expecting the issue to be that the companion app needed an update not the http(local)/https(remote) HAProxy setup I have.
Suspect a fair amount will fall into this http config issue and wonder why audio doesn’t work for them.
1 reply
Piper started OK with this but Whisper is failing to start with the following. Anyone also had this issue or have a solution?
INFO:__main__:Downloading FasterWhisperModel.MEDIUM to /data stderr
09:54:29 Traceback (most recent call last): stderr
09:54:29 File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main stderr
09:54:29 return _run_code(code, main_globals, None, stderr
09:54:29 File "/usr/lib/python3.9/runpy.py", line 87, in _run_code stderr
09:54:29 exec(code, run_globals) stderr
09:54:29 File "/usr/local/lib/python3.9/dist-packages/wyoming_faster_whisper/__main__.py", line 132, in <module> stderr
09:54:29 asyncio.run(main()) stderr
09:54:29 File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run stderr
09:54:29 return loop.run_until_complete(main) stderr
09:54:29 File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete stderr
09:54:29 return future.result() stderr
09:54:29 File "/usr/local/lib/python3.9/dist-packages/wyoming_faster_whisper/__main__.py", line 75, in main stderr
09:54:29 model_dir = download_model(model, args.download_dir) stderr
09:54:29 File "/usr/local/lib/python3.9/dist-packages/wyoming_faster_whisper/download.py", line 41, in download_model stderr
09:54:29 with urlopen(model_url) as response: stderr
09:54:29 File "/usr/lib/python3.9/urllib/request.py", line 214, in urlopen stderr
09:54:29 return opener.open(url, data, timeout) stderr
09:54:29 File "/usr/lib/python3.9/urllib/request.py", line 523, in open stderr
09:54:29 response = meth(req, response) stderr
09:54:29 File "/usr/lib/python3.9/urllib/request.py", line 632, in http_response stderr
09:54:29 response = self.parent.error( stderr
09:54:29 File "/usr/lib/python3.9/urllib/request.py", line 561, in error stderr
09:54:29 return self._call_chain(*args) stderr
09:54:29 File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain stderr
09:54:29 result = func(*args) stderr
09:54:29 File "/usr/lib/python3.9/urllib/request.py", line 641, in http_error_default stderr
09:54:29 raise HTTPError(req.full_url, code, msg, hdrs, fp) stderr
09:54:29 urllib.error.HTTPError: HTTP Error 404: Not Found
No internet in the container so it can’t download the required model also not all combinations of models are available.
1 reply
Yes look up the thread. That is what I am trying to setup
Is there a list of supported models? I am after English anything that will run ok on a Synology 920+
Yes I believe so, I was trying to point to the directory with the other hardware support too.
I hit the same with the medium model too in en.
These two tries gave the 404:
--model medium --language en
--model medium
This one did not give a 404 and starts fine:
--model small --language en
Which is weird because the faster-whisper python package downloads from guillaumekln (Guillaume Klein) which has both medium.en and small.en
1 reply
Every page of documentation has a link to submit an update. There you go.
I have seen the live stream, and it is great that we now have the esphome voice assistant client (the old Rhasspy Satellite)
but is there also coming a client (satelite) for the raspberry pi or other pc software?
In my living room I have a raspberry pi touchscreen with a WM8960_Audio_HAT this one has mics build in. I bought this always with in my mind to run it as a rhasspy satelite. but now that this is build in home assistant, I do not want to install Rhasspy any more.
currently I see the falsest why to run this is to install some kind of voip/sip client on my raspberry pi. it would be nice to have some sort of voice client software.(can be in the browser it is running chrome)
@synesthesiam which is the correct address to call Home Assistant Assist using the android sip client mizudroid?
1 reply
2023.5 is released on Wednesday.
The answer is in your post. You aren’t running the required version of HA for this.
We’ve added a new [Voice Assistant](https://esphome.io/components/voice_assistant.html) component that allows you to use ESPHome devices as an input for [assist](https://www.home-assistant.io/docs/assist/) in Home Assistant 2023.5 or later.
is it work for someone with 2023.5.0dev core? is someone try it?
Because im still in 2023.4.6 core i understand i have to wait tomorow.
But in my second instance of HA which is 2023.5.0dev still not work when i press the button, i mean i doesnt have the error about start voice_assistant could not start, i mean, it seems it start but nothing append in HA, no response.
Thanks
1 reply
You should wait for the release, or seek support on discord, as you were already told on the ESPHome github. No need to cross-post the same question.
Once the voip integration is installed, it should be the IP of your Home Assistant server with port 5060.
Ah ok, my test HA is not SSL enabled. Will try that, thanks.
I have to re-ask here. Is there a thread or area where such questions (why is this or that in commands) not working oder not working as expected or …?
So instead of opening separate threads here where most probably the rule knowing people, language repsonsibles, etc. will not even see them.
2 replies
Doesn’t matter either. [] are optional parts.
but to skip this language topic, let the rule be
"aaa [bbb|ccc] ddd eee fff [ggg] <name>"
shouldn’t
"aaa ddd eee fff <name>"
work if “Heizung Büro” is the name of the climate entity?
1 reply
Sorry Tom, you are on a wrong path. Then you don’t know that this is only a rules engine, which do a word by word comparison with further rules like |, [], <>, … (see above link as well) It is completely not relevant, which are the words here. There is no meaning behind it for the filling weorkds in this rules. Even your example would work, if there is a rule, which corresponds to this schema". But your schema doen’t exist, so my answer would be “your rule does not exist” or wrongly called. The rule I want to call is existing (see link) and I call it with the example, which (in my opinion) should work because of the optional parts and the 1:1 comparison of the words, etc.". So the question is here, why it is not working or should it work and is is perhaps because of the two word entity name or …
And of course, I gave the most specific questions, even with the link to the rule.
If perhaps only the german template is not working (or the path of calling and working with the result), it doen’t make any sense to translate them. Because then I use a rule, which is not existing or another rule, which is perhaps working but without an aswer, why this specific rule is not working. Same if you would ask someone to translate jinja2 if, then, else to another language. Then the answer would be, that it is not working, because it has to be if then else.
I appreciate your help always and here as well, but if you don’t know the rule engine, then it is ofc completely fine and perhaps another one has an idea for my example.
1 reply