Raspberry Pi as a CHAPTER 5 voice assistant

I have been using it for 2 weeks, and for me, the respeaker 2 hat is not good enough, the record have a lot of noise, even when I use the noise suppression, faulty are still too much.
I tried with the hikvision ds u02 webcam and it is more way better.

By the way I am finding a solution for continous conversation. On esphome we can use method “start_continous” for trigger listenning. Do we have any simple way here for doing the same?

Yes, that is exactly what I have to do.

Yes I did & the reason I bought the ReSpeaker 2-mic HAT. Silly me thinking I’d get it to work perfectly if I had the same hardware as the instructions. :rofl:

One thing I noticed is, the LEDs on the HAT do not turn on when starting up from a power failure (or turned off/on) but do as soon as I reload the device in the Wyoming Integration.

Right now, it’s not a show stopper & will play with it as I get the time. Now I’m playing with custom sentences to control my thermostat since there’s no built in intents for that yet. I know voice in HA is still in it’s infancy (but am impressed how much they accomplished so far) and am excited to see it improve going forward.

I do hope in the future, Nabu-Casa create their own satellite hardware for voice assistants similar to what they did for HA Blue, Yellow etc… The small audio coming from various ESP devices doesn’t work for me. My ultimate goal will be to ditch my various Google/Nest assistants to control my household.

Thanks again for your help.

Thought I’d provide an update. I created my 1st kiosk based on a touch screen and RPi4. After getting that to work, I install the mic/sound hat & the Wyoming voice services & got them working too.

Guess what, the RPi4 voice satellite reconnects to HA automatically after powering down & back up. Go figure. :rofl:

2 Likes

I too am using an Anker PowerConf S330, and have the same never-ending arecord issue. Did you get past it?

For now I’ve reverted back to the deprecated homeassistant-satellite build.

sadly not.

@ ignacio82 I’ve had the homeassistant-satellite build back running since yesterday, and it works reasonably well. I notice that it too has a continuously-present arecord process, so I now assume that’s just how it works and is not an symptom of something working incorrectly.

However, my wyoming-satellite build did have issues:

  • initially it responded well, but while replying to my request, would concurrently reissue the awake sound and wait on a second request; this behaviour was fairly consistent, until…
  • later, it would only respond occasionally to the wake word, and either never return from my request, or return after a minutes-long pause with the “sorry, I didn’t understand” message.

These made the wyoming-satellite build unusable for me at the present time.

Hi everyone, question. Is it possible to have the wyoming satellite pass Assist’s response to another media_player? I know how to using an esphome. I would like to use the pi for just input.

I achieved this by using the synthesize-command flag to forward the generated text to a custom event in home assistant and picking that up with node-red to send it to any media_player with tts.speak.
I can send you the config when i am home later, if that approach sounds interresting to you. It is somewhat roundabout though…

1 Like

If it works, it works. Yes, I’d really like that. Thanks :+1:t2:

Sorry, totally spaced on getting back to you. So here it is:

Assuming /home/pi is where you "git clone"ed to:
create the file
/home/pi/wyoming-satellite/examples/commands/synthesize_custom.sh
with the content

#!/usr/bin/env sh

text="$(cat)"
echo "Text to speech text: ${text}"

token='LLA_TOKEN'
echo "${token}"

curlData='{
  "event":"synthesize",
  "satellite": "snapcast-livingroom",
  "text": "'$text'"
}';
echo "$curlData" | jq '.'

curl \
  -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $token" \
  -d "$curlData" \
  https://HASS_FQDN/api/events/satellite_tts

create yourself a long-lived access token and put it for LLA_TOKEN, and fill in the FQDN of your home assistant instance for HASS_FQDN.

Then, assuming you installed on an OS using systemd, make your

/etc/systemd/system/wyoming-satellite.service

look like so:

[Unit]
Description=Wyoming Satellite
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
ExecStart=/home/pi/wyoming-satellite/script/run \
  --debug \
  --name 'snapcast-livingroom' \
  --uri 'tcp://0.0.0.0:10700' \
  --mic-command 'arecord -D plughw:CARD=Device,DEV=0 -r 16000 -c
1 -f S16_LE -t raw' \
  --snd-command 'aplay -D null' \
  --synthesize-command 'examples/commands/synthesize_custom.sh'

WorkingDirectory=/home/pi/wyoming-satellite
Restart=always
RestartSec=1

Restart=always

[install]

Note the synthesize-command-line and sending snd to null.

Now, anytime the satellite gets send synthesized text, it should create an event in home assistant with the name of the satellite and the text to be spoken. I use this information within node-red like so:

[{"id":"c57a4a8e82851f13","type":"server-events","z":"a533baf02563f05c","name":"","server":"2452f89c.1b7828","version":3,"exposeAsEntityConfig":"","eventType":"satellite_tts","eventData":"","waitForRunning":true,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"eventData"},{"property":"topic","propertyType":"msg","value":"$outputData(\"eventData\").event_type","valueType":"jsonata"}],"x":130,"y":400,"wires":[["8a0a12e62be88756"]]},{"id":"eb18cd0b8af1bf7d","type":"api-call-service","z":"a533baf02563f05c","name":"","server":"2452f89c.1b7828","version":5,"debugenabled":false,"domain":"tts","service":"speak","areaId":[],"deviceId":[],"entityId":["tts.piper"],"data":"{\"message\":\"{{payload.event.text}}\",\"media_player_entity_id\":\"media_player.snapcast_player\"}","dataType":"json","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","x":540,"y":400,"wires":[["50f6c0501580df66"]]},{"id":"8a0a12e62be88756","type":"switch","z":"a533baf02563f05c","name":"satellite_name","property":"payload.event.satellite","propertyType":"msg","rules":[{"t":"eq","v":"snapcast-livingroom","vt":"str"},{"t":"eq","v":"snapcast-kitchen","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":360,"y":400,"wires":[["eb18cd0b8af1bf7d"],["b142937ea983f692"]]},{"id":"2452f89c.1b7828","type":"server","name":"Home Assistant","addon":true}]

Basically listen for the event type “satellite_tts”, route by satellite and call the tts.speak service on whatever compatible media_player you like.
Hope this helps you along :slight_smile:

2 Likes

Thanks! Im going to try that this weekend.

So I am using a reSpeaker and a Pi 2 W. I setup everything according to the tutorial. What I am confused about is that the wake word specified in my systemd unit file is not being used. It seems to be overriden by the Assist pipeline

[Unit]
Description=Wyoming Satellite
Wants=network-online.target
After=network-online.target
Requires=wyoming-openwakeword.service


[Service]
Type=simple
ExecStart=/home/pi/wyoming-satellite/script/run \
        --name 'kitchen-voice' \
        --uri 'tcp://0.0.0.0:10700' \
        --mic-command 'arecord -D plughw:CARD=seeed2micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw' \
        --snd-command 'aplay -D plughw:CARD=seeed2micvoicec,DEV=0 -r 22050 -c 1 -f S16_LE -t raw'
        --mic-auto-gain 5 \
        --mic-noise-suppression 2 \
        --wake-uri 'tcp://127.0.0.1:10400' \
        --wake-word-name 'hey_jarvis'

WorkingDirectory=/home/pi/wyoming-satellite
Restart=always
RestartSec=1
[Install]
WantedBy=default.target

Instead of trigging on Jarvis it triggers on Alex which is defined in my pipeline:

What I thought was supposed to happen is that the wake word on the PI triggers and then sends the audio over to HA. What seems to be happening is that the audio is being sent directly to HA, and it is in charge of determining whether it should process the audio.

Any pointers?

I am not exactly sure what made the difference here. I ended up removing the satellite from HA. Then I adjusted the systemd file like this:

[Unit]
Description=Wyoming Satellite
Wants=network-online.target
After=network-online.target
Requires=wyoming-openwakeword.service
Requires=2mic_leds.service
[Service]
Type=simple
ExecStart=/home/pi/wyoming-satellite/script/run \
        --debug \
        --name 'kitchen-voice' \
        --uri 'tcp://0.0.0.0:10700' \
        --mic-command 'arecord -D plughw:CARD=seeed2micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw' \
        --snd-command 'aplay -D plughw:CARD=seeed2micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw' \
        --snd-command-rate 16000 \
        --snd-command-channels 1 \
        --wake-uri 'tcp://127.0.0.1:10400' \
        --wake-word-name 'hey_jarvis' \
        --event-uri 'tcp://127.0.0.1:10500'
WorkingDirectory=/home/pi/wyoming-satellite
Restart=always
RestartSec=1
[Install]
WantedBy=default.target

I did find that the event-uri has or systemd complains about unknown arguments.

The device is now working as I expect

My guess is that (like me) you may have missed a system restart. Personally I am not confident with linux stopping and starting services, so do a full reboot just to make sure :wink:

I am not using event-uri, and I guess that it is not working for you either - so try removing that argument, restart the service, and see if that removes the error messages or stops the service. The trick is to change only one thing at a time - which becomes very tedious.

FYI, to prove that my system is no longer using Openwakeword on the HA machine, I disabled it there. When I am happy with this HA Voice Assist I will re-enable it for use by my RasPi Zero satellite

I have Pi and M5Atom. The Pi I use local wake word and notice that I have to say wake word, wait for it to activate then say command. With the AtomEchoM5, I can almost flow the wake word and request.
I assume the local wake word with Pi is the reason as the M5 is always streaming and HA/Wyoming is doing the processing.

I originally thought having local wake word would end up being better/faster, but now I am not totally convinced.

Except for running it as a satellite, is there a way to get a respeaker mic (I have the 4 array USB version) working directly connected to a RPI running HAA?

Mike do what works best for you in your situation. Local wake word reduces data over the network (more important if multiple satellites over wi-fi) - but at the expense of requiring more CPU at the satellite. You didn’t mention whether you are using one of the older slower RasPi models - or one of the faster more expensive models. There are always trade-offs.


You mean like was discussed in the Chapter 4 blog back in October 2023, and other forum topics ?

Just doing another fresh install, and come across a problem … so i will update my original post to use the correct branch of HinTak’s reSpeaker driver for the current kernel version.

The current (at time of writing) RasPi OS Bookworm (dated March 15th 2024) uses the v6.6 kernel.

Raspberry Pi OS with desktop

    Release date: March 15th 2024
    System: 32-bit
    Kernel version: 6.6
    Debian version: 12 (bookworm)

However it seems that the reSpeaker driver install currently defaults to the v6.1 kernel - which is correct for the older RasPi OS, now referred to as “Legacy” or “Bullseye”

Fortunately this is easy to fix.

On your Raspberry Pi use the “uname -r” command to find out which kernel version is currently running on your RasPi. The first two numbers are the kernel version. If it starts “6.1.” then the default branch is for you; but my newly updated RasPi 4 replied uname -r with “6.6.31+rpt-rpi-v8” so I have kernel v6.6

pi@raspi4:~ $ uname -r
6.6.31+rpt-rpi-v8

HinTak has provided branches in his gitbub repository named with the kernel versions, so it is simple to add the “–branch < version >” option to the git clone command:

git clone --branch v6.6 https://github.com/HinTak/seeed-voicecard.git

or to add a “git checkout < version >” command before running the install script:

git clone https://github.com/HinTak/seeed-voicecard
cd seeed-voicecard
git checkout v6.6
sudo ./install.sh

Still happening for me on my Voice Assist test RasPi4. Thinking that there may have been an update to wyoming-satellite or openwakeword in the past 6 months, I have started again to re-build from scratch.

Installed the respeaker driver usijng the github v6.6 branch; tested audio with arecord and aplay; installed and configures wyoming-satellite and wyoming-openwakeword. All good so far.

I speak the wakeword and hear the awake-wav sound. In HA > Wyoming > services the “Assist in progress” turns On and I speak my command. But “Assist in progress” does not turn off when I stop speaking. It seems to time out after about 15 seconds - and then I hear the done-wav sound, “Assist in progress” turns off, my command is recognised and actioned very quickly (so not because of HA on a slow CPU).

Possibly the microphone is picking up just enough noise to stop VAD (voice activity detection) from triggering ?

I have been distracted with other projects for several months … but now It’s time for me to find the best settings for my microphone … and that means also documenting, so others new users can short-cut that learning curve.

Alas, I am still using Rhasspy on 2 of my RasPi satellites because my HA Voice Assist wyoming-satellite is still taking 15 seconds before doing commands. There has been no improvement to HA code, and no response to the various github issues reporting this issue, in the past 4 months.