Rhasspy offline voice assistant toolkit

Hi,
I´ve installed Rhasspy on my Raspberry Pi, set the basic configuration and startet to write the sentences for intend recognition. But even when i copy and past the sentence from the sentence.ini file, I don’t get an intend. It works for the example sentences, but not for my own ones. I’ve also trained the model several times but always the same problem. I’m using fsticuffs. Do I have to install something special before using fsticuffs?

No bother, sorry I didn’t get back to you.

The error with the sentences.ini file you mentioned has now been fixed. Still not sure about the download issue, but I do have something to try. In your Rhasspy Settings, go to “Speech to Text” and switch to using Kaldi. Once you restart, this will download some different files. If that fails, we will at least know it’s something with the download code and not just the other files.

1 Like

Hi @david1998. There shouldn’t be anything extra you need to install. How did you install Rhasspy? Docker, Hass.io?

Three days later, and Rhasspy is allowing voice control of lights and televisions through Home Assistant. :star_struck: :star_struck: :star_struck: (psst! the WAF is amazing)
My config:

  • Intel Nuc i3, Debian Buster Server, Docker, Hass.io, Rhasspy add-on for speech-to-text processing and HA interface
  • RPi 3B+, Jabra 410 USB Speaker/Mic, Raspbian Buster, Docker, Rhasspy voice ‘client’ / satellite

A few technical Items worked out during the configuration process:

  • ALSA .asoundrc to handle the Jabra 410/510/710 microphone properly. It records at 16KHz by default, which is about 1/3 too ‘slow’. Adding ‘rate 48000’ to the pcm.input or pcm.microphone section is the solution.
  • SSL/HTTPS/TLS security - needed to use the –add-node option in Docker commands to start both the Rhasspy client and server instances. Without this, it was not possible to use simple IP addresses to talk among hass,io, Rhasspy client, and Rhasspy server

Many gracious thanks to @synesthesiam , @koan, and others in the Rhasspy forum for all their effort and experience. Users from the Snips community have brought with them a wealth of experience, helping make Rhasspy even better.

Great things are on the horizon, and Rhasspy has a thriving community that is growing. This is the best thing about Free and Open Source Software projects. Many good minds working together. :boom: Magic is happening here.

3 Likes

I‘ve installed Rhasspy out of the Docker image on fresh raspbian buster.

Hi,

Firstly, thanks for this, it’s great to have offline voice commands

Everythings working (at the moment) but I’m having some volume issues with Rhasspy, when playing audio (the default beep_hi.wav) from the CLI using aplay beep_hi.wav it plays back at the volume set in alsamixer but when it plays the same file from rhasspy after using the wake word it is super loud.

Should this be linked or is volume something I should be setting in the profile?

Great to hear such a positive experience :slight_smile: It’s hard to get much better than a high WAF! My greatest success to date (according to my wife, I’m sure) has been a red LED that shows when we left the garage door open :smiley:

I’m thinking I should start collecting these nuggets of wisdom somewhere. A Rhasspy wiki might be in order soon.

OK, then there shouldn’t be anything extra. Pi 3? Do you see any errors in the log?

Rhasspy just uses aplay, so I’m surprised that there would be any difference in the volume. Do you happen to have a different ALSA device configured for audio output in Rhasspy? It might be the same hardware device, but I’ve noticed ALSA sometimes lists things with or without hardware conversions. I’m thinking maybe one of those variations is not respecting the alsamixer volume.

I had the same thought, ALSA is listing several options but none of the others successfully playback any audio at all.

Yes on Raspberry Pi 3b+ and I didn‘t see errors in the log. I‘ll setup the pi again and see if the error will remain.

Dear @synesthesiam ,

thanks for the updates to Rhasspy - it is very pleasant that pico tts does not crack anymore. I thought for a while that my speaker is broken.
Also the error sound is great if an intention is not detected.
Thanks a lot for these christmas presents. Please never sell Rhasspy to Sonos or any other company - that would be a pity. And if you do, at least force them to give the product the name Zoidberg (imagine the hotword would be: “hey zoidberg”).

I noticed a special behavior in the hotword about porcupine (you once said, this one has the best performance). After a restart the Hotword is recognized once, also the further processing works wonderfully. But after that the hotword detection does not work anymore.
Something seems to be wrong with porcupine in the log, right? It looks as if rhasspy recognizes the hotword, but then does not continue as it (he? she?) does after a restart.

[DEBUG:1566570] PorcupineWakeListener: Hotword detected (True)
[INFO:1557837] quart.serving: 127.0.0.1:37150 GET / 1.1 200 1029 6422
[INFO:1527436] quart.serving: 127.0.0.1:36742 GET / 1.1 200 1029 6968
[INFO:1496984] quart.serving: 127.0.0.1:36332 GET / 1.1 200 1029 6297
[INFO:1466604] quart.serving: 127.0.0.1:36000 GET / 1.1 200 1029 6778
[INFO:1436180] quart.serving: 127.0.0.1:35572 GET / 1.1 200 1029 6835
[INFO:1405758] quart.serving: 127.0.0.1:35190 GET / 1.1 200 1029 8183
[INFO:1375333] quart.serving: 127.0.0.1:34854 GET / 1.1 200 1029 5557
[INFO:1344877] quart.serving: 127.0.0.1:34406 GET / 1.1 200 1029 7032
[INFO:1314426] quart.serving: 127.0.0.1:34046 GET / 1.1 200 1029 4977
[INFO:1283924] quart.serving: 127.0.0.1:33714 GET / 1.1 200 1029 4834
[DEBUG:1272645] PorcupineWakeListener: Hotword detected (True)
[INFO:1253458] quart.serving: 127.0.0.1:33236 GET / 1.1 200 1029 7309
[INFO:1223037] quart.serving: 127.0.0.1:32898 GET / 1.1 200 1029 10200
[INFO:1192603] quart.serving: 127.0.0.1:60792 GET / 1.1 200 1029 9388
[INFO:1162142] quart.serving: 127.0.0.1:60326 GET / 1.1 200 1029 9080
[INFO:1131687] quart.serving: 127.0.0.1:59988 GET / 1.1 200 1029 4740
[INFO:1101252] quart.serving: 127.0.0.1:59658 GET / 1.1 200 1029 10576
[INFO:1070759] quart.serving: 127.0.0.1:59176 GET / 1.1 200 1029 9578
[INFO:1040242] quart.serving: 127.0.0.1:58850 GET / 1.1 200 1029 9768
[INFO:1009710] quart.serving: 127.0.0.1:58512 GET / 1.1 200 1029 11096
[INFO:979177] quart.serving: 127.0.0.1:58038 GET / 1.1 200 1029 19948
[DEBUG:958556] PorcupineWakeListener: Hotword detected (True)
[INFO:953150] quart.serving: 172.20.0.1:45698 POST /api/text-to-speech 1.1 200 16 2157775
[DEBUG:953144] PorcupineWakeListener: started -> listening
[DEBUG:953134] PicoTTSSentenceSpeaker: speaking -> ready
[DEBUG:951231] PorcupineWakeListener: Loaded porcupine (keyword=[PosixPath('/profiles/de/porcupine/hey_pico_raspberrypi.ppn')]). Expecting sample rate=16000, frame length=512
[DEBUG:951211] PorcupineWakeListener: listening -> started
[DEBUG:951159] APlayAudioPlayer: ['aplay', '-q']
[DEBUG:951156] PicoTTSSentenceSpeaker: ready -> speaking
[DEBUG:950996] PicoTTSSentenceSpeaker: ['pico2wave', '-w', '/tmp/tmpmcr2u3rg.wav', '-l', 'de-DE', 'Okay, Flur   an.']
[DEBUG:950969] PorcupineWakeListener: started -> listening

You’re welcome :santa:

Since it’s all open source, I guess the only thing they’d be buying is me. I’d be flattered, but I’m not for sale :slight_smile: I do think, though, that the next version of Rhasspy should be code-named Project Zoidberg :crab:

Should be fixed now in the latest version!

Blockquote
You’re welcome :santa:

Best present this year :gift: :slightly_smiling_face:

Blockquote
Since it’s all open source, I guess the only thing they’d be buying is me. I’d be flattered, but I’m not for sale :slight_smile: I do think, though, that the next version of Rhasspy should be code-named Project Zoidberg :crab:

First I was a bit confused, because snips was sold - but snips was only partly open source. Besides, Sonos must have been interested in the development team and the existing closed source solution. Fortunately, the days of traditional slavery in more developed countries are over, so I don’t worry if someone wants to buy you. :crazy_face:
How about Zoidberg’s typical wav files as standard feedback sounds? That would be great - but my wife would probably not be amused.
Here’s a source of inspiration: Zoidberg Soundboard

Blockquote
Should be fixed now in the latest version!

Works out wonderfully now. Thank you!

1 Like

Lol I love it.

1 Like

Maybe we could write an automation to switch to Zoidberg sounds when the wife leaves…:smiley:

1 Like

Maybe we could write an automation to switch to Zoidberg sounds when the wife leaves…:smiley:

Accompanied by the woop-woop-woop sound of Zoodberg? Definitely! :rofl: :heart_eyes:

Seriously… You just have to exchange the wav file - a restart of rhasspy is not necessary. It works quite simply - I’ve added an option that allows not only a Zoidbergfication but also a Barneyfication. The logic is as follows: Wife leaves the house, then rhasspy is provided with the appropriate wav according to the input select.

Rhasspy Side
Filesystem

  1. Active wav (error.wav, recorded.wav, wake.wav) files are in:
    /docker/Rhasspy/de/wav/
  2. Wav files with the same name (error.wav, recorded.wav, wake.wav), but folder-dependent content in corresponding subfolders:
    /docker/Rhasspy/de/wav/regular/
    /docker/Rhasspy/de/wav/zoidberg/
    /docker/Rhasspy/de/wav/barney/

Frontend - settings - point to corresponding files
${RHASSPY_PROFILE_DIR}/wav/error.wav
${RHASSPY_PROFILE_DIR}/wav/recorded.wav
${RHASSPY_PROFILE_DIR}/wav/wake.wav

Home Assistant Side
input_select.rhasspy_feedback

name: Rhasspy Feedback
options:
 - Normal
 - Zoidberg
 - Barney

shell_command

docker_master_rhasspy_feedback_regular: "ssh [email protected] 'rsync -r /docker/Rhasspy/de/wav/regular/ /docker/Rhasspy/de/wav/'"
docker_master_rhasspy_feedback_zoidberg: "ssh [email protected] 'rsync -r /docker/Rhasspy/de/wav/zoidberg/ /docker/Rhasspy/de/wav/'"
docker_master_rhasspy_feedback_barney: "ssh [email protected] 'rsync -r /docker/Rhasspy/de/wav/barney/ /docker/Rhasspy/de/wav/'"

automation

- alias: rhasspy_zoidbergfication
  initial_state: true
  trigger:
    platform: state
    entity_id: person.wife
  action:
    - service_template: >-
        {%- if states.person.wife.state == "home" -%}
          shell_command.docker_master_rhasspy_feedback_regular
        {%- else -%}
          {%- if states.input_select.rhasspy_feedback.state == "Zoidberg" -%}
            shell_command.docker_master_rhasspy_feedback_zoidberg
          {%- elif states.input_select.rhasspy_feedback.state == "Barney" -%}
            shell_command.docker_master_rhasspy_feedback_barney
          {%- else -%}
            shell_command.docker_master_rhasspy_feedback_regular
          {%- endif %}
        {%- endif %}
3 Likes

(SOLVED see below) I am attempting to get Rhasspy running on a Respeaker Core v2.0 using the docker method. It looks like it is hearing the wake word but not able to play sound. So far googling it hasn’t helped me, the host is able to run aplay/arecord and the libraries appear to be installed.

Here is the error:

[ERROR:899591] APlayAudioPlayer: on_receive
Traceback (most recent call last):
  File "/usr/share/rhasspy/rhasspy/actor.py", line 169, in on_receive
    self._state_method(message, sender)
  File "/usr/share/rhasspy/rhasspy/audio_player.py", line 67, in in_started
    self.play_file(message.wav_path)
  File "/usr/share/rhasspy/rhasspy/audio_player.py", line 90, in play_file
    subprocess.run(aplay_cmd, check=True)
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['aplay', '-q', '/usr/share/rhasspy/etc/wav/beep_hi.wav']' returned non-zero exit status 1.

I am also seeing these errors in the docker log on startup:

[2020-01-01 19:32:18,189] ASGI Framework Lifespan error, continuing without Lifespan support,
WARNING:quart.serving:ASGI Framework Lifespan error, continuing without Lifespan support,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.front,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround40,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround41,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround50,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround51,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline,
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline,
ALSA lib pcm_dmix.c:1052:(snd_pcm_dmix_open) unable to open slave,

This is the error I get if I try to run aplay from within the container:

aplay beep_hi.wav
aplay: main:788: audio open error: No such file or directory

This however does play within the container:

aplay -Dplughw:0,1 beep_error.wav
Playing WAVE 'beep_error.wav' : Signed 16 bit Little Endian, Rate 44100 Hz, Stereo

SOLVED with the following asound.conf file:

pcm.!default {
  type plug
  slave {
    pcm "hw:0,1"
  }
}
ctl.!default {
  type hw
  card 1
}

Added this as a volume to the docker command:

-v "$HOME/rhasspy/asound.conf:/etc/asound.conf" \
1 Like

I need to adapt this as a Rhasspy tutorial for the docs :smiley:

1 Like

I’d be honored! :grin:
Suitable wav files can be downloaded here:
Soundboard
Soundboard #2

I recommend files from Soundboard #2 because the quality is much better and the files have a suitable length. Proposal for a selection:

wake
Zoidberg - Go on.wav

recorded
Zoidberg - Hooray.wav
Zoidberg - Aha.wav

error
Zoidberg - Whoop.wav