Rhasspy offline voice assistant toolkit

im running hassOS on a raspberry pi3

Hass.io running on raspberry 3

Rhasspy Hass.IO Alpha

OK, I finally have an alpha version of Rhasspy ready for anyone using Hass.IO! Supported languages include English, German, Spanish, Dutch, French, and Russian. Since I only speak one of these, feedback would be greatly appreciated!

Please add my repository URL (https://github.com/synesthesiam/hassio-addons.git) to your add-on repositories list, refresh, and then install the “Rhasspy Assistant” under “Synesthesiam Hass.IO Add-Ons” (all the way at the bottom of the Add-On Store screen).

NOTE: Beware that on a Raspberry Pi 3, the add-on can take 10-15 minutes to build and around 1-2 minutes to start. Watch the system log for a message like “Build 8e35c251/armhf-addon-rhasspy:1 done”. If the “Open Web UI” link on the add-on page doesn’t work, please check the log for errors, wait a minute, and try again.

I also have an introductory YouTube video showing how to use the web interface. Here’s the demo Home Assistant config used in the video. I’ll try to add more tutorials as time allows. In the meantime, see the main repo README file as well as the documentation on Rhasspy profiles and the sentences file format.

For testers and contributors: it would be great to have translations of the example English sentences into all of the supported languages. For the brave, I also need examples of common words in your language that make use of each phoneme in the language’s dictionary. The English profile has some good examples.

Some things missing from this early alpha release:

  1. The wake system is currently disabled. I can’t use snowboy for technical reasons on Hass.IO, so I’m looking into alternatives. It’s working with pocketsphinx, but has way too many false positives to be useful.
  2. RasaNLU (the intent recognizer) is not used. Just trying to install it on HassOS causes it to hang due to low memory. If anyone knows how to increase the swap space on HassOS, I’d love to know. I have a cheesy intent recognizer I wrote myself that will work for testing.
  3. More documentation is sorely needed. Rhasspy can still work in a client/server configuration, for example, but I haven’t written up how to do it (hint: use the “remote” system for speech to text and intent recognition in your profile).
4 Likes

I tried to find a detailed description of how to do a client-server install that’s mentioned earlier but couldn’t.

Is it possible to install Rhasspy on one Pi3 and run HA on another?
Or rout the connection through MQTT?

I cannot afford to mess with my Live HA machine while trying out this very promising sounding tool, it’s supposed to manage the house :grin:
I’d get in huge troubles with the boss; the WAF would drop through the floor if I kept messing with HomeAssistant constantly.

Hi .
I installed in my docker file
when i run the error
please help me
HTTPConnectionPool(host=‘hassio’, port=80): Max retries exceeded with url: /homeassistant/api/events/rhasspy_ChangeLightColor (Caused by NewConnectionError(’<urllib3.connection.HTTPConnection object at 0x7f313e7c3b70>: Failed to establish a new connection: [Errno -2] Name or service not known’,))

Thanks for trying out Rhasspy!

I’ve added a new Settings page that should address the issues @chairstacker and @Cao_Hoa mention above. If you update and click the “Settings” tab, you’ll see “Home Assistant”, “Speech Recognition”, and “Intent Recognition” sections.

In the “Home Assistant” section, put in the URL to wherever you have Home Assistant running (including the port). The default setting is for Hass.IO, where Home Assistant is running in the hassio docker container. If you’re running it on a different machine, just put in something like http://my-hass-server:8123 The “API Password” field lets you set the password for the remote server (I’ll have to update this soon since they’re deprecating the API password authentication).

For a client/server set up, where Rhasspy is listening on a Raspberry Pi and doing the speech/intent processing on a server, take a look at the “Speech Recognition” and “Intent Recognition” sections. You’ll need to put in the URL of the server where Rhasspy is running with the API endpoint (e.g., /api/speech-to-text). The Rhasspy client (Raspberry Pi) will automatically pass along the profile name.

1 Like

ok. thank you very much
I do always

Sounds promising.

Do I get it right that the Dutch profile has to be contributed? If so, then I’ll help you out as with the previous version :blush:

Will you be integrating the intent script component again or only use regular automations using the events?

I get this error
Expecting ‘,’ delimiter: line 5 column 9 (char 113)

Yes, please! I went ahead and added the sentences you’d previously given to the Dutch profile on Github: https://github.com/synesthesiam/rhasspy-profiles/blob/master/nl/sentences.ini

There are three “unknown” words in these sentences: garagepoort, garageverlichting, and woonkamerverlichting. We’ll need to add custom dictionary entries for them, which will be done on the “Words” tab with the phoneme table or by looking up the phonemes for known words. This will definitely be the topic of an upcoming guide.

Please update your Docker image and try using the new Settings tab. I realized too late that exposing the JSON directly could lead to a state where Rhasspy would be unable to load your profile.json (if it contains any minor syntax errors).

The new interface should look like this:

This is an excellent question. I really wanted to use the intent script component again, but I can’t figure out a way to do that outside of Home Assistant! There doesn’t seem to be any REST API support for it, and Hass.IO won’t let you install custom components anymore.

If anyone knows of a way to push an intent in via the REST API, I would love to hear it. As it stands, Rhasspy just creates a rhasspy_<INTENT NAME> event for every intent, and sends that it for automation processing.

Sorry, I still don’t get it - as long as the way to start Rhasspy is:

Running Rhasspy
It's time to try and run Rhasspy for the first time. Make sure you're in your Home Assistant directory and that your virtual environment is still activated:

cd /home/pi/homeassistant/
hass -c config

it seems like I still have to install HomeAssistant on the RPi I want to run Rhasspy on - I just don’t use this instance for the HomeAutomation part then and refer to my live Pi in the config?

Ah, I see the problem. The previous version of Rhasspy ran inside Home Assistant as a set of custom components. The Hass.IO version of Rhasspy runs as a standalone web application and communicates remotely with Home Assistant.

Hmmm - while the create_venv.sh seems to actually set up a venv, the run-venv.sh seems to refer to a docker image.
Given that I’m NOT a Linux Wiz, I don’t really know what that means.

2nd question:
I assume I have to download the requirements.txt file (where to?) so that the script can actually carry our the line
python3 -m pip install -r requirements.txt

3rd question:
the Docker installation shows both, a client and a server install (whatever that means) - if I want to run only run Rhasspy on my Pi (and only there) do i install the server version and set up its config so that it links to my live HA setup?

Sorry for the confusion. The run-venv.sh script refers to Docker images for the supporting tooling that Rhasspy needs (opengrm and phonetisaurus). Install all three .deb and setting the RHASSPY_NODOCKER variable will by-pass it, but I can just make the create-venv.sh script install the native binaries.

2nd question: Did you not get the requirements.txt in the git repo? It’s here.

3rd question: You’ll want the client install on your Pi. You can link it to your live HA set up directly via the Settings tab. Don’t worry about the server install (it does machine learning stuff that the Pi doesn’t support well). I’m not sure which names I should give the client/server Docker images.

I’ve not done anything yet - it’s all just not clear to me yet and I don’t want to screw up what I have on my secondary Pi already.

I can see that the venv script refers to the repo, but only AFTER the line that’s supposed to install requirements:

python3 -m pip install -r requirements.txt
python3 -m pip install https://github.com/synesthesiam/pocketsphinx-python/releases/download/v1.0/pocketsphinx-python.tar.gz

Originally, I wanted to install the Venv version, now it seems Docker might be the better way to go for the Pi.

I think the easiest way going forward for the moment would be to install Docker on your Pi, but then use run-venv.sh as is. This will do all the speech/intent stuff natively on the Pi, but will call out to the scripts in bin/docker to do training. I did it this way so you wouldn’t have to install anything outside of the virtual environment (except for python3-dev, etc.).

The second line of the venv script you see where it goes out to Github actually just installs a Pocketsphinx module for Python. It’s a version I made based on the pocketsphinx Python library, but doesn’t need a bunch additional stuff to support microphones.

Also, I forgot that the current full Docker client image of Rhasspy is only supported on amd64 machines (laptops/desktops). I need to build an armhf specific image for the Pi. I’ll let you know when I have that uploaded.

It was not clear at first that you have another repo with the profiles in. I downloaded that one also, chose the Dutch and French profiles and merged them into the Rhasspy Hass.io addon.

I receive some errors due to the fact that some files are missing from the Dutch profile like the following files:

  • custom_words.txt
  • dictionary.txt
  • espeak_phonemes.txt
  • intent_examples.json
  • language_model.txt

I copied them over from the English profile, recreated the add-on and now it starts. I’m not convinced that this is the correct approach as espeak_phonemes.txt at least seems to be language specific.

Do you have these files available for Dutch as well?