Rhasspy offline voice assistant toolkit

geoffrey · December 10, 2018, 11:13pm

I retried today using your repo and was able to use the English profile. When switching to nl I received errors again indicating some files are missing like the following ones:

[Errno 2] No such file or directory: 'profiles/nl/phoneme_examples.txt'

[Errno 2] No such file or directory: 'profiles/nl/unknown_words.txt'

Also the example sentences where missing, so I added them in the textarea manually leading to the following notification:

Wrote 501 byte(s) to /data/rhasspy/profiles/nl/sentences.ini.

Finally, when I try to record a Dutch sentence, the log file shows the following message:

ERROR: "acmod.c", line 79: Folder 'profiles/nl/acoustic_model' does not contain acoustic model definition 'mdef' ERROR:root:new_Decoder returned -1

What am I doing wrong with the selection of the profile?

infiniteloop · December 11, 2018, 12:09am

I think it would be useful a more clear guide on how to install Rhasspy on a stand alone raspberry pi.
As @chairstacker I would like to avoid to mess with my working hassio while testing at this stage, so I’ve made a fresh install of Raspbian stretch lite on a second Pi.
Now what’s the next step to take?

Thanks for your work, I really like this project and hope to contribute with translation.

chairstacker · December 11, 2018, 12:13am

Hi @synesthesiam, here’s what I did today:

I created the the script create-venv.sh in the directory /usr/local/bin
Made it executable
Ran it.

And here are the error messages (including the reference to the file requirements.txt I asked about earlier).

Error: missing `server' JVM at `/usr/lib/jvm/java-8-openjdk-armhf/jre/lib/arm/server/libjvm.so'.
Please install or use the JRE or JDK that contains these missing components.
dpkg: error processing package ca-certificates-java (--configure):
 subprocess installed post-installation script returned error exit status 4
dpkg: dependency problems prevent configuration of openjdk-8-jre-headless:armhf:
 openjdk-8-jre-headless:armhf depends on ca-certificates-java; however:
  Package ca-certificates-java is not configured yet.

dpkg: error processing package openjdk-8-jre-headless:armhf (--configure):
 dependency problems - leaving unconfigured

Error: missing `server' JVM at `/usr/lib/jvm/java-8-openjdk-armhf/jre/lib/arm/server/libjvm.so'.
Please install or use the JRE or JDK that contains these missing components.
E: /etc/ca-certificates/update.d/jks-keystore exited with code 1.
done.
Errors were encountered while processing:
 ca-certificates-java
 openjdk-8-jre-headless:armhf
E: Sub-process /usr/bin/dpkg returned an error code (1)
/usr/local/bin/.venv

Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
Successfully built pocketsphinx
Installing collected packages: pocketsphinx
Successfully installed pocketsphinx-0.1.15

What do I need to do to fix this?
How do I go forward from here?

chairstacker · December 11, 2018, 2:28am

Tried a few things on my own:

Ran the command sudo apt-get install -y openjdk-8-jre-headless that is inside the script again - might have fixed the issue.
Created a file requirements.txt in my pi home folder and referenced it from the script, looks like it installed them when I ran the whole script again, albeit with an eror message for each one: Cache entry deserialization failed, entry ignored
Created the script run-venv.sh and made it executable.
Running it with sudo resulted in the following messages:

Using Docker scripts.
 * Serving Flask app "app.py"
 * Environment: production
   WARNING: Do not use the development server in a production environment.
   Use a production WSGI server instead.
 * Debug mode: off
Usage: flask run [OPTIONS]

Error: Could not import "app".

That might be all I have time for today

synesthesiam · December 11, 2018, 3:34am

I’ve tried bumping the version of the Rhasspy Hass.IO add-on to 1.1. Hass.IO seems to be caching the old Dockerfile, which may be messing everything up. Also, be careful that the profiles directory in your /addons directory doesn’t have lingering files. If all goes well, the profiles should be downloaded from Github during the build process.

Ultimately, your profiles/nl directory should look identical to the one in the profiles repo, which includes most of the missing files you mentioned. Maybe I should just add the ability to download profiles from within Rhasspy?

synesthesiam · December 11, 2018, 4:03am

I’ll be testing the virtual environment scripts out on a fresh Raspbian tomorrow. For now, let’s try doing this through Docker entirely. Here are some instructions I’ve been working on:

Make sure you have Docker installed:

curl -sSL https://get.docker.com | sh

and that your user is part of the docker group:

sudo usermod -a -G docker $USER

Be sure to reboot after adding yourself to the docker group!

Next, start the Rhasspy Docker image in the background:

docker run -d -p 12101:12101 \
      --restart unless-stopped \
      -e RHASSPY_PROFILES=/profiles \
      -v "$HOME/.rhasspy:/profiles" \
      -v /dev/snd:/dev/snd \
      --privileged \
      synesthesiam/rhasspy-hassio-addon:armhf

The web interface should now be accessible at http://localhost:12101

I just tried this on my Raspberry Pi 3 running the very latest Raspbian lite (headless), and it worked just fine with my PS3 eye microphone!

infiniteloop · December 11, 2018, 11:49am

Thanks now it’s way more clear.
Just one question: before of this, should I first pull the Rhasspy client with
docker pull synesthesiam/rhasspy-client ?

Cao_Hoa · December 11, 2018, 12:04pm

I do not have access to the interface since I updated and rebooted

Cao_Hoa · December 11, 2018, 12:06pm

This is my console

synesthesiam · December 11, 2018, 1:55pm

The first docker run will automatically do a docker pull when it can’t find the Docker image. Also, please use synesthesiam/rhasspy-hassio-addon:armhf if you’re on a Raspberry Pi. I really need to rename the images…

synesthesiam · December 11, 2018, 1:56pm

Looks like it’s running, at least. What do you see when you click into the log for rhasspy-server?

thundergreen · December 11, 2018, 2:06pm

Dear @synesthesiam.

First: BIG Thanks for this awesome piece of work! By reading his post here I got confused, cleared, confused and at the end … confused There are several ways to setup Rhasspy. I run the Docker image. I have access to the webinterface. So looking good for the moment.

My next question is:

If I well understood is Rhasspy running all components --> no custom components needed in Homeassistant right?
What I’m supposed to do next to get this running with hass. I already put in my Hass url:port No I can start doing what exactly and where exactly? Your repo has so many example files that I totally lost the overview
I know writing a software doesn’t include having also good docs and sometimes writing the manual is even harder But if I’m not wrong there will be Hassio addon (which is kind of nested docker) and pure Docker image. I think manual install should be depreciated to keep this project easy to setup for everybody. People either use venv, docker manual install or hassio. I assume that Hassio user are more like beginners where venv install users are more advanced. So for me the Docker install method / documentation seems to be rather sufficient.
–> leads to a request to a clean write - up for either DOCKER or HASSIO. ()what has to be done on the side of RhassPy and what on the side of Homeassistant (no matter which install method)
If you need some help with GERMAN translations or whatever in German… Let me know… I’M willed to help as I reallllllyyyyy appreciate your project as my personal intention is having homeassistant running WITHOUT ANY CLOUD SERVICE

gpbenton · December 11, 2018, 3:09pm

I have also just brought up the docker image, and I am rather stuck.

I agree with @thundergreen that there needs to be some instructions on what to do next (or make them more prominent if they already exist ).

But also, the HA login is asking for an api password, which is deprecated (if it is even still supported). Future access to HA api is going to be through long lived access tokens, so they do need to be supported.

And why is the --privileged flag used in the command line? Is there something specific this is needed for?

This project looks just what I wanted - once I can get it going.

Cao_Hoa · December 11, 2018, 3:09pm

File “/usr/share/rhasspy/app.py”, line 83, in

load_default_profile()

File “/usr/share/rhasspy/app.py”, line 70, in load_default_profile

default_profile = Profile(default_profile_name, profiles_dirs)

File “/usr/share/rhasspy/profiles.py”, line 11, in init

self.load_profile()

File “/usr/share/rhasspy/profiles.py”, line 30, in load_profile

recursive_update(self.json, json.load(profile_file))

File “/usr/lib/python3.6/json/init.py”, line 299, in load

parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)

File “/usr/lib/python3.6/json/init.py”, line 354, in loads

return _default_decoder.decode(s)

File “/usr/lib/python3.6/json/decoder.py”, line 339, in decode

obj, end = self.raw_decode(s, idx=_w(s, 0).end())

File “/usr/lib/python3.6/json/decoder.py”, line 355, in raw_decode

obj, end = self.scan_once(s, idx)

json.decoder.JSONDecodeError: Expecting ‘,’ delimiter: line 5 column 9 (char 113)

This is my error file

thundergreen · December 11, 2018, 4:25pm

but Here’s avery positive point: This project seems to become very popular and beloved Let’s make this piece of software a common part of Home Assistant If there’s anything I can do… Let me know.

synesthesiam · December 11, 2018, 5:46pm

It looks like you got caught in a state where there’s some malformed JSON in your profile.json. Depending on your docker-compose configuration, your profiles could either be inside the Docker container or on your file system. Wherever they are, try deleting them and starting from a fresh Docker pull or copying from the Github profiles repo.

synesthesiam · December 11, 2018, 6:01pm

Thank you! I’m kind of building this plane as I’m flying it, so to speak. So I understand the confusion. I’ll try to answer your questions as best I can.

Correct, no custom components are needed with the new Rhasspy. Everything is self-contained, and communication with Home Assistant is done via the HTTP REST API.
This will be my next video tutorial. The next step is to create one or more intents and some example sentences like in the English profile. Then, after re-training, should be able to speak those sentences in the Speech tab and have Rhasspy send an event to Hass. You’d then add to automations.yml to do something with the event, like turn some device on/off.
I agree. The only problem I’ve found with the manual Docker install is that sometimes microphones/speakers don’t work right (this is why /dev/snd is mapped in and why the privileged flag is set (per the question by @gpbenton). So the venv option is a last resort that let’s Rhasspy access the hardware natively instead of through Docker.
Yes, I will be cleaning up the Docker images (the names make no sense right now), and writing a single document describing the manual Docker and Hass.IO methods.
German translations would be very welcome! I don’t have anything yet. The phoneme_examples.txt file in the German profile was a best guess at common German words and their pronunciations (from base_dictionary.txt).

synesthesiam · December 11, 2018, 6:08pm

The access tokens in on my TODO list. It was easy in Hass.IO where the token is provided via an environment variable. For external HA access, I have to add in the whole websocket back and forth to create a token, I guess? Is there any way to create a long-lived token from within the HA interface that I could just have the user paste into a Rhasspy profile?

As best I can tell, this is necessary to give the Docker container read/write privileges to /dev/snd so it can access the microphone and speakers of the host system. You might try doing docker run without it and see if it works. If so, that would be great, since I don’t want more access to the host system than is strictly necessary.

gpbenton · December 11, 2018, 6:50pm

Yes, creating the long lived token is normally done in HA, from the user profile page. Its normally just a matter of pasting it somewhere.

I have, and it seems to work fine without privileges - I was just wondering if there was definitely something I was missing.

FYI my docker-compose entry is

    rhasspy:
        image: "synesthesiam/rhasspy-server:amd64"
        restart: unless-stopped
        container_name: rhasspy-server
        environment:
            RHASSPY_PROFILES: "/profiles"
        volumes:
            - "./rhasspy_config:/profiles"
        ports:
            - "12101:12101"
        devices:
            - "/dev/snd:/dev/snd"

I have got further now, and almost everything seems to be working, except that when I speak into the web interface, it correctly recognizes I said What time is it, but then converts that in to a getTemperature intent.

"intent":
"entities":
"intent":
"confidence": 0.5575017645851849
"name": "GetTemperature"
"intent_ranking":
0:
"confidence": 0.5575017645851849
"name": "GetTemperature"
1:
"confidence": 0.31922910283110323
"name": "GetTime"
2:
"confidence": 0.0552980109184647
"name": "GetGarageState"
3:
"confidence": 0.0403309328804052
"name": "ChangeLightState"
4:
"confidence": 0.027640188784842316
"name": "ChangeLightColor"
"model": "model_20181211-173604"
"project": "default"
"text": "what time is it"
"time_sec": 0.03763532638549805

So something is wrong in translating the sentence to the intent, but I have re-trained and reloaded without effect.

infiniteloop · December 11, 2018, 6:54pm

Ok, up and running.
How can I help you with translation now?