Rhasspy offline voice assistant toolkit

This is still a major weakness of Rhasspy, and not something I’ll probably be able to solve for some time by myself. Here are the best tips I have:

  • If you’re using pocketsphinx, I’m adding a “sensitivity” setting that you can adjust. Check out the graph in this article, though, to see just how terrible pocketspinx is.
  • For snowboy, it’s best to train a personal model (.pmdl) using the exact hardware you plan to use it on. I originally trained it using my laptop, but got better performance by training it from my Pi with the PS3 Eye. There’s also a sensitivity setting that you can play with.

Mycroft Precise appears to be just about to release a new version. I’m going to see if it finally can do better than snowboy. If anyone has some spare time, I’d also be interested to hear your experience with porcupine. I don’t like that your wake word “expires” after some period of time, but his results sound impressive.

Oh, and in case of an accidental activation, there are things you can do on the Rhasspy side. In both the speech to text and intent recognition settings, you’ll find “Minimum Confidence” values. When they’re 0, they let anything through. Increasing one or the other will help stop some things from getting accidentally recognized. I’m working on making the speech/intent confidences print out on the main web interface so you can properly tune them (you can see them printed in the log, however).

Probably, glad you fixed it

Yes, a headphone 3.5 jack and two small connectors:
https://matrix-io.github.io/matrix-documentation/matrix-voice/overview/

I was going to try porcupine indeed! No, where was my spare time again… I’m sure I had it.

1 Like

Things that I found:

  1. Every time the addon is restarted, in the Rhasspy URL appears the message “Some files must be downloaded for your selected profile ()”. (Nothing inside the parenthesis. I believe should be “en”, since is the profile that I am using).
    I don’t if it planned to do so. Why it has to download something, and everytime?

  2. I tryed to deal with the “minimum confidence” field in Speech Recognization. I understand that it should be a number from 0 to 1 (what is the default value?). I tryed “0.6”. Then it did not worked anymore, even restarting addon, and in Rhasspy URL restart.
    Reloading the Rhasspy URL, the value was “0,6” instead of "0.6). Using the up and down arrows from the field an saving, the field keeps blank, but it gets saved in “profile.json”.

  3. Then I leaved the field blank, saved, but could not make rhasspy work again. I am having some errors reproduced below:

    Expression ‘ret’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 1736
    Expression ‘AlsaOpen( &alsaApi->baseHostApiRep, params, streamDir, &self->pcm )’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 1904
    Expression ‘PaAlsaStreamComponent_Initialize( &self->capture, alsaApi, inParams, StreamDirection_In, NULL != callback )’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 2171
    Expression ‘PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags, userData )’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 2840
    ERROR:PyAudioRecorder:receiveMessage
    Traceback (most recent call last):
    File “/usr/share/rhasspy/rhasspy/actor.py”, line 103, in on_receive
    self._state_method(message, sender)
    File “/usr/share/rhasspy/rhasspy/audio_recorder.py”, line 115, in in_started
    self.transition(“recording”)
    File “/usr/share/rhasspy/rhasspy/actor.py”, line 155, in transition
    getattr(self, transition_method)(from_state)
    File “/usr/share/rhasspy/rhasspy/audio_recorder.py”, line 141, in to_recording
    frames_per_buffer=self.frames_per_buffer,
    File “/usr/local/lib/python3.6/site-packages/pyaudio.py”, line 750, in open
    stream = Stream(self, *args, **kwargs)
    File “/usr/local/lib/python3.6/site-packages/pyaudio.py”, line 441, in init
    self._stream = pa.open(**arguments)
    OSError: [Errno -9985] Device unavailable
    DEBUG:PocketsphinxWakeListener:Hotword detected (okay rhasspy)

As you see, the hotword is detected, but no audio signal can be heard, and the sentences are not recognized anymore. Sorry but for some reason the format to code did not worked. I am now using a PS3 Eye Camera.

Fresh install Hassio and Rhasppy.

Start Rhasspy (version 2.01):

DEBUG:PocketsphinxDecoder:Loading decoder with hmm=profiles/en/acoustic_model, dict=profiles/en/dictionary.txt, lm=profiles/en/language_model.txt
ERROR (Input():snowboy-io.cc:315) Fail to open input file “profiles/en/computer.umdl”
terminate called after throwing an instance of ‘std::runtime_error’
what(): ERROR (Input():snowboy-io.cc:315) Fail to open input file “profiles/en/computer.umdl”
[stack trace: ]
Backtrace has not been implemented yet.
/run.sh: line 20: 14 Aborted (core dumped) flask run --host=0.0.0.0 --port=12101

I have uploaded a computer.umdl into /share/rhasspy/profiles/en/ and then it worked.
After that, when I went into the webui, issue 1 of FunkyBot stated (en).
Before, it was ()
But I was kicked out with the message that de/snowboy.umdl was not available.
So I have uploaded that to /share/rhasspy/profiles/de/ (not sure why the language is DE)
Then the web wanted to download files for DE, which failed:

subprocess.CalledProcessError: Command ‘[‘bash’, ‘/usr/share/rhasspy/profiles/de/download-profile.sh’]’ returned non-zero exit status 1.

Then I gave up for now :wink:

OK, I’ve pushed a new version that will hopefully fix some of the problems. I guess git doesn’t care for symbolic links without some extra babysitting (the snowboy model issue).

I’ll look into the other issues as I have time!

No worries, keep up the good work :slight_smile:

Removed the Addon and all files from /share/rhasppy, installed the new one.
After a while, it fails with error message below

I am making sure to remove any old stuff, so that I can test it with a fresh install every time :slight_smile:

When I click cancel and then Train, Rhasspy trains the sentences correctly. The problems seems to be in downloading snowboy in this error message.

The Addon update is still not appearing for me in Hassio.

Edit:
Now I could update.
After some restarts, it started to work again. I may change de Pocketsphinx sensitivity if I feel necessary.
Now I have to experiment a little more to see how it goes. Thanks!

New version goes well when default EN!

Switching to Dutch gives:

sorry man :wink:

Rhasspy 2.03
I tryed to change the Pocketsphinx sensitivity, from 0.7 to 1 (default 0.6), but I couldn’t notice any difference.
As many false positives as before, even when using 1 and with low volume on TV (only music included), since Eye Camera is pretty sensitive.

For every change I made, I had to restart the addon and download the needed files in rhasspy page, since restarting in the rhasspy page only was not enough.

At least I’m getting closer! Next attempt is up :slight_smile:

I also added a “Re-Download” button on the settings page that will delete the download cache and start from scratch.

I’m surprised by the restart problem. Rhasspy should be writing your profile to the /share directory in Hass.io. Do you see any Rhasspy files there?

Yes, I have many files in /share/rhasspy/profiles/en
which includes profile.json.

Re-Download button is nice.
EN works out of the box, switching to nl gives:

Traceback (most recent call last):
File “/usr/share/rhasspy/app.py”, line 207, in api_download_profile
download_cmd, cwd=core.profile.write_path(), stderr=subprocess.STDOUT
File “/usr/local/lib/python3.6/subprocess.py”, line 356, in check_output
**kwargs).stdout
File “/usr/local/lib/python3.6/subprocess.py”, line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[‘bash’, ‘/usr/share/rhasspy/profiles/nl/download-profile.sh’, ‘–delete’]’ returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/usr/local/lib/python3.6/site-packages/flask/app.py”, line 1813, in full_dispatch_request
rv = self.dispatch_request()
File “/usr/local/lib/python3.6/site-packages/flask/app.py”, line 1799, in dispatch_request
return self.view_functionsrule.endpoint
File “/usr/share/rhasspy/app.py”, line 213, in api_download_profile
raise Exception(output)
Exception: Downloading Dutch (nl) profile (sphinx)
Extracting acoustic model (/usr/share/rhasspy/profiles/nl/download/cmusphinx-nl-5.2.tar.gz)
mv: cannot stat ‘/usr/share/rhasspy/profiles/nl/cmusphinx-nl-5.2/model_parameters/voxforge_nl_sphinx.cd_cont_2000/’: No such file or directory

Version 2.04:

Pocketsphinx is at the moment too sensitive to the point to be very annoying, since it detects too many false positives. So I decided to try snowboy again. Using “computer.umdl” the wake word worked MUCH better, but:

  1. After a restart in the addon page (with no download in the rhasspy page), it detects the wake word giving both de wake word confirmation and end of sentence signals. But no automation is triggered.
  2. In the rhasspy page, if I download the files and restart, no signals can be heard, but it detects the wake word (seen in the log), but as before no action is taken.

Below are some errors in me log in the hope that it can be helpful:

Expression 'ret' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1736
Expression 'AlsaOpen( &alsaApi->baseHostApiRep, params, streamDir, &self->pcm )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1904
Expression 'PaAlsaStreamComponent_Initialize( &self->capture, alsaApi, inParams, StreamDirection_In, NULL != callback )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2171
Expression 'PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags, userData )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2840
ERROR:PyAudioRecorder:receiveMessage
OSError: [Errno -9985] Device unavailable

OK, I think I have the profile/restart/download issues mostly resolved. There was some non-trival interplay between the download scripts and the multi-directory profile structure Rhasspy has. I hope it’s fixed!

@FunkyBoT, I’m not sure yet about the audio errors. I need to see if something has changed recently in Hass.io. I’m really hoping they didn’t switch to PulseAudio. In the mean time, you might try switching to the ALSA audio recorder just to see if it helps.

Can you tell me how I can do that?
Edit:
I upgraded to version 2.05.
Now I got it to work after addon restart. If I download the files in rhasspy page, it stops from giving me audio signals and triggering automation, but wake word is still detected (on the log). This sounds a bit weird.
Restart again in the addon page and it is working again.
It seems that the wake word “computer” (snowboy) is a bit harder to be detected now. I will try to adjust snowboy sensitivity.
I still have many errors in my log.

Update:
If I change snowboy wake word sensitivity to 0.3, it can’t detect the wake word anymore. If I bring it back to 0.5, then it works again, but I am having to speak loud and in proximity of the speaker.

Update:
Snowboy.umdl seems to work better for me. It is detected easier. So I should go with this one for now.

I run by command: ./run-venv.sh. And go to web by address localhost:12101. See error: [Errno 2] No such file or directory: ‘/rhasspy20190412/dist/index.html’.
Help me!