Rhasspy offline voice assistant toolkit

Sure, but if you like I can create an english version easily.

1 Like

Yes, that would be great. I could only guess what you were saying :slight_smile:

Working on it :slight_smile:

1 Like

As I’ve dug more into the issues with audio streaming/synchronization in Rhasspy, I’ve realized that a great deal of those problems would be better solved with something like gstreamer. It has rich support for audio pipelines, and would allow me to simplify Rhasspy’s audio code.

Does anyone here have experience with gstreamer in Hass.io? I don’t want to adopt something that won’t work for everyone out of the gate.

I know you can add it as component, but have no experiance whatsoever.

I’m having great success so far with gstreamer. It has support for both PulseAudio and ALSA, so I’m expecting it to work fine on both Hass.io and other systems.

2 Likes

Well, I have opened an issue for HassIO with gstreamer, currently I do not think it works ok.

:frowning:

Gstreamer is apparantly not installed in hassio, so not usable I assume.

1 Like

Hopefully it will still work in an add-on, but it’s a shame they don’t have it installed by default.

I’m facing a strange behavior issue with Rhasspy, I’ve got a docker container running using the following command on a Raspberry :

docker run -d -p 12101:12101 \
      --restart unless-stopped \
      -v "$HOME/.config/rhasspy/profiles:/profiles" \
      --device /dev/snd:/dev/snd \
      synesthesiam/rhasspy-server:armhf \
      --user-profiles /profiles \
      --profile fr

When my container start I can connect to the web ui, I can see an “en” profile in settings tabs. After a few minutes the web ui stop working but the container does not seem to show any error if I use docker log.

I am not sure if this is related but I cannot curl my container neither from the docker host.

Any advices ?

Edit : Here are my docker logs

Docker logs

2019-07-23T05:09:16.848090709Z DEBUG:main:Namespace(host=‘0.0.0.0’, port=12101, profile=‘fr’, set=, ssl=None, system_profiles=‘/usr/share/rhasspy/profiles’, user_profiles=‘/profiles’)
2019-07-23T05:09:17.002381593Z DEBUG:RhasspyCore:Loaded profile from /usr/share/rhasspy/profiles/fr/profile.json
2019-07-23T05:09:17.003027785Z DEBUG:RhasspyCore:Profile files will be written to /profiles/fr
2019-07-23T05:09:17.003743507Z DEBUG:root:Loading default profile settings from /usr/share/rhasspy/profiles/defaults.json
2019-07-23T05:09:17.006372077Z DEBUG:WebSocketObserver: → started
2019-07-23T05:09:17.010333030Z DEBUG:DialogueManager: → started
2019-07-23T05:09:17.010999743Z DEBUG:DialogueManager:started → loading
2019-07-23T05:09:17.011310626Z DEBUG:DialogueManager:Loading actors
2019-07-23T05:09:17.021715895Z DEBUG:DialogueManager:Actors created. Waiting for [‘recorder’, ‘player’, ‘speech’, ‘wake’, ‘command’, ‘decoder’, ‘recognizer’, ‘handler’, ‘sentence_generator’, ‘speech_trainer’, ‘intent_trainer’, ‘word_pronouncer’] to start.
2019-07-23T05:09:17.025617058Z DEBUG:APlayAudioPlayer: → started
2019-07-23T05:09:17.027219751Z DEBUG:EspeakSentenceSpeaker: → started
2019-07-23T05:09:17.033224540Z DEBUG:PocketsphinxWakeListener: → started
2019-07-23T05:09:17.034521716Z DEBUG:FsticuffsRecognizer: → started
2019-07-23T05:09:17.036350553Z ERROR: ReadFst: Can’t open file: profiles/fr/intent.fst
2019-07-23T05:09:17.037218150Z DEBUG:PocketsphinxSpeechTrainer: → started
2019-07-23T05:09:17.039567659Z DEBUG:FsticuffsIntentTrainer: → started
2019-07-23T05:09:17.040695566Z DEBUG:PhonetisaurusPronounce: → started
2019-07-23T05:09:17.041856701Z DEBUG:PyAudioRecorder: → started
2019-07-23T05:09:17.043278146Z DEBUG:WebrtcvadCommandListener: → started
2019-07-23T05:09:17.045824009Z DEBUG:PocketsphinxDecoder: → started
2019-07-23T05:09:17.047293839Z DEBUG:DummyIntentHandler: → started
2019-07-23T05:09:17.048302632Z DEBUG:JsgfSentenceGenerator: → started
2019-07-23T05:09:17.049253040Z DEBUG:DialogueManager:player started
2019-07-23T05:09:17.050286780Z DEBUG:EspeakSentenceSpeaker:started → ready
2019-07-23T05:09:17.051693695Z WARNING:FsticuffsRecognizer:preload: Read failed: ‘profiles/fr/intent.fst’
2019-07-23T05:09:17.052441604Z DEBUG:DialogueManager:speech_trainer started
2019-07-23T05:09:17.054293098Z DEBUG:FsticuffsRecognizer:started → loaded
2019-07-23T05:09:17.055332099Z DEBUG:DialogueManager:intent_trainer started
2019-07-23T05:09:17.056038082Z DEBUG:DialogueManager:word_pronouncer started
2019-07-23T05:09:17.056426829Z DEBUG:DialogueManager:recorder started
2019-07-23T05:09:17.056886460Z DEBUG:DialogueManager:handler started
2019-07-23T05:09:17.057896451Z DEBUG:DialogueManager:sentence_generator started
2019-07-23T05:09:17.060874236Z DEBUG:DialogueManager:speech started
2019-07-23T05:09:17.062518752Z DEBUG:DialogueManager:recognizer started
2019-07-23T05:09:17.127312380Z ERROR:PocketsphinxWakeListener:loading wake decoder
2019-07-23T05:09:17.127453524Z Traceback (most recent call last):
2019-07-23T05:09:17.127484774Z File “/usr/share/rhasspy/rhasspy/wake.py”, line 117, in to_started
2019-07-23T05:09:17.127534982Z self.load_decoder()
2019-07-23T05:09:17.127564304Z File “/usr/share/rhasspy/rhasspy/wake.py”, line 226, in load_decoder
2019-07-23T05:09:17.127593262Z with open(dict_path, “r”) as dict_file:
2019-07-23T05:09:17.127643106Z FileNotFoundError: [Errno 2] No such file or directory: ‘profiles/fr/dictionary.txt’
2019-07-23T05:09:17.128380859Z DEBUG:PocketsphinxWakeListener:started → loaded
2019-07-23T05:09:17.130176937Z DEBUG:DialogueManager:wake started
2019-07-23T05:09:17.131632080Z DEBUG:PocketsphinxDecoder:Loading decoder with hmm=profiles/fr/acoustic_model, dict=profiles/fr/dictionary.txt, lm=profiles/fr/language_model.txt
2019-07-23T05:09:17.197435906Z WARNING:PocketsphinxDecoder:preload: new_Decoder returned -1
2019-07-23T05:09:17.203840744Z DEBUG:PocketsphinxDecoder:started → loaded
2019-07-23T05:09:17.206927434Z DEBUG:DialogueManager:decoder started
2019-07-23T05:09:17.738867841Z DEBUG:WebrtcvadCommandListener:started → loaded
2019-07-23T05:09:17.740223037Z DEBUG:DialogueManager:command started
2019-07-23T05:09:17.740437983Z INFO:DialogueManager:Actors loaded
2019-07-23T05:09:17.740735115Z DEBUG:DialogueManager:loading → ready
2019-07-23T05:09:17.741286881Z INFO:DialogueManager:Automatically listening for wake word
2019-07-23T05:09:17.741558285Z DEBUG:DialogueManager:ready → asleep
2019-07-23T05:09:17.744964348Z INFO:main:Started
2019-07-23T05:09:17.746355845Z DEBUG:main:Starting web server at http://0.0.0.0:12101
2019-07-23T05:09:17.750557265Z ERROR:PocketsphinxWakeListener:receiveMessage
2019-07-23T05:09:17.750662525Z Traceback (most recent call last):
2019-07-23T05:09:17.750692316Z File “/usr/share/rhasspy/rhasspy/actor.py”, line 117, in on_receive
2019-07-23T05:09:17.750721378Z self._state_method(message, sender)
2019-07-23T05:09:17.750750337Z File “/usr/share/rhasspy/rhasspy/wake.py”, line 125, in in_loaded
2019-07-23T05:09:17.750778670Z self.load_decoder()
2019-07-23T05:09:17.750806169Z File “/usr/share/rhasspy/rhasspy/wake.py”, line 226, in load_decoder
2019-07-23T05:09:17.750834502Z with open(dict_path, “r”) as dict_file:
2019-07-23T05:09:17.750863200Z FileNotFoundError: [Errno 2] No such file or directory: ‘profiles/fr/dictionary.txt’
2019-07-23T05:09:58.927530126Z Expression ‘alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 924
2019-07-23T05:09:58.935447606Z Expression ‘alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 924
2019-07-23T05:09:58.935873383Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.front
2019-07-23T05:09:58.936504679Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
2019-07-23T05:09:58.936538950Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
2019-07-23T05:09:58.936568793Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
2019-07-23T05:09:58.936598064Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
2019-07-23T05:09:58.936628012Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
2019-07-23T05:09:58.936656345Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround40
2019-07-23T05:09:58.936684678Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround41
2019-07-23T05:09:58.936717594Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround50
2019-07-23T05:09:58.936749313Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround51
2019-07-23T05:09:58.936778323Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71
2019-07-23T05:09:58.936806812Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
2019-07-23T05:09:58.936834833Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
2019-07-23T05:09:58.936864155Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
2019-07-23T05:09:58.936892541Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
2019-07-23T05:09:58.936920665Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
2019-07-23T05:09:58.936948946Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
2019-07-23T05:09:58.936983529Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
2019-07-23T05:09:58.937012175Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
2019-07-23T05:09:58.937039727Z ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
2019-07-23T05:09:58.940283135Z Expression ‘alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 924
2019-07-23T05:09:58.959468064Z ERROR:DialogueManager:receiveMessage
2019-07-23T05:09:58.959618531Z Traceback (most recent call last):
2019-07-23T05:09:58.959649677Z File “/usr/share/rhasspy/rhasspy/actor.py”, line 117, in on_receive
2019-07-23T05:09:58.959698895Z self._state_method(message, sender)
2019-07-23T05:09:58.959727801Z File “/usr/share/rhasspy/rhasspy/dialogue.py”, line 251, in in_asleep
2019-07-23T05:09:58.959777696Z self.handle_any(message, sender)
2019-07-23T05:09:58.959875195Z File “/usr/share/rhasspy/rhasspy/dialogue.py”, line 520, in handle_any
2019-07-23T05:09:58.959905299Z self.handle_forward(message, sender)
2019-07-23T05:09:58.959952330Z File “/usr/share/rhasspy/rhasspy/dialogue.py”, line 535, in handle_forward
2019-07-23T05:09:58.959983736Z mics = recorder_class.get_microphones()
2019-07-23T05:09:58.960011809Z File “/usr/share/rhasspy/rhasspy/audio_recorder.py”, line 244, in get_microphones
2019-07-23T05:09:58.960060871Z default_name = audio.get_default_input_device_info().get(“name”)
2019-07-23T05:09:58.960089777Z File “/usr/local/lib/python3.6/dist-packages/pyaudio.py”, line 949, in get_default_input_device_info
2019-07-23T05:09:58.960138839Z device_index = pa.get_default_input_device()
2019-07-23T05:09:58.960166703Z OSError: No Default Input Device Available
2019-07-23T05:36:09.660822435Z DEBUG:main:Namespace(host=‘0.0.0.0’, port=12101, profile=‘fr’, set=, ssl=None, system_profiles=‘/usr/share/rhasspy/profiles’, user_profiles=‘/profiles’)
2019-07-23T05:36:09.802289061Z DEBUG:RhasspyCore:Loaded profile from /usr/share/rhasspy/profiles/fr/profile.json
2019-07-23T05:36:09.803008742Z DEBUG:RhasspyCore:Profile files will be written to /profiles/fr
2019-07-23T05:36:09.803831130Z DEBUG:root:Loading default profile settings from /usr/share/rhasspy/profiles/defaults.json
2019-07-23T05:36:09.806567040Z DEBUG:WebSocketObserver: → started
2019-07-23T05:36:09.812305942Z DEBUG:DialogueManager: → started
2019-07-23T05:36:09.812458284Z DEBUG:DialogueManager:started → loading
2019-07-23T05:36:09.812490836Z DEBUG:DialogueManager:Loading actors
2019-07-23T05:36:09.822654069Z DEBUG:DialogueManager:Actors created. Waiting for [‘recorder’, ‘player’, ‘speech’, ‘wake’, ‘command’, ‘decoder’, ‘recognizer’, ‘handler’, ‘sentence_generator’, ‘speech_trainer’, ‘intent_trainer’, ‘word_pronouncer’] to start.
2019-07-23T05:36:09.825967526Z DEBUG:PyAudioRecorder: → started
2019-07-23T05:36:09.828216045Z DEBUG:APlayAudioPlayer: → started
2019-07-23T05:36:09.833857240Z DEBUG:EspeakSentenceSpeaker: → started
2019-07-23T05:36:09.835382069Z DEBUG:PocketsphinxDecoder: → started
2019-07-23T05:36:09.847062214Z DEBUG:FsticuffsRecognizer: → started
2019-07-23T05:36:09.848552095Z ERROR: ReadFst: Can’t open file: profiles/fr/intent.fst
2019-07-23T05:36:09.849222609Z DEBUG:DummyIntentHandler: → started
2019-07-23T05:36:09.854213966Z DEBUG:FsticuffsIntentTrainer: → started
2019-07-23T05:36:09.855289685Z DEBUG:PhonetisaurusPronounce: → started
2019-07-23T05:36:09.856668109Z DEBUG:WebrtcvadCommandListener: → started
2019-07-23T05:36:09.866257025Z DEBUG:PocketsphinxWakeListener: → started
2019-07-23T05:36:09.867372118Z DEBUG:JsgfSentenceGenerator: → started
2019-07-23T05:36:09.868461586Z DEBUG:DialogueManager:recorder started
2019-07-23T05:36:09.869503607Z DEBUG:PocketsphinxSpeechTrainer: → started
2019-07-23T05:36:09.871625566Z DEBUG:EspeakSentenceSpeaker:started → ready
2019-07-23T05:36:09.872932584Z WARNING:FsticuffsRecognizer:preload: Read failed: ‘profiles/fr/intent.fst’
2019-07-23T05:36:09.874591265Z DEBUG:PocketsphinxDecoder:Loading decoder with hmm=profiles/fr/acoustic_model, dict=profiles/fr/dictionary.txt, lm=profiles/fr/language_model.txt
2019-07-23T05:36:09.875272196Z DEBUG:DialogueManager:player started
2019-07-23T05:36:09.876240885Z DEBUG:FsticuffsRecognizer:started → loaded
2019-07-23T05:36:09.877548215Z DEBUG:DialogueManager:handler started
2019-07-23T05:36:09.878727475Z DEBUG:DialogueManager:intent_trainer started
2019-07-23T05:36:09.880022722Z DEBUG:DialogueManager:word_pronouncer started
2019-07-23T05:36:09.880577821Z DEBUG:DialogueManager:sentence_generator started
2019-07-23T05:36:09.881228804Z DEBUG:DialogueManager:speech_trainer started
2019-07-23T05:36:09.881842392Z DEBUG:DialogueManager:speech started
2019-07-23T05:36:09.886464013Z DEBUG:DialogueManager:recognizer started
2019-07-23T05:36:09.939063859Z WARNING:PocketsphinxDecoder:preload: new_Decoder returned -1
2019-07-23T05:36:09.945261402Z DEBUG:PocketsphinxDecoder:started → loaded
2019-07-23T05:36:09.985626369Z ERROR:PocketsphinxWakeListener:loading wake decoder
2019-07-23T05:36:09.985771524Z Traceback (most recent call last):
2019-07-23T05:36:09.985820846Z File “/usr/share/rhasspy/rhasspy/wake.py”, line 117, in to_started
2019-07-23T05:36:09.985851835Z self.load_decoder()
2019-07-23T05:36:09.985880793Z File “/usr/share/rhasspy/rhasspy/wake.py”, line 226, in load_decoder
2019-07-23T05:36:09.985929231Z with open(dict_path, “r”) as dict_file:
2019-07-23T05:36:09.985958032Z FileNotFoundError: [Errno 2] No such file or directory: ‘profiles/fr/dictionary.txt’
2019-07-23T05:36:10.005989241Z DEBUG:DialogueManager:decoder started
2019-07-23T05:36:10.021396329Z DEBUG:PocketsphinxWakeListener:started → loaded
2019-07-23T05:36:10.022643608Z DEBUG:DialogueManager:wake started
2019-07-23T05:36:10.338425965Z DEBUG:WebrtcvadCommandListener:started → loaded
2019-07-23T05:36:10.339692931Z DEBUG:DialogueManager:command started
2019-07-23T05:36:10.340161260Z INFO:DialogueManager:Actors loaded
2019-07-23T05:36:10.340667713Z DEBUG:DialogueManager:loading → ready
2019-07-23T05:36:10.341378696Z INFO:DialogueManager:Automatically listening for wake word
2019-07-23T05:36:10.341852962Z DEBUG:DialogueManager:ready → asleep
2019-07-23T05:36:10.345742924Z INFO:main:Started
2019-07-23T05:36:10.346962391Z DEBUG:main:Starting web server at http://0.0.0.0:12101
2019-07-23T05:36:10.350663344Z ERROR:PocketsphinxWakeListener:receiveMessage
2019-07-23T05:36:10.350887092Z Traceback (most recent call last):
2019-07-23T05:36:10.350921779Z File “/usr/share/rhasspy/rhasspy/actor.py”, line 117, in on_receive
2019-07-23T05:36:10.350970997Z self._state_method(message, sender)
2019-07-23T05:36:10.351001570Z File “/usr/share/rhasspy/rhasspy/wake.py”, line 125, in in_loaded
2019-07-23T05:36:10.351049434Z self.load_decoder()
2019-07-23T05:36:10.351078340Z File “/usr/share/rhasspy/rhasspy/wake.py”, line 226, in load_decoder
2019-07-23T05:36:10.351126673Z with open(dict_path, “r”) as dict_file:
2019-07-23T05:36:10.351156464Z FileNotFoundError: [Errno 2] No such file or directory: ‘profiles/fr/dictionary.txt’

Edit 2 : I can give more details, when I restart my container I can curl it from the docker host, after the first connection to the web ui from my browser everything stop working : The UI and the curl.

Thank you for the logs and for giving Rhasspy a try! The errors I see in the log are expected until you do the training for the first time. But, of course, the web UI should not be locking up.

Is this running on a Pi 3B or a Pi 3B+? I’m assuming Raspbian. Is there a microphone attached?

Well I have to thank you first for your answer.

Anyway, I have a Pi 3B and no microphone attached. My goal was to deploy a standalone Rhasspy and “plug” some Hermes server to act as client.

I assume that since my previous post I did not have time to investigate more. But I plan tospend some time today on Rhasspy.

By the way, Is there a dedicated Rhasspy forum or Discord ?

I’ve updated the Docker images with a few things, but the one you may be interested in is Rhasspy defaulting to the “dummy” microphone system. This doesn’t try to load PyAudio, which seems to be the most likely cause of your problem. If you get a chance to update and try again, please let me know the results!

There isn’t currently. Any advice on setting one up?

Thank for that ! I updated and met the exact same problem, a curl is possible then nothing.

Is there any logs I can check for the web client ?

I don’t know if that can be a clue but, I don’t find any files in my directory /.config/rhasspy/profiles/fr.

No, not really :sweat: I’m just an ordinnary Discord user and never have set up or manage a server.

Edit: Actually, I can curl my container indefinetly, the moment I try to access it with a browser I lost the possibility to curl… IE when I do “Local computer [Browser]” > “Docker Host:12101” > “Rhasspy:12101” I lost the ability to curl. I’m currently playing with the docker macvlan option trying to find a solution.

I’m not sure if it will help, but I’ve pushed another fix trying to address the errors I see in your logs. The web server itself should be outputting everything to the same logs you’re already seeing. When Rhasspy goes brain dead, do you see any high CPU usage?

If you can, please pull the latest update and share your logs again if there are still problems. Thanks!

Hi there,

I was recently switching from a Pi to my NAS, which has AMD64 cpu.
When I try to build the addon I get:
Can’t build 75f2ff60/amd64-addon-rhasspy:2.20: unauthorized: authentication required

I have checked the code, and then did a manual pull of the rhasspy-server.
Somewhere halfway the error occurs and the pull stops. Pulling on other archirechtures is fine, it tried intell as well.

Any ideas? This was on version 2.19 and also in 2.20 by the way.

I have no idea. I re-built the Docker images and bumped the Hass.IO version number just in case. I don’t see any notification on my DockerHub page about any problems (though I was surprised to see Rhasspy has 50k+ pulls!).

The only suggestion I saw when Googling around was something about the host clock being off. I wonder if your Hass.IO doesn’t have access to /etc/localtime and that’s causing problems?

That is why I ran a manual docker pull on the NAS.
I have an Asustor with entware installed, first thing I though that the building was an issue, but when I did a manual pull of rhasspy-server:latest, I got the same message.

It fails somewhere halfway, I will try to find out the problem

My CPU usage was flat but now It works ! I pull the latest image and now everything seems fine. I’m gonna play for a while with this now! Thanks a lot!