Great, thank you for your time! I’ll add some more MQTT functionality once we get these fundamental issues sorted.
The critical line to look for in the log is this:
INFO:werkzeug: * Running on http://0.0.0.0:12101/ (Press CTRL+C to quit)
This indicates that the Flask web server started, which comes at the very end of the start-up process. I just pushed a new version that has a 15 second timeout, so the web server should start even if one of the sub-components (microphone, etc.) fails to start.
OK, I’m finally getting a chance to test again on Hass.IO and I’m really disappointed in snowboy. It seems to be useless on ARMv7 due to the unexpected reloc type 0x03 bug, and it won’t even install on ARMv8 (aarch64)!
I can at least try and back off to an ARMv6 docker base for Rhasspy on the Pi 3, but I’m not sure what to do for the Pi 3 B+. There is clearly an aarch64 library for snowboy, but it doesn’t seem to work. Mycroft Precise is also out on the Pi’s unless I can get tensorflow installed there. Any thoughts?
Okay, indeed, with previous version this line never appeared.
One other thing that was in cause I think too is that I had set language to ‘fr’ in the addon config, and this seems to break the start.
while ‘fr’ is set as profile :
* Serving Flask app “app.py”
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: off
DEBUG:app:Namespace(profile=None, set=[])
DEBUG:RhasspyCore:Loaded profile from profiles/fr/profile.json
INFO:root:++++ Actor System gen (3, 9) started, admin @ ActorAddr-Q.ThespianQ
DEBUG:root:Thespian source: /usr/local/lib/python3.6/site-packages/thespian/init.py
DEBUG:DialogueManager:Actors created
DEBUG:DialogueManager:started -> loading
DEBUG:DialogueManager: -> started
DEBUG:HermesMqtt: -> started
DEBUG:PhonetisaurusPronounce: -> started
DEBUG:FuzzyWuzzyIntentTrainer: -> started
DEBUG:PocketsphinxSpeechTrainer: -> started
DEBUG:JsgfSentenceGenerator: -> started
DEBUG:HomeAssistantIntentHandler: -> started
DEBUG:FuzzyWuzzyRecognizer:started -> loaded
DEBUG:APlayAudioPlayer: -> started
DEBUG:FuzzyWuzzyRecognizer: -> started
DEBUG:WebrtcvadCommandListener:started -> loaded
DEBUG:PyAudioRecorder: -> started
DEBUG:WebrtcvadCommandListener: -> started
INFO:PocketsphinxDecoder:Loading decoder with hmm=profiles/fr/acoustic_model, dict=profiles/fr/dictionary.txt, lm=profiles/fr/language_model.txt
DEBUG:PocketsphinxWakeListener:Loading wake decoder with hmm=profiles/fr/acoustic_model, dict=profiles/fr/dictionary.txt
ERROR:PocketsphinxWakeListener:receiveMessage
Traceback (most recent call last):
File “/usr/share/rhasspy/rhasspy/actor.py”, line 38, in receiveMessage
self.transition(‘started’)
File “/usr/share/rhasspy/rhasspy/actor.py”, line 59, in transition
getattr(self, transition_method)(from_state)
File “/usr/share/rhasspy/rhasspy/wake.py”, line 49, in to_started
self.load_decoder()
File “/usr/share/rhasspy/rhasspy/wake.py”, line 131, in load_decoder
self.decoder = pocketsphinx.Decoder(decoder_config)
File “/usr/local/lib/python3.6/site-packages/pocketsphinx/pocketsphinx.py”, line 275, in init
this = _pocketsphinx.new_Decoder(*args)
RuntimeError: new_Decoder returned -1
ERROR:PocketsphinxDecoder:receiveMessage
Traceback (most recent call last):
File “/usr/share/rhasspy/rhasspy/actor.py”, line 38, in receiveMessage
self.transition(‘started’)
File “/usr/share/rhasspy/rhasspy/actor.py”, line 59, in transition
getattr(self, transition_method)(from_state)
File “/usr/share/rhasspy/rhasspy/stt.py”, line 50, in to_started
self.load_decoder()
File “/usr/share/rhasspy/rhasspy/stt.py”, line 87, in load_decoder
self.decoder = pocketsphinx.Decoder(decoder_config)
File “/usr/local/lib/python3.6/site-packages/pocketsphinx/pocketsphinx.py”, line 275, in init
this = _pocketsphinx.new_Decoder(*args)
RuntimeError: new_Decoder returned -1
INFO:werkzeug: * Running on http://0.0.0.0:12101/ (Press CTRL+C to quit)
and then while ‘en’ is set :
* Serving Flask app “app.py”
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: off
DEBUG:app:Namespace(profile=None, set=[])
DEBUG:RhasspyCore:Loaded profile from /data/rhasspy/profiles/en/profile.json
INFO:root:++++ Actor System gen (3, 9) started, admin @ ActorAddr-Q.ThespianQ
DEBUG:root:Thespian source: /usr/local/lib/python3.6/site-packages/thespian/init.py
DEBUG:DialogueManager:Actors created
DEBUG:DialogueManager:started -> loading
DEBUG:DialogueManager: -> started
DEBUG:PhonetisaurusPronounce: -> started
DEBUG:FuzzyWuzzyIntentTrainer: -> started
DEBUG:PocketsphinxSpeechTrainer: -> started
DEBUG:JsgfSentenceGenerator: -> started
DEBUG:HomeAssistantIntentHandler: -> started
DEBUG:FuzzyWuzzyRecognizer:Loaded examples from /usr/share/rhasspy/profiles/en/intent_examples.json
DEBUG:WebrtcvadCommandListener:started -> loaded
INFO:PocketsphinxDecoder:Loading decoder with hmm=/usr/share/rhasspy/profiles/en/acoustic_model, dict=/usr/share/rhasspy/profiles/en/dictionary.txt, lm=/usr/share/rhasspy/profiles/en/language_model.txt
DEBUG:APlayAudioPlayer: -> started
DEBUG:PyAudioRecorder: -> started
DEBUG:WebrtcvadCommandListener: -> started
DEBUG:FuzzyWuzzyRecognizer:started -> loaded
DEBUG:PocketsphinxWakeListener:Loading wake decoder with hmm=/usr/share/rhasspy/profiles/en/acoustic_model, dict=/usr/share/rhasspy/profiles/en/dictionary.txt
DEBUG:FuzzyWuzzyRecognizer: -> started
INFO:werkzeug: * Running on http://0.0.0.0:12101/ (Press CTRL+C to quit)
DEBUG:PocketsphinxWakeListener:started -> loaded
DEBUG:PocketsphinxWakeListener: -> started
DEBUG:PocketsphinxDecoder:started -> loaded
DEBUG:PocketsphinxDecoder: -> started
INFO:DialogueManager:Actors loaded
INFO:DialogueManager:Automatically listening for wake word
DEBUG:DialogueManager:ready -> asleep
DEBUG:DialogueManager:loading -> ready
DEBUG:PocketsphinxWakeListener:loaded -> listening
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘cards.bcm2835_alsa.pcm.front.0:CARD=0’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM front
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘cards.bcm2835_alsa.pcm.surround51.0:CARD=0’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround21
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘cards.bcm2835_alsa.pcm.surround51.0:CARD=0’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround21
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘cards.bcm2835_alsa.pcm.surround40.0:CARD=0’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround40
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘cards.bcm2835_alsa.pcm.surround51.0:CARD=0’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround41
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘cards.bcm2835_alsa.pcm.surround51.0:CARD=0’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround50
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘cards.bcm2835_alsa.pcm.surround51.0:CARD=0’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround51
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘cards.bcm2835_alsa.pcm.surround71.0:CARD=0’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround71
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘cards.bcm2835_alsa.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM iec958
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘cards.bcm2835_alsa.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM spdif
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘cards.bcm2835_alsa.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM spdif
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
DEBUG:PyAudioRecorder:Recording from microphone (PyAudio, device=None)
DEBUG:PyAudioRecorder:started -> recording
It goes further with ‘en’ or default profile than when I set ‘fr’
OK, so I brilliantly forgot to include all of the other profiles 
Fixed now in version 1.25.
Also, if the microphone fails to work, try switching from PyAudio to Arecord in the settings. The very latest version of Hass.IO is giving me problems too…
1.25 started fine on my Hassio RPi3B!
I will start trying the MQTT stuff, see if I can get it to work 
1 Like
Great! The MQTT “microphone” thing has been working for me, but Rhasspy needs to be woken up first before it starts recording. For now, please use the Wake Up button in the web interface (it should beep at you if you have speakers connected and an output device set).
For MQTT + wake word, Rhasspy was originally streaming out audio data. I disabled that after it seemed like folks were getting confused. Do you need that, or just need Rhasspy to wake up when it gets the Hermes wake word detected event?
I need Rhasspy to listen to a MQTT audiostream, like it seems to do now
And indeed, do hotword detection on that audio stream. But it is possible to do that in another way.
It seems there is something wrong with saving settings though, it looks ok but when I do a restart, settings are back to default.
It should actually do hotword detection on that stream, though snowboy is still not working 
I’m experiencing the same problem with settings every time I restart the container. Isn’t /data a persistent place to write in Hass.IO?
Yes, that is indeed the place.
Snowboy is not a real issue at this point, since I am still experimenting
OK, it seems that you have to have config:rw in the map section of the config file for the add-on before Hass.io actually mounts /data. I was able to get settings to stick around between restarts using this, but updates to the add-on would blow them away. So I switched to writing to /share/rhasspy in hopes of keeping settings between updates and restarts.
Also, I added support for waking up on the Hermes hotword detected event.
1 Like
Good improvement
I have limited time right now, but am testing some stuff out
Hi there,
Can you explain a bit more about yoyr configuration / tests with hermes protocol?
I can’t seem to get it to work. I have Rhasspy connected to my MQTT broker:
INFO:HermesMqtt:Connected to 192.168.178.194:1883
when I press wake up button this appears in the log:
DEBUG:HermesWakeListener:listening -> loaded
DEBUG:HermesAudioRecorder:Recording from microphone (hermes) DEBUG:HermesAudioRecorder:started -> recording
Seems good to me, but then? I get this in my log:
DEBUG:DialogueManager:awake -> decoding
DEBUG:WebrtcvadCommandListener:listening -> loaded DEBUG:PocketsphinxDecoder:rate=16000, width=2, channels=1. DEBUG:PocketsphinxDecoder:Decoded WAV in 0.0011963844299316406 second(s) DEBUG:DialogueManager:
DEBUG:DialogueManager:decoding -> recognizing
DEBUG:APlayAudioPlayer:[‘aplay’, ‘-q’, ‘etc/wav/beep_lo.wav’]
DEBUG:DialogueManager:{‘text’: ‘’, ‘intent’: {‘name’: ‘’}, ‘entities’: {}} DEBUG:DialogueManager:recognizing -> handling WARNING:HomeAssistantIntentHandler:Empty intent. Not sending to Home Assistant
I push a wave file to MQTT via node red. The exact same wav file which work fine when I upload it via the web ui.
How did you test this? I can then try to do it the same way.
Are you sending the WAV file all at once, or in chunks? I was assuming in chunks (as if it were streaming), so I passed it through my default voice command listener (which needs silence to detect when the command is over).
I’ve since added a “oneshot” voice command listener that assumes the entire voice command is in a single chunk of audio data. I’m currently putting together an MQTT example with docker compose that has all of the relevant configuration information. The example profile.json sets command.system to oneshot, enables MQTT (make sure to change the host), and sets both microphone.system and wake.system to hermes. I then have a bash script that wakes Rhasspy up and sends in a WAV file via MQTT (again, change the host).
Hope this helps for now!
I have tried both
The wav file as one shot, after I tried the chunks way with my Matrix Voice Audio Streamer.
The latter works good with Snips, so I am certain it sends small wave files.
But I see you say send 1 wav file in chunks, my software (and snips) sends every small chunk as a wav file. By default 256 frames per wave message I believe.
I will try your example and continue from there, I think you are doing great work.
Currently I am trying Dutch and this works very well.
I am using Rhasspy as a Hassio AddOn, I see that in your docker compose you use a separate docker for rhasspy and home assistant.
I was a bit confused by the docs though, I was assuming that I should create an intent_script like in de documentation, but in the AddOn documentation I read that I should use events and automations. After I created that, it worked excellent
While I am still testing with MQTT, I am building automations for my devices via events
Great, thank you for sticking with it!
Regarding intents vs. events, it seems like intents can only be fired from within Home Assistant, so a component is required. I used events because Hass.io doesn’t allow for custom components.
However, I realized this morning that that Snips.ai component will generate intents from the Hermes MQTT messages with no extra components required So…I added MQTT intent publishing into Rhasspy today and updated the example to use an intent instead of an event. Seems like a good approach if you have a lot of intent scripts you’ve already invested in.
That is indeed a great idea, I do not have that much so it was no problem for me.
But in general hook into the snips component is a nice feature.
1 Like
On my never-ending quest to make Rhasspy ever more complicated, I’ve added the ability to swap out most of the internal components (speech-to-text, intent recognition, etc.) with calls to external programs! You can even have Rhasspy run a program instead of (or in addition to) contacting Home Assistant.
I am also slowly creating a set of example configurations with docker compose to demonstrate how to use Rhasspy by itself, in a client-server set up, etc.
Lastly, if anyone can read one of the supported profile languages, I’d love to hear feedback on the sentences in sentences.ini. Most of them (excluding English and Dutch) were done using Google Translate 
1 Like
Quick question, does the MQTT audio recorder expect 16bit, 16 kHz mono or stereo?
The short answer is mono.
The long answer is, anything but 16-bit, 16 Khz mono will be converted automatically. This probably isn’t a problem when the MQTT message contains an entire WAV file, but it could be a major performance hit when streaming.
I use sox to do the WAV conversion, and I just pushed an update where I’m using stdin/stdout instead of temporary files. Hopefully that will help with any performance problems. If not, maybe you know of a faster way to do the conversion in Python?