Rhasspy offline voice assistant toolkit

Share your Projects! Custom Integrations
21

You’re welcome! There is a way to test Mycroft’s precise using a tool called precise-listen that gets installed into your HA virtual environment. If you activate the virtualenv and then run precise-listen /path/to/okay-rhasspy.pb it should listen with the default microphone and print !!! when you say the wake word (“okay rhasspy”).

You might also try the hotword_snowboy and hotword_pocketsphinx components to see if either of those works. hotword_pocketsphinx isn’t as good (lots of false positives) but it does let you select a specific audio device in the configuration.

22

Could this work on hassio?

1 Like
23

@fernandex00 it seems not:

24

When I run precise-listen, I get the following error:

Traceback (most recent call last):
File “/home/rhasspy/.homeassistant/deps/bin/precise-listen”, line 7, in
from precise.scripts.listen import main
ModuleNotFoundError: No module named ‘precise’

Is there any additional installation necessary to have that module ‘precise’? Maybe that is the reason why it does not detect the hotword…

25

It could work on Hass.IO in principle, but it would require re-architecting rhasspy into a set of Docker containers. Are most people using Hass.IO these days?

26

I bumped the version of mycroft-precise to 0.2.0 inside hotword_precise.py (it was 0.1.1). It needs another Python library called precise-runner that is also listed in hotword_precise.py, but maybe the versions were mismatched.

27

Ir could be good to use this on hassio, most people uses hassio

1 Like
28

I don’t know if anyone knows the true answer to that but it is certainly the direction anyone new to HA is guided down.

29

This looks very intresting, thanks for the great work.
I will need a server/client setup tho as my home assistant installation is running on my nas server and there is no good way to get a microphone to this setup that will be useable in the rest of my home.

But i like the idea of simple raspberrz pis which work as client only, maybe combined with a speaker for tts output that i could place around the the flat.

30

A front-end/back-end type setup would also allow for multiple microphones (with real cheap low-end hardware) in multiple rooms around the house, while the “heavy” lifting could be done by the back-end.

Course, then you start potentially running into multiple microphones hearing you…

31

i just found the client server documentation on github, missed that.
But as far as i understand this requires home assistant on both machines or is this still incomplete?

Optimal setup would be exactly like mrand described, minimal microphone client that will only record and just sends the audiodata to my much more powerfull server.

32

The audio data is just POSTed via HTTP to the server, so it would be pretty straightforward to adapt the existing code to run without Home Assistant.

33

Small update: I’ve added a quickstart that uses Home Assistant’s built-in conversation component instead of rasaNLU. While less powerful and flexible, it should make installation a lot easier!

34

I use which client_server? conversation? raspberry_pi? single_machine?

35

Depends on your goals:

  • If you’re running everything on a Raspberry Pi, then use raspberry_pi
  • If you have a Pi for recording and a server for processing, use client_server
  • If your Home Assistant is on a desktop computer or laptop, you can use single_machine or conversation
    • Try to get conversation working first - it doesn’t require as many dependencies
    • single_machine uses the more flexible rasaNLU for intent recognition, but it’s harder to get installed right
1 Like
36

thank you very much

37

I have the same issue which I’d be grateful for any advice on. I’ve installed a fresh raspberian OS with a PS3 style USB microphone array and using the audio out from the pi itself.

aplay and arecord work fine once I’d created a suitable .aroundrc file. I can use the extra rhasspy services in HA to fire off events to make sounds on the speaker OK as well.

I followed your guide to install on raspberry using the raspberrypi config files and while it starts up OK, I also get no response when talking. A test with snowboy directly works fine and the word snowboy gets recognised, but running HA gives me nothing when I say OK RHASSPY. I wanted to do an initial diagnosis and so I tried the precise-listen command you mentioned in my HA venev but that isn’t there. I installed via https://github.com/MycroftAI/mycroft-precise, but again that executable isn’t there.

When hass starts up, there are lots of warnings printed along the lines of

ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘cards.bcm2835_alsa.pcm.front.0:CARD=0’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM front
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘cards.bcm2835_alsa.pcm.surround51.0:CARD=0’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
<>

Do you have any generic advice on diagnosing what I think is a very standard pi config? If you do, I’m quite happy to spend time working through things. My first guess is that somehow pyaudio needs to be told which device to listen to, but I have no idea how to configure that (and my guess is probably wrong!) :slight_smile:

38

EDITED WITH CORRECTIONS

Those warnings are pretty typical; I usually ignore them. I wonder if the libpulse dependency is messing with ALSA somehow? I remember having to manually edit sphinxbase/__init__.py inside my virtual environment at one point to force pocketsphinx to use ALSA.

Have you run the microphones.py script I added to the repo? It prints all the PyAudio devices and their device index. stt_pocketsphinx has a config property called audio_device that lets you pick your microphone. Try changing that to 0, 1, etc and see what happens. Confusingly, audio_device is either an index or a name depending on whether pocketsphinx is using ALSA or Pulse.

39

Also if snowboy is working for you, you might just switch to using the hotword_snowboy component instead of hotword_precise (see the Raspberry Pi quickstart for an example). Another one to try is hotword_pocketsphinx, which you can configure like this;

hotword_pocketsphinx:
  hotword: "okay raspy"  # purposely misspelled
  acoustic_model: $RHASSPY_TOOLS/pocketsphinx/cmusphinx-en-us-5.2
  dictionary: $RHASSPY_ASSISTANT/data/mixed.dict
40

Cheers. I found and ran microphones.py and it pumps out the same sort of initial things I saw in the HA log (and I’m ignoring them!) At the end it pumps out a list and the two with microphones are

Device 2 (USB Camera-B4.09.24.1: Audio (hw:1,0))

{‘defaultHighInputLatency’: 0.096,
‘defaultHighOutputLatency’: -1.0,
‘defaultLowInputLatency’: 0.0239375,
‘defaultLowOutputLatency’: -1.0,
‘defaultSampleRate’: 16000.0,
‘hostApi’: 0,
‘index’: 2,
‘maxInputChannels’: 4,
‘maxOutputChannels’: 0,
‘name’: ‘USB Camera-B4.09.24.1: Audio (hw:1,0)’,
‘structVersion’: 2}

and

Device 4 (mic)

{‘defaultHighInputLatency’: 0.096,
‘defaultHighOutputLatency’: -1.0,
‘defaultLowInputLatency’: 0.0239375,
‘defaultLowOutputLatency’: -1.0,
‘defaultSampleRate’: 16000.0,
‘hostApi’: 0,
‘index’: 4,
‘maxInputChannels’: 128,
‘maxOutputChannels’: 0,
‘name’: ‘mic’,
‘structVersion’: 2}

The reason there arew two is because like many other people, I simply couldn’t get the USB microphone to work with arecord until I had set up an ~/.asoundrc file which has the following:

pcm.!default {
type asym
capture.pcm “mic”
playback.pcm “speaker”
}
pcm.mic {
type plug
slave {
pcm “plughw:1,0”
}
}
pcm.speaker {
type plug
slave {
pcm “hw:0,0”
}
}

I tried all sorts of other combination, but this is the only one that allowed aplay and arecord to ‘just work’. I’m happy to try removing this or other changes recommended.

I changed the ha configuration.yaml to have the following. I couldn’t seem to find any docs on stt_pocketsphix for HA, so I simply did the following. Feel free to correct me!

stt_pocketsphinx:
acoustic_model: /home/pi/ha/rhasspy-tools/pocketsphinx/cmusphinx-en-us-ptm-5.2
language_model: /home/pi/ha/rhasspy-assistant/data/examples.lm
dictionary: /home/pi/ha/rhasspy-assistant/data/mixed.dict
audio_device: 4

I tried both the value 2 and the value 4 restarting HA each time. No joy and to be fair I don’t actually know if either of these values have been picked up and are in use. Is there a way to tell?