Rhasspy offline voice assistant toolkit

Iā€™ve made a subcategory request in the voice assistant forum.

If all of you like the request, it may show the interest from many to Rhasspy development and make the admin decide to create the dedicated subforum ?

1 Like

@synesthesiam

I had a question for you.
I wanted to make an intent that allowed for a variable number. For example ā€œSet a timer for X minutesā€ (this variable would be passed to a Node-Red automation by its tag). What would I use for the X? Originally I saw that pipes allowed for multiple options, but listing all the numbers would be a big list. I see there is a new feature ā€œslotsā€ and from what I read in the docs this might work, but I think would still require creating the list.

Is there another way, or would one of these be the best option?

Thanks!
DeadEnd

damn @synesthesiam just got round to installing and up and running in like 10 mins.

Docker server (NUC)
Docker client (pi 3B with Seeed Mic Array)
(Using Node-Red)

Honestly I spent too much time reading before I jumped in! Awesome job :smiley:
Write up here

Couple of questions for the seasoned users:

  1. How do you break out the split of responsibility? (Wake, voice detection and mic on pi) (Speech to Text & Intent on Server?). Figured it may be easier to process the Speech to Text locally and send the result?
  2. How do people handle switches that are lights?
  3. I have the same issue as here based on ā€œJarvisā€
  4. Anyone fancy sharing some of their sentences for inspiration?
2 Likes

Regarding 2:

Depending on your setup you can call the service homeassistant which takes whole entity ids like switch.light_outside.

1 Like

I have a basic setup I hope to expand in the future.
Currently everything is run on a min-ATX server (it is on the opposite side of a wall from the main living area, so I ran a USB through the wall for a speakerphone).

I disabled MQTT and Home Assistant intent handling since I use Node-Red for all automation. I thought I would share since you said you are using Node-Red as well.

I setup a web-socket listener and then a switch node that filters based on the intent. From this you can use tagged variables that are passed - I use a switch node to change names if necessary. Then you can call Home Assistant services, or whatever else you need.

Farther up on this forum I have my first working example explained in short details.

Cheers!
DeadEnd

Until Rhasspy supports builtin grammar based entities (like datetime, number, etc) you can look at the timer recipe from voice2json for inspiration.

Cheers :slight_smile:

2 Likes

Awesome!
Thanks for the link!

Quick questionā€¦ can someone explain why they broke out one to fifty nine vs two to fifty nine?
in the minute/second expressions, couldnā€™t they have just dropped the ((one:1){minutes}) or second and used the one to fifty nine instead?

ā€¦ lastly, for anyone who uses this timer designā€¦ how long did training take? 8 million possibilities is a lot - but my system is taking FOREVER to do the training. It is a mini-ATX, not a Piā€¦ didnā€™t expect it to take more than a few minutes.

Thanks, I had a feeling it was likely best to have ā€œlightā€ in Rhasspy and based on entity convert that to ā€œswitchā€ based on the entity being called through NR.
Alexa has a pretty graceful way of doing it by surfacing the entity config (essentially allows the user to surface a switch as a light).

Are you using the fsticuffs intent recognizer or fuzzywuzzy? The fsticuffs training shouldnā€™t take very long, since it doesnā€™t generate all possible sentences up front (like all the others have to).

Good Catch.
I was using fuzzywuzzy - Iā€™ll change to fsticuffs.
I think I was using fuzzywuzzy as fsticuffs was having issues getting the wrong intent.

Iā€™ll see how it behaves with fsticuffs and this larger sentence pool.
ā€¦ rhasspy seemed to be locking up so I restarted the containerā€¦ after that fsticuffs trained in under a second. I tried going back to fuzzywuzzy and it still wasnā€™t able to do it - but since the documents outline it is intended for 12-100 sentences, I think this is acceptableā€¦ it just isnā€™t designed for it.

Thanks!

1 Like

Iā€™m working on improving fsticuffs in the very near term, so it will (hopefully) do a better job at getting the right intent. Ideally, it will be possible for other people to add some better matching algorithms.

fuzzywuzzy is very flexible, but doesnā€™t scale at all since it basically stores every sentence in a big JSON file and has to go through each one every time it processes a sentence.

Thanks for keeping with it!

Hi All,
Iā€™ve just tried install Rhasspy and so far Iā€™ve had no luck, I wonder if anyone could cast their eye over this and suggest a way forward. Iā€™m using a headless (ssh only) Pi Zero W with Respeaker 2-Mics Pi Hat and this is the full command history of this pi (after pi bakery did the wifi/hostname etcā€¦)

sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get update
git clone https://github.com/respeaker/seeed-voicecard.git
cd seeed-voicecard
sudo ./install.sh
sudo reboot
# arecord -l successfully shows the hat as a microphone
curl -sSL https://get.docker.com | sh
sudo usermod -a -G docker $USER
sudo reboot
docker run -d -p 12101:12101 \
      --restart unless-stopped \
      -v "$HOME/.config/rhasspy/profiles:/profiles" \
      --device /dev/snd:/dev/snd \
      synesthesiam/rhasspy-server:latest \
      --user-profiles /profiles \
      --profile en

Effectively following the instructions for Respeaker then Rhasspy. My problem is none of the following commands return anything at all


pi@mic:~ $ docker run -it -p 12101:12101       --restart unless-stopped       -v "$HOME/.config/rhasspy/profiles:/profiles"       --                               device /dev/snd:/dev/snd       synesthesiam/rhasspy-server:latest       --user-profiles /profiles       --profile en
pi@mic:~ $ cat < /dev/tcp/127.0.0.1/12101
pi@mic:~ $ ls .config/rhasspy/profiles/
pi@mic:~ $ 

the docker run seems to silently succeed but thereā€™s nothing running on the port. Nothing created in profiles. Trying from a web browser on another machine I get refused.
Any ideas what Iā€™ve missed? Or what I can run to get more debug info?

hmmā€¦ the profiles etc. I do not believe will be created until you do the first training. What error message are you getting in the browser? This would be my first step is to get to the web interface.

DeadEnd

Unable to connect
Firefox canā€™t establish a connection to the server at 192.168.1.154:12101.
The site could be temporarily unavailable or too busy. Try again in a few moments.
If you are unable to load any pages, check your computerā€™s network connection.
If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.

after starting the container (docker run -d -p 12101:12101ā€¦) give it a minute and then run:

docker container ls

Take a look and see if you container is still up and running.
Do you use other containers? If so, are they up?
Lastly, do you use Portainer? Useful GUI for container management.

DeadEnd

Hi @dans, sorry things arenā€™t working for you. Can you confirm that a Pi Zero can actually run an armv7l Docker container? I thought Pi Zeroā€™s were ARMv6 CPUs. What does uname --m output for you?

CONTAINER ID        IMAGE                                COMMAND                  CREATED             STATUS                            PORTS               NAMES
476d0d4178ce        synesthesiam/rhasspy-server:latest   "/run.sh --user-profā€¦"   22 hours ago        Restarting (139) 47 seconds ago                       recursi

yup, looks like the container is live and well

You are correct. Running uname --m returns armv6l

No, it is restarting - that means it is failing to start and looping.
It should be online or something like that.

Sounds like synesthesiam is taking you down the correct path of questions :slight_smile:

OK, I think Iā€™ll have to build a special version for the Pi Zero. Unfortunately, Ubuntu doesnā€™t have an official image for anything before armv7, but it looks like Debian itself does.