Rhasspy offline voice assistant toolkit

Sure, if you let Rhasspy publish all intents to your MQTT broker, you can let any program respond to any MQTT message, so you can mix them.

ok will put something together that is a bit more general, my current example is full of custom HA stuff :slight_smile:

PS: anyone else experiencing problems with wavenet right now? Just noticed that rhasspy fell back to using picottts. I got the following erros in my log:


[DEBUG:72710992] GoogleWaveNetSentenceSpeaker: ready -> speaking
[DEBUG:72710992] GoogleWaveNetSentenceSpeaker: Falling back to PicoTTSSentenceSpeaker
[ERROR:72710990] GoogleWaveNetSentenceSpeaker: speak
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 690, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 592, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
	status = StatusCode.INTERNAL
	details = "Received RST_STREAM with error code 2"
	debug_error_string = "{"created":"@1575364820.905986736","description":"Error received from peer ipv4:172.217.19.74:443","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Received RST_STREAM with error code 2","grpc_status":13}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/share/rhasspy/rhasspy/tts.py", line 519, in in_ready
    self.wav_data = self.speak(message.sentence)
  File "/usr/share/rhasspy/rhasspy/tts.py", line 604, in speak
    response = client.synthesize_speech(synthesis_input, voice, audio_config)
  File "/usr/local/lib/python3.6/dist-packages/google/cloud/texttospeech_v1/gapic/text_to_speech_client.py", line 322, in synthesize_speech
    request, retry=retry, timeout=timeout, metadata=metadata
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/gapic_v1/method.py", line 143, in __call__
    return wrapped_func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/retry.py", line 277, in retry_wrapped_func
    on_error=on_error,
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/retry.py", line 182, in retry_target
    return target()
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/timeout.py", line 214, in func_with_timeout
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
    six.raise_from(exceptions.from_grpc_error(exc), exc)
  File "<string>", line 3, in raise_from
google.api_core.exceptions.InternalServerError: 500 Received RST_STREAM with error code 2
1 Like

Well Snips finally ends - Console will be closing down on 31 Dec 2020. What a shame!
https://forum.snips.ai/t/important-message-regarding-the-snips-console/4145

There goes my Jarvis!

1 Like

I think Rhasspy has grown enough in popularity to warrant its own discussion site. I was thinking of setting up a Discourse droplet on Digital Ocean. Does anyone have any experience with this? Any other suggestions?

BTW, Discourse is what this site is using. Seems pretty nice to me.

1 Like

Iā€™m with you, that Rhasspy deserves a lot more space than this one thread. :slight_smile: But in general the advantages in having a well run forum and itā€™s members outweigh the ā€œneed for spaceā€.

See, you would have to run the droplet, somebody needs to moderate the discussions and the most important point, others are not that often there to help. If a thread is here on this forum, a lot of people ā€œjust take a lookā€ and can answer at least some questions and stay up to date.

If I would be asked, I would vote for a new subforum for Rhasspy under ā€œThird Party Integrationsā€. Would make (my) life easier and I wouldnā€™t need to check another site for news that are interesting or necessary for me.

But thatā€™s just my two cents, no hard feelings, please! :slight_smile:

2 Likes

I didnā€™t realize that was an option. Thanks! Iā€™ll check into it.

1 Like

I second that idea.
It keeps this (strong) community connected.
Have a sub topic or whatever they call it in third partyā€¦ like ESPHome, etc.

Cheers!
DeadEnd

Good news, Paulus (the HA founder), has agreed to create a general Voice Assistant category! Iā€™ll post a link as soon as I get more info.

This should be generally useful for discussions/help on Rhasspy and Ada.

5 Likes

Its there About the Voice Assistant category

I think the first thing would be to move this fine thread there :slight_smile:

2 Likes

Discord is a pretty popular alternative.

1 Like

@synesthesiam

Quick question on persistent data.
While working through my configurations I found an interesting situationā€¦ and I wanted to confirm I understand it correctly. I am running Rhasspy in a container. I have the profiles folder as a mapped volume so that I can make changes, backups, etc.

So, I decided to clear my config folder and start from scratch (for the Kaldi issue I had). I stopped my container, deleted the mapped directory, recreated the directory, and then started the Rhasspy container up again.

To my surprise, my sentences, custom words, etc. were still there even though my sentences.ini, custom_words.txt, etc. were deleted (and actually have still not been recreated).

So, my assumption is that the container when it is trained stores this in another location. Since I only stopped it and did not delete the container itself, this data was still present in a database somewhere. Does all that make sense? and am I correct?

So the only way to clear it correctly would be to delete the folder AND the container?

Thanks!
DeadEnd

1 Like

Rhasspy should only be writing to the directory specified in the --user-profiles argument. If youā€™re seeing something else, this might be a bug or a problem with configuration.

Iā€™d be interested to see your docker run command and any log messages you have when Rhasspy starts (it should print out the profile directories). Thanks :slight_smile:

Hi @synesthesiam, youā€™re doing an awesome job with Rhasspy, but I think you could use some help, as more and more users are coming in, especially from Snips. I guess there are a lot of people, like me, that want to contribute more but donā€™t know well what tasks they could pick up.

I think itā€™s a good idea to put some issues on GitHub with a description of some basic tasks that should be implemented, enhanced, fixed, tested or documented (all of these aspects are important in my opinion) in Rhasspy.

I especially like the approach of LibreOfficeā€™s Easy Hacks, which has lists of ā€œeasy hacksā€ that newcomers can implement, ordered by skill, difficulty level or topic. Itā€™s a great way to familiarize yourself with the codebase and significantly lowers the barrier to contributors.

So if you would put some issues on GitHub with a label easyhack or something like that, people would see that they should be able to tackle this issue even if they arenā€™t very experienced with the Rhasspy codebase. It could pull in some new contributors or trigger occasional contributors like me more to dig in because much of Rhasspyā€™s code (especially the actor model youā€™re using) is way over my head and Iā€™m still a bit lost :slight_smile:

4 Likes

Wellā€¦ I tried to reproduce what happened, and it didnā€™tā€¦
So, I must have done something weird to cause it, but I stopped the container, deleted the config directory, and started the container again, and as expected everything was default.

I really donā€™t know how I did itā€¦ but unless I can reproduce it consider it user error. :man_shrugging:

Thanks!
DeadEnd

1 Like

Thanks, this is something Iā€™d like to address as soon as possible. One of my first steps is splitting out pieces of Rhasspy into separate libraries, so contributors can focus on smaller pieces. For example, Iā€™m almost ready to release the rhasspy-nlu library that encapsulates the sentences.ini parsing and fsticuffs intent recognition.

Iā€™d also like to scale back whatā€™s in the core Rhasspy distribution, and instead make it easy to point Rhasspy at an external service (Home Assistant, HTTP, MQTT, etc) for each function (speech to text, etc.). That will keep the main Docker image slim, and make it clear where the boundaries are.

I understand the actor model can be confusing. I need to pull that out as a separate library, with some actual documentation. Each actor is really just a thread with an ā€œinboxā€ and a current state. The state determines which message handling function is called in the Python class, and is used so the actor can react differently to messages over time. Really, their just microservices in the same process :slight_smile:

3 Likes

As I think more about it, there are three places I could definitely use some help:

  1. Documentation (tutorials, videos, setups)
    • Rhasspyā€™s documentation really needs a set of focused tutorials, showing users how to go from nothing to a functioning integration with HA, NodeRED, or Hermes/MQTT.
    • More videos demonstrating working systems would be great, so potential users can decide if itā€™s worth their time.
    • A collection of usersā€™ current (working) setups (hardware, mics, software, settings)
  2. Web interface
    • The current Rhasspy web interface is a Vue app I put together while learning Vue. I donā€™t really like doing web development, so anyone is welcome to improve it or make a new one!
  3. Testing
    • Iā€™d like to hear suggestions for ways to automatically test Rhasspy. There are so many variations in languages, CPU architectures, installation methods (Docker, Hass.io, etc.), and settings that I have a hard time not missing something each release.
3 Likes

Hi @synesthesiam first, let me congratulate you on your impressive project.
Itā€™s incredible what youā€™ve already achieved and how fast development is progressing. :grinning: :+1:
I also think the last steps towards modularization and streamlining the core are right and good. iā€™ve been watching it for a year, but unfortunately i have to say that for a project of mine i first put on snips. that was a mistake as i saw in the last days.

I tried rhasspy on several test systems and everything works fine under docker. On my target project i have to reduce all overhead so i tried the pyton env install as described.
Unfortunately, I canā€™t get it to work. For safety I tested it on a raspberry pi 3 with the latest
buster version without modifications and special drivers and it comes to the same error.

git cloning works. download-dependencies also works. at ./create-venv.sh the system installs well for a long time but in the end there are some bugs that I couldnā€™t figure out and fix. see the screenshot from the SSH console.

Hope so we can get this to work i would like to spend a lot of time on rhasspy and also actively help with testing and extending it.

Hello,

Iā€™m an former user of snipsā€¦ I have been missing for years that snips made only some parts of it open-source for marketing purpose and took care to keep the rest privately held.

After looking for truly opensource alternatives local voice assistant, I want to give rhasspy a tryā€¦ and if I can, help to make it grow.
I think you could get motivated developpers coming from snips. Do you think you could drive us ? Would it make sense to fork snips-nlu, train it with the snips data each snips user could export from its snips account, and integrate it to rhasspy ?

2 Likes

welcome @farfade
Iā€™m also such a Snip ā€œvictim.ā€ Have you tried Rhasspy?
The installation with Docker on a Raspberry Pi 3 is very easy, fast done and runs impressively good. I think the Rhasspy concept is great. You can choose the best suitable components for all applications. There are several intent recognition systems to choose from. With the standard config (fsticuffs) I achieve amazingly good results. What distinguishes Rhasspy from Snips at the moment is that it does not support any skills directly. But that is depending on the desired application no big problem at all. Fortunately Rhasspy supports the Hermes (MQTT) protocol like snips. This is how you can ā€œaddā€ Skill functionality. Run your own Phyton script (also as a service) and listen to the Hermes protocol, at the desired intent you start an action and also return a response to TTS. If you select PicoTTS in the Rhasspy setting, you will have exactly the same voice like Snips.

2 Likes

Hi @thinker, welcome! Thank you for the encouragement :slight_smile:

I think Iā€™ve tracked down the source of the problem, and it looks like I forgot to update some of the dependencies on Github (so thank you). Please do a git pull, delete rhasspy-tools_armhf.tar.gz from the ā€œdownloadā€ directory in your rhasspy repo, and re-run create-venv.sh.

Iā€™m also working on some experiment Debian packages for rhasspy, which should really simplify the non-Docker installation. I have one up for amd64, but Iā€™m still working on one for the Raspberry Pi (armhf). Iā€™ll post when I have it uploaded.

1 Like