Kaldi + Fsticuffs (rhasspy-nlu) in HA like with Rhasspy v2?

Hello there.

First of all, I’m German an therefore have the problems I have with voice recognition.
I’m pretty sure it works better with the English language.

Anyway… I’m using Rhasspy v2 for many years now and it works like a charm.
But I want to switch to the Voice Assistant-Features built-in HA because they give me more direct integration, the Names of devices can be used instantly without manual setup, and so on.
And as I want to have satellites in different rooms now, I would like to use wyoming-sattelite, because it’s so easy to setup.

But the problem is, that the VA doesn’t recognize even the simplest words/names for the devices, or anything at all… like giving me the current time or the weather.
I can’t even get it to turn on a simple light.
I tried the default, so

  • wyoming-whisper
  • wyoming-piper
  • wyoming-openwakeword

wyoming-openwakeword works great, I’m happy with that.
wyoming-piper does sound horrible, I exchanged that with wyoming-opentts. But both of them work, no problem here.

But wyoming-whisper is the problem. As I said, it doesn’t recognize anything for me.
So I tried wyoming-vosk, because it should be more like Kaldi (what I’ve been using for years now). It was better, but really not good enough.

So my questions are:
As I have a totally functional Rhasspy v2 setup, is it possible to integrate that in the “HA-VA workflow”?
I know that v2 uses MQTT and v3 the Wyoming protocol.
But are there any chances to integrate that with each other?

The wokflow should be like this:

  • in HA: wyoming-satellites ==> wyoming-openwakeword ==> record or stream speech
  • then it should send/stream the speech to Rhasspy v2
  • in Rhasspy v2: Kaldi + Fsticuffs ==> Intents ==> back to HA
  • In HA: Intents are processed and optonal there is feedback via TTS (wyoming-piper or wyoming-opentts)

So that the “STT and intent recognition”-part is done by rhasspy v2, because it simply just works for me.

Any ideas and suggestions are welcome.
Thanks in advance.

Here is a screenshot of my current Rhasspy v2-setup.
If anyone needs more information, I’m happy to provide it.

1 Like