Speech-to-phrase fallback to faster-whisper (no cloud)

Configuration
1

Replace my google voices with Home Assistant Nabu. I ran speech-to-phrase for a while, but since you need exact phrases sometimes it failed to understand. I then switched over to whisper, but smaller models have worse voice to text understanding than speech to phrase for some reason. Faster-whisper-medium has perfect voice recognition for me and executes things well but on my server it runs quite slow.

Is there anyway I can run speech-to-phrase and then fallback to faster-whisper-medium when speech-to-phrase fails to understand a command? I tried looking up but it seems all fallback documentation and discussions are only if you are using a cloud based option that falls back to local.

I have both running in docker containers right now and have nabu activate speech and jarvis activate whisper.

Anyway to fallback within the Wyoming protocol itself or anything like that?

2

Of course, after spending hours looking over the past few weeks. After I crack and decide to post this I found the solution. Seems there is no built in integration in HA.

I ended up implementing Add stt-fallback by danielrheinbay · Pull Request #17 · rhasspy/wyoming-addons · GitHub which works great. Hope this will be merged eventually.

This also seems to be an alternative solution for anyone else trying to find the answer.

1 Like