Raspberry Pi as a HA Voice Assist CHAPTER 4 satellite

You sir are a life saver. I did go through the Seeed Studio wiki when it first arrived but again that was for windows just to test it out. But the goal was eventually for it to be a mic for the smart home. I did find that Hintaks repo is still being updated and the latest is setting up the kernel to a 6.1.x version in his install script.

I was installing 64bit right of the bat for the 3B but after the initial seed 64bit install didnt work and I learned that hintaks was 32bit I grabbed the 32bit versions of the OS. dont need 64bit anyways if rhasspy is the only ting its being used for. IF Hintaks updated version doesnt seem to work with your tips I will just go older versions that I can find to put on it,

Its a shame Seeed let this product go, it seemed like it would have made a good dent in the assistant market these days. lol the bit the bullet too soon.

Again thank you for taking to time to share your tips and tricks with me… and the rest of the community involvement from what Ive seen.

One last thing I forgot to mention as I was reading over your reply again. probably wont relay to your current setup but I guess there was an issue with the channels on the 4mic hat getting jumbled around on reboots and uses, and someone made a tweak to 2 of the driver files to correct this. Here is the github thread that has the link for the adjusted files. just in case you were unaware or decide to use that mic in another setup someday and run into that issue…

[Bug]: reference channels shifts (disordered) over multiple recordings · Issue #309 · respeaker/seeed-voicecard (github.com)

I’m getting SSL certificate errors when I try to run. Is there a way to disable checking? (Sorry about formatting I don’t have much experience posting code)

Blockquote
ERROR:main:Unknown exception in the main thread
Traceback (most recent call last):
File “/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/connector.py”, line 980, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs) # type: ignore[return-value] # noqa
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.11/asyncio/base_events.py”, line 1112, in create_connection
transport, protocol = await self._create_connection_transport(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.11/asyncio/base_events.py”, line 1145, in _create_connection_transport
await waiter
File “/usr/local/lib/python3.11/asyncio/sslproto.py”, line 575, in _on_handshake_complete
raise handshake_exc
File “/usr/local/lib/python3.11/asyncio/sslproto.py”, line 557, in _do_handshake
self._sslobj.do_handshake()
File “/usr/local/lib/python3.11/ssl.py”, line 979, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for ‘192.168.1.168’. (_ssl.c:1002)

BlockquoteThe above exception was the direct cause of the following exception:

Blockquote Traceback (most recent call last):
File “/home/admin/homeassistant-satellite/homeassistant_satellite/main.py”, line 234, in main
async with HAConnection(
File “/home/admin/homeassistant-satellite/homeassistant_satellite/ha_connection.py”, line 51, in aenter
self.__websocket = await self._websocket_context.aenter()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/client.py”, line 1141, in aenter
self._resp = await self._coro
^^^^^^^^^^^^^^^^
File “/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/client.py”, line 779, in _ws_connect
resp = await self.request(
^^^^^^^^^^^^^^^^^^^
File “/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/client.py”, line 536, in _request
conn = await self._connector.connect(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/connector.py”, line 540, in connect
proto = await self._create_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/connector.py”, line 901, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/connector.py”, line 1206, in _create_direct_connection
raise last_exc
File “/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/connector.py”, line 1175, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/connector.py”, line 982, in _wrap_create_connection
raise ClientConnectorCertificateError(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorCertificateError: Cannot connect to host 192.168.1.168:8123 ssl:True [SSLCertVerificationError: (1, “[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for ‘192.168.1.168’. (_ssl.c:1002)”)]
Recording raw data ‘stdin’ : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
ERROR:asyncio:Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x758a4710>

I’ve had a quick look and don’t see anywhere in Voice Assist or the RasPi Homeassistant-satellite you could have turned SSL on.

Are you using https: to access your Home Assistant server at https://192.168.1.168:8123 ?
Are you using Docker on your homeassistant-satellite ?

I am trying to connect to my home assistant server using https.

No docker on my pi.

You’re connecting to the internal IP, which usually does not have SSL. If you have --protocol https in your run command, remove it.

I agree with cnose. I access my own HA as http://192.168.1.98:8123/ … so SSL is not needed for basic access to HA over a LAN (and I try to keep mine as simple as possible so i can understand it).

I guess that your satellite RasPi is not using SSL, but your HA server is expecting an SSL key to be sent to it; however you didn’t show the commands you are running for Homeassistant-satellite and wyoming-openwakeword.

You can try:

  • If you have included the --protocol option, in the commands you are running for Homeassistant-satellite and wyoming-openwakeword; please remove it.
  • turn off SSL in Home Assistant (though this might affect other things); or
  • enable SSL on your RasPi.

I don’t get as far without --protocol https.

This command script/run --host 192.168.1.XX --token HNuI1UfEKX…wUSXzn8xkVKQwgsCDQ results in what you see below. I am running barebones to establish a connection can be made.

ERROR:__main__:Unknown exception in the main thread
Traceback (most recent call last):
  File "/home/admin/homeassistant-satellite/homeassistant_satellite/__main__.py", line 235, in main
    async with HAConnection(
  File "/home/admin/homeassistant-satellite/homeassistant_satellite/ha_connection.py", line 51, in __aenter__
    self.__websocket = await self._websocket_context.__aenter__()
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/client.py", line 1141, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/client.py", line 779, in _ws_connect
    resp = await self.request(
           ^^^^^^^^^^^^^^^^^^^
  File "/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/client.py", line 560, in _request
    await resp.start(conn)
  File "/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 899, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/admin/homeassistant-satellite/.venv/lib/python3.11/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
Recording raw data 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
ERROR:asyncio:Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x75899710>

My server cannot be accessed locally without https. I believe it was required to set up duckdns remote connections. I’ll look into setting up SSL on my pi.

Jon, how did you get on with setting up SSL on your RasPi ? Did that solve the problem ?

Hi @narselon
I believe I was having that same issue and this post helped me to enable http local access while preserving https for outside connections.

I changed how my Home Assistant SSL works and they now connect after encountering several other errors that required upgrading my pi to Bullseye.

I can use the wake word and initiate my conversation with Assist. There’s a bit of a delay to the overall process. Would there be a way to possibly circumvent the wake word requirement? I plan on using a motion sensor nearby. Could I use it or a button that could switch it to active?

Also is there a way to continue the conversation from the initial response? I’m using PALM and I would like to respond to it’s response.

Certainly. Back in Year of the Voice chapter 2 the M5STACK ATOM Echo was promoted as a satellite with push button to activate … though (having got so used to my RasPi’s with Rhasspy and wakeword detection) I have not paid attention to posts about how to activate a voice session on a RasPi with homeassistant-satellite. I assume it is possible.

This has been asked several times here and in the Rhasspy forum, without any easy answer.

Rhasspy is intended as a voice toolkit, with which people can develop their own applications … if you have the expertise to go down that rabbit hole :wink:

So, after some back and forth, I managed to get this up and running. @Don you are a star!!! Thanks for such a detailed write-up!

Rather than going offtopic, could I kindly ask you to point me an article or discussion on how to improve accuracy? Voice detection works and is done locally on my raspberry, but results are rather mediocre. Unless one uses carefully chosen words, the system is not responding as I would expect… I’m sure I need to work on evolving the model, or do something to make it useful for me and my family.

Thanks!!

Alas, that is where I am also at currently :frowning:
I have swapped Whisper to the largest available model (“small”) to improve accuracy … but Jarvis on the test RasPi in my study is like a teenager finding so many inventive ways to misunderstand “turn on the light”, and with 15 second delay from my finishing speaking until it processes the command.

Admittedly I have put it aside for a couple of weeks, so these issues may have been resolved already.

I am now eagerly awaiting the next chapter in a couple of days.

Can I ask what hardware you tried this on?
I have heard many mention 10-15 second delay after speaking. I have not witnessed this so I sm curious about it.
From the time I finish saying “what time is it?” To the time I get a response is about 3 seconds.

Oh! I see. The truth is that as it currently works, I wouldn’t call my interactions as something natural.

I improved the behavior by leveraging OpenAI and with OpenAI interactions are way more natural but HA refuses to interact with my home when OpenAI is being used.

Don, if you don’t mind, drop me a line here if you ever manage to get a better response from your system. I’m clearly in the same boat :slight_smile:

LOL this is funny. In my case my voice assistant refuses to tell me what time it is, when I ask that :slight_smile:

I wonder what may I have messed up in my assistant pipeline… I can turn lights on and off but time won’t work :slight_smile:

I am running HAOS in the only VM under proxmox on a used Optiplex 7050 (i5-7500T). VM uses 4 CPU threads and 16BG RAM. Wyoming, Whisper, Piper, OpenWakeWord are all installed as add-ons. Also ESPHome, Mosquitto, Node-RED and Rhasspy Assistant 2.5.11 are installed add-ons. Whisper uses the largest model available to us (ironically called “small” :wink: ) yet the whole VM rarely goes over 15% CPU usage even when processing voice.

At the satellite end, it’s a RasPi 3B with Raspberry Pi OS Lite 32-bit on a fresh micro-SD card. I am using a decent USB mic, and headphones connected to the 3.5mm audio jack. Both homeassistant-satellite and wyoming-openwakeword are running as services.

The hardware seems well up to the job. My guess is that there’s some setting not exposed in wyoming-openwakeword for the length of silence to signal the end of command.

I’m guessing that these Voice Assist satellites can’t also act as media players… Are you able to confirm?

I’m running homeassistant-satelita on a raspberry pi 3b+ which also acts as a shairport-sync player, that is, I’m able to stream music to it and it detects the wake word, plays confirmation sounds and then responds, so yes, you can.