Send audio to assist_pipeline over websocket

Configuration Voice Assistant
1

I am struggling to figure out how to send audio data to Assist using websockets. The documentation on Assist Pipelines | Home Assistant Developer Docs (home-assistant.io) doesn’t seem to provide much information either. I tried to just send the starting JSON followed by raw audio, but that results in an error in Home Assistant: " Received binary message for non-existing handler 5".

I also trawled through the Websocket and Esphome code on Github but I couldn’t figure out how they managed to do it. Does anyone know more about how to do this?

2

Did you check esphome/voice_assistant.cpp at ed024a0aa5f029b788af3398dbd382dcf5445036 · esphome/esphome · GitHub ?

It seems plain UDP, not websocket.

3

I was able to setup an app to send audio over the websockets to Home-Assistant. Using Porcupine to detect a wake word and send the audio to a voice assistant pipeline.

1 Like