OpenAI Realtime (gpt-realtime-2) voice assistant for the Home Assistant Voice PE

Share your Projects! Custom Integrations
, ,
1

Hi all,

I've turned my Home Assistant Voice PE into a natural, low-latency voice assistant
running on OpenAI's Realtime API (gpt-realtime-2). The device streams audio to a
small HA add-on that runs the speech-to-speech session and controls your home through
the official MCP Server integration — lights, scenes, climate — and it has web
search built in (weather, news, quick facts). It's a real back-and-forth conversation
rather than the classic pipeline.

It comes in two parts :

A full step-by-step install guide (INSTALL.md) is linked right from the firmware repo's
README, and each repo points to the other.

Setup is mostly one-click once the add-on and a small ESPHome stub are in place. Heads-up:
it's an experimental hobby project and runs on OpenAI's paid API, so you'll need an
OpenAI key with billing enabled.

Would love for people to try it and share feedback!

5 Likes
2

Hi!
How does it work so far? I’m curious why not many people use your work because I think it can make Voice Assistant much smoother .

Do you have a video showing how it works?

3

This look interesting to me! What about if, for what reason, I want to switch back to the default firmware, is that possible @Xander ?

4

I’m trying to edit my HA Voice PE with the DHCP yaml as provided in the Install Guide, but entering the yaml gives me the following error:

Could not download from https://github.com/kahrendt/microWakeWord/releases/download/v2.1_models/alexa.json: 404 Client Error: Not Found for url: https://github.com/kahrendt/microWakeWord/releases/download/v2.1_models/alexa.json.

5

Is this still in active development and improvements, or is it working so well you consider it finished? Are you still using it in practice, and how well does it work for you after a couple months?