After getting frustrated with Alexa, I recently bought a Raspberry Pi 5 and an AI HAT+2 to build my own Voice Assistant, which would run locally. I planned to have a standard pipeline with Whisper + Home Assistant + Ollama + Piper. And the challenges began with using NPUs on the AI HAT+2. Once I had everything running, my relief didn’t last long. That was because things didn’t go as anticipated, with hallucinations from Ollama and its response latency.
With all that, I pivoted that original architecture and built a self-healing Whisper that works well with Home Assistant's intents. Feel free to go through the blog post here. I might have overshared a bit on the post. However, since there isn’t enough documentation on AI HAT+2, I had to.