I've been building this for my own place for about a year and finally cleaned it up enough to share. It's AGPL, free, self-hostable in full, and it exists because I wanted what the cloud-AI integrations do — but with nothing about my family leaving the house.
Hestia is a single stateful "brain": a local LLM (Ollama, resident Qwen3-14B on one consumer GPU) wrapped in an agent loop with ten scoped tools. Home Assistant is how it touches the physical house lights, sensors, the shopping list and every window into it (my phone, a terminal, the kitchen voice satellite) talks to the same brain over Tailscale. Nothing runs in the cloud and nothing is exposed to the internet.
The design idea, since this crowd will recognize the tension immediately: most "AI for the home" points the model at the things it's worst at remembering schedules, watching thresholds, firing a reminder at the right minute. Home Assistant is already excellent at all of that deterministically. So Hestia hands anything schedule- or threshold-shaped to HA automations, timers, and database rows, and the LLM only does what only an LLM can do: judgment and conversation. "The trash goes out Tuesday" is an automation. "Which garden bed needs water most?" is the model reading six soil-moisture sensors through HA and giving a judgment. The goal was never a smarter brain, it's a more reliable one.
What it does day to day at our place:
- Morning briefing at 7:10 — weather, what's due, what emerged overnight in the garden pest model — pushed to my phone and announced on the kitchen Voice PE
- "We're out of olive oil" by voice → HA's
todo.shopping_list, deduped and split - Logs household records by voice into SQLite — "vaccinated the dogs today," "got a new puppy, Biscuit, she's a corgi" → entities plus a dated event log
- Walks my wife through recipes hands-free in the kitchen, holding the conversation open between steps (
continue_conversation) so she never re-wakes it - Reads soil-moisture channels and freeze/rain watches for the garden beds
- Runs the media stack (Plex + *arr + subtitles) — "grab the new season of X" just works
- Files trail-cam photos and tracks wildlife sightings
How it relates to HA specifically: HA runs on a little Dell appliance; the brain runs on a separate GPU box. Hestia is a conversation agent from HA's point of view. Assist STT/TTS pipe into it, and the Voice PE in the kitchen is the main family interface. The brain's home tool calls HA's API for actuation and state. If you already run HA, Hestia doesn't replace anything, it sits on top and treats your HA install as its hands and senses.
The honest requirements and caveats, up front:
- It wants a real GPU for the resident model (mine is a single RTX 5080; the 14B runs ~77 tok/s). Smaller models work, I have eval data on exactly what you lose — but this is not a Pi-sized workload.
- There is no built-in auth, so it must stay on a private network (LAN/Tailscale). That's a documented trade-off (SECURITY.md), not an oversight. Don't expose it.
- There is deliberately no shell tool, the brain can act in the house but can't run arbitrary commands.
- Voice quality: Chatterbox-Turbo for TTS with Piper as CPU fallback; STT is Whisper via the Assist pipeline.
Repo: GitHub - thefullnacho/hestia: Local-first, self-hosted home & records assistant: one LLM brain, scoped tools, durable memory. AGPL-3.0. · GitHub. Happy to answer anything.