Hi all! I’ve been building JARVIS, a HACS integration that gives Home Assistant something closer to Stark’s JARVIS: an assistant you can talk to, that also watches the house on its own and decides what’s worth telling you.
The best summary came from a user in the HA Facebook group:
“It takes a while to just monitor your day to day and eventually starts making suggestions based off your actions.”
That’s the core idea: suggest, don’t act.
Install: it’s in the HACS default store. Search JARVIS AI Assistant, install, restart, then add it under Settings → Devices & Services.
GitHub: GitHub - sam3gp8/jarvis-aio: MCU-inspired autonomous AI butler for Home Assistant — voice, vision, per-person memory, goals, and a full HUD panel. · GitHub
What makes it different from a regular LLM conversation agent
Most LLM integrations answer when you speak to them. JARVIS does that too (it plugs into the normal Assist voice pipeline). It also works in the background:
- Learns your routine and suggests automations. It only starts doing something on its own after you’ve accepted the same suggestion three times, and you can revoke that at any time.
- Decides what’s worth your attention. It classifies every home event by urgency, based on your home’s own history. The kitchen light at 7 a.m. is routine. The basement window opening at 3 a.m., when it never has before, gets flagged, and escalated if you’re away.
- Keeps working offline. If the cloud provider drops, a local reasoning brain takes over event decisions. Or run everything on Ollama and nothing leaves your network.
- Cameras (optional). Doorbell-press analysis, package and mail announcements, and quiet visitor learning, over Frigate (and Nest through it).
- Answers from your paperwork. Drop appliance manuals and receipts into a folder and ask “what filter does the furnace take?”
- Knows your calendar. It flags overlapping and back-to-back events, and can look things up on the web (DuckDuckGo, or your own SearXNG).
- Iron Man HUD dashboard, with a live 3D house, per-room occupancy and an event feed.
Getting started is small
You need HA 2024.10+ and one LLM provider: Groq (free tier), OpenAI, Anthropic, Gemini, Ollama, or any OpenAI-compatible endpoint. You can be talking to it in about five minutes. Cameras, voice satellites and GPUs are all optional add-ons. It speaks in the language you address it in, and the UI is translated into 20 languages.
Everything it learns stays in /config/jarvis/ on your own instance: no JARVIS cloud, no telemetry.
What I’d love feedback on
- What should it notice in your home that it doesn’t yet?
- How much personality do you want? There’s a banter setting (plain / dry / full), and it always goes serious for smoke alarms and the like.
- Anything confusing in setup. I’d like the first five minutes to be painless.
Bugs and ideas are welcome here or on GitHub Issues. I’ll post notable releases in this thread.