UPDATE 10/1/2026 - As of 9/21, fully local speech options were added. More recently (as of today) added the beginning of a web UI that provides control over settings, prompts, and the ability to review session transcripts, video clips, and snapshots. Still very early stages, but is working well for me so far. YMMV
=========================ORIGINAL POST BELOW=========================
Just wanted to share a project I started working on recently. My friend Matt created a similar project (Doorman: A Privacy-Focused Agentic Doorbell) and inspired me to try to make it work using my Amcrest AD410 and Frigate/Home Assistant. I’m sure this can be adapted to just about any camera that supports 2-way audio. Link to my project is at the bottom of the post.
Doorman Deluxe is an AI-speaking agentic doorbell for a privacy-focused homelab. When someone approaches or rings, Doorman Deluxe talks through the doorbell camera speaker using a Gemini Live two-way voice session, greets them per household policy, can snapshot the visitor, and notifies the homeowner on a phone.
Everything runs locally except the voice engine (currently): Frigate (person/face/cat/dog detection + snapshots), go2rtc WebRTC talkback to the doorbell, Home Assistant (doorbell-press trigger, notifications, camera snapshots), and a Gemini Live cloud voice session.
I plan on experimenting more with local TTS/STT for a fully local/private setup, but this current stack is minimally invasive and is working great for now!
And yet somehow privacy focused by sending your commands thru the cloud?
Very interesting.
How do you do you keep your privacy while passing all your commands and responses thru an AI and…
I don’t actually have a problem with what you are doing, just the explanation and titles seem a bit misleading to me.
The intent for the project is to eventually have a fully local and fully private two-way voice session option. Right now, the only non-private thing is your visitor’s voices going to Google for transcription, and the response coming back. Everything else is local. This was just the fastest and simplest way to fully stand this project up on day 1.
I’m currently in the process of evaluating local voice options and hope to have this thing fully local/fully private in the near future.
Personally, I don’t really care if the solicitors who I am trying to keep out/away have any of their privacy violated by this And in my case, I set mine up so that it really only tries to talk to unexpected guests.
I overall like the direction. Just be upfront about it up at the ‘privacy focus’ part.
There are those among us (including me) who do not consider anything about free hyperscaler inference as ‘private’. Heard on future direction. But now is now.
Initial fully-local options added. While the local AI voice responses are not quite as on-point as the ones I was getting from Gemini, it is getting pretty close, and is usable. Still very much in experimental stages, but totally works right now. Lots more to do to turn this into something others can easily make use of, but will continue to share progress as time permits.
And in my case, I set mine up so that it really only tries to talk to unexpected guests.