DIY Smart Video Doorbell using Raspberry Pi with Home Assistant & Frigate Integration

1

Hello everyone,

I am planning to build a DIY smart video doorbell and looking for suggestions from the community regarding the hardware setup, software architecture, and integrations .

Objective

The goal is to build a fully self-hosted smart video doorbell using a Raspberry Pi , with deep integration into Home Assistant and Frigate for AI detection. The system should support two-way video intercom communication through the Home Assistant mobile app on a phone or tablet.

Main Requirements

1. Integration with Home Assistant

The doorbell system should integrate completely with Home Assistant and support automations such as:

  • Sending instant notifications when the doorbell button is pressed
  • Opening the live video stream in the Home Assistant mobile app
  • Triggering automations such as lights, recordings, or announcements

2. Integration with Frigate

I plan to use Frigate NVR for AI-based object detection and event recording.

Frigate should handle:

  • Person detection
  • Motion events
  • Snapshot and video recording
  • Event triggers for Home Assistant automations

3. Two-Way Video Intercom

The system should support two-way communication between the visitor and the user.

Requirements:

  • Live video stream
  • Two-way voice communication
  • Access through the Home Assistant mobile app
  • Should work on mobile phones and tablets
  • Preferably low latency

Door Unit Hardware (Planned)

For the door unit I plan to use:

  • Raspberry Pi (Pi Zero 2 W or Raspberry Pi 4)
  • High-quality Raspberry Pi camera module
  • Microphone
  • Speaker with small amplifier
  • Physical doorbell button
  • Status LED
  • Weatherproof enclosure for outdoor use
  • Wi-Fi or Ethernet connectivity

Smart & AI Features

Person Detection

Using Frigate , detect when a person approaches the door and trigger:

  • Home Assistant notification
  • Snapshot capture
  • Video recording

Automatic Light Control

When a person is detected at night , Home Assistant should automatically:

  • Turn porch lights ON
  • Improve camera visibility

Face Recognition

I would also like to add face recognition to identify known visitors such as:

  • Family members
  • Friends
  • Regular visitors

This could trigger automations like announcements.

Alexa Announcements

Integration with Alexa smart speakers to announce events such as:

  • “Someone is at the front door”
  • “Doorbell was pressed”
  • “Arya is at the front door” (if face recognition identifies a known person)

Other AI Features (Optional)

Possible additional features:

  • Package detection
  • Unknown person alerts
  • Visitor snapshot in notification
  • Motion zone detection
  • Event history

Additional Features

  • Live camera stream in Home Assistant
  • Doorbell press automation
  • Snapshot notifications
  • Video recording
  • Motion alerts
  • Secure local-only system without cloud dependency

Questions

  1. What would be the best way to implement two-way audio/video with Home Assistant?
  2. Which Raspberry Pi camera module would work best for this use case?
  3. What is the recommended approach for low-latency streaming to the Home Assistant app?
  4. Should AI processing run on the Raspberry Pi , or be handled by the Home Assistant / Frigate server ?
  5. Are there any existing open-source projects implementing a similar DIY doorbell?

Planned System Architecture (Concept)

Door Unit

  • Raspberry Pi
  • Camera
  • Microphone
  • Speaker
  • Doorbell button

Processing & Automation

  • Home Assistant server
  • Frigate NVR for AI detection
  • Automations for notifications, lights, and announcements

User Interface

  • Home Assistant mobile app
  • Tablets for monitoring
  • Alexa speakers for voice announcements

Goal

To build a reliable, privacy-focused, locally controlled smart doorbell with advanced AI features while maintaining strong integration with Home Assistant and Frigate .

Any suggestions regarding hardware choices, software stack, or implementation approach would be greatly appreciated.

Thank you!

1 Like
2

Congratulations on your 6th year anniversary here on the Home Assistant Forum next month. How has your Home Assistant Journey been? What have you learned?

I know you probably read this when you started out here, but perhaps read this again. Having an AI talk (a lot) is a perfect way to get no one to answer you. I would suggest starting out with a short version of your story, what you have running, what you want to accomplish, your budget, and your physical location and then you might get some help. As it is that post is TA;DR.

1 Like
3

I have a similar goal. I have been trying for quite some time, but got stuck in the WEbRTC blackhole. Luckily with AI assistants (Deepseek and Claude), I have made more progress this month.

My setup:

Raspberry Pi 4, with camera and sound card
I am still not sure if to go with a webcam or with a GPIO cam, most likely the latter. I have configured for both. It will ultimately depend on the situation during installation.

MediaMTX server
This publishes the camera feed to RTSP, which is picked up by Frigate. That way, I can have more flexibility, including recording, Motion detection, etc. Frigate is on another Raspberry at the moment, but will be transferred to a Docker on HP eliteDesk

Janus gateway
Installed on a Docker so far, will be transferred to Proxmox after testing. I am using a videoroom plugin.

Raspberry Pi has a python script that captures the RTSP, and feeds into the Janus gateway. That way, I have used WebRTC to connect multiple clients in a browser.

One ringer is pressed, the python scripts connects to one client (intercom) to play a chime. This will be always on, on a tablet. Still to figure out details.

It also connects to HA through a webhook, to send a Critical message to phones. Once pressed, these will then launch a browser with the feed.

there are some security features. One time Tokens are used in the notification messages, so that even if some one has the link, they also have to have the token for the door to be opened.

I am around 85% there. Still to do:

  1. failure detection and notification if Janus server is down
  2. Doorbell control from browser.
  3. Door opening notifications

See it at: