Camera Notification V0.8 with Actionable Buttons Android IOS - Frigate LLM Vision AI

Blueprints Exchange
, , , ,
1

Frigate LLM Notification v0.8

Home Assistant
Frigate
LLM Vision

A Home Assistant blueprint that combines Frigate, LLM Vision AI, and actionable mobile notifications for smart, context-aware security alerts. Fully iOS and Android aware.

Features

  • iOS & Android aware – HLS (.m3u8) clips on iOS, MP4 on Android, chosen automatically per device.
  • Two-stage AI analysis – fast snapshot summary on the new event, deeper video analysis on the end event.
  • Interactive notifications – 3 configurable buttons per event (View Live, Silence, Call, etc.), smart silence with a custom timer, and click-through to snapshot/clip/live view.
  • Smart filtering – required objects, zones, sub-labels, severity, zone-vacancy (person-away) checks, plus a custom condition slot.
  • Flexible AI provider – works with any LLM Vision provider/model (OpenAI, Anthropic, Google, Groq, local, etc.); memory & timeline integration optional.
  • Optional local caching – use the Downloader integration to store snapshots/clips locally before analysis instead of streaming the URL directly.
  • Parallel-safe – multiple cameras/events can run through the same automation instance without blocking each other.

Prerequisites

Notification Buttons

Action What it does
VIEW_LIVE Opens the live camera feed
VIEW_SNAPSHOT Opens the event snapshot
VIEW_CLIP Opens the full clip (iOS/Android aware)
SILENCE Pauses this automation; reply with a number to set custom minutes (default 5)
CALL Dials the configured phone number

Useful Template Variables

{{ camera_name }}     # "Front Door"
{{ objects }}         # Objects detected, e.g. ["person"]
{{ zone_names }}      # Zones required, e.g. ["front_porch"]
{{ video }}           # MP4 clip URL (Android)
{{ video_ios }}       # HLS clip URL (iOS)
{{ id }}              # Frigate event ID

Choosing a Model

The blueprint’s model dropdowns list the current models supported by the LLM Vision providers used when this repo was last updated. Pick a smaller/faster model (a “mini”, “flash-lite”, or “haiku” variant) for the snapshot analysis if you want near-instant alerts, and a larger model for the clip analysis if you want richer detail. Custom model IDs are also accepted if your provider adds something new later.

Version: v0.8 · Author: whag · Compatibility: Home Assistant 2024.1+, Frigate 0.13+

Open your Home Assistant instance and show the blueprint import dialog with a specific blueprint pre-filled.

10 Likes
2

Hello Whag,

Thanks for contributing to the community with a new Blueprint.
I have a suggestion for you. Many people who are not familiar with directory structures will have problems installing this without the Home Assistant MY tools.
Adding a MY link for this Blueprint to your top post would help them a lot.
Here is the link to make that.
Create a link – My Home Assistant
Note: if the original is in the forums here, only 1 code block can be in the top post ot the MY link tool will not work.

3

Delay time reduced before LLM video analysis

4
  • Added sub directory option for downloader.
  • Inputs added for delays before image and clip processing.
  • Variable description changes and spelling corrections.
  • Added Links for frigate and LLM Vision.
5

I can’t seem to get this to work and the traces seem to indicate that it fails on the download.
Does this require using the frigate add-on within haos, or can it use a separate frigate instance? The frigate server config input shows a default port of 8123, which is the ha default port. My frigate server is on a separate server and uses the default frigate port of 8971. Would it not also need a frigate username and password to login to the frigate server? I am not familiar with how the api works. Or, am I supposed to put in the ha server IP here?
Thanks for your work.

EDIT: I got this to work by putting my HomeAssistant server address into the frigate server configuration box. I used https with my internal FQDN and port 8123.

6

Hi Bobby in my instance i am using the HA frigate addon so its all on the same machine so the default value i have entered in the BP was based off my setup, although you have it sorted now anyway. thanks

7

Hi, I am not using the frigate addon in ha, i am using frigate in a container on proxmox but the blueprint wants a frigate camera. How do I go about this?

8

Hi, Are you using the frigate integration this is how the cameras are exposed to HA.

9
10

I already have frigate in a docker, do I still need to use the home assistant integration?

11

Yes you need to install the frigate integration which i linked previously and ensure you enable mqtt in your frigate config.

12

Thanks. That seemed to work, but now i’m getting a couple of warnings and no notification

image
image1996×724 58.2 KB

13

Can you try the url in a browser and if possible the machine running HA to see if the url is accessible.

Regarding the zones have you added a zone in the input field.

14

Yes I have added a zone in the input field. also that url gives me a 404 not found. I am attaching my frigate config.

image
image2242×556 34 KB

mqtt:
  host: 192.168.1.91
  port: 1883
  user: mqtt-user
  password: Looking123
  client_id: frigate
  topic_prefix: frigate

ffmpeg:
  hwaccel_args: preset-vaapi

detectors:
  ov:
    type: openvino
    device: AUTO

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  path: /openvino-model/ssdlite_mobilenet_v2.xml
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

record:
  enabled: true
  retain:
    days: 7
    mode: motion
  alerts:
    retain:
      days: 30
  detections:
    retain:
      days: 30
detect:
  enabled: true

snapshots:
  enabled: true
  retain:
    default: 30

cameras:
  kitchen:
    snapshots:
      enabled: true
      retain:
        default: 10
    detect:
      width: 1280
      height: 720
      fps: 5
    ffmpeg:
      inputs:
        - path: rtsp://rafsanjani:[email protected]:554/stream1
          roles:
            - record
        - path: rtsp://rafsanjani:[email protected]:554/stream2
          roles:
            - detect
    zones:
      zone_1:
        coordinates: 0.014,0.019,0.012,0.98,0.992,0.975,0.984,0.02
        loitering_time: 0
    review:
      alerts:
        required_zones: zone_1
version: 0.15-1
15

Hi there I have not setup frigate on a separate machine before so I do not have a direct means of testing your issues. I can only assume you need the frigate proxy add on or port 5000 needs opened on frigate server side.

Regarding the zone_names error… Did you select a zone from the predefined list or did you manually enter it.

16

I selected the zone from the predefined list. That warning seems to have gone now but it seems the media is still not getting downloaded.

17

Okay well that’s one thing.

I would advise opening port 5000 on frigate and maybe look at installing the proxy add-on or do some research on the external access

18

Is it giving the same error on the trace.

19

Updates: Version 0.2

  • Added sections.

  • Input entry for custom zones.

  • Altered template for custom zone entry as well as the default device selector.

  • Extra models added to models default list.

  • Option to disable LLM Vision Image Analysis of initial notification, along with a custom message for the notification. should this be set to false.

  • Prompt and related options split for image and clip analysis.

  • Altered downloader directory and sub directory variables to add “/” if missing.

  • Variable spelling correction in logbook.log action.

  • Some variable names changed along with a few unused removed.

  • Minor description changes.

To do: add input option for all zones required rather then just any zone matched

20

Hi, i am trying to setup the automation and it triggeres. But never got the notification to fire. it starts a loop and i think when it checks if the clip exist, it does not. is ther something i am missing? I’m not sure about my downloader configuration… is there a way to access the files directly on my frigate box?