Camera, Frigate: Intelligent AI-powered notifications

Blueprints Exchange
, , , ,
186

You don’t need MQTT for this. Just set up the integration in Home Assistant so your Frigate cameras are visible in Home Assistant.

187

Thank you, finally got it working and I’m getting notifications, but the description (AI portion) never follows. I’ve got Google Gemini integration and using gpt-4o-mini as the provider and model.

Any suggestions?

188

gpt-4o-mini is not served by Google. That would be an OpenAI model. If you’ve set up Google as provider you’ll need to use one of their models. See the documentation here: Cloud Providers | LLM Vision | Getting Started

189

Thank you for this information. I have tried a few models for Gemini but still struggling. I do restart the Frigate add-on everytime I try a new model. I don’t know if I’m getting something wrong if someone will be so kind as to give me the provider and model they are using (free preferably as pricing in South Africa is rediculously expensive)

The latest one I tried was gemini-2.5-pro-exp-03-25

Thank you everyone

190

This is what I have in my Frigate config …

genai:
  enabled: true
  provider: gemini
  api_key: !secret squirrel
  model: gemini-2.0-flash

then in cameras:
front: ← your camera

    genai:
      enabled: true
      prompt: Analyze the {label} in these images. Focus on the actions and potential
        intent of the {label}.

I hope this helps you along the way :slight_smile:

1 Like
191

You really want to publish your personal api_key ? :roll_eyes:

192

Some of us have nothing to hide :wink:

(thanks)

194

This is for the Frigate configuration. However, this topic is about the LLM Vision blueprint.

195

True, but the ‘model: gemini-2.0-flash’ is applicable there as well (having a free tier)

196

I think I’ve confused 2 threads :slight_smile:

This is one of my working LLM Vision Automations, take a look and see if there is anything different.

alias: Front Door LLM
description: ""
use_blueprint:
  path: homeassistant/event_summary.yaml
  input:
    mode: Frigate
    provider: 01JCXVTJZMY3P61PJKBW7YQQ6F
    remember: false
    notify_device:
      - 374faf582efb05ee36d6feef9c9f6288
    camera_entities:
      - camera.front
    motion_sensors:
      - binary_sensor.front_door_person_occupancy
    preview_mode: Snapshot
    cooldown: 5
    max_tokens: 75
    temperature: 0.5
    message: >-
      Summarize what's happening in the camera feed (two sentences max). Don't
      describe the scene! If there is a person, describe what they're doing and
      what they look like. If they look like a courier mention that! If nothing
      is happening, say so.
    target_width: 1024
    model: gemini-2.0-flash
    input_mode: Frigate

I hope this helps :slight_smile:

1 Like
197

Is it possible to not notify if not activity is observed

198

I don’t have the frigate plugin in home assistant. I am only running frigate in an LXC container so the part where I am supposed to pick a camera entity comes up as blank.

Is there a way to still use this blueprint without directly installing the home assistant plugin?

199

Increase the tokens.

200

Two questions or just some advice.

Background

  • I have the blueprint up and running with Frigate, and this thing is awesome, so thank you.
  • I have 6 cameras.
  • Each camera has a mask/zone for alerting. <camera_name>_alert
  • Each camera has Person and Car occupancy and a few have additional, especially “Bear”. I live in North East US and the most important alert is there is a bear.
  • I have setup a separate automation for each camera, for maximum flexibility. (but maybe i don’t need to).

Question #1 - If I only have 1 camera in an automation, can I have multiple Motion sensor triggers? Meaning alert “Person Occupancy” and “Car Occupancy” Or do I have to (or should I) use the “All Occupancy” for each camera?

Question #2 - The picture in the alert is always late in the cycle of the event. My front door camera almost always send a picture with no one in it but describes the Amazon delivery guy. Sometimes I can see the person’s foot. Amazon delivery people are quick but they are still in frame for 10-15 seconds so I am wondering why I get a photo without the person but a valid description. I am guessing it has to do with the settings for Duration and Max Frames. I am using 8 for duration and 3 for max frames. Is there any kind of advice here?

Thanks - YodaCoda

1 Like
201

Hi I keep getting this error
Stopped because an error was encountered at June 12, 2025 at 7:27:38 PM (runtime: 7.31 seconds)

Error rendering data template: UndefinedError: ‘dict object’ has no attribute ‘key_frame’
Im on the newest versions of the integration and blueprint, also I get an error when enabling timeline.
Config entry not found. Please create the ‘Timeline’ config entry first.
It looks like it doesnt want to AI summarize anything… not sure what I did wrong

202

I get the #2 issue you’ve been experiencing as well.

204

Having the same issue. Did you figured it out ?

205

It seems to be better with flip flopping and doing more frames than seconds. I was doing 8 seconds 3 frames. Now I am doing 10 seconds 30 frames. It seems to be picking something a little more than half way through the alert but I am at least seeing a person.

206

The automation that I have created with this script, which was working fine previously now gets stuck infinitely. There is no error message or anything, just keeps running for days.

Any way to troubleshoot it? I can provide a trace if that helps

207

@valentinfrlch is it possible to not notify “No activity observed” ?
Thanks!

2 Likes