Generative AI camera snapshot notification

Hi.
Would be great if there was an option to choose another sensor than motion to the trigger. Examble i have person and car detection on my cameras and would like to have a snapshot only when a person is detected in my yard and not from every motion detection. :blush:

1 Like

How can i select a different sensor, my Reolink is facing the street and i dont need a description of every car that drives by but if i can activate on person detection instead of all Motion it will be a lot better, same for cars vs trees moving in the wind and pets etc. Seems only motion binary sensors are allowed not any other binary sensor. How can we change this?

The Reolink should expose entities for vehicle, person:

      - binary_sensor.[[YOUR CAMERA]]_vehicle
      - binary_sensor.[[YOUR CAMERA]]_person

It should respect masked area.

I have two cameras at my gate, one for plates/gate one to see inside the car and talk. Automation with AI and then sending notifications to two phones (uses a blueprint for the notification):

alias: Gate Vehicle Camera Alerts
description: ""
triggers:
  - entity_id:
      - binary_sensor.rc_intercom_vehicle
      - binary_sensor.rc_intercom_person
    id: intercom
    from: "off"
    to: "on"
    trigger: state
  - entity_id:
      - binary_sensor.rc_gate_vehicle
      - binary_sensor.rc_gate_person
    id: gate
    from: "off"
    to: "on"
    trigger: state
conditions: []
actions:
  - action: camera.snapshot
    metadata: {}
    data:
      filename: ./www/snapshots/rc_gate_fluent_snapshot.jpg
    target:
      entity_id: camera.rc_gate_fluent
  - action: google_generative_ai_conversation.generate_content
    metadata: {}
    data:
      image_filename: ./www/snapshots/rc_gate_fluent_snapshot.jpg
      prompt: >-
        Motion has been detected, very briefly describe what you see in this
        image.  If a person or vehicle is present, describe them in detail.
        Don't describe stationary objects or buildings. Keep the response short.
    response_variable: result
  - parallel:
      - metadata: {}
        data_template:
          field_notify_device: 6e7ef8ed17852d13dc932de01891ac75
          field_title: "{{reason}} at <font color='red'>{{camera_location|title}}</font>"
          field_message: "{{ result.text }}"
          field_subtitle: ""
          field_attachment_type: camera_entity
          field_attachment_camera_entity: camera.rc_{{camera_location}}_fluent
          field_timeout:
            hours: 0
            minutes: 2
            seconds: 0
          field_notification_link: app://com.mcu.reolink
          field_confirm_uri: app://com.mcu.reolink
        action: script.gate_vehicle_motion_detection
      - metadata: {}
        data_template:
          field_notify_device: fc4d03d44446c906dbe5c611795fdb98
          field_title: "{{reason}} at <font color='red'>{{camera_location|title}}</font>"
          field_message: "{{ result.text }}"
          field_subtitle: ""
          field_attachment_type: camera_entity
          field_attachment_camera_entity: camera.rc_{{camera_location}}_fluent
          field_timeout:
            hours: 0
            minutes: 2
            seconds: 0
          field_notification_link: app://com.mcu.reolink
          field_confirm_uri: app://com.mcu.reolink
        action: script.gate_vehicle_motion_detection
variables:
  camera_location: "{{trigger.id}}"
  reason: "{{trigger.entity_id.split('_')[3] | title}}"
mode: parallel

I use this for the notification:

So I love this, and really appreciate that you came up with it.
Anyone have an idea on how to log these so that you could have a dashboard of all the detections in the last 24h? Would you. have to write a custom integration and custom JSON file or something to keep track? Or does HA have a facility for something like this.

Hello, I have the same issue with iOS notifications. A long press allows me to see the notification, but if I just tap once, it opens the app, and no notification is readable. Couldn’t we have an option to make notifications persistent? Thanks, great work, by the way!

1 Like

My temporary workaround on my Pixel 8 Pro is using a phone app called “Recent Notification” by LibApps. It shows all recent phone notifications in full, and shows the image and full notification text. As I said, temporary workaround, but at least I can see all the text.

1 Like

Good morning, your work is very nice, I have two small problems. The first is that I can’t see the message after the three dots (like the previous posts). The second is that in the message that I receive the AI ​​gives the correct description of the snapshot photo but it always attaches an old photo.
is it possible to change the snapshot path?
thanks

1 Like

Hey guys,

I’ve used this for months now and it works really really well.
Unfortunately I have seen that the amount of request is too high and I will now have to pay for the requests from Google.

I’ve turned it down to 3 snapshots, but I still see this on my google billing:

The funny thing though is, if I look at the “Generative Language API” I only see 759 requests the last 30 days :question:

I don’t use Generative AI to anything else than this Automation.

Does anyone know why this behavior and what can I do about it?

same issue… dont understand why

I’ve been trying this out for the last few days and it works great. However, one thing I would like to change.

If there is no meaningfull change it still sends me a notification, but I would prefer it to just do nothing. But based on the code it seems it should already be doing that? But I’m still getting the notification

  - conditions:
              - condition: template
                value_template: >
                  {{ generated_content['text'] != 'Camera has detected motion
                  however no obvious motion observed comparing snapshots.' }} 
1 Like

same been looking, or wnt a blueprint to use frigates built-in genai

Anyone knows how to fix this?
Since recent HA 2025.3.0 update I get this message:

The ‘image_filename’ parameter in Google Generative AI actions is deprecated. Please edit scripts and automations to use ‘filenames’ intead.

Yes, I just resolved this today. Just move the file names from image filenames to attachment filenames.

2 Likes

filenames works now in 2025.3.1

Is there a way to modify this to pull images from 2 cameras (from 2 different angles) to compare and describe? only seeing the one snapshot in the notification is fine.

Were did you finde these settings I cant finde were to change it

2 Likes

Every time i run the action i get the following error:
Error generating content: 429 RESOURCE_EXHAUSTED. {‘error’: {‘code’: 429, ‘message’: ‘Resource has been exhausted (e.g. check quota).’, ‘status’: ‘RESOURCE_EXHAUSTED’}}

Did you ever find a solution for the 429 RESOURCE_EXHAUSTED. {‘error’: {‘code’: 429, ‘message’: ‘Resource has been exhausted (e.g. check quota).’, ‘status’: ‘RESOURCE_EXHAUSTED’}} error?? I just setup the blueprint and getting the same error. Thanks!

No unforntunately not yet. I have delete the google AI integration and now i’m looking for another integration. Sometime the google AI works but most of the time i received the error

Can I edit this to also send Text to Speech? Or is this needed to be done in your code?