Guide for CCTV Snapshot on motion, send to Google Generative AI & get notification with description & snapshot

I have a similar issue with only one camera (Unifi) where the snapshot is quite static, so most of the time in the notification I see one of the previous snapshot image but the generated text is correct… I’m using iPhone for notification, so maybe Apple mixes something in the soup

1 Like

I came on here for the same issue…it’s my blink thumbnail photo it uses instead of a snap shot. I wonder can it not take a snap shot when my video camera is taking video…or is home assitant just not compatible with blink (and maybe same for your brand)

EditL for me i think that answer is here How to configure Blink XT 2 to take pictures or play a stream - #4 by Salty
if I can figure it out.

Same error here!

Can anyone can help with the cached image issue?

It’s hard to say as it works fine here, are you using Cloudfare ? Or DuckDNS ?

I am using a iPhone and it works fine on my end. Sounds like your notification is firing off before the image is saved maybe? You can try add a delay of 0.5 of a second before the notification part and see if that helps.

Hey @Aussie_Adam, thanks for getting back to me.
Yes I am using Cloudflare for security to access HA remotely, but not when I am at home. So 2 serves addresses, local and external using Cloudflare. However the issue still persists even when home.

I see the updated snapshot in the folder but for some reason its pushing what I assume is a cached image. Where it’s getting it from I don’t know, because its not from the folder where the images are stored.

I tried adding the timestamp code which someone mentioned early up in the thread, but its giving me an error.

Just wanted to say thanks, this is awesome!

1 Like

Ok I don’t use it but I think Cloudfare is the issue here, you can try setting temporarily to “developer mode” and seeing if that helps, I am guessing it will, which means you will need to create a new cache rule to bypass cache for the snapshot image calls.

More info on cache rules is here:

Create a cache rule in the dashboard

Hope this helps!

1 Like

Hello,

Do you know why always when a person is in front of camera send me same pictures from yesterday? Not update pictures?

Thanks

To always get correct images, timestamp them:

alias: Camera - Back Alley - Snapshot, AI & Notification
sequence:
  - variables:
      trig_ts: "{{ now().strftime('%Y%m%d-%H%M%S') }}"
  - action: camera.snapshot
    metadata: {}
    data:
      filename: /config/www/snapshots/alley_{{trig_ts}}_snapshot1.jpg
    target:
      entity_id:
        - camera.alley
    enabled: true
  - delay:
      hours: 0
      minutes: 0
      seconds: 1
      milliseconds: 0
    enabled: true
  - action: camera.snapshot
    metadata: {}
    data:
      filename: /config/www/snapshots/alley_{{trig_ts}}_snapshot2.jpg
    target:
      entity_id:
        - camera.alley
    enabled: true
  - delay:
      hours: 0
      minutes: 0
      seconds: 1
      milliseconds: 0
    enabled: true
  - action: camera.snapshot
    metadata: {}
    data:
      filename: /config/www/snapshots/alley_{{trig_ts}}_snapshot3.jpg
    target:
      entity_id:
        - camera.alley
    enabled: true
  - action: google_generative_ai_conversation.generate_content
    metadata: {}
    data:
      prompt: >-
        PROMPT_HERE
      image_filename:
        - ./www/snapshots/alley_{{trig_ts}}_snapshot1.jpg
        - ./www/snapshots/alley_{{trig_ts}}_snapshot2.jpg
        - ./www/snapshots/alley_{{trig_ts}}_snapshot3.jpg
    response_variable: generated_content

Manage old files

Of course, now you need to manage the old images. There is an add-on to add actions to delete files and folders with filters like X hours old, which I use in an automation that fires daily.

I also delete files that don’t have anything of interest. For the following example to work, you need to install the add-on.

Delete images on fail

  - if:
      - condition: template
        value_template: "{{ 'No Obvious Motion Detected.' in generated_content.text }}"
    then:
      - sequence:
          - parallel:
              - action: delete.file
                data:
                  file: /config/www/snapshots/alley_{{trig_ts}}_snapshot1.jpg
              - action: delete.file
                data:
                  file: /config/www/snapshots/alley_{{trig_ts}}_snapshot2.jpg
              - action: delete.file
                data:
                  file: /config/www/snapshots/alley_{{trig_ts}}_snapshot3.jpg
          - stop: Gemini AI saw nothing of interest

Delete old files daily

The default time is set to 24 hours, so any files in the folder older than 24 hours will be deleted in this example automation that runs at 3 AM daily:

description: "Delete old Gemini AI CCTV images that are older than 24 hours"
mode: single
triggers:
  - trigger: time
    at: "03:00:00"
conditions: []
actions:
  - action: delete.files_in_folder
    data:
      folder: /config/www/snapshots

See the add-on repo for the other filters available.

1 Like

Thanks a ton, implemented this using snapshot.

alias: AI - Main Door - Snapshot on motion
description: ""
triggers:
  - trigger: state
    entity_id:
      - binary_sensor.main_door_person
    from: "off"
    to: "on"
conditions: []
actions:
  - action: script.toggle
    metadata: {}
    data: {}
    target:
      entity_id: script.main_door_snapshot_ai_notification
mode: single
alias: Main Door - Snapshot, AI & Notification
sequence:
  - metadata: {}
    data:
      filename: /media/local/img/snapshot.jpg
    target:
      entity_id: camera.main_door_fluent
    action: camera.snapshot
  - metadata: {}
    data:
      prompt: >
        Motion has been detected, very briefly describe what you see in this
        image from my driveway camera number 1. What do you think caused the
        motion alarm? If a person is present, describe them in detail. Don't
        describe stationary objects or buildings. Your message needs to be short
        enough to fit in a phone notification.
      image_filename: /media/local/img/snapshot.jpg
    response_variable: generated_content
    action: google_generative_ai_conversation.generate_content
  - metadata: {}
    data:
      title: Main Door Motion Detected
      message: "\"{{ generated_content['text'] }}\""
      data:
        image: /media/local/local/img/snapshot.jpg
    action: notify.IOS_DEVICES
mode: single
description: ""

I am using a custom folder to store the snapshot. There is no need to clean up. The previous snapshot is deleted each time a motion is triggered.

The need to clean up is because each image has a timestamp in its filename in my script. In the yaml you posted, the filename is the same, so the file is overwritten each time and is why sometimes all notifications in the notification drawer show the same image.

Basically, adding a timestamp to the image name (alley_20241117-195941_snapshot1.jpg) forces each notification you send to use a specific image, instead of whatever the image for alley_snapshot1.jpg is at the time the image is retrieved.

Hi how do I get the text of what it detects on multiple lines when I receive the notification? and above all when I click on the notification how do I recover the description of the photo taken? Thank you

This is great, thank you for sharing!

I seem to be getting this error when attempting to save the script.

When I review my mobile ID, I’ve tried both the ID that appears in the address bar and also the device name ‘Alex’ to no avail.

Would anyone know what I might be doing wrong?

Thanks in advance.

@Aussie_Adam I believe I have it set correctly. But still no image.

This how I save the snapshots…

action: camera.snapshot
metadata: {}
data:
  filename: www/img/frigate/ai_parkering_1.jpg
target:
  entity_id:
    - camera.frigate_parkering_3

Have you tried without the "- ./" in the filename?

On another forum, someone has posted the exact same procedure, but for their prompt, they have the following, which I don’t understand how to insert it, as the yaml doesn’t like the bullet points. Show how do I use the following prompt:

Describe precisely and in detail what is visible in the image sequence , which consists of three photos taken by my surveillance camera at the front door. The camera was triggered when the doorbell was pressed.
• No static objects/buildings.
• Nothing moving in all images? Answer solely with “Nothing detected.”
• No obvious contexts without details (e.g., “Person at the front door, ringing the bell, making a movement”).
• Do not list analysis criteria or mention what was not done or detected.
• No time or date.
• Person: clothing (uniforms, logos, colors), gender, identifiable expression, emotion, gestures.
• Movement direction: to the left toward the garage, to the right from the driveway, through the door into the house.
• Interactions: knocking, ringing, leaving a package/delivery, tools/flyers in hand, conversations, official actions.
• Posture: upright, bent, searching, delivering, repairing.
• Other moving objects: vehicles (delivery vans, cars with company logos), animals, people.
• Answer with “Secret code” if a person shows the “OK” sign (👌🏼).

Always respond in the following format:
Title: Maximum of 60 characters, a short title for the push notification.
Message: Message for the push notification, maximum 250 characters.
Code: Respond only with “Nothing detected,” “Secret code,” “Delivery service,” “Person,” or “Other.”

I just removed the bullets and it is fine for AI :slight_smile:

I’ve been facing huge delays between calling the service and getting a response. It took 6min to call the notification with Gemini’s response. Is there any parameters I can change to make it answer faster?