Guide for CCTV Snapshot on motion, send to Google Generative AI & get notification with description & snapshot

Nice work.
But this here looks like a nice neet Blueprint works great.

This is absolutely crazy, thank you for sharing this script/automation!!

1 Like

I think you need to use actions now instead of service.

else:
      - action: notify.mobile_app_952505fcee7aafb7d5a574051c2a852e

That’s how mine is. I was getting that error before. I haven’t got mine to work yet though overall…but i think it’s because the photos don’t seem to show anything different. I think maybe the blink app recording video automatically could be stopping me taking photos in time.

Hi Everyone, I have just set this up for my 4 cameras (4 individual scripts and a custom automation). However, I am having a slight issue with the notification. It is displaying a cached image which doesn’t exist anymore and not the recent snapshot. The AI does describe the latest snapshot which I can see in the snapshots folder.

Any ideas on how to fix this?

I have a similar issue with only one camera (Unifi) where the snapshot is quite static, so most of the time in the notification I see one of the previous snapshot image but the generated text is correct… I’m using iPhone for notification, so maybe Apple mixes something in the soup

1 Like

I came on here for the same issue…it’s my blink thumbnail photo it uses instead of a snap shot. I wonder can it not take a snap shot when my video camera is taking video…or is home assitant just not compatible with blink (and maybe same for your brand)

EditL for me i think that answer is here How to configure Blink XT 2 to take pictures or play a stream - #4 by Salty
if I can figure it out.

Same error here!

Can anyone can help with the cached image issue?

It’s hard to say as it works fine here, are you using Cloudfare ? Or DuckDNS ?

I am using a iPhone and it works fine on my end. Sounds like your notification is firing off before the image is saved maybe? You can try add a delay of 0.5 of a second before the notification part and see if that helps.

Hey @Aussie_Adam, thanks for getting back to me.
Yes I am using Cloudflare for security to access HA remotely, but not when I am at home. So 2 serves addresses, local and external using Cloudflare. However the issue still persists even when home.

I see the updated snapshot in the folder but for some reason its pushing what I assume is a cached image. Where it’s getting it from I don’t know, because its not from the folder where the images are stored.

I tried adding the timestamp code which someone mentioned early up in the thread, but its giving me an error.

Just wanted to say thanks, this is awesome!

1 Like

Ok I don’t use it but I think Cloudfare is the issue here, you can try setting temporarily to “developer mode” and seeing if that helps, I am guessing it will, which means you will need to create a new cache rule to bypass cache for the snapshot image calls.

More info on cache rules is here:

Create a cache rule in the dashboard

Hope this helps!

1 Like

Hello,

Do you know why always when a person is in front of camera send me same pictures from yesterday? Not update pictures?

Thanks

To always get correct images, timestamp them:

alias: Camera - Back Alley - Snapshot, AI & Notification
sequence:
  - variables:
      trig_ts: "{{ now().strftime('%Y%m%d-%H%M%S') }}"
  - action: camera.snapshot
    metadata: {}
    data:
      filename: /config/www/snapshots/alley_{{trig_ts}}_snapshot1.jpg
    target:
      entity_id:
        - camera.alley
    enabled: true
  - delay:
      hours: 0
      minutes: 0
      seconds: 1
      milliseconds: 0
    enabled: true
  - action: camera.snapshot
    metadata: {}
    data:
      filename: /config/www/snapshots/alley_{{trig_ts}}_snapshot2.jpg
    target:
      entity_id:
        - camera.alley
    enabled: true
  - delay:
      hours: 0
      minutes: 0
      seconds: 1
      milliseconds: 0
    enabled: true
  - action: camera.snapshot
    metadata: {}
    data:
      filename: /config/www/snapshots/alley_{{trig_ts}}_snapshot3.jpg
    target:
      entity_id:
        - camera.alley
    enabled: true
  - action: google_generative_ai_conversation.generate_content
    metadata: {}
    data:
      prompt: >-
        PROMPT_HERE
      image_filename:
        - ./www/snapshots/alley_{{trig_ts}}_snapshot1.jpg
        - ./www/snapshots/alley_{{trig_ts}}_snapshot2.jpg
        - ./www/snapshots/alley_{{trig_ts}}_snapshot3.jpg
    response_variable: generated_content

Manage old files

Of course, now you need to manage the old images. There is an add-on to add actions to delete files and folders with filters like X hours old, which I use in an automation that fires daily.

I also delete files that don’t have anything of interest. For the following example to work, you need to install the add-on.

Delete images on fail

  - if:
      - condition: template
        value_template: "{{ 'No Obvious Motion Detected.' in generated_content.text }}"
    then:
      - sequence:
          - parallel:
              - action: delete.file
                data:
                  file: /config/www/snapshots/alley_{{trig_ts}}_snapshot1.jpg
              - action: delete.file
                data:
                  file: /config/www/snapshots/alley_{{trig_ts}}_snapshot2.jpg
              - action: delete.file
                data:
                  file: /config/www/snapshots/alley_{{trig_ts}}_snapshot3.jpg
          - stop: Gemini AI saw nothing of interest

Delete old files daily

The default time is set to 24 hours, so any files in the folder older than 24 hours will be deleted in this example automation that runs at 3 AM daily:

description: "Delete old Gemini AI CCTV images that are older than 24 hours"
mode: single
triggers:
  - trigger: time
    at: "03:00:00"
conditions: []
actions:
  - action: delete.files_in_folder
    data:
      folder: /config/www/snapshots

See the add-on repo for the other filters available.

1 Like

Thanks a ton, implemented this using snapshot.

alias: AI - Main Door - Snapshot on motion
description: ""
triggers:
  - trigger: state
    entity_id:
      - binary_sensor.main_door_person
    from: "off"
    to: "on"
conditions: []
actions:
  - action: script.toggle
    metadata: {}
    data: {}
    target:
      entity_id: script.main_door_snapshot_ai_notification
mode: single
alias: Main Door - Snapshot, AI & Notification
sequence:
  - metadata: {}
    data:
      filename: /media/local/img/snapshot.jpg
    target:
      entity_id: camera.main_door_fluent
    action: camera.snapshot
  - metadata: {}
    data:
      prompt: >
        Motion has been detected, very briefly describe what you see in this
        image from my driveway camera number 1. What do you think caused the
        motion alarm? If a person is present, describe them in detail. Don't
        describe stationary objects or buildings. Your message needs to be short
        enough to fit in a phone notification.
      image_filename: /media/local/img/snapshot.jpg
    response_variable: generated_content
    action: google_generative_ai_conversation.generate_content
  - metadata: {}
    data:
      title: Main Door Motion Detected
      message: "\"{{ generated_content['text'] }}\""
      data:
        image: /media/local/local/img/snapshot.jpg
    action: notify.IOS_DEVICES
mode: single
description: ""

I am using a custom folder to store the snapshot. There is no need to clean up. The previous snapshot is deleted each time a motion is triggered.

The need to clean up is because each image has a timestamp in its filename in my script. In the yaml you posted, the filename is the same, so the file is overwritten each time and is why sometimes all notifications in the notification drawer show the same image.

Basically, adding a timestamp to the image name (alley_20241117-195941_snapshot1.jpg) forces each notification you send to use a specific image, instead of whatever the image for alley_snapshot1.jpg is at the time the image is retrieved.

Hi how do I get the text of what it detects on multiple lines when I receive the notification? and above all when I click on the notification how do I recover the description of the photo taken? Thank you

This is great, thank you for sharing!

I seem to be getting this error when attempting to save the script.

When I review my mobile ID, I’ve tried both the ID that appears in the address bar and also the device name ‘Alex’ to no avail.

Would anyone know what I might be doing wrong?

Thanks in advance.

@Aussie_Adam I believe I have it set correctly. But still no image.