Guide for CCTV Snapshot on motion, send to Google Generative AI & get notification with description & snapshot

This is brilliant. Thanks so much for posting.

I’ve gone from just hearing the doorbell ring, to having home assistant send me a snapshot of who rang the doorbell, to having AI analyse motion at the front door, all in the space of a week.

This really does get the very best out of the technology available.

Aside from that, this morning, I’ve been helping a guy in South Africa get this up and running too. I love how this community connects people from all over the world.

2 Likes

I get the image and text via the companion app, but not telegram.

Here is my complete script:

alias: AI Driveway camera - Snapshot, AI & Notification (Duplicate)
sequence:
  - metadata: {}
    data:
      filename: ./www/snapshots/driveway_snapshot1.jpg
    target:
      device_id: f60ae99c7139f23e79cd9a74e4b875d7
    enabled: true
    action: camera.snapshot
  - delay:
      hours: 0
      minutes: 0
      seconds: 0
      milliseconds: 500
    enabled: true
  - metadata: {}
    data:
      filename: ./www/snapshots/driveway_snapshot2.jpg
    target:
      device_id: f60ae99c7139f23e79cd9a74e4b875d7
    enabled: true
    action: camera.snapshot
  - delay:
      hours: 0
      minutes: 0
      seconds: 0
      milliseconds: 500
    enabled: true
  - metadata: {}
    data:
      filename: ./www/snapshots/driveway_snapshot3.jpg
    target:
      device_id: f60ae99c7139f23e79cd9a74e4b875d7
    enabled: true
    action: camera.snapshot
  - metadata: {}
    data:
      prompt: >-
        Motion has been detected, compare and very briefly describe what you see
        in the following sequence of images from my driveway camera number 1.
        What do you think caused the motion alarm? If a person or car is
        present, describe them in detail. Do not describe stationary objects or
        buildings. If you see no obvious causes of motion, reply with "No
        Obvious Motion Detected." Your message needs to be short enough to fit
        in a phone notification. Ignore animals and rain.
      image_filename:
        - ./www/snapshots/driveway_snapshot1.jpg
        - ./www/snapshots/driveway_snapshot2.jpg
        - ./www/snapshots/driveway_snapshot3.jpg
    response_variable: generated_content
    action: google_generative_ai_conversation.generate_content
  - if:
      - condition: template
        value_template: "{{ 'No Obvious Motion Detected.' in generated_content.text }}"
    then:
      - stop: ""
    else:
      - metadata: {}
        data:
          title: Driveway camera Motion Detected
          message: "{{generated_content['text'] }}"
          data:
            image: ./www/snapshots/driveway_snapshot2.jpg
            file: ./www/snapshots/driveway_snapshot2.jpg
        action: notify.telegram_notifier4
      - action: telegram_bot.send_photo
        metadata: {}
        data:
          authentication: digest
          file: ./www/snapshots/driveway_snapshot2.jpg
          target: notify.telegram_notifier4
mode: single
description: ""

I’m not familiar with telegram, sorry.

I would imagine if it works via the companion app but not telegram, then the problem is with telegram :thinking: :man_shrugging:

A novice here, giving this a go.
I’ve got issues with the script saving the snapshot.
I got the error:
“Failed to perform the action script/garage_cam_snapshot_ai_notification. Cannot write filename: ./www/snapshots/garagecam_snapshot1.jpg, no access to path; allowlist_external_dirs may need to be adjusted in configuration.yaml

I went into the configuration file and added the lines:

allowlist_external_dirs:
- www/snapshots/

I created the folder as per instructions so it is there, have double checked spelling!

Here is the start of my code:

alias: Garage Cam - Snapshot, AI & Notification
description: AI work on snapshots
sequence:
  - action: camera.snapshot
    metadata: {}
    data:
      filename: "filename: ./www/snapshots/garagecam_snapshot1.jpg"
    target:
      device_id: 4ec962ba7d4f71f8b342558bd4332922
       enabled: true

Not a clue why I couldn’t copy and paste the code into yaml but it disappeared everytime i saved it or switched to visual interface, so was obviously doing something wrong. I used the visual interface to pick the entities and got the code above. That bit isn’t too important right now though :grinning: one problem at a time!

Edit1: Fixed indentation omission in the yaml file update

allowlist_external_dirs:
  - www/snapshots/

Edit 2: I have now corrected to:

homeassistant:
  allowlist_external_dirs:
    - /config/www/snapshots

Edit 3: Spotted my mistake, I had copied the word filename twice in the script. So I can take snap shots…not to get the rest working!

“Message malformed: Service notify.mobile cdc49305af8725b720f979ee0c23196a does not match format . for dictionary value @ data[‘sequence’][6][‘else’][0][‘action’]”

anyone getting that error?

HA 11.0

1 Like

Nice work.
But this here looks like a nice neet Blueprint works great.

This is absolutely crazy, thank you for sharing this script/automation!!

1 Like

I think you need to use actions now instead of service.

else:
      - action: notify.mobile_app_952505fcee7aafb7d5a574051c2a852e

That’s how mine is. I was getting that error before. I haven’t got mine to work yet though overall…but i think it’s because the photos don’t seem to show anything different. I think maybe the blink app recording video automatically could be stopping me taking photos in time.

Hi Everyone, I have just set this up for my 4 cameras (4 individual scripts and a custom automation). However, I am having a slight issue with the notification. It is displaying a cached image which doesn’t exist anymore and not the recent snapshot. The AI does describe the latest snapshot which I can see in the snapshots folder.

Any ideas on how to fix this?

I have a similar issue with only one camera (Unifi) where the snapshot is quite static, so most of the time in the notification I see one of the previous snapshot image but the generated text is correct… I’m using iPhone for notification, so maybe Apple mixes something in the soup

1 Like

I came on here for the same issue…it’s my blink thumbnail photo it uses instead of a snap shot. I wonder can it not take a snap shot when my video camera is taking video…or is home assitant just not compatible with blink (and maybe same for your brand)

EditL for me i think that answer is here How to configure Blink XT 2 to take pictures or play a stream - #4 by Salty
if I can figure it out.

Same error here!

Can anyone can help with the cached image issue?

It’s hard to say as it works fine here, are you using Cloudfare ? Or DuckDNS ?

I am using a iPhone and it works fine on my end. Sounds like your notification is firing off before the image is saved maybe? You can try add a delay of 0.5 of a second before the notification part and see if that helps.

Hey @Aussie_Adam, thanks for getting back to me.
Yes I am using Cloudflare for security to access HA remotely, but not when I am at home. So 2 serves addresses, local and external using Cloudflare. However the issue still persists even when home.

I see the updated snapshot in the folder but for some reason its pushing what I assume is a cached image. Where it’s getting it from I don’t know, because its not from the folder where the images are stored.

I tried adding the timestamp code which someone mentioned early up in the thread, but its giving me an error.

Just wanted to say thanks, this is awesome!

1 Like

Ok I don’t use it but I think Cloudfare is the issue here, you can try setting temporarily to “developer mode” and seeing if that helps, I am guessing it will, which means you will need to create a new cache rule to bypass cache for the snapshot image calls.

More info on cache rules is here:

Create a cache rule in the dashboard

Hope this helps!

1 Like

Hello,

Do you know why always when a person is in front of camera send me same pictures from yesterday? Not update pictures?

Thanks

To always get correct images, timestamp them:

alias: Camera - Back Alley - Snapshot, AI & Notification
sequence:
  - variables:
      trig_ts: "{{ now().strftime('%Y%m%d-%H%M%S') }}"
  - action: camera.snapshot
    metadata: {}
    data:
      filename: /config/www/snapshots/alley_{{trig_ts}}_snapshot1.jpg
    target:
      entity_id:
        - camera.alley
    enabled: true
  - delay:
      hours: 0
      minutes: 0
      seconds: 1
      milliseconds: 0
    enabled: true
  - action: camera.snapshot
    metadata: {}
    data:
      filename: /config/www/snapshots/alley_{{trig_ts}}_snapshot2.jpg
    target:
      entity_id:
        - camera.alley
    enabled: true
  - delay:
      hours: 0
      minutes: 0
      seconds: 1
      milliseconds: 0
    enabled: true
  - action: camera.snapshot
    metadata: {}
    data:
      filename: /config/www/snapshots/alley_{{trig_ts}}_snapshot3.jpg
    target:
      entity_id:
        - camera.alley
    enabled: true
  - action: google_generative_ai_conversation.generate_content
    metadata: {}
    data:
      prompt: >-
        PROMPT_HERE
      image_filename:
        - ./www/snapshots/alley_{{trig_ts}}_snapshot1.jpg
        - ./www/snapshots/alley_{{trig_ts}}_snapshot2.jpg
        - ./www/snapshots/alley_{{trig_ts}}_snapshot3.jpg
    response_variable: generated_content

Manage old files

Of course, now you need to manage the old images. There is an add-on to add actions to delete files and folders with filters like X hours old, which I use in an automation that fires daily.

I also delete files that don’t have anything of interest. For the following example to work, you need to install the add-on.

Delete images on fail

  - if:
      - condition: template
        value_template: "{{ 'No Obvious Motion Detected.' in generated_content.text }}"
    then:
      - sequence:
          - parallel:
              - action: delete.file
                data:
                  file: /config/www/snapshots/alley_{{trig_ts}}_snapshot1.jpg
              - action: delete.file
                data:
                  file: /config/www/snapshots/alley_{{trig_ts}}_snapshot2.jpg
              - action: delete.file
                data:
                  file: /config/www/snapshots/alley_{{trig_ts}}_snapshot3.jpg
          - stop: Gemini AI saw nothing of interest

Delete old files daily

The default time is set to 24 hours, so any files in the folder older than 24 hours will be deleted in this example automation that runs at 3 AM daily:

description: "Delete old Gemini AI CCTV images that are older than 24 hours"
mode: single
triggers:
  - trigger: time
    at: "03:00:00"
conditions: []
actions:
  - action: delete.files_in_folder
    data:
      folder: /config/www/snapshots

See the add-on repo for the other filters available.

1 Like