Guide for CCTV Snapshot on motion, send to Google Generative AI & get notification with description & snapshot

Hi all,
A few weeks I posted a rough guide on a couple HA facebook pages giving folks the basics of getting a CCTV camera snapshot taken by HA, sending that to Google AI and having the description and image come back as a notification. Not sure if anyone has posted anything similar on here so here I am… :slight_smile:

When I originally posted, the script used one snapshot image and you got alot of false notifications… So now I’ve updated it so it takes 3 snapshots 0.5 seconds apart, sends those 3 images google to compare and if google notices no significant motion, the notification WILL NOT get sent.

This hopefully cuts down on false notifications or ones from trees etc moving! It also helps the AI notice actual movement of things like cars…

This works on Iphone, unsure if the image will come through with the notification on Android.

Anyway, I see lots of others asking about this so I thought I’d share my YAML I use as an easy way to do it, as well as creating the actual snapshot WITHOUT using Frigate or your camera to take snapshots.

Your cameras (and phone notifications) will of course need to be already added in HA and they will need to support motion detection and HA must have an entity to be able to see that motion alarm status…

If so, read on.

All the YAML for both the script and automation live here:
AdamGit69/code-snippets (github.com)

To get it all running follow these instructions:

First up install the integration from here: Google Generative AI Conversation - Home Assistant

Once you have that installed, create a directory under your www folder called “snapshots”.

Ensure your camera has motion detection turned on and that HA can see the alarm when motion is detected.

Then create a AUTOMATION named: “Camera 1 - Snapshot on motion” using the following code: code-snippets/cctv_ai_notifications_automation at e963791be142fdf3352f676b5d2cf934c01edb1d · AdamGit69/code-snippets · GitHub

Edit to suit, switching to visual editor will help you find your camera and entity id’s. This automation is to run the below script when HA receives the motion alarm from your camera.

After that create a SCRIPT called: “Camera 1 - Snapshot, AI & Notification” with the following code: code-snippets/cctv_ai_notification_script_UPDATE at e963791be142fdf3352f676b5d2cf934c01edb1d · AdamGit69/code-snippets · GitHub

BE AWARE: If you change the google prompt, especially the "If you see no obvious causes of motion, reply with “Camera has detected motion however no obvious motion observed comparing snapshots”. part, you will also have to change the IF statement part to suit.

ALSO NOTE: this sends 3 images to google, so if you get a “Error generating content: 400 Request payload size exceeds the limit” error, if you manually run the script, your camera snapshots are too big, and you have to adjust it back to 2 images by removing the last snapshot action, the 0.5 second delay before it and then removing the reference to the 3rd image from under image_filename in the google ai service part…

Phew, I hope that all makes sense! :slight_smile:

Happy HA’ing! :smiley:




5 Likes

Love this use of Ai, thanks for sharing.

Just when I think I have finished a particular project someone comes along and shows something way better.

1 Like

Thanks, will look this up for a future project.

I was just sitting in bed last night wondering how I could be notified if a white :llama: was on my driveway. :metal::slightly_smiling_face:

1 Like

Isn’t that what AI stands for? Alpaca Identification? or am I doing it wrong? hahaha :rofl: :rofl:

2 Likes

Love it! Thanks!

Interestingly, it seems to think the sequence is opposite of what it is – a vehicle pulling into my driveway is seen as “backing out”.

Could fixing this just be a matter of reversing the image sequence as sent to Google?

Possibly. I noticed mine does this as well on one camera when I enter the drive. I will have a play and report back. :slight_smile: