With the increase errors “429 Resource has been exhausted (e.g. check quota)”
I have been able to modify the automation to do a bit of error handling or ignore, while still sending me the image without the Google Generative text.
Next stop is too look into other options for image analysis, bit for now sticking to it.
Could you share your automation.yaml with us? I’m in the same ‘429’ situation and would like to know how you modified the automation. Thnx in advanced!
Hi @goprojojo
Here you go, this is an unredacted version, so does a bit of checking of doors, rate limiting etc. to avoid excessive triggering of notifications.
You will notice the key to making this work is the statement in the Google and Input.text actions
continue_on_error: true
This allows the the automation to proceed, while the error is written to the log, and not a hard stop in the automation.
alias: Notification - Snapshot & AI - Car Port - Person
description: ""
triggers:
- trigger: state
entity_id:
- binary_sensor.the_car_port_person_detection
to: "on"
conditions:
- condition: template
value_template: >-
{{ now() -
state_attr('automation.notification_snapshot_ai_car_port_person_or_vehicle'
, 'last_triggered') | default(now(), true) > timedelta(seconds=60) }}
- condition: state
entity_id: input_boolean.motion_detection_paused
state: "off"
- condition: state
entity_id: binary_sensor.back_door_door
state: "off"
enabled: true
- condition: state
entity_id: cover.garage_door_door
state: closed
enabled: true
- condition: state
entity_id: lock.front_door_lock
state: locked
enabled: true
actions:
- action: input_text.set_value
metadata: {}
data:
value: Google Generative Image Analysis failure
target:
entity_id: input_text.google_response_car_port
- action: camera.snapshot
metadata: {}
data:
filename: /config/www/snapshot/{{ trigger.from_state.name }}_1.jpg
target:
device_id: 4dd374dfa223b7ae03644f7720f9960e
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
- action: camera.snapshot
metadata: {}
data:
filename: /config/www/snapshot/{{ trigger.from_state.name }}_2.jpg
enabled: true
target:
device_id: 4dd374dfa223b7ae03644f7720f9960e
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
enabled: true
- action: camera.snapshot
metadata: {}
data:
filename: /config/www/snapshot/{{ trigger.from_state.name }}_3.jpg
enabled: true
target:
device_id: 4dd374dfa223b7ae03644f7720f9960e
- action: google_generative_ai_conversation.generate_content
continue_on_error: true
metadata: {}
data:
prompt: >-
Motion has been detected, compare and very briefly describe what you see
in the following sequence of images from my Car Port camera. What do
you think caused the motion alarm? If a person, car or animal is
present, describe them in detail. Do not describe stationary objects or
buildings. If you see no obvious causes of motion, reply with "No
obvious motion observed". Your message needs to be short enough to fit
in a phone notification.
image_filename:
- ./www/snapshot/{{ trigger.from_state.name }}_1.jpg
- ./www/snapshot/{{ trigger.from_state.name }}_2.jpg
- ./www/snapshot/{{ trigger.from_state.name }}_3.jpg
response_variable: generated_content
enabled: true
- action: input_text.set_value
continue_on_error: true
metadata: {}
data:
value: "{{ generated_content['text'] }}"
target:
entity_id: input_text.google_response_car_port
- if:
- condition: template
value_template: "{{ 'No obvious motion observed.' in generated_content.text }}"
then:
- stop: ""
else:
- action: notify.pushover
metadata: {}
data:
target: S20
title: Car Port Motion Detected
message: "{{states('input_text.google_response_car_port')}}"
data:
attachment: /config/www/snapshot/{{ trigger.from_state.name }}_1.jpg
enabled: true
mode: single
Update…
Been doing testing with LLM Vision + Google Gemini
I’m unable to get the model to respond with “No Obvious Motion observed” it keeps on coming up with something to infer there was possible motion, small animal, rats, birds etc…
I have adjusted the Temperature between integrations to match (1), however have seen additional options for Top K and Top P which is not present in the LLm Vision + Gemini integration.
So the above automation only been proven to with with Google Generative AI integration currently, to prevent trigger on no motion observed in images.
With LLM vision and ChatGPT, this prompt works for me. I doubled up on the no motion instruction; it could probably be clearer, but mostly I dont want it telling me the dog is wandering around the house. The annoying part about chatgpt is that it insists on telling me there are no cars visible in the living room.
Motion has been detected, compare and very briefly describe any people or
cars that you see in the following sequence of images from my {{ camera_name
}} camera. If you do not see a person, reply with “No obvious motion
detected.” Do not describe stationary objects or buildings. If you see no
obvious causes of motion, reply with “No Obvious Motion Detected.” Your
message needs to be short enough to fit in a phone notification.
Since HA 2025.3.0 this warning message appear:
The ‘image_filename’ parameter in Google Generative AI actions is deprecated. Please edit scripts and automations to use ‘filenames’ intead.
Hey @haus
could you please paste your script as a reference for me please. trying to use your path instead of google as I’m always getting 429 responses
Here you go. This depends on camera_name and sub_entity being passed from my automation (based on the triggering device):
This is an ongoing project so some things aren’t currently used; for example, trigger_timestamp was used to create distinct image for each event, but a bug in LLM Vision at the moment means I have to pass the camera stream to LLM vision instead of taking snapshots and naming them, but I’m leaving it in case I revert back to snapshots later.
sequence:
- variables:
generated_content:
response_text: No AI Available
trigger_timestamp: "{{ now().strftime('%Y%m%d-%H%M%S') }}"
- action: llmvision.image_analyzer
metadata: {}
data:
remember: true
use_memory: false
include_filename: false
max_tokens: 100
temperature: 0.2
expose_images: true
generate_title: true
message: >-
Motion has been detected. Briefly describe any people or cars that you
see in this image from my {{ camera_name }} camera. If you do not see a
person, reply with "No obvious motion detected." Do not describe
stationary objects or buildings. If you see no obvious causes of motion,
reply with "No Obvious Motion Detected." Your message needs to be short
enough to fit in a phone notification.
provider: REDACTED
image_entity:
- "{{ sub_entity }}"
response_variable: generated_content
- if:
- condition: template
value_template: >-
{{ ("no obvious motion" not in
generated_content['response_text']|lower) }}
then:
- metadata: {}
data:
title: "{{ camera_name }}"
message: "{{ generated_content['response_text'] }}"
data:
group: Motion
image: "{{generated_content.key_frame.replace('/config/www/','/local/') }}"
priority: high
ttl: 0
clickAction: noAction
actions:
- action: URI
title: Notification History
uri: settings://notification_history
- action: URI
title: Cameras Page
uri: /lovelace/cameras
sticky: true
notification_icon: mdi:motion-sensor
action: notify.REDACTED
alias: General Camera - Snapshot AI & Notification (Simplified)
trace:
stored_traces: 40
mode: parallel
description: ""
max: 3
And I just realized that “{{ camera_name }} camera” results in something like “Front Walkway Camera camera” so I will modify that because all my camera names have “camera” in them.
Thanks, just saw your reply and I actually managed to get mine going by myself. Will check over your config too to see what you’re doing. I like the approach of being able to select a different provider too
I have added my config for the LLM portion of the script using the visual editor if that help someone in the future
I hope this post will help a lot of people with 429 Resource has been exhausted and 503 Service Unavailable . There is nothing you can do to prevent these errors unless you are a corporate entity paying hundreds of thousands of dollars per day to use the service.
Free account users get what few computing cycles that are left over after the corporate accounts are serviced. I switched over to a paid account, and it doesn’t appear to make any difference on the 429 errors, but at least I don’t have to worry about getting my account locked for too many queries in succession. For what it’s worth, my last month’s bill was $0.04.
Now, for the part that will help. I have finally been able to handle errors from the generative AI call and have the script finish and at least send the image and notification. Even though there is no AI description, and it might be a false positive without any actual motion, I still get the alert message.
I’ve been testing this for the last few days and it appears to work.
That day of testing, I received three 429 errors, and I also received three MMS text messages with the caption “Google AI Error”. Very promising.
I previously posted that I changed the automation and script to handle multiple cameras with just a single automation and a single script. That is the code that I’ll be posting below. If you are using Adam’s original code, you’ll have to break out the error handling routines and add them to his code.
Here’s the calling automation. It is configured to queue up to 4 motion calls in case multiple cameras trigger close to each other.
alias: Unifi Cameras Motion AI SMS Snapshot
description: ""
triggers:
- trigger: state
entity_id:
- binary_sensor.ai_pro_driveway_motion
- binary_sensor.ai_pro_backyard_motion
- binary_sensor.ai_pro_frontyard_motion
from: "off"
to: "on"
conditions: []
actions:
- action: script.camera_snapshot_ai_notification
metadata: {}
data:
camera_id: "{{ device_id(trigger.entity_id) }}"
camera_name: "{{ device_attr(trigger.entity_id, 'name') }}"
mode: queued
max: 4
And this is the script that handles the snapshot creation. This runs in single mode so there is no chance that the snapshots we create will be stomped on by a second automation call before this finishes handling the current one.
alias: Camera - Snapshot AI & Notification
sequence:
- metadata: {}
data:
filename: ./www/snapshots/{{ camera_name }}_snapshot1.jpg
target:
device_id: "{{ camera_id }}"
enabled: true
action: camera.snapshot
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
enabled: true
- metadata: {}
data:
filename: ./www/snapshots/{{ camera_name }}_snapshot2.jpg
target:
device_id: "{{ camera_id }}"
enabled: true
action: camera.snapshot
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
enabled: true
- metadata: {}
data:
filename: ./www/snapshots/{{ camera_name }}_snapshot3.jpg
target:
device_id: "{{ camera_id }}"
enabled: true
action: camera.snapshot
- metadata: {}
data:
prompt: >-
Motion has been detected, compare and very briefly describe what you see
in the following sequence of images from my {{ camera_name }} camera.
What do you think caused the motion alarm? If a moving person or moving
car is present, describe them in detail. Do not describe stationary
objects or buildings. If motion was caused by snow or rain reply with
"No Obvious Motion Detected". If you see no significant motion, or no
obvious causes of motion, reply with "No Obvious Motion Detected". Your
message needs to be short enough to fit in a phone notification.
image_filename:
- ./www/snapshots/{{ camera_name }}_snapshot1.jpg
- ./www/snapshots/{{ camera_name }}_snapshot2.jpg
- ./www/snapshots/{{ camera_name }}_snapshot3.jpg
response_variable: generated_content
continue_on_error: true
action: google_generative_ai_conversation.generate_content
- variables:
filtered_content: >-
{{ 'Google AI Error' if generated_content == empty else
generated_content.text }}
- if:
- condition: template
value_template: "{{ 'no obvious motion detected' in filtered_content | lower}}"
then:
- stop: ""
else:
- action: shell_command.fixup_jpeg
data:
image_file: ./www/snapshots/{{ camera_name }}_snapshot2.jpg
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
- metadata: {}
data:
title: "{{ camera_name }} Motion Detected"
message: "{{filtered_content}}"
data:
images:
- /config/www/snapshots/{{ camera_name }}_snapshot2.jpg
action: notify.family_sms
mode: single
description: ""
The important parts are the continue_on_error: true line before the generative ai call, and the variables block after the call. The filtered_content variable will contain “google AI Error” if the generative AI call failed, or the response text if it succeeded.
The check to see if no motion occured, and the actual notification message now need to use the new filtered_content variable and not the previous generated_content.text.
That’s all you need to do to at least get a notification that something occurred on your cameras even if Google fails. Good luck.
If you are still reading at this point, I also noticed something curious during my testing and I’m looking into it further. During my testing I was also trying out various prompts for Gemini (trying to get it to recognize that raindrops are not “motion”), and I noticed that when I was using a short prompt, I received fewer 429 Resource has been exhausted errors. I don’t know at this point if there really is a cause and effect relationship, or if it was just coincidence. I’m testing further and I’ll let you know.
Cheers!
EDIT - it was pointed out in another thread that my used of == empty only works by accident. The proper test should be is not defined.
Can I just say thanks this is awesome beyond belief… And updating the prompt to use Aussie slang is ‘bloody ripper mate’
I loved this idea and implemented it to my smart home.
At first i got error 429 alot but then i reduced the images from 3 to 2 and shortend the phrase alot. and now it works 9/10 times.
One question, is it possible to get the generated_content variable to print in the logbook? if so how?
You could just create a persistent notification something like this:
- action: persistent_notification.create
data:
title: "AI Generated Content"
message: "Response Variable: {{ generated_content }}"
I use persistent notifications when I’m trying to debug a script. Very useful when you need to confirm the variables being generated and passed.
Thanks. that’s way better!
I do something similar to see the responses, with this the response goes to to a Input text helper
action: input_text.set_value
continue_on_error: true
metadata: {}
data:
value: "{{ google.response_text }}"
target:
entity_id: input_text.google_response_car_port
Which I use to debug but also enrich the photo on the dashboard I have for my camera’s
Thanks for sharing all the scripts and yaml code!
I was wondering if someone has managed to create a notification for (GoogleTV) smart tv? If yes, please share your code and steps.
Does anyone know how to get this to work with the new Ai Task Integration?
I’ve tried to modify it myself but I have no idea how to attach 3 photo’s and it also seems to be an issue using the www folder.
Same here. Everything is broken 
Here’s what I did to switch to the new ai_task.generate_data service. You can’t just replace google_generative_ai_conversation.generate_content with the new call. Not only has the service name changed, but so has the format of the data as well as the location of the attachments.
First, you will need to create the “media” directory under config. I’m running HAOS with File Editor 5.8.0 addon. I created both a media directory under homeassistant, and then created a snapshots directory within the new media directory.
Next, you need to tell Home Assistant to use the media directory by editing your configuration.yaml file. The new media_dirs is required because the new service uses media content ID URLs for the attachments instead of the file names. The allow_external_directories needs to be updated to the new media directory as well.
homeassistant:
allowlist_external_dirs:
- "/config/media/snapshots"
media_dirs:
local: "/config/media"
Finally, are the changes to your CCTV script. For each snapshot you create, you need to change the path from ./www/snapshots to ./media/snapshots.
For example:
- service: camera.snapshot
metadata: {}
data:
filename: ./www/snapshots/driveway1_snapshot2.jpg
target:
device_id: ***YOUR_CAMERAS_DEVICE_ID***
becomes:
- service: camera.snapshot
metadata: {}
data:
filename: ./media/snapshots/driveway1_snapshot2.jpg
target:
device_id: ***YOUR_CAMERAS_DEVICE_ID***
Next are the changes to the service itself. Previously you had something like this:
- service: google_generative_ai_conversation.generate_content
metadata: {}
data:
prompt: >-
Motion has been detected, compare and very briefly describe what you see
in the following sequence of images from my driveway camera number 1.
What do you think caused the motion alarm? If a person or car is
present, describe them in detail. Do not describe stationary objects or
buildings. If you see no obvious causes of motion, reply with "No
Obvious Motion Detected." Your message needs to be short enough to fit
in a phone notification.
image_filename:
- ./www/snapshots/driveway1_snapshot1.jpg
- ./www/snapshots/driveway1_snapshot2.jpg
- ./www/snapshots/driveway1_snapshot3.jpg
response_variable: generated_content
This gets changed to this:
- metadata: {}
data:
task_name: Video Snapshot Sequence Analysis
instructions: >-
Motion has been detected, compare and very briefly describe what you see
in the following sequence of images from my driveway camera number 1.
What do you think caused the motion alarm? If a person or car is
present, describe them in detail. Do not describe stationary objects or
buildings. If you see no obvious causes of motion, reply with "No
Obvious Motion Detected." Your message needs to be short enough to fit
in a phone notification.
attachments:
- media_content_id: >
media-source://media_source/local/snapshots/driveway1_snapshot1.jpg
media_content_type: image/jpeg
- media_content_id: >
media-source://media_source/local/snapshots/driveway1_snapshot2.jpg
media_content_type: image/jpeg
- media_content_id: >
media-source://media_source/local/snapshots/driveway1_snapshot3.jpg
media_content_type: image/jpeg
response_variable: generated_content
action: ai_task.generate_data
We had to add a task_name, the prompt changes to instructions, the image_filename changes to attachments, the actual file names become media URLs, and the service changes to the new action.
For the last of the script changes, you will need to update the image that’s included in the notification:
- service: notify.mobile ***** YOUR PHONE ID ********
metadata: {}
data:
title: Driveway 1 Motion Detected
message: "{{generated_content['text'] }}"
data:
image: /local/snapshots/driveway1_snapshot2.jpg
Would become:
- service: notify.mobile ***** YOUR PHONE ID ********
metadata: {}
data:
title: Driveway 1 Motion Detected
message: "{{generated_content['text'] }}"
data:
image: ./media/snapshots/driveway1_snapshot2.jpg
That should be all of the changes necessary to your script. The very last thing to do before we restart HA is to set the default script actions. In HA, select Settings, then System, then General. In the AI Suggestions section, select Google AI Task from the dropdown for each of the entries. Save those changes, and restart Home Assistant.
Hopefully that will take care of the deprecated message and get your CCTV AI Notification working again.
Do you get notfications with image in home assistant companion app? I got an error. URLSessionTask failed with error: URL is not supported. Very strange
Hi all,
Like with several other people, AI generated responses to Camera events simply stopped working for us a while ago.
I’ve looked through several media and tried all kinds of solutions to the issue, including the above.
The good news is, I am once again receiving photo’s from triggering events, the bad, I am not receiving any generated output along with the images.
I am genuinely out of ideas and hope someone can help me in the right direction.
Here’s the Automation YAML (Obscured some items, like the AI provider (Google))
alias: Melding bij detectie Camera deurbel (AI)
description: ""
triggers:
- type: turned_on
device_id:
entity_id:
domain: binary_sensor
trigger: device
conditions:
- condition: and
conditions:
- condition: time
after: "04:30:00"
before: "23:00:00"
actions:
- alias: Camera - Deurbel - Snapshot, AI & Notification
sequence:
- action: camera.snapshot
target:
entity_id: camera.deurbel_voordeur_vloeiend
data:
filename: /config/media/snapshots/snapshot_doorbell1.jpg
enabled: true
- delay:
milliseconds: 500
enabled: true
- action: camera.snapshot
target:
entity_id: camera.deurbel_voordeur_vloeiend
data:
filename: /config/media/snapshots/snapshot_doorbell2.jpg
enabled: true
- delay:
milliseconds: 500
enabled: true
- action: camera.snapshot
target:
entity_id: camera.deurbel_voordeur_vloeiend
data:
filename: /config/media/snapshots/snapshot_doorbell3.jpg
enabled: true
- action: llmvision.image_analyzer
metadata: {}
data:
remember: false
use_memory: false
include_filename: true
target_width: 1280
max_tokens: 100
generate_title: false
expose_images: true
provider:
message: >-
De deurbel camera heeft beweging gedetecteerd. Beschrijf op een
grappige manier aan de hand van de beelden wat er staat.
Beschrijf geen stilstaande objecten of gebouwen of struiken. Als het
een voorbijganger betreft of niemand gezien, antwoord dan alleen
"Voorbijganger". Je bericht moet kort genoeg zijn om in een
telefoonmelding te passen.
image_file: |-
/config/media/snapshots/snapshot_doorbell1.jpg
/config/media/snapshots/snapshot_doorbell2.jpg
/config/media/snapshots/snapshot_doorbell3.jpg
model: Video Snapshot Sequence Analysis
image_entity:
- camera.deurbel_voordeur_vloeiend
response_variable: output
- if:
- condition: template
value_template: "{{ 'Voorbijganger' in output.text }}"
then:
- stop: Voorbijganger gezien, geen melding naar telefoons
else:
- alias: Stuur via Telegram naar George
action: telegram_bot.send_photo
data:
authentication: digest
config_entry_id:
caption: "{{ output['text'] }}"
file: /config/media/snapshots/snapshot_doorbell1.jpg
mode: single