With the increase errors “429 Resource has been exhausted (e.g. check quota)”
I have been able to modify the automation to do a bit of error handling or ignore, while still sending me the image without the Google Generative text.
Next stop is too look into other options for image analysis, bit for now sticking to it.
Could you share your automation.yaml with us? I’m in the same ‘429’ situation and would like to know how you modified the automation. Thnx in advanced!
Hi @goprojojo
Here you go, this is an unredacted version, so does a bit of checking of doors, rate limiting etc. to avoid excessive triggering of notifications.
You will notice the key to making this work is the statement in the Google and Input.text actions
continue_on_error: true
This allows the the automation to proceed, while the error is written to the log, and not a hard stop in the automation.
alias: Notification - Snapshot & AI - Car Port - Person
description: ""
triggers:
- trigger: state
entity_id:
- binary_sensor.the_car_port_person_detection
to: "on"
conditions:
- condition: template
value_template: >-
{{ now() -
state_attr('automation.notification_snapshot_ai_car_port_person_or_vehicle'
, 'last_triggered') | default(now(), true) > timedelta(seconds=60) }}
- condition: state
entity_id: input_boolean.motion_detection_paused
state: "off"
- condition: state
entity_id: binary_sensor.back_door_door
state: "off"
enabled: true
- condition: state
entity_id: cover.garage_door_door
state: closed
enabled: true
- condition: state
entity_id: lock.front_door_lock
state: locked
enabled: true
actions:
- action: input_text.set_value
metadata: {}
data:
value: Google Generative Image Analysis failure
target:
entity_id: input_text.google_response_car_port
- action: camera.snapshot
metadata: {}
data:
filename: /config/www/snapshot/{{ trigger.from_state.name }}_1.jpg
target:
device_id: 4dd374dfa223b7ae03644f7720f9960e
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
- action: camera.snapshot
metadata: {}
data:
filename: /config/www/snapshot/{{ trigger.from_state.name }}_2.jpg
enabled: true
target:
device_id: 4dd374dfa223b7ae03644f7720f9960e
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
enabled: true
- action: camera.snapshot
metadata: {}
data:
filename: /config/www/snapshot/{{ trigger.from_state.name }}_3.jpg
enabled: true
target:
device_id: 4dd374dfa223b7ae03644f7720f9960e
- action: google_generative_ai_conversation.generate_content
continue_on_error: true
metadata: {}
data:
prompt: >-
Motion has been detected, compare and very briefly describe what you see
in the following sequence of images from my Car Port camera. What do
you think caused the motion alarm? If a person, car or animal is
present, describe them in detail. Do not describe stationary objects or
buildings. If you see no obvious causes of motion, reply with "No
obvious motion observed". Your message needs to be short enough to fit
in a phone notification.
image_filename:
- ./www/snapshot/{{ trigger.from_state.name }}_1.jpg
- ./www/snapshot/{{ trigger.from_state.name }}_2.jpg
- ./www/snapshot/{{ trigger.from_state.name }}_3.jpg
response_variable: generated_content
enabled: true
- action: input_text.set_value
continue_on_error: true
metadata: {}
data:
value: "{{ generated_content['text'] }}"
target:
entity_id: input_text.google_response_car_port
- if:
- condition: template
value_template: "{{ 'No obvious motion observed.' in generated_content.text }}"
then:
- stop: ""
else:
- action: notify.pushover
metadata: {}
data:
target: S20
title: Car Port Motion Detected
message: "{{states('input_text.google_response_car_port')}}"
data:
attachment: /config/www/snapshot/{{ trigger.from_state.name }}_1.jpg
enabled: true
mode: single
Update…
Been doing testing with LLM Vision + Google Gemini
I’m unable to get the model to respond with “No Obvious Motion observed” it keeps on coming up with something to infer there was possible motion, small animal, rats, birds etc…
I have adjusted the Temperature between integrations to match (1), however have seen additional options for Top K and Top P which is not present in the LLm Vision + Gemini integration.
So the above automation only been proven to with with Google Generative AI integration currently, to prevent trigger on no motion observed in images.
With LLM vision and ChatGPT, this prompt works for me. I doubled up on the no motion instruction; it could probably be clearer, but mostly I dont want it telling me the dog is wandering around the house. The annoying part about chatgpt is that it insists on telling me there are no cars visible in the living room.
Motion has been detected, compare and very briefly describe any people or
cars that you see in the following sequence of images from my {{ camera_name
}} camera. If you do not see a person, reply with “No obvious motion
detected.” Do not describe stationary objects or buildings. If you see no
obvious causes of motion, reply with “No Obvious Motion Detected.” Your
message needs to be short enough to fit in a phone notification.
Since HA 2025.3.0 this warning message appear:
The ‘image_filename’ parameter in Google Generative AI actions is deprecated. Please edit scripts and automations to use ‘filenames’ intead.
Hey @haus
could you please paste your script as a reference for me please. trying to use your path instead of google as I’m always getting 429 responses
Here you go. This depends on camera_name and sub_entity being passed from my automation (based on the triggering device):
This is an ongoing project so some things aren’t currently used; for example, trigger_timestamp was used to create distinct image for each event, but a bug in LLM Vision at the moment means I have to pass the camera stream to LLM vision instead of taking snapshots and naming them, but I’m leaving it in case I revert back to snapshots later.
sequence:
- variables:
generated_content:
response_text: No AI Available
trigger_timestamp: "{{ now().strftime('%Y%m%d-%H%M%S') }}"
- action: llmvision.image_analyzer
metadata: {}
data:
remember: true
use_memory: false
include_filename: false
max_tokens: 100
temperature: 0.2
expose_images: true
generate_title: true
message: >-
Motion has been detected. Briefly describe any people or cars that you
see in this image from my {{ camera_name }} camera. If you do not see a
person, reply with "No obvious motion detected." Do not describe
stationary objects or buildings. If you see no obvious causes of motion,
reply with "No Obvious Motion Detected." Your message needs to be short
enough to fit in a phone notification.
provider: REDACTED
image_entity:
- "{{ sub_entity }}"
response_variable: generated_content
- if:
- condition: template
value_template: >-
{{ ("no obvious motion" not in
generated_content['response_text']|lower) }}
then:
- metadata: {}
data:
title: "{{ camera_name }}"
message: "{{ generated_content['response_text'] }}"
data:
group: Motion
image: "{{generated_content.key_frame.replace('/config/www/','/local/') }}"
priority: high
ttl: 0
clickAction: noAction
actions:
- action: URI
title: Notification History
uri: settings://notification_history
- action: URI
title: Cameras Page
uri: /lovelace/cameras
sticky: true
notification_icon: mdi:motion-sensor
action: notify.REDACTED
alias: General Camera - Snapshot AI & Notification (Simplified)
trace:
stored_traces: 40
mode: parallel
description: ""
max: 3
And I just realized that “{{ camera_name }} camera” results in something like “Front Walkway Camera camera” so I will modify that because all my camera names have “camera” in them.
Thanks, just saw your reply and I actually managed to get mine going by myself. Will check over your config too to see what you’re doing. I like the approach of being able to select a different provider too
I have added my config for the LLM portion of the script using the visual editor if that help someone in the future
I hope this post will help a lot of people with 429 Resource has been exhausted and 503 Service Unavailable . There is nothing you can do to prevent these errors unless you are a corporate entity paying hundreds of thousands of dollars per day to use the service.
Free account users get what few computing cycles that are left over after the corporate accounts are serviced. I switched over to a paid account, and it doesn’t appear to make any difference on the 429 errors, but at least I don’t have to worry about getting my account locked for too many queries in succession. For what it’s worth, my last month’s bill was $0.04.
Now, for the part that will help. I have finally been able to handle errors from the generative AI call and have the script finish and at least send the image and notification. Even though there is no AI description, and it might be a false positive without any actual motion, I still get the alert message.
I’ve been testing this for the last few days and it appears to work.
That day of testing, I received three 429 errors, and I also received three MMS text messages with the caption “Google AI Error”. Very promising.
I previously posted that I changed the automation and script to handle multiple cameras with just a single automation and a single script. That is the code that I’ll be posting below. If you are using Adam’s original code, you’ll have to break out the error handling routines and add them to his code.
Here’s the calling automation. It is configured to queue up to 4 motion calls in case multiple cameras trigger close to each other.
alias: Unifi Cameras Motion AI SMS Snapshot
description: ""
triggers:
- trigger: state
entity_id:
- binary_sensor.ai_pro_driveway_motion
- binary_sensor.ai_pro_backyard_motion
- binary_sensor.ai_pro_frontyard_motion
from: "off"
to: "on"
conditions: []
actions:
- action: script.camera_snapshot_ai_notification
metadata: {}
data:
camera_id: "{{ device_id(trigger.entity_id) }}"
camera_name: "{{ device_attr(trigger.entity_id, 'name') }}"
mode: queued
max: 4
And this is the script that handles the snapshot creation. This runs in single mode so there is no chance that the snapshots we create will be stomped on by a second automation call before this finishes handling the current one.
alias: Camera - Snapshot AI & Notification
sequence:
- metadata: {}
data:
filename: ./www/snapshots/{{ camera_name }}_snapshot1.jpg
target:
device_id: "{{ camera_id }}"
enabled: true
action: camera.snapshot
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
enabled: true
- metadata: {}
data:
filename: ./www/snapshots/{{ camera_name }}_snapshot2.jpg
target:
device_id: "{{ camera_id }}"
enabled: true
action: camera.snapshot
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
enabled: true
- metadata: {}
data:
filename: ./www/snapshots/{{ camera_name }}_snapshot3.jpg
target:
device_id: "{{ camera_id }}"
enabled: true
action: camera.snapshot
- metadata: {}
data:
prompt: >-
Motion has been detected, compare and very briefly describe what you see
in the following sequence of images from my {{ camera_name }} camera.
What do you think caused the motion alarm? If a moving person or moving
car is present, describe them in detail. Do not describe stationary
objects or buildings. If motion was caused by snow or rain reply with
"No Obvious Motion Detected". If you see no significant motion, or no
obvious causes of motion, reply with "No Obvious Motion Detected". Your
message needs to be short enough to fit in a phone notification.
image_filename:
- ./www/snapshots/{{ camera_name }}_snapshot1.jpg
- ./www/snapshots/{{ camera_name }}_snapshot2.jpg
- ./www/snapshots/{{ camera_name }}_snapshot3.jpg
response_variable: generated_content
continue_on_error: true
action: google_generative_ai_conversation.generate_content
- variables:
filtered_content: >-
{{ 'Google AI Error' if generated_content == empty else
generated_content.text }}
- if:
- condition: template
value_template: "{{ 'no obvious motion detected' in filtered_content | lower}}"
then:
- stop: ""
else:
- action: shell_command.fixup_jpeg
data:
image_file: ./www/snapshots/{{ camera_name }}_snapshot2.jpg
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
- metadata: {}
data:
title: "{{ camera_name }} Motion Detected"
message: "{{filtered_content}}"
data:
images:
- /config/www/snapshots/{{ camera_name }}_snapshot2.jpg
action: notify.family_sms
mode: single
description: ""
The important parts are the continue_on_error: true line before the generative ai call, and the variables block after the call. The filtered_content variable will contain “google AI Error” if the generative AI call failed, or the response text if it succeeded.
The check to see if no motion occured, and the actual notification message now need to use the new filtered_content variable and not the previous generated_content.text.
That’s all you need to do to at least get a notification that something occurred on your cameras even if Google fails. Good luck.
If you are still reading at this point, I also noticed something curious during my testing and I’m looking into it further. During my testing I was also trying out various prompts for Gemini (trying to get it to recognize that raindrops are not “motion”), and I noticed that when I was using a short prompt, I received fewer 429 Resource has been exhausted errors. I don’t know at this point if there really is a cause and effect relationship, or if it was just coincidence. I’m testing further and I’ll let you know.
Cheers!
EDIT - it was pointed out in another thread that my used of == empty only works by accident. The proper test should be is not defined.
Can I just say thanks this is awesome beyond belief… And updating the prompt to use Aussie slang is ‘bloody ripper mate’