googleAI.yaml
alias: Camera 1 - Snapshot, AI & Notification
sequence:
- metadata: {}
data:
filename: ./www/snapshots/driveway1_snapshot1.jpg
target:
device_id: d8ccb88c128b0b47abd14a9a651f49a3
enabled: true
action: camera.snapshot
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
enabled: true
- metadata: {}
data:
filename: ./www/snapshots/driveway1_snapshot2.jpg
target:
device_id: d8ccb88c128b0b47abd14a9a651f49a3
enabled: true
action: camera.snapshot
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
enabled: true
- metadata: {}
data:
filename: ./www/snapshots/driveway1_snapshot3.jpg
target:
device_id: d8ccb88c128b0b47abd14a9a651f49a3
enabled: true
action: camera.snapshot
- metadata: {}
data:
prompt: >-
Motion has been detected, compare and very briefly describe what you see
in the following sequence of images from my driveway camera number 1.
What do you think caused the motion alarm? If a person or car is
present, describe them in detail. Do not describe stationary objects or
buildings. If you see no obvious causes of motion, reply with "No
Obvious Motion Detected." Your message needs to be short enough to fit
in a phone notification.
image_filename:
- ./www/snapshots/driveway1_snapshot1.jpg
- ./www/snapshots/driveway1_snapshot2.jpg
- ./www/snapshots/driveway1_snapshot3.jpg
response_variable: generated_content
action: google_generative_ai_conversation.generate_content
- if:
- condition: template
value_template: "{{ 'No Obvious Motion Detected.' in generated_content.text }}"
then:
- stop: ""
else:
- action: telegram_bot.send_photo
data:
file: /config/www/img/frigate/ai_parkering_2.jpg
caption: >-
{{ generated_content.text }} snap_id:
{{trigger.payload_json["after"]["id"]}}
title: PARKERING AI
data:
message_tag: "{{trigger.payload_json[\"after\"][\"start_time\"]|int}}"
mode: single
Iām doing this on mobile so indentation may be wrong. Test and if it doesnāt work, when Iām home I can do proper formatting. You should get the gist of the idea though. You need to define a action to call the service and pass it the correct data.
While Ive been sitting here I have it 80% working, Ill post my latest attempt up, its running without error, parsing the 3 photos and then forwarding one photo to Telegram BUT*** I no lo nger have the Google AI text but I cant be far away,
Im using the Visual Code editor which is now highlighting any formatting errors so Im learning as I go ā¦
I cant see if its actually going to Google AI as I cant get debugging to workā¦
Thanks for looking at this, Terry
alias: Telegram Test
sequence:
- metadata: {}
data:
filename: ./www/snapshots/garden_snapshot1.jpg
target:
entity_id: camera.garden_trackmix_fluent
enabled: true
action: camera.snapshot
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
enabled: true
- metadata: {}
data:
filename: ./www/snapshots/garden_snapshot2.jpg
target:
device_id: 7497f7c8e839a8c919a947fea2969f00
enabled: true
action: camera.snapshot
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
enabled: true
- metadata: {}
data:
filename: ./www/snapshots/garden_snapshot3.jpg
target:
device_id: 7497f7c8e839a8c919a947fea2969f00
enabled: true
action: camera.snapshot
- metadata: {}
data:
prompt: >-
Motion has been detected, compare and very briefly describe what you see
in the following sequence of images from my driveway camera. What do you
think caused the motion alarm? If a person or car is present, describe
them in detail. Do not describe stationary objects or buildings. If you
see no obvious causes of motion, reply with "No Obvious Motion Detected."
Your message needs to be short enough to fit in a phone notification.
image_filename:
- ./www/snapshots/garden_snapshot1.jpg
- ./www/snapshots/garden_snapshot2.jpg
- ./www/snapshots/garden_snapshot3.jpg
response_variable: generated_content
action: google_generative_ai_conversation.generate_content
- if:
- condition: template
value_template: "{{ 'No Obvious Motion Detected.' in generated_content.text }}"
then:
- stop: ""
else:
- metadata: {}
action: telegram_bot.send_photo
data:
title: Garden Motion Detected
message: "{{generated_content['text'] }}"
data:
authentication: digest
file: /config/www/snapshots/garden_snapshot2.jpg
mode: single
iām trying to setup the second script as described, but Iām running into following error:
Message malformed: extra keys not allowed @ data['sequence']
alias: Camera - Driveway 1 - Snapshot, AI & Notification
sequence:
- variables:
trig_ts: "{{ now().strftime('%Y%m%d-%H%M%S') }}"
- service: camera.snapshot
metadata: {}
data:
filename: ./www/snapshots/door_{{trig_ts}}_snapshot1.jpg
target:
device_id: 423c54b976c3a55b3768edf3062b29c3
enabled: true
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
enabled: true
- service: camera.snapshot
metadata: {}
data:
filename: ./www/snapshots/door_{{trig_ts}}_snapshot2.jpg
target:
device_id: 423c54b976c3a55b3768edf3062b29c3
enabled: true
- delay:
hours: 0
minutes: 0
seconds: 0
milliseconds: 500
enabled: true
- service: camera.snapshot
metadata: {}
data:
filename: ./www/snapshots/door_{{trig_ts}}_snapshot3.jpg
target:
device_id: 423c54b976c3a55b3768edf3062b29c3
enabled: true
- service: google_generative_ai_conversation.generate_content
metadata: {}
data:
prompt: >-
Motion has been detected, compare and very briefly describe what you see
in the following sequence of images from my driveway camera number 1.
What do you think caused the motion alarm? If a person or car is
present, describe them in detail. Do not describe stationary objects or
buildings. If you see no obvious causes of motion, reply with "No
Obvious Motion Detected." Your message needs to be short enough to fit
in a phone notification.
image_filename:
- ./www/snapshots/door_{{trig_ts}}_snapshot1.jpg
- ./www/snapshots/door_{{trig_ts}}_snapshot2.jpg
- ./www/snapshots/door_{{trig_ts}}_snapshot3.jpg
response_variable: generated_content
- if:
- condition: template
value_template: "{{ 'No Obvious Motion Detected.' in generated_content.text }}"
then:
- stop: ""
else:
- service: notify.mobile_app_iphone_patrick
metadata: {}
data:
title: Driveway 1 Motion Detected
message: "{{generated_content['text'] }}"
data:
image: /local/snapshots/door_{{trig_ts}}_snapshot2.jpg
mode: single
Fist thing I can advise is use the Visual Code Editor in HA as that will highlight all the errors just in spacing / indentation, its a whole new worldā¦
Firstly, thankyou to fellow Aussie @Aussie_Adam for starting this guide and really inspiring me to do this project for myself. Thankyou to everyone who has contributed to this thread as Iāve used bits and pieces of it to refine my own script as Iāve gained an understanding of what each part does, in particular thanks for alerting me to the cached image issue when using Cloudflared, and the need to timestamp my snapshots, and thanks for @BambamNZ for mentioning the āClearā stream that available on Reolink cameras.
In the interest of sharing and hopefully helping out, hereās a couple of my own personal tips (as a HA native automation and script newbie) Iāve picked up along the way:
I would start off with just doing the SCRIPT component first and running it from the menu to make sure that the SCRIPT can run standalone.
For those (like myself) that donāt use the HA native AUTOMATION this then means no need to implement the AUTOMATION. I use NodeRED and just call the SCRIPT from there.
If youāre using NodeRED and want to purge the snapshot directory periodically, you can just use the INJECT node and set the interval (say hourly) to trigger an EXEC node with the command:
rm -f /homeassistant/www/snapshots/*.jpg
I found the HA Companion App with its tiny snapshot and instantly disappearing nature very unfriendly for my use case, so I (as a non-Signal/Telegram user) I implemented the notification to go via Whatsapp to a group chat (where I can add or remove members as need be).
b) If you link the API step with your personal phone number you wonāt get notified (notification bar, or sounds) of new messages on the chat, so you may want to buy a cheap long life SIM (since it will never make calls)
c) So that you donāt fill your phone storage, as the admin of the group you can set the messages to disappear by default after a set period (say 7 days) and you can also mark individual messages to be kept regardless of the default.
d) You may have troubles finding the right path reference for the snapshot - the following YAML may be useful to you in your SCRIPT:
