Generative AI camera snapshot notification

Hi.
Would be great if there was an option to choose another sensor than motion to the trigger. Examble i have person and car detection on my cameras and would like to have a snapshot only when a person is detected in my yard and not from every motion detection. :blush:

1 Like

How can i select a different sensor, my Reolink is facing the street and i dont need a description of every car that drives by but if i can activate on person detection instead of all Motion it will be a lot better, same for cars vs trees moving in the wind and pets etc. Seems only motion binary sensors are allowed not any other binary sensor. How can we change this?

The Reolink should expose entities for vehicle, person:

      - binary_sensor.[[YOUR CAMERA]]_vehicle
      - binary_sensor.[[YOUR CAMERA]]_person

It should respect masked area.

I have two cameras at my gate, one for plates/gate one to see inside the car and talk. Automation with AI and then sending notifications to two phones (uses a blueprint for the notification):

alias: Gate Vehicle Camera Alerts
description: ""
triggers:
  - entity_id:
      - binary_sensor.rc_intercom_vehicle
      - binary_sensor.rc_intercom_person
    id: intercom
    from: "off"
    to: "on"
    trigger: state
  - entity_id:
      - binary_sensor.rc_gate_vehicle
      - binary_sensor.rc_gate_person
    id: gate
    from: "off"
    to: "on"
    trigger: state
conditions: []
actions:
  - action: camera.snapshot
    metadata: {}
    data:
      filename: ./www/snapshots/rc_gate_fluent_snapshot.jpg
    target:
      entity_id: camera.rc_gate_fluent
  - action: google_generative_ai_conversation.generate_content
    metadata: {}
    data:
      image_filename: ./www/snapshots/rc_gate_fluent_snapshot.jpg
      prompt: >-
        Motion has been detected, very briefly describe what you see in this
        image.  If a person or vehicle is present, describe them in detail.
        Don't describe stationary objects or buildings. Keep the response short.
    response_variable: result
  - parallel:
      - metadata: {}
        data_template:
          field_notify_device: 6e7ef8ed17852d13dc932de01891ac75
          field_title: "{{reason}} at <font color='red'>{{camera_location|title}}</font>"
          field_message: "{{ result.text }}"
          field_subtitle: ""
          field_attachment_type: camera_entity
          field_attachment_camera_entity: camera.rc_{{camera_location}}_fluent
          field_timeout:
            hours: 0
            minutes: 2
            seconds: 0
          field_notification_link: app://com.mcu.reolink
          field_confirm_uri: app://com.mcu.reolink
        action: script.gate_vehicle_motion_detection
      - metadata: {}
        data_template:
          field_notify_device: fc4d03d44446c906dbe5c611795fdb98
          field_title: "{{reason}} at <font color='red'>{{camera_location|title}}</font>"
          field_message: "{{ result.text }}"
          field_subtitle: ""
          field_attachment_type: camera_entity
          field_attachment_camera_entity: camera.rc_{{camera_location}}_fluent
          field_timeout:
            hours: 0
            minutes: 2
            seconds: 0
          field_notification_link: app://com.mcu.reolink
          field_confirm_uri: app://com.mcu.reolink
        action: script.gate_vehicle_motion_detection
variables:
  camera_location: "{{trigger.id}}"
  reason: "{{trigger.entity_id.split('_')[3] | title}}"
mode: parallel

I use this for the notification:

So I love this, and really appreciate that you came up with it.
Anyone have an idea on how to log these so that you could have a dashboard of all the detections in the last 24h? Would you. have to write a custom integration and custom JSON file or something to keep track? Or does HA have a facility for something like this.

Hello, I have the same issue with iOS notifications. A long press allows me to see the notification, but if I just tap once, it opens the app, and no notification is readable. Couldn’t we have an option to make notifications persistent? Thanks, great work, by the way!

1 Like

My temporary workaround on my Pixel 8 Pro is using a phone app called “Recent Notification” by LibApps. It shows all recent phone notifications in full, and shows the image and full notification text. As I said, temporary workaround, but at least I can see all the text.

1 Like

Good morning, your work is very nice, I have two small problems. The first is that I can’t see the message after the three dots (like the previous posts). The second is that in the message that I receive the AI ​​gives the correct description of the snapshot photo but it always attaches an old photo.
is it possible to change the snapshot path?
thanks

Hey guys,

I’ve used this for months now and it works really really well.
Unfortunately I have seen that the amount of request is too high and I will now have to pay for the requests from Google.

I’ve turned it down to 3 snapshots, but I still see this on my google billing:

The funny thing though is, if I look at the “Generative Language API” I only see 759 requests the last 30 days :question:

I don’t use Generative AI to anything else than this Automation.

Does anyone know why this behavior and what can I do about it?