Camera Detection of FedEx Truck or Unique Object?

Configuration
, ,
1

What’s up everyone!?
I’m super pissed at FedEx right now and I’m not going to get into it LOL
But I just thought it would be awesome if home assistant had the ability to leverage my Reolink camera(s) to detect unique motion of a specific object, in this case the FedEx truck, and then notify me.
Is there a way to train home assistant with an integration and AI to accomplish this?

2

Frigate NVR

Version 15 you can search and get reasonable results for things like “white truck” or “FedEx” with the free version

Supposedly paid version does this but I never used

1 Like
3

Hello Lee,

This is interesting. Planning on trying it soon.

2 Likes
4

I wonder if that’s possible with my Reolink NVR. I’ll research that anyhow.

5

Send any picture to the right AI with the right engine running and it will do that. Some are better at images than others.

1 Like
6

Tbh e NVR has rtsp restream that you can use or direct from camera rtsp stream

Sir_goodenough is correct. You can send an image to any number of ai engines and have it give what you want

1 Like
7

Thanks for this link - this works great for me :+1:

2 Likes
8

So I am testing this blueprint as well and it is successfully detecting U-Haul trucks with my Reolink doorbell camera.
Now I just need to figure out how to incorporate a conditional rule to notify using a different Android notification channel and only if photo AI response include keyword ‘FedEx’ truck.

9

Hello everybody.

Can anybody tell us step by step about this installation and integration? I want to translate other languages and make some changes.

I need help. Thank you

10

This is what I followed, read this: Generative AI camera snapshot notification

11

I got around to playing to taking control of this blueprint (Generative AI camera snapshot notification) and editing the yaml to only notify if there is a Fedex truck present:

alias: Driveway camera AI Detection - away (fedex)
description: ""
triggers:
  - entity_id:
      - binary_sensor.driveway_motion
    to: "on"
    trigger: state
    for:
      hours: 0
      minutes: 0
      seconds: 15
conditions:
  - condition: and
    conditions:
      - condition: state
        entity_id: input_boolean.away
        state: "on"
actions:
  - if:
      - condition:
          - type: is_motion
            condition: device
            device_id: 5180f5ecf43ce6278ee0c5a209c5b5ad
            entity_id: b224ece5e6d5eede10e2b7c6d477f582
            domain: binary_sensor
    then:
      - repeat:
          count: "{{ num_snapshots }}"
          sequence:
            - data:
                filename: >-
                  ./www/snapshots/{{ camera_path }}_snapshot{{ repeat.index
                  }}.jpg
              target:
                entity_id: "{{ camera_device }}"
              action: camera.snapshot
            - delay:
                hours: 0
                minutes: 0
                seconds: 0
                milliseconds: 500
      - data:
          prompt: "{{ ai_prompt }}"
          image_filename: >
            {% set snap_count = 3 %} {% set ns = namespace(images=[]) %} {% for
            i in range(1, snap_count + 1) %}
              {% set image = "./www/snapshots/" ~ camera_path ~ "_snapshot" ~ i ~ ".jpg" %}
              {% set ns.images = ns.images + [image] %}
            {% endfor %} {{ ns.images }}
        response_variable: generated_content
        action: google_generative_ai_conversation.generate_content
      - choose:
          - conditions:
              - condition: template
                value_template: |
                  {{ "fedex truck" in generated_content['text'] | lower }}
            sequence:
              - device_id: c648fb71087f55213f355094c5828664
                domain: mobile_app
                type: notify
                title: "{{ motion_name }} Detected"
                message: "{{ generated_content['text'] }}"
                data: >
                  {% set android_data = {"image": snapshot_access_file_path} %}
                  {% set ios_data = {"attachment": {"url":
                  snapshot_access_file_path, "content_type": "JPEG"}} %} {{
                  ios_data if is_ios else android_data }}
  - delay:
      hours: 0
      minutes: 10
      seconds: 0
variables:
  camera_device: camera.driveway_fluent
  camera_name: "{{ state_attr(camera_device, 'friendly_name') }}"
  camera_path: "{{ state_attr(camera_device, 'friendly_name') | lower | replace(' ', '_') }}"
  motion_sensor: binary_sensor.driveway_motion
  motion_name: "{{ state_attr(motion_sensor, 'friendly_name') }}"
  is_ios: false
  num_snapshots: 3
  snapshot_access_file_path: /local/snapshots/{{ camera_path }}_snapshot1.jpg
  ai_prompt: >
    Motion has been detected, compare and what you see in the following sequence
    of images from my {{ camera_name }} camera. Only respond if you see a FedEx
    truck present, by saying "FedEx truck is here". Do not describe stationary
    objects or buildings. If you see no obvious causes of motion, do not reply.
    Your message needs to be short enough to fit in a phone notification.
  condition_auto:
    - type: is_motion
      condition: device
      device_id: 5180f5ecf43ce6278ee0c5a209c5b5ad
      entity_id: b224ece5e6d5eede10e2b7c6d477f582
      domain: binary_sensor
mode: single