Local realtime person detection for RTSP cameras

I don’t do the https://your.public.hass.address.com part. I use VPN to get in from outside my network.
So I wanted to find a way to get the images from a local source (hence the camera snapshot service) and then send them out.
I missed the part in the Readme with the link to the different examples. I’ll definitely check it out.
Thanks

You can fetch the same images directly from frigate at /events/<id>/snapshot.jpg.

First, thank you so much for building this - it’s amazing.

I have the binary sensor working and its sending an image into hass for display via MQTT, but I cant get the notifications to my phone to display the URL from the frigate server.

I have HassIO running on NUC1 and Frigate running in docker on NUC2 (separate machine). Should I be using the integration? I was under the impression that the addon would install the frigate docker container on my hass server.

@blakeblackshear

Hi,

Great add-on - very good docs and easy to install. Beta Integration worked without any issues.

Any chance of the integration natively incorporating a binary sensor that turns on when a person is detected? That way we can use frigate really easily with Alarmo and other alarm based integrations. Moreover, lots of blueprints have revelry appeared that would be able to be used with s binary sensor too.

In the mean time is anyone able to help create a template binary sensor for me? :rofl:

Thanks!

EDIT: Any idea why the stream entity and camera.camera_person entity looks, well, not great? The stream entity doesn’t work play properly and looks like the thumbnail and also shows green smears. I chose the substream (640x480) for detection, rtmp and the 1080p main stream for clips and record. The thumbnails of the clips don’t look great either but the actual video files are fine

Have a look at the example notifications, they should give you enough to get started on writing a specific automation for notifications.

@Eeeeeediot Check out the docs for creating binary sensors. They aren’t created automatically, because there’s a lot that you can do custom for each one.

I have a few and they’re simple to create in the configuration.yaml file in HA:

binary_sensor:
  - platform: mqtt
    state_topic: "frigate/front_patio/person"
    device_class: motion
    availability_topic: "frigate/available"
    name: "Front patio person motion"

  - platform: mqtt
    state_topic: "frigate/driveway/person"
    device_class: motion
    availability_topic: "frigate/available"
    name: "Driveway person motion"

  - platform: mqtt
    state_topic: "frigate/front_yard/person"
    device_class: motion
    availability_topic: "frigate/available"
    name: "Front yard person motion"

@paulstronaut posted docs for 0.7.x, I posted docs from 0.8.x. So check the version you’re using and use the docs accordingly :slight_smile:

Ah yes. The docs links are different, but the MQTT binary sensor configuration is the same :slight_smile:

1 Like

Please forgive my ignorance here, but what is/where can I find the <id> in /events/<id>/snapshot.jpg?

thanks

@paulstronaut @hawkeye217

Thanks, I overlooked that part because I used the custom component!

Any idea why the video stream isn’t working correctly? It works fine in HA directly from the camera

I’m getting the ID from events like below in my telegram automation - using beta3. I didn’t see much about eventid in the beta4 documentation.

  - alias: 'frigate_telegram'
    trigger:
      platform: mqtt
      topic: frigate/events
    condition:
      - "{{ trigger.payload_json['after']['label'] == 'person' }}"
#      - "{{ states('input_boolean.isalarmon') == 'on' }}"
#      - "{{ (now() - state_attr('automation.frigate_telegram','last_triggered')).seconds > 30 }}"
    action:
      - service: notify.telegram_full
        data_template:
          message: 'A {{trigger.payload_json["after"]["label"]}} with ID {{trigger.payload_json["after"]["id"]}} was detected at {{ trigger.payload_json["after"]["camera"] }} {{ "\n" -}} found at: http://<frigateip>:5000/{{ trigger.payload_json["after"]["camera"] }}-{{trigger.payload_json["after"]["id"]}}/snapshot.jpg?h=720'
          data:
            photo:
              - url: 'http://<frigateip>:5000/events/{{trigger.payload_json["after"]["id"]}}/snapshot.jpg?format=android'
                caption : 'A {{trigger.payload_json["after"]["label"]}} was detected at {{ trigger.payload_json["after"]["camera"] }} '

When I get home I’ll try

- url: 'http://<frigateip>:5000/events/{{trigger.payload_json["after"]["id"]}}/snapshot.jpg?format=android'

But for Pushover instead of Telegram because I don’t use Telegram. The problem is I have to use the built-in HA downloader to first download that snapshot.jpg and then attach that local file to Pushover. Its not a big deal, just an extra step that isn’t documented here by Blake (though that’s not his fault). The devs of HA disabled the ability to attach outside URLs to things like Pushover claiming it was a security risk.

You might want to check the notification section in the beta documentation - I think black has better support for your case than what I posted:

            image: 'https://your.public.hass.address.com/api/frigate/notifications/{{trigger.payload_json["after"]["id"]}}.jpg?format=android'

This one might also be a good read:
https://github.com/blakeblackshear/frigate/discussions/406

You may not have to download the file since homeassistant can access it directly. Telegram sends the actual content of the image in the notification too, and I was able to give it that internal url without downloading.

I tried to copy this example for notifications that you have in the examples:

automation:
  - alias: Notify of events
    trigger:
      platform: mqtt
      topic: frigate/events
    action:
      - service: notify.telegram_full
        data_template:
          message: 'A {{trigger.payload_json["after"]["label"]}} was detected.'
          data:
            photo:
              # this url should work for addon users
              - url: 'http://ccab4aaf-frigate:5000/events/{{trigger.payload_json["after"]["id"]}}/snapshot.jpg'
                caption : 'A {{trigger.payload_json["after"]["label"]}} was detected on {{ trigger.payload_json["after"]["camera"] }} camera'

I don’t get anything through to telegram using it though.
Should we (beta addon users) be adding the -beta after ccab4aaf-frigate like we had to do for the integration in the URL here:
- url: 'http://ccab4aaf-frigate:5000/events/{{trigger.payload_json["after"]["id"]}}/snapshot.jpg'

When I try to view the snapshot in a browser from another PC on the network using the URL: http://<frigateip>:5000/events/{{trigger.payload_json["after"]["id"]}}/snapshot.jpg
I just get: Event not found

You need to add the -beta when using the beta addon. If you get “Event not found” then your event id is wrong. /events will return events from the database.

1 Like

So with Pushover, one can’t attach a file via a URL any longer since about the Spring of 2020. You must first use the downloader and download the file to the /config/www/ directory in the HA VM/container/whatever which is publicly available and whitelisted by HA. You can then attach the file this way and send it via Pushover with no problem. This is a known issue and the HA devs don’t seem receptive to ever allowing a URL attachment again. I don’t know why or how Telegram is able to use URL and Pushover can not.

So that means the only way to attach an image to pushover is to use the HA downloader to download it first. The issue I have here is that I don’t always get the correct snapshots.

Here is my full configuration.yaml:

automation:
  - alias: "driveway notfication"
    trigger:
      platform: mqtt
      topic: frigate/events
    condition: 
      - "{{ trigger.payload_json['after']['label'] == 'person' }}"
      - "{{ ( as_timestamp(now()) - as_timestamp(state_attr('automation.driveway_notfication', 'last_triggered')) |int(0) ) > 45 }}"
    action:
      - service: downloader.download_file
        data_template:
          url: 'http://<frigate-ip>:5000/events/{{trigger.payload_json["after"]["id"]}}/snapshot.jpg?format=android'
          filename: "snapshot.jpg"
          overwrite: true
      - service: notify.rodman_phone
        data:
          message: "Person in Driveway"
          data:
            attachment: /config/www/snapshot.jpg
            priority: 0
      - service: notify.mobile_app_sm_g950u
        data:
          message: 'A {{trigger.payload_json["after"]["label"]}} was detected in the driveway.'
          data:
            image: 'http://<HA-IP>:8123/api/frigate/notifications/{{trigger.payload_json["after"]["id"]}}.jpg?format=android'
            tag: '{{trigger.payload_json["after"]["id"]}}'
            data:
              ttl: 0
              priority: high

Once the conditions are met (person is detected) I then download the snapshot.jpg from the Frigate IP, attach it and send it to Pushover, and then send it again via the built-in HA notification just for good measure.

However, i’m not always getting the same snapshot.jpg being sent between the Pushover attachment and the HA notification. Most of the time, I’m behind one motion “event” with the Pushover snapshot.jpg attachment. In other words, Pushover is attaching an old snapshot.jpg file. I’m assuming that is because Pushover is acting too fast and Frigate hasn’t written out the “new” snapshot.jpg file yet or the HA Downloader service is too slow before Pushover attaches the one it sees and sends it off. I can somewhat prevent this by adding a delay of 1 second (- delay 00:00:01) to the above automation between the HA downloader and attaching the image to Pushover. That makes Pushover pause for 1 second in hopes that Frigate was able to write out a new snapshot.jpg.

I may have made this more complicated than it is, but I see no other way to attach an image to Pushover since I can’t take directly from a URL. Perhaps I need to setup Telegram instead.

I’m a bit confused by all this. I’m no programmer so that isn’t surprising!
I was under the impression that the {{trigger.payload_json["after"]["id"]}} was parsing the latest event to pull the id out and include that snapshot in the notification.
I can indeed go to /events and find an id number, then paste it into the URL http://<frigateip>:5000/events/the_id_I_got_from_events/snapshot.jpg and see the snapshot.
I can send it using the telegram integration in the services tab of HA with the following service data:

data:
  photo:
    - caption: lorem ipsum
      url: 'http://<frigateIP>:5000/events/<id>/snapshot.jpg'
message: lorem ipsum

So what can I do to troubleshoot the full URL of http://<frigateip>:5000/events/{{trigger.payload_json["after"]["id"]}}/snapshot.jpg not working?

Thanks

First of all - I really like Blakes approach of using different threads, e.g. in here. blakeblackshear/frigate · Discussions · GitHub
I’m sure that its just because of small brain problems - but I’m having trouble following multiple people doing similar, but different things here, potentially with different versions.

So what can I do to troubleshoot the full URL of
http://:5000/events/{{trigger.payload_json[“after”][“id”]}}/snapshot.jpg
not working?

That looks like a copy & paste from my config - but its yours & its not working? Can you share the full section including the frigate version that you’re using?

When i figured how to do it I had telegram send the URL instead of the actual picture, that way I could quickly figure out if I had typos. Just send the url as telegram text.

You could do the same thing with the entire payload.

ItsMee

How do you measure ‘behind’ here? The way you’re using eventid here the snapshot should be specific? Or do you mean that the event takes long enough to appear on your phone so another event was triggered meanwhile?