Local realtime person detection for RTSP cameras

I have a couple of detected people with only the upper part of the body visible, but I can’t understand why it never detects a postman :slight_smile:
My fence is 160 cm high and I assume that the postman walks close to the fence and can only see his head.

- alias: Send end event notification
  trigger:
    platform: mqtt
    topic: frigate/events
  condition:
    - "{{ trigger.payload_json['after']['entered_zones']|length > 0 }}"
    - condition: state
      entity_id: input_boolean.notification_mobile
      state: "on"
    - "{{ trigger.payload_json['type'] == 'end' }}"
  action:
    - service: notify.mobile_app_s10
      data_template:
        message: A {{trigger.payload_json["after"]["label"]}} ({{ (trigger.payload_json["after"]["top_score"] | float * 100) | round(0) }}%) was detected on the {{trigger.payload_json["after"]["camera"]}} camera.
        data:
          image: https://xxx.ui.nabu.casa/api/frigate/notifications/{{trigger.payload_json["after"]["id"]}}/thumbnail.jpg?format=android
          tag: '{{trigger.payload_json["after"]["id"]}}'
          clickAction: https://xxx.ui.nabu.casa/api/frigate/notifications/{{trigger.payload_json["after"]["id"]}}/snapshot.jpg
          group: Front Motion
          channel: frigate
          color: "#2DF56D"
          actions:
            - action: URI
              title: View Video
              uri: https://xxx.ui.nabu.casa/api/frigate/notifications/{{trigger.payload_json["after"]["id"]}}/{{trigger.payload_json["after"]["camera"]}}/clip.mp4
            - action: URI
              title: View Snapshot
              uri: https://xxx.ui.nabu.casa/api/frigate/notifications/{{trigger.payload_json["after"]["id"]}}/snapshot.jpg
            - action: URI
              title: HomeAssistant
              uri: app://io.homeassistant.companion.android
2 Likes

Not that it will help, but I had it detect the postal worker IN the truck the other day in addition to the vehicle, LOL. Wasn’t expecting that :slight_smile:

1 Like

Brilliant. Thanks a bunch!

Thanks. Works great!

Odd issue. The date shown for clips in media browser is 8 hours ahead of the time of the clips. The frigate NVR tab shows the correct time for the clips. My lovelace UI has an analog clock that displays the correct time.

Host OS - date (this is the correct date)
dtla@homeassist : ~ $ date
Mon Mar 1 07:41:59 PST 2021

Host OS - timedatectl

dtla@homeassist:~$ timedatectl 
               Local time: Mon 2021-03-01 07:43:29 PST
           Universal time: Mon 2021-03-01 15:43:29 UTC
                 RTC time: n/a
                Time zone: America/Los_Angeles (PST, -0800)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

Interesting observation. timedatectrl in Host OS “RTC in local TZ= no”. I thought this may be the answer but when changing to yes I got this warning

Warning: The system is configured to read the RTC time in the local time zone.
         This mode cannot be fully supported. It will create various problems
         with time zone changes and daylight saving time adjustments. The RTC
         time is never updated, it relies on external facilities to maintain it.
         If at all possible, use RTC in UTC by calling
         'timedatectl set-local-rtc 0'.

I changed the RTC back to Universal time for host stability.
So it seems like media clips in the media browser are using Universal time and when viewed from Frigate they display Local time. All other Home Assistant time seems to be in Local time.

Update:
I also found that timestamps applied by frigate are also on UTC and not local time

Am I missing something? Is this a frigate or a HA issue?

What’s your configuration (camera resolution, person: min_area, max_area, min_score and threshold)?
Do you have coral?

Hi,

i have some issue with the automation, if i add this image
image: ‘http://IP:5000/api/camera1/person/best.jpg

it works, but if i try the string from the documentation

http://IP:5000/api/frigate/notifications/{{trigger.payload_json[“after”][“id”]}}/thumbnail.jpg?format=android

it doesn’t. i get the message but not the picture

i also tried this

http://IP:5000/api/{{trigger.payload_json[“after”][“camera”]}}/notifications/{{trigger.payload_json[“after”][“id”]}}/thumbnail.jpg?format=android

but i still not getting the image

any ideas?

regards

I was actually able to recover the old configuration. In the addon configuration tab, I clicked the three dots in the upper right corner and chose Edit in YAML. I have no idea where it dug it up from, but the configuration appeared in the text area. I quickly copied it to safety.

1 Like

The endpoints you are trying are for the homeassistant integration. You need to use your homeassistant host, not ip:5000. See the http section of the docs for the corresponding frigate api endpoints.

I struggled with similar issue on iOS for a day or so. Found this guide very helpful

Read the whole section on attachments. I’m sure you’ll find your answer there as well as other cool tricks.

Sorry, I think I’m confused
When I access the api to get person/best.jpg I’m pointing to the frigate instance, but the other api is on the HA instance; correct?

The documentation provides full details but the gist of it is:

/api/... endpoints are for your Frigate IP
/api/frigate/... endpoints are for your HA IP

https://blakeblackshear.github.io/frigate/usage/api
https://blakeblackshear.github.io/frigate/usage/home-assistant/

1 Like

Thank you. Now is clear

This is what I’m doing right now. Detect on substream, clips and record on mainstream. If I’m correctly understanding what you’re trying to to this part of the documentation should help you.

Hi all
Sorry for dumb and OT question, why retain all mqtt messages from frigate? Is this something configurable?
Now I want to purge my mqtt from retained snapshot, do you have some magic command to clear all frigate topic?

Thank you
Dario

Seem to be getting a lot of these errors in the logs after a while, any ideas please where to look?

ffmpeg.drivewaycam.detect
ERROR   : [segment @ 0x55bea6f77000] Non-monotonous DTS in output stream 0:0;
previous: 210092538, current: 210092463; changing to 210092539.
This may result in incorrect timestamps in the output file.

Running Frigate 0.0.1 integration and Frigate 1.13 NVR on latest HA

This is part of frigate.yml:


ffmpeg: {}
cameras:
  drivewaycam:
    ffmpeg:
      inputs:
        - path: 'rtsp://user:[email protected]:554/ch1/sub/av_stream'
          roles:
            - detect
            - clips
    height: 288
    width: 352
    fps: 5
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true

Thanks in anticipation.

1 Like

@dapuzz

I use MQTT-Explorer for that (http://mqtt-explorer.com/), nice tool for exploring and removing MQTT topics

Same as @ValnillaJack, this is the exact same thing I’m investigating.

If the driver were in the HassOS running on the NUC (or laptop in my case) and the coral is a pcie device, is that all that is needed for Frigate to pick it up and run with it? It looks like the driver code is in buildroot, just not enabled for the HassOS build.

I recently re-activates frigate and I’m really impressed of the progress the project made since my last try some month ago.
Its now super simple to get all the needed info.

On the previous solution with shinoby i manually saved an image locally to the pi if a person was detected and could show all this images in a nice Gallery Card

With the frigate component i have the clips in the media browser, but how to show them in a gallery card in lovelace?

If i ran this in my hassio running in win10 inside a Linux virtual box (installed frigate as an add-on on hassio), can i store the data on a hard drive? From my understanding a virtual box only uses Ram for storage…

I keep getting low disk space errors after adding frigate…
Even though i dedicated 5gb from my Ram in virtual box

Available space is less than 1GB!

Available space is 0.0GB, see https://www.home-assistant.io/more-info/free-space for more information.