Local realtime person detection for RTSP cameras

OK, I got the RTMP stream working for my sub stream, but do I have to add the main stream as a different camera? They obviously have different heights and widths and FPS, as well as different roles

Do I need to do something like front_door_sub and fill that out and another one named front_door_main?

Have a look at the example. You can add multiple camera streams as inputs and specify the role you want for each input.

yes, thats where I am basing all of my work on, trust me, Id much rather have this working on my own instead of asking for help on the forum.

however, my substream has a different height/width/fps from my main stream, when inputting two paths like the example, and input a height/width/fps for either one of my streams, I get a broken frame error.

If I remove one input for example the main stream, and give it the correct width/height/fps it works, however, I want to set it up as in the example with each cam having a sub/main stream.

The fps and resolution only applies to the detect input. Everything else is pass-through from the camera. Check the comments in the full example config.

Also, if youā€™re getting a broken frame error, you can enable debugging to see exactly whatā€™s going wrong.

Well, iā€™ve been using it on my ZoneMinder setup with various results.
I finally made shut down on that system and handed over the surveillance work to Frigate!
With that saidā€¦ i now have a free Coral laying around.

Does a second Coral have any benefit to Frigate?

Iā€™ve just had a look through the documentation again and couldnā€™t find it. Is it possible to change the date format to day/month/year ? Also how do i show the media browser in the sidebar as shown in some of the examples in the documentation?

Absolutely. You can add a second coral in the detector config. Both will process the detection queue in parallel, so your max fps will be over 200fps if you are getting <10ms inference times.

1 Like

If we up the fps does this affect CPU? What are the advantages of a higher detect fps? Iā€™ve got a nuc style mini pc which has a slow Intel n4000 CPU but itā€™s cheap and has 2 x m.2 PCIe slots and Sata! So naturally Iā€™ve got both slots with corals in now :grin:

From what I can tell, from my experiments with Frigate, Coral and docker is that the M.2 versions need the driver installing. HassOS doesnā€™t have this afaik. The USB version however, doesnā€™t need the driver. Iā€™m now running on Openmediavault which is Debian but I had to install the proxmox kernel to get the Coral drivers to compile correctly. OMV is pretty nice though, most things are one click install and my Frigate box now doubles up as a file server!

Thanks for the infoā€¦ 2 corals warming up the USB ports at the momentā€¦ :slight_smile:

VLC can open a directory and will play the 1 min files in order. You can increase playback speed to scan through.

Just out of curiousity is it possible to use Frigate from HASS if Frigate is a seperate PC lest say a remote machine? Some configs?

Yes I can run frigate docker on remote machine and point the component in Hass to this server where frigate is running

Hi there, Is there a way to specify a custom folder to save snapshots for each camera?

1 Like

I ended up doing this, although it isnā€™t as quick or as painless as BI makes it to scrub through days of video at a fast speed.

I have built locally amd64nvidia and it is running well, each ffmpeg consumes 4% cpu, this is on an old quad core haswell with nvidia GT640.
I want to run this as an addon to get the ingress functionality. While looking for a solution I see that the amd64nvidia build is on docker hub. So I thought it may be easier to try to get home assistant pull that instead, so I changed Dockerfile in addons/git/ccab4aaf to reference that. Then I guess there are other options needed, like the NVIDIA* environment variables, also runtime=nvidia. I am having trouble mapping the docker run command I use to config.json - Is that the best way?

edit: adding nvidia on the end the the line on Dockerfile causes home assistant to pull that image from the docker hub. However ffmpeg will not run with -c:v h264_cuvid because it wants libncuvid. I think this is to do with passing the gpu into dockerā€¦

edit2: Got it working by adding the NVIDIA env stuff to config.json and adding a default runtime of nvidia in /etc/docker/daemon.json as documented in this thread.

1 Like

Hello.
I want my clip recordings to be longer, I am finding one event gets spread over two or three clips.
Part of the problem may be that the object (such as a delivery person) walks into the front gate up to the door and then disappears when they are immediately under the porch. They then ring the bell and leave the package, and even though itā€™s only a few seconds this whole sequence gets cut into two or three clips.

Iā€™ve tried playing with min_score and thresholds, max_disappeared, and max_seconds. But I am not having much success so far. I have max_disappeared set to 5fps x 10s = 50, and I thought that was the key but I have not seen a difference.

Can I do what I am looking for and what are the key parameters I should be using?

Hereā€™s my config?

# Detector configuration and hardware acceleration flags for RPi4 64-bit
detectors:
  cpu1:
    type: cpu
  cpu2:
    type: cpu

ffmpeg:
  hwaccel_args:
    - -c:v
    - h264_v4l2m2m

# What to look for and how much confidence to require
objects:
  track:
    - person
    - dog
    - car
  filters:
    person:
      min_score: 0.5
      threshold: 0.7
    dog:
      min_score: 0.4
      threshold: 0.6
    car:
      min_score: 0.5
      threshold: 0.7

cameras:
  front:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.50.135:554/12
          roles:
            - detect
            - rtmp
        - path: rtsp://192.168.50.135:554/11
          roles:
            - record
            - clips
    width: 640
    height: 352
    motion:
      mask:
        - 640,0,640,80,0,82,0,0
    clips:
      enabled: true
    snapshots:
      enabled: true
    fps: 5

detect:
  # Number of frames without detection to consider object gone 10s x 5fps = 50
  max_disappeared: 50

clips:
  max_seconds: 300
  retain:
    default: 10

side note - Raspberry Pi 4, running only CPU detectors and the ffmpeg acceleration options. One camera at 5fps and the baseload is sitting on about 0.6 (Homeassistant has a bunch of other appdaemon automation too).

This might be useful to others. Its an IOS notification to the HA app, where the notification action opens the application to frigate. It uses the single mode and a delay to prevent rapid retriggers which is simpler to get working than a templat condition. I wasnā€™t able to get notifications with embedded working when remote from my home network, but this is a good workaround. It would be nice if I could give the URL the event ID too, but the events page does not seem to be URL addressable.

- id: "1616359370503"
  alias: Notify of frigate object detection and send clip
  description: ""
  trigger:
    - platform: mqtt
      topic: frigate/events
  condition:
    - "{{ trigger.payload_json['type'] == 'new' }}"
  action:
    - service: notify.mobile_app_jason_s_iphone
      data:
        message: "A {{trigger.payload_json['after']['label']}} ({{ (trigger.payload_json['after']['top_score']| float * 100) | round(0) }}%) was detected on the {{trigger.payload_json['after']['camera']}} camera"
        data:
          url: /xxxxxxxx_frigate/dashboard
    - service: notify.mobile_app_leonas_iphone
      data:
        message: "A {{trigger.payload_json['after']['label']}} ({{ (trigger.payload_json['after']['top_score']| float * 100) | round(0) }}%) was detected on the {{trigger.payload_json['after']['camera']}} camera"
        data:
          url: /xxxxxxxx_frigate/dashboard
    - delay: 180
  mode: single
  max_exceeded: silent
2 Likes

Hi @jasebob,
if i properly understand you are running Frigate on a Rasperry pi 4, thatā€™s correct?
itā€™s the raspi4 dedicate and doesnā€™t host the HA, correct?

if thatā€™s true, what raspi OS are you running? and what docker image are you using?

regards

So you open Frigate from HASS itself? Now that is amazing, would you mind share config where exactly it is set and how does it show? In sidebar also? :open_mouth: much :heart:

Are u on discord? My name is thundergreen Iā€™ll guide you if you want