Local realtime person detection for RTSP cameras

I run Frigate in a VM as well. USB passthrough of the Coral was problematic. I have sold that and switched to the PCIe card and inference times now vary between 6.5 to 8 ms with no problems.

Does this mean my usb coral is actively being used:

I get detector: coral even when using cpu. I would verify that it says TPU found on Frigate startup

Can you help me where to find that?

i have home assistant running as a virtual machine on a unraid server. Can i use frigate with the coral Mini PCIe Accelerator, or will it now be able to take advantage of the coral cpu?

I’m not sure how you have HA setup, but mine is supervised so I go to supervisor, frigate, log at the upper right hand corner and hopefully you’ll see something like this in there:

detector.coral                 INFO    : Starting detection process: 37
frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate.edgetpu                INFO    : TPU found
1 Like

Mine is running as docker container on ubuntu, how can I see there?

Those settings refer to the bounding box size, so I did a quick check and the person shown in your image, resized to 640x356px, appears to be about 81x148px=11988 (very close to the 12054 value shown). The person is also holding something, so the bounding box may be a bit larger than it would otherwise be.

Standing outside of the green box, as a best guess, I’m getting figures like 35x72=2520. With the min_area value set to 5000, it would ignore an object of this size and explain the situation you are facing. So reducing the min_area value should, in theory, allow Frigate to detect things outside of the green box (though maybe not as far as the gate, you’d have to test it).

This is indeed an extremely low resolution to try this with, and not all cameras are equal, however with my own setup I have 2 cameras with the min_area value set to 0. One of those cameras quite happily detects people at a distance of ~100m (as far away as someone could possibly stand in the image), with an object size of ~2464 @ ~76%. With the other camera (a wide angle, catch all camera), I’m seeing object area values as low as 800 @ ~65%.

So from my own real world testing and use, I’d recommend drastically lowering or removing the min_area value and seeing what results you get as it appears that Frigate is capable of reliably detecting extremely small objects in an image. You may want to configure masks etc, however, and then refine your settings based on the results.

Ok, should I change the -detect to to mainstream, 1920x1080 for detecting further away?

Not necessarily, simply reducing the min_area value should increase your detection distance from what I can tell, based on my own results. The default min_area value is actually 0.

# Optional: minimum width*height of the bounding box for the detected object (default: 0)
      min_area: 5000

Switching to 1080p would provide a significant improvement, though, and allow you to catch things with a much greater accuracy based on your field of view. It would be like catching people at the gate vs catching people in the distance approaching the gate, I think.

Why not add the main stream as a separate camera, and compare the results from both streams side by side?

I’m actually testing dual streams from one of my cameras at the moment, it’s quite a different scenario to your own with a much smaller FOV, but in my case there is no meaningful difference between detections. In fact, the 640x480 substream with no filters tends to outperform my refined 1080p main stream, and detections are consistently faster. Frigate is incredible :grin:

It’s Oct 2020 since I ran this configuration (Ubuntu + Frigate in Docker), but running “docker logs frigate” used to work.

Hey everyone,

Frigate is working great for me but for some reason it no longer seems to record events. I see the events on this screen & I get notifications via automations but if I click events or the person / car / etc buttons down the bottom there is either nothing there or one or two things randomly.

Any ideas? Thank you.

Works great for me and is a lot simpler than you might inagine.

I’m running Unraid with Home Assistant in a VM and Frigate in a Docker container using the Coral Mini PCI.

You need to install the Community Applications plug in if you haven’t already. Then search for Coral Accelerator Module Drivers and install it. Next, install the Frigate Unraid community application Docker container (not the HA Supervisor Add On).

All you have to do to make Frigate use the Coral Mini PCI is put the following in your Frigate config.yaml:

detectors:
  coral:
    type: edgetpu
    device: pci

Hi guys, is there a way to trigger object detection on a camera manually rather than through motion detection? For example, through a HA service call or API call.

Thanks for any ideas!

can someone help me out here, still doesn’t seem to work.

No. Motion must be detected first.

Anybody have links to buy coral PCI in europe? Thank you

https://www.rs-online.com/

How about RS?

1 Like

Hi, anyone with hikvision cameras can help me out?
I’m only getting green screen for hikvision rtsp stream (substream) and choppy images (main stream). I can see the steams working fine with vlc.

Anything i need to properly configure to get it to display properly?

mqtt:
  host: 192.168.2.120
  user: !secret mqtt_user
  password: !secret mqtt_password
  
cameras:
  backyard:
    ffmpeg:
      hwaccel_args:
        - -hwaccel
        - vaapi
        - -hwaccel_device
        - /dev/dri/renderD128
        - -hwaccel_output_format
        - yuv420p
      inputs:
        - path: rtsp://user:[email protected]:554/Streaming/channels/102
          roles:
            - detect
        - path: rtsp://user:[email protected]:554/Streaming/channels/102 
          roles:         
            - rtmp
    width: 720
    height: 576
    fps: 5
    objects:
      track:    
       - person
       - cat
       
  back_lawn:
    ffmpeg:
      hwaccel_args:
        - -hwaccel
        - vaapi
        - -hwaccel_device
        - /dev/dri/renderD128
        - -hwaccel_output_format
        - yuv420p
      inputs:
        - path: rtsp://user:[email protected]:554/Streaming/channels/102
          roles:
            - detect
        - path: rtsp://user:[email protected]:554/Streaming/channels/102 
          roles:         
            - rtmp
    width: 720
    height: 576
    fps: 5
    objects:
      track:    
       - person
       - cat

detectors:
  cpu1:
   type: cpu
  cpu2:
   type: cpu

device: mac mini late 2012 on debian
ipcamera: hikvision DS-2CD3145F
ha supervised

Try an extra “/” after 102 in the rtsp link first.