Local realtime person detection for RTSP cameras

Yes, thank you - but “_all” always shows “0 objects” and “_all_motion” is always “off”. I also tried adding the “all” object to the detection list (as I see references to it in the integration code) - it made no difference. Essentially I’m just trying to get “motion” events from Frigate in addition to “object detection” events. I opened a GitHub issue for this - thought I’d ask here in case it was something simple.

Issue resolved - these sensors are meant for occupancy and require frigate .11. But the motion sensors I really need will come with frigate integration 3.0.

Just figured I’d add to the thread that I was able to get the mini PCIe Coral working using the StarTech PEX2MPEX that @ukro had mentioned. My motherboard is the Gigabyte Aorus Z390 Pro Wifi and I’m running Ubuntu. Just needed to follow the instructions on the Coral website to get the driver installed.

Note for anyone else that adds this card with this mobo, after installing it and turning the power back on, the board flashed some white lights about 5-10 times over the course of about 30-45 seconds. During this time the machine would not boot - this cost me a lot of time being impatient. Just give it a minute, eventually you should see a green light on the PCI board turn on. The board should be ready to boot then.

I found the mini PCIe to PCIe board at Provantage for $33. It’s on Amazon for a few bucks more.
It required a small tweak to fit on the board though :joy:

1 Like

@Jon123 Mine is better :stuck_out_tongue: :smiley:


1 Like

Potentially dumb question but I couldn’t see an answer in the documentation:

Is there a way to set the main stream of the camera to be used for snapshots?

The current sub stream is not great for snapshots and I’m not super keen on recording as I’m using a Raspberry Pi & don’t want the excessive disk writes.
I can see you can set a different stream for record but I don’t see an option to do the same for snapshots

I read something along the line of…

You can use a higher resolution for detect and set the detect resolution (width and height) lower. By doing this the snapshots, etc have higher resolution but detect uses the lower resolution that was set.

Doing this puts greater strain on cpu.

I haven’t tested this but sound doable

EDIT

A question and a bug…
How can I use the RTMP stream from Frigate? Like in a card for viewing.
The bug:
I was trying to use the RTMP feed in a generic camera integration. Pasting “rtmp://192.168..*/live/driveway_cam” into the stream source crashes Home Assistant instantly when clicking submit. Done several times to verify.
I running HAAS OS on a bare metal PC.

Set up a generic camera using the feed from the camera. Ignore the snapshot line in the configuration. I could never get that to work…
In the snapshot automation configuration, use entity, not device. EG:

service: camera.snapshot
data:
  filename: //media/images/temp.jpg
target:
  entity_id: camera.porch_generic

For UK based folk - RS currently have stock of USB coral devices:

https://uk.rs-online.com/web/p/development-tool-accessories/2017821

FWIW I’ve used a cheap pcie to mini pcie adapter board designed for wifi cards like this one:

Watch out for versions that only have screw holes populated for full length boards though.

2 Likes

hmm i was considering that, but when it had the wifi antena i tought it will not work. But nice. Thx.

I just ditched the whole bracket including the antenna sockets to keep it entirely internal.

1 Like

@ukro You do have me beat! I was staring at the thing for a few minutes after excitedly opening the package then going “well… shit, how am I gonna get this to stay?” and for some reason zip ties never came to mind - but cutting and drilling some thin metal apparently made a lot of sense at the time.

Show off :wink:

2 Likes

@Jon123 xD ahahaha :smiley:

Did you ever create a model for trash can detection? I was thinking about trying the same thing, but I have a lot to learn before I get that working.

Hey All,
I’m using 2 streams for frigate. 1 to detect (low quality) and the other to record (high quality). When getting a snapshot, it’s always take a snapshot using the low quality stream. Is there a way to get the high quality snapshot? Using them for face recognition…

@blakeblackshear There is a new Jetson called Orin with support of 22x 1080p30 (H.265) video decode. https://www.waveshare.com/jetson-agx-orin-developer-kit.htm It isnt cheap but maybe the best solution for large number of cameras and ARM based low power consumption. Will it work with Frigate in a way that capable of utilizing the hardware acceleration of Orin?

Maybe someone is able to point me in the right direction.

Just started with frigate and currently trying to get going with 0.11.0 beta (full access).
Got a Reolink Duo in the garden and added both cameras to my frigate instance on a Home Assistant OS, Rpi 4, Google Coral USB setup.

Everything is working well, except for HW acceleration and without HW acceleration the cpu usage of frigate is at 90%.

Log is telling, that frigate cannot allocate the neccessary gpu memory. I added gpu_mem=256 to my config.txt on the boot partition. Also tried 512mb without success.

[2022-08-01 11:45:05] ffmpeg.reolinkduo2.detect      ERROR   : Error while decoding stream #0:0: Cannot allocate memory

Here is my current fresh frigate.yml:

mqtt:
  host: 192.168.1.21
  user: XXXX
  password: XXXX
  
detectors:
  coral:
    type: edgetpu
    device: usb
  
cameras:
  reolinkduo1:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/h264Preview_01_main
          roles:
            - detect
            - rtmp
      hwaccel_args: -c:v h264_v4l2m2m
    detect:
      width: 640
      height: 360
      fps: 7
    snapshots:
      enabled: True
      
      
  reolinkduo2:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/h264Preview_02_main
          roles:
            - detect
            - rtmp
      hwaccel_args: -c:v h264_v4l2m2m
    detect:
      width: 640
      height: 360
      fps: 7
    snapshots:
      enabled: True
      
objects:
  track:
    - person
    - dog
    - cat
  filters:
    person:
      threshold: 0.75

Also tried h264_mmal, because I am running a 32-bit instance on my Rpi 4 but unfortunately without success. Then the log is displaying another error, that h264_mmal cannot be found. I think it is not part in the latest beta of frigate / latest version of ffmpeg.

Thank you in advance, maybe someone got the right idea.

Edit: Full Access is also granted

I I also have problem with high cpu usage. As I found out on the net permanent solution for this problem will be using coral usb accelerator but this is pretty expensive.

I’m using a Google Coral usb accelerator, which frigate is successfully using for detection. I think my high cpu usage is coming from decoding the streams without hardware acceleration.