Local realtime person detection for RTSP cameras

This is my holy grail… unfortunately not achievable under a VM as /dev/dri/renderD128 is not passed through from the host to the VM

I really want to run supervisor on the OS(Ubuntu 20 in my instance) or LXC to capitalise on hwaccel but both installation methods are unsupported.

Ah…! that is unfortunate. Are there any issues using a Coral when running in a VM (I was eyeing one of the M.2 devices - very cheap, and I’m pretty sure I have a spare slot (designed for an M.2 wifi card))?

Another thought since you’re running an identical setup as mine albeit your CPU is 3x faster than mine Intel Core i5-7500 @ 3.40GHz vs Intel Celeron J3455 @ 1.50GHz [cpubenchmark.net] by PassMark Software

We could just run frigate outside HASS as per
https://blakeblackshear.github.io/frigate/installation
You’ll also be able to use the coral directly too and easily get -hwaccel

I nominate you to try first :smiley: I’m burnt out with my LXC failures and am a HA noob

Will take a look - I’m running a few other Dockers anyway, so should be able to spin up easily (famous last words). Maybe after I’ve had a bit of a play using the HassOS addon. Running 4 cams atm thr moment has seen my idle CPU jump from ~1 - 10% up to around 30%.

Hi Guys,
I’m try again to set this up and added everything but I’m getting the following error

ffmpeg.front.detect            ERROR   : mmal: mmal_component_create_core: could not create component 'vc.ril.video_decode' (7)
ffmpeg.front.detect            ERROR   : Error while opening decoder for input stream #0:0 : Unknown error occurred
frigate.video

I’m using hikvision and have set up a sub stream to H264 and also connected it to HA via ONVIF

here is my frigate settings running the ADD-ON RPI4 4GB

cameras:
  front:
    ffmpeg:
      inputs:
        - path: rtsp://user:[email protected]:64999/Streaming/Unicast/channels/302
          roles:
            - detect
            - rtmp
          hwaccel_args:
            - -c:v
            - h264_mmal
    width: 960
    height: 576
    fps: 5

detectors:
  cpu1:
    type: cpu
  cpu2:
    type: cpu

Could someone please help

I’ve given up getting the RPi to work without a powered USB hub and shelled out for a NUC. Trying to set up a M.2-2230-A-E-S3 (A+E key) PCIe Coral in the WiFi slot, it’s detected ok:

# lspci -nn | grep 089a
05:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]

How do I pass this through in docker? I tried /sys/bus/pci/devices/0000:05:00.0 and just /sys/bus/pci/devices/

And this in config:

detectors:
  coral_pci:
    type: edgetpu
    device: pci

ls /dev/apex_0
doesn’t list anything though…

So… docker instance of frigate now spun up. Added in the standard hwaccel_args for my Gen 7 CPU (i.e. the <10th Gen intel setting):

ffmpeg:
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p

CPU usage seems to idle at around 25-29%, so only marginally lower than running unaccelerated using the HassOS add-on (and with occasional spikes to ~65%)

Does that seem normal? Am wondering is the CPU limits on the VM were artificially repressing CPU usage.

I have ordered a Coral, but was hoping to run on CPU till it got here–seems a bit high to run 24/7 though.

Question… should the ffmpeg hwaccel_args be placed separately (like camera: and detector:) to apply to all cameras, or duplicated within the ffmpeg parameters for each individual camera?

Please follow the instructions of coral here

After installing the pcie I r ready to go

Hi,

Can you post your docker params ?
As per the documentation you do need to ensure the device is passed through eg

 --group-add=109 \
 --restart=always \
 --device /dev/dri/renderD128:/dev/dri/renderD128 \
 --device /dev/dri/card0:/dev/dri/card0 \

One thing very important to note is the render-group which in my case is 109

root@svr1:~# grep render /etc/group
render:x:109:me

I would also suggest that you read Jellyfin’s excellent work around the subject here

https://jellyfin.org/docs/general/administration/hardware-acceleration.html

Good luck :slight_smile:

1 Like

Thanks, I’ll do some reading… in the meantime I just used the posted docker run (modded for my volumes):

sudo docker run -d 
--name frigate 
--restart=unless-stopped 
--mount type=tmpfs,target=/tmp/cache,tmpfs-size=1000000000 
--device /dev/bus/usb:/dev/bus/usb 
--device /dev/dri/renderD128 
-v /srv/data/camera:/media/frigate 
-v /srv/data/app-data/frigate/config.yml:/config/config.yml:ro 
-v /etc/localtime:/etc/localtime:ro 
-e FRIGATE_RTSP_PASSWORD='XXXXXX' 
-p 5000:5000 
-p 1935:1935 
blakeblackshear/frigate:stable-amd64

That render group thing may be (partly?) my issue.

BTW use intel_gpu_top this should show if you’re using the GPU

http://manpages.ubuntu.com/manpages/trusty/man1/intel_gpu_top.1.html

I think I’m going to have to try the whole frigate setup myself

1 Like

I dialled back the resolution on the camera feeds in the interim (now 1@1024x576, 3@720x400), and getting CPU idle around 18-25%, with spikes ~45-50%. And check my render group - 109 like yours.

I’ve added…

--group-add=109
--device /dev/dri/card0:/dev/dri/card0

to my docker run with no discernible effect… guess its reading time. Thanks for all your help!

Thanks for that… GPU is doing something, but doesn’t seem to working very hard - Render/3D/0 using 1.77 to 1.95%, Video/0 using 2.57 to 3.05%.

Going by that acceleration doesn’t seem to be being used.

Decoding doesn’t work the GPU hard that’s the whole point :wink:
It only works hard for h265 or more difficult streams

This really is all negligible as in terms of overall system performance frigate is using the CPU for detection so you do need to get the coral in play

BTW hwaccel Seems to be working for me

root 2715184 2715180 17 17:09 ? 00:00:51 ffmpeg -hide_banner -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0 -r 10 -f rawvideo -pix_fmt yuv420p pipe:

Image deleted

1 Like

Thanks that’s for the USB version and runtime is installed in the container isn’t it?
I followed this

I think I’m missing the PCIe driver as apex_0 isnt there. I’m running on Libreelec as I’ve got this hair brained idea of being able to use Kodi to display the streams on an attached monitor. I will try again tomorrow on another distro.

Has anyone got the PCIe coral working under HassOS or does this have the same issue?

Does anyone have suggestions about the best way to connect a docker container running Frigate into HassOS (as opposed to running it through the HassOS addon)?

Here you go

Yea that’s the same link I posted. :face_with_hand_over_mouth: