Local realtime person detection for RTSP cameras

It’s already in there: frigate/Dockerfile.ffmpeg.amd64 at 09a4d6d030fec2f9e2d3e8be1fcb5560cd66414d · blakeblackshear/frigate · GitHub

RC2 of Hassos now support coral m.2?

1 Like

Cool, what should be the hwaccel_args i need to add to use it?

Hi all,

There has been couple of other posts regarding hardware acceleration problems with Raspberry Pi.
I have also facing same issues. When I have the hardware acceleration configuration in frigate yml file, I will get the green screen and also plenty of different errors. If I will delete the hardware acceleration config, I will not get any errors and stream is working.

Does anybody have advice?

What I have checked:
Stream settings in camera and Frigate
Protection mode is disabled in Frigate Add-On

My setup:
Raspberry Pi 4 (4GB)
Home Assistant OS 5.13 (64-bit)
Frigate Add-on 1.13

Error logs from Frigate Add-on:

frigate.video                  INFO    : frontyard: ffmpeg process is not running. exiting capture thread...
ffmpeg.frontyard.detect        ERROR   : [h264_v4l2m2m @ 0x5590337990] Could not find a valid device
ffmpeg.frontyard.detect        ERROR   : [h264_v4l2m2m @ 0x5590337990] can't configure decoder
ffmpeg.frontyard.detect        ERROR   : Error while opening decoder for input stream #0:0 : Operation not permitted
frigate.video                  INFO    : frontyard: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have

Frigate camera configuration:

cameras:
  frontyard:
    ffmpeg:
      hwaccel_args:
        - -c:v
        - h264_v4l2m2m
      inputs:
        - path: rtsp://username:[email protected]:XXXX/unicast
          roles:
            - detect
            - clips
            - rtmp
    width: 1280
    height: 720
    fps: 5
    objects:
      track:
        - person
        - car
        - dog
        - motorcycle
        - bicycle
        - cat
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 5
    clips:
      enabled: true
      retain:
        default: 5

Exact, i report the same thing, i have try a lot of thing but nothing work for me.
Work only when i remove the hardware acceleration but the CPU is consumming a lot.

My setup:
Raspberry Pi 4 (4GB)
Debian buster 10
Docker Home Assistant
Frigate Add-on 1.13

Good to know that I’m not the only one.

Now I started to think about this sentence from the Frigate documentation page:

Raspberry Pi 3/4 (64-bit OS) NOTICE : If you are using the addon, ensure you turn off Protection mode for hardware acceleration.

I have always thought that this is the same thing as turn off protection mode in the add-on.
But is it the same thing?

Correct. I upgrade to rc3 on a NUC 8 yesterday and no problems so far.

1 Like

I run HassOS in a Virtualbox VM but not Frigate in it’s own with a Wyze cam, so I’m not sure if this applies to you or not but I have no issues. What is your config? I would start with no HW accel and see if that works. The VM can add some unnecessary layers for accessing some of the underlying hardware, though I do use these on my 6th gen i3:

          hwaccel_args:
            - -hwaccel
            - qsv
            - -qsv_device
            - qsv

One thing I’ve noticed over the past (almost) year is that the Wyze RTSP might be the culprit - it’s terribly unstable at times, causing the lvalue and rvalue issue to pop. Meanwhile, I’ve tested other means of capturing the video through TinyCam and streaming the mjpeg stream and TinyCam–>Blue Iris–>Frigate. Those two work much better, but require more overhead. I wish TinyCam could restream RTSP instead of mjpeg since it uses the Wyze credentials to pull the video, maybe one day.

The RTSP firmware works for the most part directly with Frigate, but admittedly there were a couple of times it may have skipped frames and didn’t catch what it was supposed to. While the Wyze are nice and cheap, the RTSP firmware can be tough to work with at times.

I need some help…
When taking the latest picture from MQTT (frigate//car/snapshot) I always get bounding boxes ,timestamp and croped…
How do I disable them in snapshots from MQTT?
But when I view the same event in media browser the image is clear from bbox, crop and timestamp.

Modify these parameters.

image

Done that… Still have issues… :frowning:

#############################
#        PARKERING          #
#############################
  frigate_parkering:
    ffmpeg:
      inputs:
        - path: rtmp://192.168.1.216/bcs/channel0_main.bcs?channel=0&stream=1&user=admin&password=XXX
          roles:
            - clips
            - rtmp
#            - detect
        - path: rtmp://192.168.1.216/bcs/channel0_sub.bcs?channel=0&stream=1&user=admin&password=XXX
          roles:
            - detect
    width: 640
    height: 480
#    width: 2560
#    height: 1920
    fps: 4
    detect:
      enabled: true
    clips:
      enabled: true
      retain:
        default: 2      
    snapshots:
      enabled: True
      timestamp: False
      bounding_box: False
      height: 960
      crop: False
      retain:
        default: 2      
    objects:
      track:
        - person
        - cat
        - dog
        - bicycle
        - truck
        - car
      filters:
        person:
          min_score: 0.6
        car:
          min_score: 0.6
          min_area: 24000
    motion:
      mask:
        - 0,65,301,88,436,109,589,132,640,153,640,348,640,404,640,480,640,0,0,0
      threshold: 25

For MQTT images, you have to modify the settings under mqtt. See the full example here: https://blakeblackshear.github.io/frigate/configuration/cameras#full-example

The snapshot settings only apply to the files saved to disk.

Okey…
Tryed that before but all I get is error…

 * Starting nginx nginx
   ...done.
Error parsing config: extra keys not allowed @ data['mqtt']['timestamp']

Edit: Ok… Working… mqtt options is set per camera…

Hi, where is changelog for the latest frigate? when i checked the github it showed only 2lines of code was changed,is that correct? :>

Try the “False” as “false”.

Hi,

I have been using Frigate with a Coral usb on a raspberry pi 4. Everything was working well but the pi often struggled a little so I decided to upgrade. I’ve built an ASRock X300 with an AMD 4650g running Proxmox. I’ve set up a Home Assistant VM, restored the snapshot and have everything running now except Frigate.

I’ve read back through different messages here and elsewhere and understand that passing through the Coral is not recommended. My understanding would be that my best bet is to install frigate via an LXC container and pass through the Coral usb to the container? Do I also need GPU acceleration for ffmpeg and if so has anyone had any luck with AMD APUs? And if I do this, I lose the close integration with HA?

The other fallback could be to install HassOS directly without Proxmox but it would limit what I can do on the device in terms of other containers or VMs (and would be overkill for that purpose).

Any tips, hints or experiences would be very appreciated.

Thanks!

I would recommend frigate as an LXC container. The less layers between frigate and your hardware the better. You will only lose access to the frigate UI within homeassistant. I am planning to build a separate"proxy" addon that makes this work for those running HassOS but don’t run frigate as an addon.

ffmpeg hardware accel for AMD

6 Likes

Thank you @unrly for responding. I run Virtualbox in an Ubuntu 20.04 host.

The cameras work fine with hardware acceleration disabled (although I have also noticed the odd broken frame)

I don’t think it is a Wyzecam issue because I have the same result on another camera (Sercomm iCamera2).

I use the hwaccel configuration specified in the Frigate documentation, namely:

      hwaccel_args:
        - -hwaccel
        - vaapi
        - -hwaccel_device
        - /dev/dri/renderD128
        - -hwaccel_output_format
        - yuv420p

However, I am running on a 3rd generation i3 so maybe the hardware support is inadequate.

Has anyone set up a entity within home assistant that brings up a list of clips/detections? Similar to if you went to media browser → frigate → clips

Hi…
I’m wondering…
Imagine that I want to use my cam snapshot instead of Frigates as I’m using sub stream for detection.
But the low-res snapshot isn’t good enough for ALPR detection etc.

Would it be possible to create a camera entity from MAIN stream when frigate detects motion?