Local realtime person detection for RTSP cameras

holy crap, for some reason I missed the HW acceleration stuff in my config. when I added these in, I saw my nuc load go from ~5 to ~1. thanks for all your hard work on this, really appreciate it!

So I got Frigate 0.5.* working on unraid (Thanks @hollywoodford). I have just noticed the vs 0.4 I see massive smearing (Reolink RLC-410 5MP) on the image when its nighttime and using IR which makes detection impossible.
Anyone with these cameras have suggestions for configs which work?

reolink does not behave well with rtsp,
you should use rtmp:

ffmpeg:
      input: rtmp://192.168.0.199/bcs/channel0_sub.bcs?token=254&channel=1&stream=0&user=username&password=password

if i add my reolink cameras as rtmp it gives me an error and frigate does not start

Ill give it a try.

Hi there,

I would really like to have Frigate running, can someone please geve some advice on how to do this with the HA portainer add-on? So far I managed to pull the image from blakeblackshear/frigate:stable (not much, I know) but I am unsure of how to proceed from there.

Thanks in advance

Okay not sure whatā€™s going on using the command:

sudo docker run --rm \
--privileged \
--shm-size=1024m \
-v gpus=all \
-v/opt/frigate:/config:ro \
-v /etc/localtime:/etc/localtime:ro \
-p 5001:5001 \
blakeblackshear/frigate:stable

And getting the result of:

Traceback (most recent call last):
  File "detect_objects.py", line 25, in <module>
    with open('/config/config.yml') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/config/config.yml'

Iā€™ve checked the file and directory and I do have: ā€˜/config/config.ymlā€™ :worried: can anyone help please Iā€™ve been trying this for some time a straightforward method would be Great

If you are not so familiar with Docker and Portainer, you can just use the Frigate addon for HA: https://github.com/mories76/ha-addon-frigate

hello, just add that line in config.yml for the camera? shouldnā€™t I add or change an argument? I have tried adding my cameras as well, but it gives me the following error: under1: ffmpeg didnt return a frame. something is wrong. under1: ffmpeg process is not running. exiting capture thread ā€¦

Best way using portainer is to add frigate as a stack using the docker-compose script. I donā€™t have an example because I donā€™t do it this way, but itā€™s not too hard once you get the variables set right, not really much different than getting a hass.io addon working but a few more variables to contend with.

In case it helps someone else, after thinking about it I have obtained that it will work with the following configuration:

    ffmpeg:
      input: rtmp://192.168.1.6:1935/bcs/channel0_sub.bcs?channel=0&stream=0&user=xxxxxxx&password=xxxxxxx
      hwaccel_args:
        - -hwaccel
        - vaapi
        - -hwaccel_device
        - /dev/dri/renderD128
        - -hwaccel_output_format
        - yuv420p
      input_args:
        - -avoid_negative_ts
        - make_zero
        - -fflags
        - nobuffer
        - -flags
        - low_delay
        - -strict
        - experimental
        - -fflags
        - +genpts+discardcorrupt
        - -vsync
        - drop
        - -use_wallclock_as_timestamps
        - '1'
3 Likes

Okay Iā€™ve solved that issue frigate is now running but I still have an error:


[rtsp @ 0x559173fe2a20] RTP: missed 1 packets

[rtsp @ 0x559173fe2a20] max delay reached. need to consume packet

[rtsp @ 0x559173fe2a20] RTP: missed 1 packets

[rtsp @ 0x559173fe2a20] max delay reached. need to consume packet

[rtsp @ 0x559173fe2a20] RTP: missed 1 packets

[h264 @ 0x559173fe6480] concealing 3231 DC, 3231 AC, 3231 MV errors in P frame

[rtsp @ 0x559173fe2a20] max delay reached. need to consume packet

[rtsp @ 0x559173fe2a20] RTP: missed 1 packets

[rtsp @ 0x559173fe2a20] decoding for stream 0 failed

Input #0, rtsp, from 'rtsp://root:[email protected]:8554/unicast':

  Metadata:

    title           : LIVE555 Streaming Media v2020.03.06

    comment         : LIVE555 Streaming Media v2020.03.06

  Duration: N/A, start: 0.180700, bitrate: N/A

    Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1600x900, 25 fps, 25 tbr, 90k tbn, 50 tbc

    Stream #0:1: Audio: mp3, 44100 Hz, mono, s16p, 64 kb/s

[NULL @ 0x5591740da240] Unable to find a suitable output format for 'pipe:'

pipe:: Invalid argument

back: ffmpeg didnt return a frame. something is wrong.

back: ffmpeg process is not running. exiting capture thread...

Does anyone have any ideas on how I can fix this please thank youā€¦

Hi,

scstraus:
will stick to the add-on for nowā€¦

Mario:
thanks fot the heads up!

Having an add-on is great! But is not working (I am starting to hate be a noob) Anyway, installed it, created the config.yml in the proper folder and added the camera, binary sensor and sensor to my configuration.yaml. Was hoping to see it working, but pressing the start button has no effect, after a moment is back to start as opposed to stop.

What am I doing wong? Can it be an error in the config file?

Hi Blake

Iā€™ve been running v0.5.1 stable since it was released, but it has been failing on average every 4-5 days. When this happens, objects are not detected. The debug stats return all frames as ā€˜Skippedā€™, and there is no activity on the Coral.

I was waiting to catch it again as soon as it happened so I could check if there was anything meaningful in the log at the point it started, but there is not.

This evening it died at 1747.

2020-07-12T09:19:20.432438109Z Camera_process started for driveway: 36
2020-07-12T09:19:20.442843181Z Camera_process started for side: 37
2020-07-12T09:19:20.451120258Z Starting process for driveway: 36
2020-07-12T09:19:20.454022774Z Camera_process started for back: 38
2020-07-12T09:19:20.462770860Z Starting process for side: 37
2020-07-12T09:19:20.482972849Z Starting process for back: 38
2020-07-12T09:19:20.490384351Z Camera_process started for backside: 39
2020-07-12T09:19:20.491902376Z Starting process for backside: 39
2020-07-12T09:19:20.509883316Z Starting process for frontdoor: 40
2020-07-12T09:19:20.515580537Z Camera_process started for frontdoor: 40
2020-07-12T09:19:20.538748430Z  * Serving Flask app "detect_objects" (lazy loading)
2020-07-12T09:19:20.538969666Z  * Environment: production
2020-07-12T09:19:20.539018276Z    WARNING: Do not use the development server in a production environment.
2020-07-12T09:19:20.539042628Z    Use a production WSGI server instead.
2020-07-12T09:19:20.539068405Z  * Debug mode: off
2020-07-13T17:47:22.321685309Z Detection appears to be stuck. Restarting detection process
2020-07-13T17:47:22.321775216Z Waiting for detection process to exit gracefully...
2020-07-13T17:47:22.355060110Z Starting detection process: 10799
2020-07-13T17:47:25.261354539Z ../src/plasma/client.cc:182: mmap failed
2020-07-13T17:47:25.262250344Z /usr/local/lib/python3.7/dist-packages/pyarrow/libarrow.so.17(+0x614cc0)[0xac74acc0]
2020-07-13T17:47:25.262688210Z /usr/local/lib/python3.7/dist-packages/pyarrow/libarrow.so.17(_ZN5arrow4util8ArrowLogD1Ev+0x108)[0xac74ade0]
2020-07-13T17:47:25.262734709Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0xd0f4)[0xabf000f4]
2020-07-13T17:47:25.262889986Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0xdd54)[0xabf00d54]
2020-07-13T17:47:25.262919596Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0xe8ac)[0xabf018ac]
2020-07-13T17:47:25.262942189Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(_ZN6plasma12PlasmaClient3GetERKSt6vectorINS_8UniqueIDESaIS2_EExPS1_INS_12ObjectBufferESaIS7_EE+0x24)[0xabf019a4]
2020-07-13T17:47:25.262964614Z /usr/local/lib/python3.7/dist-packages/pyarrow/_plasma.cpython-37m-arm-linux-gnueabihf.so(+0x22d40)[0xabf45d40]
2020-07-13T17:47:25.262986003Z /usr/local/lib/python3.7/dist-packages/pyarrow/_plasma.cpython-37m-arm-linux-gnueabihf.so(+0x1d334)[0xabf40334]
2020-07-13T17:48:52.434941577Z Detection appears to have stopped. Restarting detection process
2020-07-13T17:48:52.458307822Z Starting detection process: 10816
2020-07-13T17:48:55.317279066Z ../src/plasma/client.cc:182: mmap failed
2020-07-13T17:48:55.318371386Z /usr/local/lib/python3.7/dist-packages/pyarrow/libarrow.so.17(+0x614cc0)[0xac74acc0]
2020-07-13T17:48:55.318888195Z /usr/local/lib/python3.7/dist-packages/pyarrow/libarrow.so.17(_ZN5arrow4util8ArrowLogD1Ev+0x108)[0xac74ade0]
2020-07-13T17:48:55.318938602Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0xd0f4)[0xabf000f4]
2020-07-13T17:48:55.318962935Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0xdd54)[0xabf00d54]
2020-07-13T17:48:55.318986305Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0xe8ac)[0xabf018ac]
2020-07-13T17:48:55.319008971Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(_ZN6plasma12PlasmaClient3GetERKSt6vectorINS_8UniqueIDESaIS2_EExPS1_INS_12ObjectBufferESaIS7_EE+0x24)[0xabf019a4]
2020-07-13T17:48:55.319031434Z /usr/local/lib/python3.7/dist-packages/pyarrow/_plasma.cpython-37m-arm-linux-gnueabihf.so(+0x22d40)[0xabf45d40]
2020-07-13T17:48:55.319053656Z /usr/local/lib/python3.7/dist-packages/pyarrow/_plasma.cpython-37m-arm-linux-gnueabihf.so(+0x1d334)[0xabf40334]
2020-07-13T17:50:22.534771178Z Detection appears to have stopped. Restarting detection process
2020-07-13T17:50:22.573770457Z Starting detection process: 10833
2020-07-13T17:50:25.429654956Z ../src/plasma/client.cc:182: mmap failed
2020-07-13T17:50:25.430757257Z /usr/local/lib/python3.7/dist-packages/pyarrow/libarrow.so.17(+0x614cc0)[0xac74acc0]
2020-07-13T17:50:25.431220066Z /usr/local/lib/python3.7/dist-packages/pyarrow/libarrow.so.17(_ZN5arrow4util8ArrowLogD1Ev+0x108)[0xac74ade0]
2020-07-13T17:50:25.431389842Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0xd0f4)[0xabf000f4]
2020-07-13T17:50:25.431422915Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0xdd54)[0xabf00d54]
2020-07-13T17:50:25.431445989Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0xe8ac)[0xabf018ac]
2020-07-13T17:50:25.431468433Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(_ZN6plasma12PlasmaClient3GetERKSt6vectorINS_8UniqueIDESaIS2_EExPS1_INS_12ObjectBufferESaIS7_EE+0x24)[0xabf019a4]
2020-07-13T17:50:25.431490896Z /usr/local/lib/python3.7/dist-packages/pyarrow/_plasma.cpython-37m-arm-linux-gnueabihf.so(+0x22d40)[0xabf45d40]
2020-07-13T17:50:25.431512414Z /usr/local/lib/python3.7/dist-packages/pyarrow/_plasma.cpython-37m-arm-linux-gnueabihf.so(+0x1d334)[0xabf40334]
2020-07-13T17:51:52.646667867Z Detection appears to have stopped. Restarting detection process
2020-07-13T17:51:52.664862689Z Starting detection process: 10851
2020-07-13T17:51:55.502596252Z ../src/plasma/client.cc:182: mmap failed
.....

Many thanks

I havenā€™t see the ā€œmmap failedā€ error from the plasma store. Not entirely sure what that could be, but it sounds like an issue with the plasma client trying to read or write from memory.

Never mind, disabled protected modeā€¦

Edit: Spoke too fastā€¦ stops after a few momentsā€¦

i donā€™t use environment variables as i just saw it easier to define that in my config file.

you can pretty much just use @blakeblackshear 's examples.

below is a sanitized version of my docker-compose
In portainer, click on stacks, add new stack, give it a name (i called this stack object-detection) then in the editor use my snippet below or the example from @blakeblackshear 's repo.

before starting the stack, you need to make sure you have a proper config file in the config folder to reference otherwise the container will just error out

version: '2'

services:
  frigate:
    container_name: frigate
    restart: unless-stopped
    privileged: true
    shm_size: '2g' # should work for 5-7 cameras
    image: blakeblackshear/frigate:0.5.1
#    image: blakeblackshear/frigate:stable
    volumes:
      - /dev/bus/usb:/dev/bus/usb
      - /etc/localtime:/etc/localtime:ro
      - <PATH_TO_YOUR_CONFIG_FOLDER>:/config
    ports:
      - "5000:5000"
    devices:
      - /dev/dri:/dev/dri

I use the ā€œmainā€ url below for my RLC-420.

input: rtmp://192.168.2.140/bcs/channel0_main.bcs?token=254&channel=0&stream=0&user=USERNAMEHERE&password=PASSWORDHERE

and hereā€™s a link to my whole frigate.yml if itā€™s helpful. Iā€™m not currently using HW accel, but need to try it out.

1 Like

Hrmā€¦I tried adding the hwaccel args, but now itā€™s just stuck trying to create the FFMPEG process. Iā€™ve tried passing both /dev/dri and /dev/dri/renderD128 to the docker, but no dice. I just got a new 10th gen Intel (i7-10700), could that have anything to do with it?

Creating ffmpeg process...
ffmpeg -hide_banner -loglevel panic -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtmp://192.168.2.140/bcs/channel0_main.bcs?token=254&channel=0&stream=0&user=admin&password=PASSWORD -f rawvideo -pix_fmt rgb24 pipe:
front: ffmpeg didnt return a frame. something is wrong.
front: ffmpeg process is not running. exiting capture thread...
Creating ffmpeg process...
ffmpeg -hide_banner -loglevel panic -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtmp://192.168.2.140/bcs/channel0_main.bcs?token=254&channel=0&stream=0&user=admin&password=PASSWORD  -f rawvideo -pix_fmt rgb24 pipe:
front: ffmpeg didnt return a frame. something is wrong.
front: ffmpeg process is not running. exiting capture thread...

edit: I should add, I have it passed through fine to my Plex docker, just canā€™t get it to work here.

You were right: I was not using a pure black color.