Local realtime person detection for RTSP cameras

Share your Projects!
3133

@blakeblackshear Hi there, I am having some issues with the latest update of frigate. I am using the very minimal configuration with just the mqtt and a camera components as explained in the docs. But I am getting this error.

 * Starting nginx nginx
   ...done.
frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.app                    INFO    : Camera processor started for back: 36
frigate.app                    INFO    : Capture process started for back: 37
detector.coral                 INFO    : Starting detection process: 33
frigate.edgetpu                INFO    : Attempting to load TPU as usb:0
frigate.edgetpu                INFO    : No EdgeTPU detected.
Process detector:coral:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 111, in __init__
    raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/edgetpu.py", line 124, in run_detector
    object_detector = LocalObjectDetector(tf_device=tf_device, num_threads=num_threads)
  File "/opt/frigate/frigate/edgetpu.py", line 63, in __init__
    edge_tpu_delegate = load_delegate('libedgetpu.so.1.0', device_config)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1.0

I am running this rpi 4 with the latest HA
Hope somebody can help.

3134

Unfortunately coral accelerators seem to be sold out everywhere in Canada so I’m stuck using my CPU. I have noticed that defining multiple cpu detectors spawns multiple detector processes. Does this provide any performance benefit? I have an i7 4790k if that matters.

detectors:
cpu1:
type: cpu
cpu2:
type: cpu

I tried using the hardware acceleration options for <10gen intel processors but I only ever get a green image appear.

3135

I have the same error, please let me know if you find a fix. I’m running hassio.

detectors:
  coral:
    device: usb
    type: edgetpu
mqtt:
  host: 10.0.0.10
  password: xxx
  topic_prefix: frigate
  user: mqtt
ffmpeg:
  input_args: >-
    -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt
    -use_wallclock_as_timestamps 1
  hwaccel_args:
    - '-hwaccel'
    - vaapi
    - '-hwaccel_device'
    - /dev/dri/renderD128
    - '-hwaccel_output_format'
    - yuv420p
cameras:
  drive:
    ffmpeg:
      inputs:
        - path: >-
            rtmp://10.0.0.87/bcs/channel0_sub.bcs?channel=0&stream=0&user=admin&password=xxx
          roles:
            - detect
    fps: 4
    motion:
      mask: >-
        0,214,0,0,640,0,639,151,640,301,508,162,445,171,386,145,357,147,417,233,318,235,170,227
    height: 352
    width: 640
  frontdoor:
    ffmpeg:
      inputs:
        - path: 'rtsp://admin:[email protected]//h264Preview_01_sub'
          roles:
            - detect
    fps: 4
    width: 640
    height: 352
    objects:
      track:
        - person
objects:
  track:
    - person
    - car
    - truck
    - bicycle
logger:
  default: info
  logs:
    frigate.mqtt: error
3136

It no longer falls back to the CPU automatically if you don’t have a Coral device. You need to update the detector in the config to be a CPU type. See the Readme.

3137

It probably will up to a certain point. Not sure what the sweet spot will be.

3138

Thank you, that worked.

detectors:
  coral:
    type: cpu
3139

i am still facing issues. The error that I reported has gone with assigning cpu as the detector but the detector is turned off. I am running this in rpi4 4gb inside HA supervised as addon

this is the log.

 * Starting nginx nginx
   ...done.
frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.mqtt                   INFO    : MQTT connected
detector.coral                 INFO    : Starting detection process: 33
frigate.app                    INFO    : Camera processor started for back: 36
frigate.mqtt                   INFO    : Turning off detection for back via mqtt
frigate.app                    INFO    : Capture process started for back: 37

this is my config

detectors:
  coral:
    type: cpu
mqtt:
  host: 192.168.33.250
  port: 1883
  user: user
  password: password
  topic_prefix: frigate
ffmpeg:
  hwaccel_args:
    - '-c:v'
    - h264_mmal
cameras:
  back:
    ffmpeg:
      inputs:
        - path: >-
            rtsp://user:[email protected]:554/cam/realmonitor?channel=19&subtype=1&unicast=true&proto=Onvif
          roles:
            - detect
            - rtmp
    height: 352
    width: 640
    fps: 5
objects:
  track:
    - person
    - car
3140

There is a switch.back_detect which you need to switch on, mine was off by default.

1 Like
3141

It looks like that did the trick. Thanks Dave

3142

What a lovely piece of code. :slight_smile:
Just for fun I did install this on my old i7 lenovo laptop (with broken screen).
It’s running quite well with average cpu load at 40-60% and 5 cams.

However, i’ve seen that there is a home integration. I’m running HA on a RPI4 and would like to run the add on but use the lenovo laptop as a host for Frigate. Is it possible?

And live streams with boundary boxes… How to show them in HA?
Edit: found it.
http://192.168.1.209:5000/api/mot_parkering?bbox=1

//Minglarn…

3143

It does indeed:

image

Looking at the rights of the folder:

eben@eben-ThinkCentre-M700:~/Docker/frigate$ ls -l
total 2932
drwx------ 2 eben eben   49152 Jan  4 00:41 cache
drwx------ 2 eben eben 2928640 Jan 19 13:04 clips
drwx------ 2 eben eben    4096 Jan 16 13:51 config
-rwx------ 1 eben eben    1061 Jan  3 01:00 docker-compose.yml
-rwx------ 1 eben eben     633 Oct 18 00:26 docker-compose.yml.0.7.0.yml
drwxr-xr-x 2 root root    4096 Jan  2 21:20 recordings

The files inside pretty much were created by root but with read access to groups and others:

-rw-r--r-- 1 root root  12026961 Jan 14 19:24 cars-1610645033.030097-ebtmj1.mp4
-rw-r--r-- 1 root root  10196073 Jan 14 19:24 cars-1610645076.340808-2qufcp.mp4

Users on the system (only root and my user listed):

eben@eben-ThinkCentre-M700:~/Docker/frigate$ cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
eben:x:1000:1000:Eben,,,:/home/eben:/bin/bash

Groups where my username is part of

eben@eben-ThinkCentre-M700:~/Docker/frigate$ cat /etc/group
root:x:0:
adm:x:4:syslog,eben
cdrom:x:24:eben
sudo:x:27:eben
dip:x:30:eben
plugdev:x:46:eben
eben:x:1000:
sambashare:x:132:eben
docker:x:134:eben
eben@eben-ThinkCentre-M700:~/Docker/frigate$

I tried changing the rights of the clips folder to be a bit more forgiving:

eben@eben-ThinkCentre-M700:~/Docker/frigate$ ls -l
total 2932
drwx------ 2 eben eben   49152 Jan  4 00:41 cache
drwxrw-rw- 2 eben eben 2928640 Jan 19 13:20 clips

and restarted the Docker container, but no difference. Files inside are written with the same permissions in any case.

3144

New config file is here: https://github.com/blakeblackshear/frigate/tree/v0.8.0-rc1#configuration
Be careful at new masks syntax.

3145

Great new release BTW.
The new webUI is awesome. Couple of issues on responsiveness, and debug is not readable using chrome night theme.

3146

Question…
When using a RTMP stream and DETECT stream…
I want to use the bigger stream for RTMP so the quality of the stream is perfect and only do motion detection on the sub stream (low qual).
Im a but confused about the width and height options.
How do I set the width and height for the MAIN stream?

  mot_parkering:
    ffmpeg:
      inputs:
######### MAIN Stream for recording....
        - path: rtmp://192.168.1.110/bcs/channel0_main.bcs?channel=0&stream=0&user=admin&password=ZXZXZX
          roles:
            - clips
            - rtmp
######### SUB stream for object detection....
        - path: rtmp://192.168.1.110/bcs/channel0_sub.bcs?channel=0&stream=0&user=admin&password=ZXZXZX
          roles:
            - detect
    width: 640
    height: 480
#    width: 2560
#    height: 1920
    fps: 5
    detect:
      enabled: true
      max_disappeared: 10
    clips:
      enabled: true

Main stream has 2560x1920 and SUB stream has 640x480 …

3147

You don’t need to set it. The width/height only need to be set for the detect role.

3148

Okey… But when Im watching the stream its sub stream with low quality…

http://192.168.1.209:5000/api/mot_parkering?bbox=1

image
image1537×948 156 KB

3149

You are watching the detect stream, not RTMP.

3150

Allright. How to watch the rmtp?
Edit: Found it.

Question: is RTMP the only choise for output stream?
I would love to have a mjpeg stream so i can show it on my surveillance monitors.

3151

Why not connect directly to the camera? You could use VLC, for example, and pull the RTSP stream from the camera and bypass Frigate totally.

3152

That’s one option. However, I’m using Chrome and php to show multiple streams at the same time. And I love the boundary boxes… RTMP is a pain to get working with HTML.

1611083527456986192750770249035
16110835274569861927507702490354032×1908 1.56 MB

1 Like