Local realtime person detection for RTSP cameras

The docs are your friend again here. Look at the COCO label map for the model that Frigate uses. That will give you a list of all of the trained objects that can be detected. Then, just add that label to your configuration (as you did with “animal” - though “animal” is not a valid object in the label map).

Thx… Just need a guidance maybe also because some kind of language barrior…

Anyway… how I clean events manually if not needed anymore after test?

Hi, How would I be able to also view the clips?

I have in config:

clips:
  # Optional: Maximum length of time to retain video during long events. (default: shown below)
  # NOTE: If an object is being tracked for longer than this amount of time, the cache
  #       will begin to expire and the resulting clip will be the last x seconds of the event.
  max_seconds: 300
  # Optional: size of tmpfs mount to create for cache files (default: not set)
  # mount -t tmpfs -o size={tmpfs_cache_size} tmpfs /tmp/cache
  # NOTICE: Addon users must have Protection mode disabled for the addon when using this setting.
  # Also, if you have mounted a tmpfs volume through docker, this value should not be set in your config.
  tmpfs_cache_size: 256m
  # Optional: Retention settings for clips (default: shown below)
  retain:
    # Required: Default retention days (default: shown below)
    default: 100
    # Optional: Per object retention days
    objects:
      person: 150

I see:
image

I can go through my images, they are saved, but no clips…?

Have you set clips role for the camera?

cameras:
  back:
    ffmpeg:
      inputs:
        - path: rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@10.0.10.10:554/live
          roles:
            - clips

Have tried this, thanks will let know.

Another question. I added a 3rd cam and now I have:

Fatal Python error: Bus error

According to documentaion I need to increase shm size according to https://blakeblackshear.github.io/frigate/installation#calculating-shm-size. But how and where can I do this when I am using portainer to start Frigate?

I calculated the shm:
image
total is 54… was 64 and I changed in the docker config file to 96 with:
image
EDIT (128 also not working)

But still it doesnt work…

What am I doing wrong now?

Whats weird, the 1st and 2nd cam seem to keep ruuning (can see the time and live image). Clips is off for the 3rd cam… but taht stops.

EDIT: it’s a matter of time… if motion is on the other cam it also stops…

The error:

frigate.app                    INFO    : Capture process started for parking: 70
Fatal Python error: Bus error
Thread 0x00007fcd6e6f2700 (most recent call first):
  File "/usr/lib/python3.8/threading.py", line 302 in wait
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 227 in _feed
  File "/usr/lib/python3.8/threading.py", line 870 in run
  File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap
Current thread 0x00007fcd70ef3700 (most recent call first):
  File "/opt/frigate/frigate/video.py", line 117 in capture_frames
  File "/opt/frigate/frigate/video.py", line 231 in run
  File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap
Thread 0x00007fcd736f4700 (most recent call first):
  File "/opt/frigate/frigate/video.py", line 205 in run
  File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap
Thread 0x00007fcd73ef5700 (most recent call first):
  File "/opt/frigate/frigate/log.py", line 71 in run
  File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap
Thread 0x00007fcd878eb740 (most recent call first):
  File "/usr/lib/python3.8/threading.py", line 1027 in _wait_for_tstate_lock
  File "/usr/lib/python3.8/threading.py", line 1011 in join
  File "/opt/frigate/frigate/video.py", line 245 in capture_camera
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108 in run
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315 in _bootstrap
  File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75 in _launch
  File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19 in __init__
  File "/usr/lib/python3.8/multiprocessing/context.py", line 277 in _Popen
  File "/usr/lib/python3.8/multiprocessing/context.py", line 224 in _Popen
  File "/usr/lib/python3.8/multiprocessing/process.py", line 121 in start
  File "/opt/frigate/frigate/app.py", line 182 in start_camera_capture_processes
  File "/opt/frigate/frigate/app.py", line 228 in start
  File "/opt/frigate/frigate/__main__.py", line 15 in <module>
  File "/usr/lib/python3.8/runpy.py", line 87 in _run_code
  File "/usr/lib/python3.8/runpy.py", line 194 in _run_module_as_main

EDIT: If there is no motion all 3 cams keep working… as soon as there is was/some motion it stops.

Hello!

So here it goes.

Running Home Assistant OS 5.12, core-2021.3.2, supervisor-2021.03.4, on Intel NUC.

I read about frigate and said to give it a try.

First I made in Unifi 2 rules for port forwarding 5000 and 1935 from anywhere to IP of NUC only TCP.

Then continuing with installation, I read the part that says “Note that HassOS Addons and custom components are different things. If you are already running Frigate with Docker directly, you do not need the Addon since the Addon would run another instance of Frigate."

So I thought I would just need to install custom component from HACS and nothing else.
Installed it in Hacs but then couldn’t add the integration:
Configuration->integrations->+add integration->Frigate-> used “http://ccab4aaf-frigate:5000” and got message “Unable to connect to Frigate.”

Question 1
Do I really need to add Frigate NVR from Supervisore->Add-on Store? If not, why I get this message? What needs to be configured and where?

Then, I tried the other way. Installed Frigate NVR from Supervisore->Add-on Store, in configuration I let 1935 and 5000 as is, created frigate.yml in config folder and entered:

mqtt:
  host: IP of NUC
  user: my mqtt user
  password: my mqtt password
cameras:
  outdoor:
    ffmpeg:
      inputs:
        - path: rtsp://<username>:<password>@<address>:<port>/Streaming/Channels/102
          roles:
            - detect
            - rtmp
    width: 640
    height: 360
    fps: 5

Then I get the below log when I start the add-on:

 * Starting nginx nginx
   ...done.
frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.app                    INFO    : Camera processor started for outdoor: 38
frigate.app                    INFO    : Capture process started for outdoor: 40
detector.coral                 INFO    : Starting detection process: 36
frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate.edgetpu                INFO    : No EdgeTPU detected.
Process detector:coral:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 111, in __init__
    raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/edgetpu.py", line 124, in run_detector
    object_detector = LocalObjectDetector(tf_device=tf_device, num_threads=num_threads)
  File "/opt/frigate/frigate/edgetpu.py", line 63, in __init__
    edge_tpu_delegate = load_delegate('libedgetpu.so.1.0', device_config)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1.0
frigate.mqtt                   INFO    : MQTT connected
frigate.watchdog               INFO    : Detection appears to have stopped. Exiting frigate...
frigate.app                    INFO    : Stopping...
frigate.events                 INFO    : Exiting event processor...
frigate.record                 INFO    : Exiting recording maintenance...
frigate.object_processing      INFO    : Exiting object processor...
frigate.events                 INFO    : Exiting event cleanup...
frigate.watchdog               INFO    : Exiting watchdog...

Question 2
What am I missing?

pardon my noobynes

The error you’re seeing in the logs is due to the fact that you don’t have a Coral device and you haven’t defined a CPU detector in the config file. Check the docs.

1 Like

Pulled the latest image and am getting a couple errors in the logs, not sure where to look for the issue.

peewee_migrate                 INFO    : There is nothing to migrate
frigate.mqtt                   INFO    : MQTT connected
detector.coral                 INFO    : Starting detection process: 42
frigate.app                    INFO    : Camera processor started for porch: 45
frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate.app                    INFO    : Capture process started for porch: 46
frigate.edgetpu                INFO    : TPU found
frigate.video                  INFO    : porch: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : porch: ffmpeg process is not running. exiting capture thread...
ffmpeg.porch.detect            ERROR   : [h264_v4l2m2m @ 0x564653183b80] Could not find a valid device
ffmpeg.porch.detect            ERROR   : [h264_v4l2m2m @ 0x564653183b80] can't configure decoder
ffmpeg.porch.detect            ERROR   : Error while opening decoder for input stream #0:1 : Invalid argument

These errors repeat in my log and my web interface is showing green background in the camera view.
The RTMP stream I am using works in vlc.
I am using a Reolink RLC-410 camera.

Thanks for the great work!

Check the docs: https://blakeblackshear.github.io/frigate/configuration/cameras#reolink-410520-possibly-others

Maybe a stupid question perhaps but. …

How do I allow Frigate to other users through Home assistant app?
It doesn’t show up when my wife opens her ha app.

I have attempted with the ReoLink Configuration example and without. I have also tried disabling all roles except detect: and I have removed all hwaccel_args:
for what it is worth I have tried both RTMP streams and RTSP streams from the camera (all display fine in vlc)
I still have a green camera display and logs are repeatable showing these now.

ffmpeg.porch.detect            ERROR   : Option rtsp_transport not found.
frigate.video                  INFO    : porch: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : porch: ffmpeg process is not running. exiting capture thread...

1 Like

Check your resolution of the cam if it matches in your config

I think I am ok here but not certain. if I have width / height correct.
image

So for what it is worth. I changed to an RTSP stream again. updated my camera FPS to match my configuration. All looks better now. Thanks for the support.

can i change the ffmpeg binaries in the frigate docker container. or can I run frigate without docker and use the host’s ffmpeg?

i’ve got a jetson nano, with a custom ffmpeg that I want to use in frigate.

Any ideas? I don’t know where to start!

I have frigate nvr integration. Is it possible to save the clips on my network share? As it is taking a lot of space on my HASS OS.

1 Like

HI all,

I have a latest person image as a button, how would i play the latest clip associated with that latest person image? I want to use the media player as a popup window.

1 Like

Hi!

Maybe this is a stupid question but I’d like to use frigate so that it records clips when there is movement (sensitivy is selectable) and also does object detection if it can. Now it seems (or don’t know how to conf) that it saves clips only when it understand configured object.

Also like to know how to manually clean old events (or just lower retention day) and why HA Media browser doesn’t show thumbnails from pics & clips (however all of these working if clicked).

I bought myself a coral usb… expect questions the next period when it arrives :-).

for now please help me with Local realtime person detection for RTSP cameras

1 Like

How do I fix the time on the clips? It’s not showing the right time…

So far, I traced the following:
Human and Car works well but…
85% dog = my wife wearing a coat with a fur hoodie and fur earmuffs
75% dog = a squirell
70% dog = some cat
Never traced an actual dog or cat, will def need training… :slight_smile:

1 Like