Local realtime person detection for RTSP cameras

Did you add any cameras to your config?

Then you are running an old version.

1 Like

I suspected this might be the case. Will update and see how things go.

Paul, sorry I am trying to learn. I have installed ubuntu, docker, portainer etc. And I am stuckā€¦

I have:

Gives me:

I have added my frigate.yml file in /etc/frigate/config/frigate.yml directory.

I briefly get container starting with published port 5000:5000 but then it disappears?

What am I doing wrong? Well can also ask the other way around: what am I doing right I got so far :slight_smile:

Iā€™d click the Logs button in portainer to see why the container is failing to start.

Hi!

What is wrong with this?


detectors:
  coral:
    type: edgetpu
    device: usb

mqtt:
  host: 192.168.1.60
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: xxx
  password: xxx

cameras:
  eteinen:
    ffmpeg:
      inputs:
        - path: rtsp://xxi:[email protected]:554/stream1
          roles:
            - detect
    width: 1920
    height: 1080
    fps: 5
    snapshots:
      enabled: true
      bounding_box: true

  takapiha:
    ffmpeg:
      inputs:
        - path: rtsp://xx:[email protected]:554/Streaming/Channels/1
          roles:
            - detect
            - rtmp
            - clips
    width: 1920
    height: 1080
    fps: 5
    clips:
      enabled: true
      pre_capture: 5
      post_capture: 5
      objects:
        - person
        - animal
      snapshots:
        enabled: true
        bounding_box: true

What Iā€™d like to achieve is object detection persons and animals (at first). Person detection is working and I am getting events but every time I am trying to anything to the yml file its give me an error like:

 * Starting nginx nginx
   ...done.
Error parsing config: extra keys not allowed @ data['cameras']['takapiha']['clips']['snapshots']

I am running HA with Intel Nuc docker and Frigate has been installed from HACS.

Check your indentation and match it up with the example config in the docs (https://blakeblackshear.github.io/frigate/configuration/cameras#full-example). Your snapshots: section for the takapiha camera needs to be moved left by two spaces so that it lines up with clips:

I just submitted a pull request to the HassOS repo to build in the gasket/apex drivers as modules. I have it running on my test machine, so hopefully they will incorporate it soon and I can put it on my main machine.

Thxā€¦ Yeapā€¦ that is something what I should check at first (indentation)ā€¦

Anybody have stepbystep instructions for more detailed object detection than just a person?

The docs are your friend again here. Look at the COCO label map for the model that Frigate uses. That will give you a list of all of the trained objects that can be detected. Then, just add that label to your configuration (as you did with ā€œanimalā€ - though ā€œanimalā€ is not a valid object in the label map).

Thxā€¦ Just need a guidance maybe also because some kind of language barriorā€¦

Anywayā€¦ how I clean events manually if not needed anymore after test?

Hi, How would I be able to also view the clips?

I have in config:

clips:
  # Optional: Maximum length of time to retain video during long events. (default: shown below)
  # NOTE: If an object is being tracked for longer than this amount of time, the cache
  #       will begin to expire and the resulting clip will be the last x seconds of the event.
  max_seconds: 300
  # Optional: size of tmpfs mount to create for cache files (default: not set)
  # mount -t tmpfs -o size={tmpfs_cache_size} tmpfs /tmp/cache
  # NOTICE: Addon users must have Protection mode disabled for the addon when using this setting.
  # Also, if you have mounted a tmpfs volume through docker, this value should not be set in your config.
  tmpfs_cache_size: 256m
  # Optional: Retention settings for clips (default: shown below)
  retain:
    # Required: Default retention days (default: shown below)
    default: 100
    # Optional: Per object retention days
    objects:
      person: 150

I see:
image

I can go through my images, they are saved, but no clipsā€¦?

Have you set clips role for the camera?

cameras:
  back:
    ffmpeg:
      inputs:
        - path: rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@10.0.10.10:554/live
          roles:
            - clips

Have tried this, thanks will let know.

Another question. I added a 3rd cam and now I have:

Fatal Python error: Bus error

According to documentaion I need to increase shm size according to https://blakeblackshear.github.io/frigate/installation#calculating-shm-size. But how and where can I do this when I am using portainer to start Frigate?

I calculated the shm:
image
total is 54ā€¦ was 64 and I changed in the docker config file to 96 with:
image
EDIT (128 also not working)

But still it doesnt workā€¦

What am I doing wrong now?

Whats weird, the 1st and 2nd cam seem to keep ruuning (can see the time and live image). Clips is off for the 3rd camā€¦ but taht stops.

EDIT: itā€™s a matter of timeā€¦ if motion is on the other cam it also stopsā€¦

The error:

frigate.app                    INFO    : Capture process started for parking: 70
Fatal Python error: Bus error
Thread 0x00007fcd6e6f2700 (most recent call first):
  File "/usr/lib/python3.8/threading.py", line 302 in wait
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 227 in _feed
  File "/usr/lib/python3.8/threading.py", line 870 in run
  File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap
Current thread 0x00007fcd70ef3700 (most recent call first):
  File "/opt/frigate/frigate/video.py", line 117 in capture_frames
  File "/opt/frigate/frigate/video.py", line 231 in run
  File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap
Thread 0x00007fcd736f4700 (most recent call first):
  File "/opt/frigate/frigate/video.py", line 205 in run
  File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap
Thread 0x00007fcd73ef5700 (most recent call first):
  File "/opt/frigate/frigate/log.py", line 71 in run
  File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap
Thread 0x00007fcd878eb740 (most recent call first):
  File "/usr/lib/python3.8/threading.py", line 1027 in _wait_for_tstate_lock
  File "/usr/lib/python3.8/threading.py", line 1011 in join
  File "/opt/frigate/frigate/video.py", line 245 in capture_camera
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108 in run
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315 in _bootstrap
  File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75 in _launch
  File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19 in __init__
  File "/usr/lib/python3.8/multiprocessing/context.py", line 277 in _Popen
  File "/usr/lib/python3.8/multiprocessing/context.py", line 224 in _Popen
  File "/usr/lib/python3.8/multiprocessing/process.py", line 121 in start
  File "/opt/frigate/frigate/app.py", line 182 in start_camera_capture_processes
  File "/opt/frigate/frigate/app.py", line 228 in start
  File "/opt/frigate/frigate/__main__.py", line 15 in <module>
  File "/usr/lib/python3.8/runpy.py", line 87 in _run_code
  File "/usr/lib/python3.8/runpy.py", line 194 in _run_module_as_main

EDIT: If there is no motion all 3 cams keep workingā€¦ as soon as there is was/some motion it stops.

Hello!

So here it goes.

Running Home Assistant OS 5.12, core-2021.3.2, supervisor-2021.03.4, on Intel NUC.

I read about frigate and said to give it a try.

First I made in Unifi 2 rules for port forwarding 5000 and 1935 from anywhere to IP of NUC only TCP.

Then continuing with installation, I read the part that says ā€œNote that HassOS Addons and custom components are different things. If you are already running Frigate with Docker directly, you do not need the Addon since the Addon would run another instance of Frigate."

So I thought I would just need to install custom component from HACS and nothing else.
Installed it in Hacs but then couldnā€™t add the integration:
Configuration->integrations->+add integration->Frigate-> used ā€œhttp://ccab4aaf-frigate:5000ā€ and got message ā€œUnable to connect to Frigate.ā€

Question 1
Do I really need to add Frigate NVR from Supervisore->Add-on Store? If not, why I get this message? What needs to be configured and where?

Then, I tried the other way. Installed Frigate NVR from Supervisore->Add-on Store, in configuration I let 1935 and 5000 as is, created frigate.yml in config folder and entered:

mqtt:
  host: IP of NUC
  user: my mqtt user
  password: my mqtt password
cameras:
  outdoor:
    ffmpeg:
      inputs:
        - path: rtsp://<username>:<password>@<address>:<port>/Streaming/Channels/102
          roles:
            - detect
            - rtmp
    width: 640
    height: 360
    fps: 5

Then I get the below log when I start the add-on:

 * Starting nginx nginx
   ...done.
frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.app                    INFO    : Camera processor started for outdoor: 38
frigate.app                    INFO    : Capture process started for outdoor: 40
detector.coral                 INFO    : Starting detection process: 36
frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate.edgetpu                INFO    : No EdgeTPU detected.
Process detector:coral:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 111, in __init__
    raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/edgetpu.py", line 124, in run_detector
    object_detector = LocalObjectDetector(tf_device=tf_device, num_threads=num_threads)
  File "/opt/frigate/frigate/edgetpu.py", line 63, in __init__
    edge_tpu_delegate = load_delegate('libedgetpu.so.1.0', device_config)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1.0
frigate.mqtt                   INFO    : MQTT connected
frigate.watchdog               INFO    : Detection appears to have stopped. Exiting frigate...
frigate.app                    INFO    : Stopping...
frigate.events                 INFO    : Exiting event processor...
frigate.record                 INFO    : Exiting recording maintenance...
frigate.object_processing      INFO    : Exiting object processor...
frigate.events                 INFO    : Exiting event cleanup...
frigate.watchdog               INFO    : Exiting watchdog...

Question 2
What am I missing?

pardon my noobynesā€¦

The error youā€™re seeing in the logs is due to the fact that you donā€™t have a Coral device and you havenā€™t defined a CPU detector in the config file. Check the docs.

1 Like

Pulled the latest image and am getting a couple errors in the logs, not sure where to look for the issue.

peewee_migrate                 INFO    : There is nothing to migrate
frigate.mqtt                   INFO    : MQTT connected
detector.coral                 INFO    : Starting detection process: 42
frigate.app                    INFO    : Camera processor started for porch: 45
frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate.app                    INFO    : Capture process started for porch: 46
frigate.edgetpu                INFO    : TPU found
frigate.video                  INFO    : porch: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : porch: ffmpeg process is not running. exiting capture thread...
ffmpeg.porch.detect            ERROR   : [h264_v4l2m2m @ 0x564653183b80] Could not find a valid device
ffmpeg.porch.detect            ERROR   : [h264_v4l2m2m @ 0x564653183b80] can't configure decoder
ffmpeg.porch.detect            ERROR   : Error while opening decoder for input stream #0:1 : Invalid argument

These errors repeat in my log and my web interface is showing green background in the camera view.
The RTMP stream I am using works in vlc.
I am using a Reolink RLC-410 camera.

Thanks for the great work!

Check the docs: https://blakeblackshear.github.io/frigate/configuration/cameras#reolink-410520-possibly-others

Maybe a stupid question perhaps but. ā€¦

How do I allow Frigate to other users through Home assistant app?
It doesnā€™t show up when my wife opens her ha app.