Local realtime person detection for RTSP cameras

Yes. Same reason as everything else.

OK :frowning_face:
Tought this could be a better way for my dashboard. At the moment Iā€™am using web-rtc, but this is not reliable for me and has quite long loading times.

Anyone got any thoughts on my CPU use? Iā€™ve got an i7 NUC with 16GB RAM and a Coral USB connected. With the Frigate add-on stopped, my sensor.processor_use sits at around 6~8%. When I activate Frigate, my CPU climbs quickly to sit around 23~30% depending on how windy it is (motion events).

My initial understanding was that all the processing would be off-loaded to the coral but this does not seem to be the case.

Frigate NVR 1.13
core-2021.6.6

Not all processing is done on the coral, CPU is needs for ffmpeg transcoding which is documented here in the Frigate docs. If ffmpeg is not enabled it will use software transcoding which uses a lot of CPU processing. The recommended way is to use CPU or external GPU card for hardware acceleration and coral for detectors. Documented here Hardware acceleration

Thanks for the prompt reply. I have the following config already for my older NUC.

ffmpeg:
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p

I did not however have ffmpeg enabled in config so I did this and rebooted. Sadly this has made no difference. The reason Iā€™m asking is that with the higher CPU, it seems that some of my automations have become a tad sluggish.

I have some indoor and outdoor cams connected to Frigate. I have set automation which send me notifications (telegram) if alarm panel is set to ā€œarmed awayā€. Something like this:

frigate:
  automation:
    - alias: Frigate notify of detected objects
      trigger:
        platform: mqtt
        topic: frigate/events
      condition:
        - "{{ trigger.payload_json['after']['entered_zones']|length > 0 }}"
        - "{{ trigger.payload_json['type'] != 'end' }}"
        - condition: state
          entity_id: alarm_control_panel.home
          state: armed_away

I want to set notifications to the outdoor cams only if alarm panel is set to the ā€œarmed homeā€ . I canā€™t figure out how to create such condition in automation. Is it even possible?

Make sure that you have also the coral detector in the config, documented as well on that website. As well it depends on the amount of cameras and the picture resolution and frames per second. I have configured my 5 cams for h264 video and 1080p resolution at 10fps. In my frigate config I have the parameter to take second frame which give Frigate 5fps for processing per camera.

Using this brilliantly on my dvr using rtsp. But as I have a problem with speeding vehicles id like to collate info on vehicle speed. Is there a detector for that or can it be added?

i would suggest to use frigate car object clip generated to parse the file to some other API soft to do this kind of detection
first google link

Or just use opencv :smiley:

I have a custom 2nd stream on all 5 cameras in h.264 @ 720P and this config. Logs tell me coral has been detected.

    width: 1280
    height: 720
    fps: 5
detectors:
  coral:
    type: edgetpu
    device: usb

Whatā€™s your CPU load like with 5 cameras?

what camera do you use?

I have a Lenovo mini pc with i5-6500T with 5 cams at 1080p it sitting at 13% CPU usage.

1 Like

5 8MP Cams on NUC8i5 with proxmox-lxc-docker and 7 other vms at 25% average (with coral).

1 Like

11x dahua cameras 1080p 5fps
22-40% 4CPUs - no GPU
with coral

2 Likes

Iā€™ve noticed thereā€™s been no updates to my Frigate NVR 1.13, is this still the latest version?

Yes it is the latest

Just had a brain wave! Datasheet says Coral is USB3 and Iā€™m sure mine is plugged into the front of my NUC which might be USB2!

[EDIT] Bugger, all four external USB ports are USB3ā€¦

Just set it up and it keeps crashing, here is the log from the Addon itself:

* Starting nginx nginx
   ...done.
frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.mqtt                   INFO    : MQTT connected
detector.coral                 INFO    : Starting detection process: 45
frigate.app                    INFO    : Camera processor started for front: 48
Process detector:coral:
frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate.app                    INFO    : Capture process started for front: 49
frigate.edgetpu                INFO    : No EdgeTPU detected.
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 111, in __init__
    raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/edgetpu.py", line 124, in run_detector
    object_detector = LocalObjectDetector(tf_device=tf_device, num_threads=num_threads)
  File "/opt/frigate/frigate/edgetpu.py", line 63, in __init__
    edge_tpu_delegate = load_delegate('libedgetpu.so.1.0', device_config)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1.0
frigate.watchdog               INFO    : Detection appears to have stopped. Exiting frigate...
frigate.app                    INFO    : Stopping...
frigate.events                 INFO    : Exiting event processor...
frigate.record                 INFO    : Exiting recording maintenance...
frigate.object_processing      INFO    : Exiting object processor...
frigate.events                 INFO    : Exiting event cleanup...
frigate.watchdog               INFO    : Exiting watchdog...
frigate.stats                  INFO    : Exiting watchdog...
peewee.sqliteq                 INFO    : writer received shutdown request, exiting.
root                           INFO    : Waiting for detection process to exit gracefully...
frigate.video                  INFO    : front: exiting subprocess

And the log from the System itself:

21-07-02 15:52:06 WARNING (MainThread) [supervisor.misc.tasks] Watchdog found a problem with ccab4aaf_frigate!
21-07-02 15:52:06 INFO (SyncWorker_4) [supervisor.docker.interface] Cleaning addon_ccab4aaf_frigate application
21-07-02 15:52:08 INFO (SyncWorker_4) [supervisor.docker.addon] Starting Docker add-on ccab4aaf/amd64-addon-frigate with version 1.13
21-07-02 15:52:46 INFO (MainThread) [supervisor.snapshots] Found 0 snapshot files
21-07-02 15:52:46 INFO (MainThread) [supervisor.snapshots] Found 0 snapshot files
21-07-02 15:53:38 WARNING (MainThread) [supervisor.misc.tasks] Watchdog found a problem with ccab4aaf_frigate!
21-07-02 15:53:38 INFO (SyncWorker_3) [supervisor.docker.interface] Cleaning addon_ccab4aaf_frigate application
21-07-02 15:53:40 INFO (SyncWorker_3) [supervisor.docker.addon] Starting Docker add-on ccab4aaf/amd64-addon-frigate with version 1.13
21-07-02 15:54:09 INFO (SyncWorker_2) [supervisor.docker.interface] Stopping addon_ccab4aaf_frigate application
21-07-02 15:54:21 INFO (SyncWorker_2) [supervisor.docker.interface] Cleaning addon_ccab4aaf_frigate application
21-07-02 15:55:06 INFO (SyncWorker_1) [supervisor.docker.addon] Starting Docker add-on ccab4aaf/amd64-addon-frigate with version 1.13
21-07-02 15:55:40 ERROR (MainThread) [supervisor.api.ingress] Ingress error: Cannot connect to host 172.30.33.5:5000 ssl:default [Connect call failed ('172.30.33.5', 5000)]
21-07-02 15:55:40 WARNING (MainThread) [supervisor.misc.tasks] Watchdog found a problem with ccab4aaf_frigate!
21-07-02 15:55:40 INFO (SyncWorker_5) [supervisor.docker.interface] Cleaning addon_ccab4aaf_frigate application
21-07-02 15:55:41 ERROR (MainThread) [supervisor.utils] Can't execute stop while a task is in progress
21-07-02 15:55:43 INFO (SyncWorker_5) [supervisor.docker.addon] Starting Docker add-on ccab4aaf/amd64-addon-frigate with version 1.13
21-07-02 15:56:05 INFO (SyncWorker_5) [supervisor.docker.interface] Cleaning addon_ccab4aaf_frigate application
21-07-02 15:56:08 INFO (SyncWorker_5) [supervisor.docker.addon] Starting Docker add-on ccab4aaf/amd64-addon-frigate with version 1.13
21-07-02 15:57:43 WARNING (MainThread) [supervisor.misc.tasks] Watchdog found a problem with ccab4aaf_frigate!
21-07-02 15:57:43 INFO (SyncWorker_4) [supervisor.docker.interface] Cleaning addon_ccab4aaf_frigate application
21-07-02 15:57:45 INFO (SyncWorker_4) [supervisor.docker.addon] Starting Docker add-on ccab4aaf/amd64-addon-frigate with version 1.13
21-07-02 15:59:15 WARNING (MainThread) [supervisor.misc.tasks] Watchdog found a problem with ccab4aaf_frigate!
21-07-02 15:59:15 INFO (SyncWorker_2) [supervisor.docker.interface] Cleaning addon_ccab4aaf_frigate application
21-07-02 15:59:17 INFO (SyncWorker_2) [supervisor.docker.addon] Starting Docker add-on ccab4aaf/amd64-addon-frigate with version 1.13

I have tried in both privileged and non modes and no difference, it keeps crashing.

Running HassOS on ESXi with the Frigate addon, I did increase the RAM and CPU allocated to the system but that has made no difference.

How do I find out what is causing the addon to crash?

How did you make the USB port available to your VM? The ID changes on first use. So if you share by ID, it wonā€™t work. Share the whole USB port instead.

Hello,
I have the PI4 4GB with HassOS. 6 Camera Streams in Frigate. It works but the CPU is at 80%.
To reduce the load I wanted to run 3 cameras via a second PI4 with Frigate. But how can I configure ā€œSupport for multiple Frigate instancesā€ in Settings Integration?