Local realtime person detection for RTSP cameras

Did you figure out how to reduce FPS in output_args?

Thanks, I got it running on my DS918+ with DSM 7, getting FPS of about 50 with 2K camera.
Did you set up some hwaccel?
Passed coral as USB 3 device to VMM on my Synology…

1 Like

Nope… output_args simply doesn’t work referring to my previous post.

Hi I am having this error


detector.coral                 INFO    : Starting detection process: 39
frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate.app                    INFO    : Capture process started for office2: 45
frigate.edgetpu                INFO    : No EdgeTPU detected.
Process detector:coral:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 111, in __init__
    raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/edgetpu.py", line 124, in run_detector
    object_detector = LocalObjectDetector(tf_device=tf_device, num_threads=num_threads)
  File "/opt/frigate/frigate/edgetpu.py", line 63, in __init__
    edge_tpu_delegate = load_delegate('libedgetpu.so.1.0', device_config)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1.0

This is my config


detectors:
  coral:
    type: edgetpu
    device: usb
    num_threads: 3

I did install a Google Coral USB, on my USB port (proxmox) I have this 1a6e:089a

any hint?

if I do lsusb


root@xxx:~# lsusb
Bus 002 Device 003: ID 18d1:9302 Google Inc.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 004: ID 0658:0200 Sigma Designs, Inc. Aeotec Z-Stick Gen5 (ZW090) - UZB
Bus 001 Device 003: ID 0403:6015 Future Technology Devices International, Ltd Bridge(I2C/SPI/UART/FIFO)
Bus 001 Device 002: ID 8087:0aaa Intel Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
root@xxx:~#

EDIT: SOLVED in proxmox use passthrough instead of vendor id and now Corsl is recogfnized

Having a problem getting the Docker container to start up. I am running HA Core, which makes the install more complicated, I assume.

I am also a complete Docker NOOB and therefore am basically fumbling my way through here.

Here are my config files and logs, if that helps.

mqtt:
  host: localhost
  port: 1883
  topic_prefix: frigate
  user: frigate
  password: *****
  stats_interval: 60
cameras:
  garage:
    ffmpeg:
      inputs:
        - path: rtsp://admin:*****@192.168.1.***:554//h264Preview_01_sub
          roles:
            - detect
            - rtmp
            - record
      input_args:
        - -avoid_negative_ts
        - make_zero
        - -fflags
        - nobuffer
        - -flags
        - low_delay
        - -strict
        - experimental
        - -fflags
        - +genpts+discardcorrupt
        - -use_wallclock_as_timestamps
        - "1"
        - -rw_timeout
        - "5000000"
      hwaccel_args:
        - -hwaccel
        - vaapi
        - -hwaccel_device
        - /dev/dri/renderD128
        - -hwaccel_output_format
        - yuv420p
    width: 1280
    height: 720
    fps: 5
    detect:
      enabled: True
      max_disappeared: 25
    record:
      enabled: True
      retain_days: 3
    rtmp:
      enabled: True
    objects:
      track:
        - person
        - car
        - motorcycle
    clips:
      enabled: False
      pre_capture: 5
      post_capture: 5
      tmpfs_cache_size: 256m
      objects:
        - person
        - car
      retain:
        default: 10
        objects:
          person: 15
version: "3.6"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: blakeblackshear/frigate:0.9.0-rc2-amd64
    devices:
      - /dev/bus/usb:/dev/bus/usb
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /config/frigate.yml:/config/config.yml:ro
      - /media/TV/Cams:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "1935:1935"
    environment:
      FRIGATE_RTSP_PASSWORD: *****

frigate    | [fix-attrs.d] applying ownership & permissions fixes...
frigate    | [fix-attrs.d] done.
frigate    | [cont-init.d] executing container initialization scripts...
frigate    | [cont-init.d] done.
frigate    | [services.d] starting services
frigate    | [services.d] done.
frigate    | Starting migrations
frigate    | [2021-09-04 13:59:34] peewee_migrate                 INFO    : Starting migrations
frigate    | There is nothing to migrate
frigate    | [2021-09-04 13:59:34] peewee_migrate                 INFO    : There is nothing to migrate
frigate    | [Errno 99] Cannot assign requested address
frigate    | [cmd] python3 exited 1
frigate    | [cont-finish.d] executing container finish scripts...
frigate    | [cont-finish.d] done.
frigate    | [s6-finish] waiting for services.
frigate    | [s6-finish] sending all processes the TERM signal.
frigate    | [s6-finish] sending all processes the KILL signal and exiting.
frigate    | [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
frigate    | [s6-init] ensuring user provided files have correct perms...exited 0.
frigate    | [fix-attrs.d] applying ownership & permissions fixes...
frigate    | [fix-attrs.d] done.
frigate    | [cont-init.d] executing container initialization scripts...
frigate    | [cont-init.d] done.
frigate    | [services.d] starting services
frigate    | [services.d] done.
frigate    | Starting migrations
frigate    | [2021-09-04 14:00:38] peewee_migrate                 INFO    : Starting migrations
frigate    | There is nothing to migrate
frigate    | [2021-09-04 14:00:38] peewee_migrate                 INFO    : There is nothing to migrate
frigate    | [Errno 99] Cannot assign requested address
frigate    | [cmd] python3 exited 1
frigate    | [cont-finish.d] executing container finish scripts...
frigate    | [cont-finish.d] done.
frigate    | [s6-finish] waiting for services.
frigate    | [s6-finish] sending all processes the TERM signal.
frigate    | [s6-finish] sending all processes the KILL signal and exiting.

Thanks!

EDIT:
So it seems the issue is connection with the MQTT server. I did confirm the username and password work when I use the mosquitto_sub command to listen to the frigate topic and publish some stuff from the HA plugin. So, not sure why frigate can’t connect.

Just for test, try change localhost to the IP adress of the mqtt server

No, I haven’t set hwaccel. I did try, had some issues and left that part for whenever I get more time.

OK, then the difference in fps might come from different resolution of the camera stream and the difference between DS918+ and DS920+

The MQTT server is local. But just as a test I did change to the IP of the machine and it also does not work.

OK. Solved the issue. I have NAT loopback on my router and thus had to reference the local DNS name rather than the IP. That works.

1 Like

I have fixed FPS to 7 fps. Recommendation from documentation is 5 fps for recognition. But since my cameras don’t have option of 5fps, only 4 or 7, used that value.

A small question…
To use the Frigate (blakeblackshear/frigate:stable-amd64nvidia) docker, is the
nVidia closed source driver still required when running on Ubuntu 20 LTE?

Sorry I ment Coral Inference Speed

Any one managed to set up frigate with Nvidia?
Trying but it fails big time… My Linux knowledge must be worse then a … :frowning:

Using ubuntu 20.4 LTE and have a Nvidia 1050 card.

@Minglarn
Below my docker compose. You must have the nvidia drivers installed in the host

  frigate: # https://blakeblackshear.github.io/frigate/
    container_name: frigate
    image: blakeblackshear/frigate:0.9.0-rc2-amd64nvidia 
    restart: unless-stopped
    privileged: true
    hostname: UNRAID  
    shm_size: '8gb'
    volumes:
    - /mnt/cache/appdata/cctv/frigate:/config
    - /etc/localtime:/etc/localtime:ro
    - type: bind
      source: /mnt/disks/storage/cctv/frigate
      target: /media/frigate
      bind:
        propagation: slave
    - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
      target: /tmp/cache
      tmpfs:
        size: '2g'
    environment: 
    - FRIGATE_RTSP_PASSWORD=<>
    - NVIDIA_VISIBLE_DEVICES=<>
    - NVIDIA_DRIVER_CAPABILITIES=all
    ports:
    - 5000:5000
    - 1935:1935 # RTMP feeds
    labels:
    - "com.centurylinklabs.watchtower.enable=true"  
    runtime: nvidia
1 Like

Nvidia drivers installed… I think…

root@ubuntuserver:/home/minglarn/frigate# nvidia-smi
Mon Sep  6 16:42:21 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:01:00.0 Off |                  N/A |
| 30%   26C    P8    N/A /  75W |     13MiB /  1999MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1084      G   /usr/lib/xorg/Xorg                  8MiB |
|    0   N/A  N/A      1260      G   /usr/bin/gnome-shell                2MiB |
+-----------------------------------------------------------------------------+

Seems right?

  frigate: # https://blakeblackshear.github.io/frigate/
    container_name: frigate
    image: blakeblackshear/frigate:0.9.0-rc2-amd64nvidia 
    ...
    runtime: nvidia

[/quote]
When adding “runtime” I get a error:

ERROR: for frigate  Cannot create container for service frigate: Unknown runtime specified nvidia

And all the fun stops here… Dont really know what to do… :frowning:

Maybe?

I managed to do something…
Seems that I was missing nvidia-docker2

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:01:00.0 Off |                  N/A |
| 30%   36C    P0    N/A /  75W |    931MiB /  1999MiB |     26%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1084      G   /usr/lib/xorg/Xorg                  8MiB |
|    0   N/A  N/A      1260      G   /usr/bin/gnome-shell                2MiB |
|    0   N/A  N/A    272960      C   ffmpeg                             58MiB |
|    0   N/A  N/A    272966      C   ffmpeg                             58MiB |
|    0   N/A  N/A    272967      C   ffmpeg                             58MiB |
|    0   N/A  N/A    272984      C   ffmpeg                             58MiB |
|    0   N/A  N/A    272996      C   ffmpeg                            257MiB |
|    0   N/A  N/A    273001      C   ffmpeg                            257MiB |
|    0   N/A  N/A    273002      C   ffmpeg                             58MiB |
|    0   N/A  N/A    273003      C   ffmpeg                             49MiB |
|    0   N/A  N/A    273007      C   ffmpeg                             58MiB |
+-----------------------------------------------------------------------------+

Looks good… :slight_smile:

2 Likes

When trying to install this via docker-compose I keep getting:

[+] Running 0/1
 - Container frigate  Creating                                                                                    40.3s
Error response from daemon: Post "http://ipc/filesharing/share": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Then in docker it shows:
Error parsing config: [Errno 2] No such file or directory: '/config/config.yml'

It used to work but after updating docker nothing works…