Local realtime person detection for RTSP cameras

tc23 · August 14, 2021, 12:31am

Strange, I got the message with brower_mod, variable, and frigate. Which is weird since frigate has an integration. I tried multiple restarts but all got the same message. I upgraded from 2021.5.4. did I miss anything in between?

jhart003 · August 14, 2021, 12:49am

My frigate integrations stopped working when I upgraded to 2021.8.3 last weekend so I went back to 2021.7.4. There was a message posted in this thread about 16 days ago that indicated a fix was already in the works. I will try upgrading to 2021.8.6 tomorrow.

tc23 · August 14, 2021, 2:46am

Good to know I’m not the only one. Let me know if you experience the same thing. I didn’t see any updates on the aforementioned integration, so that why I wanted to see if it was isolated to me or others were experiencing it

mcpaha · August 14, 2021, 8:40am

Sorry for probably stupid question. I really don’t understand why this object mask doesn’t enabled (at the right place&shape) even if I restart Frigate. Config.yml:

  Mokki2:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:1111/cam/realmonitor?channel=1&subtype=0
          roles:
            - clips
            - rtmp
            - detect
    width: 1920
    height: 1080
    fps: 5
    clips:
      enabled: true
      pre_capture: 15
      post_capture: 15
      retain:
        default: 2
    motion:
      mask:
        - 255,28,259,143,37,197,0,181
        - 1858,89,1858,33,1444,38,1447,92
    snapshots:
      enabled: true
      bounding_box: true
    objects:
      track:
        - person
        - dog
        - car
        - bicycle
        - bird
      filters:
        car:
          mask:
            - 1237,537,1550,559,1567,465,1246,430

What I see when checking:

So mask is totally wrong shape and wron position and I get multiple events from parked car:

And one thing more… if I want add (to specific camera) option save audio also… how to do that (I know " need to override the output args to remove -an") but I don’t have any output args anywhere so how to override that?

No new messages at 4d??? What’s going on??

lsaranto · August 18, 2021, 3:19pm

I really like the concept of Frigate. However, I’ve never gotten good video playback with Frigate. I’m using an RTMP stream from Reolink RLC-410W camera. The stream is set to be 1920x1080@15fps. Ffprobe shows 1920x1072@15fps. The Frigate UI live preview shows “Displaying at 2.6fps” (on average). Why? The recorded clips play back at a similar framerate.

The host CPU is 50% max and I’m using Coral USB which appears to be working.

cweakland · August 18, 2021, 4:43pm

Go into “debug” > the camera in question (Mokki2 ? ) > show options > Mask & Zone creator > and click on the mask you created, it should show up on the image above. I see you have a motion mask and an object mask, try and isolate them, perhaps remove them all and add them back one at a time.

As for the output args, you can find them here.

mcpaha · August 18, 2021, 5:59pm

I managed create object mask to the right place for putting it twice to the config.yml and after that removed second one…

What comes to the output args - I know where I can find them… Is that audio argument global argument or per camera?

Also… sorry my poor English…How can I remove output args -an if I don’t have any output args? This is something I don’t understand…

So here is start of my config… If I want to add audio “etupiha” camera what should be read and where? And what if I want to save audio for all cameras?

detectors:
  coral:
    type: edgetpu
    device: usb

mqtt:
  host: 192.168.1.60
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: xxx
  password: xxx

cameras:
  eteinen:
    ffmpeg:
      inputs:
        - path: rtsp://xxx:[email protected]:554/stream1
          roles:
            - rtmp
            - detect
    width: 1920
    height: 1080
    fps: 5

  etupiha:
    ffmpeg:
      inputs:
        - path: rtsp://xxx:[email protected]:554/Streaming/Channels/1
          roles:
            - clips
            - rtmp
            - detect
    width: 1920
    height: 1080
    fps: 5
    clips:
      enabled: true
      pre_capture: 15
      post_capture: 15
      retain:
        default: 2
    motion:
      mask: 0,771,257,420,407,351,625,162,694,138,673,0,670,0,656,0,344,0,167,0,0,0
    snapshots:
      enabled: true

lesterleong · August 19, 2021, 3:35pm

Look like really hard to get hold an Coral USB , everywhere out of stock. I’m thinking to get the m.2 version and run frigate on docker container. Any recommendation on hardware that can use m.2 version and to handle 8 unit cameras?

hasshoolio · August 20, 2021, 1:52am

Frigate 8, I did a pull from the hass integration repo and with 2021.8.8 here and same issue

Setup failed for frigate: Unable to import component: cannot import name ‘Message’ from ‘homeassistant.components.mqtt.models’ (/usr/src/homeassistant/homeassistant/components/mqtt/models.py)

Found this 2021.8.0 - Unable to import component: cannot import name 'Message' from 'homeassistant.components.mqtt.models' · Issue #124 · blakeblackshear/frigate-hass-integration · GitHub will see what I’ve got wrong

solved - it helps if i properly copy the files to the hass directory

SpikeyGG · August 20, 2021, 3:34am

I’ve been watching my Detection FPS chart since I swapped my USB Coral Accelerator from using USB 2.0 → USB 3.0. The overall Detection FPS ceiling has risen to 80 fps (for 7 cameras) but I’ve noticed something that I don’t understand. I’m using all my cameras at 8 fps but the detection fps chart in Home Assistant shows that a single camera can have a detection fps of 40+ on its own… how does that work? Shouldn’t each camera top out at 8 if that’s what the limit is?

Overall:

Per camera:

iMiMx · August 20, 2021, 6:04am

I assume, that as it doesn’t send the entire image for object detection - but rather the sections that are resized to 300x300 - there can/are multiple object detections, per entire image, at once often happening.

birca1987 · August 20, 2021, 9:20am

Defenetly not a job for a Intel NUC or similar. I am running frigate on a Lenovo ThinkCentre M700 Tiny that has m.2 port plus SATA and i5-6500T.
With 5 * 1080p cams the CPU is idle at around 12%.

SpikeyGG · August 20, 2021, 10:27am

That makes much more sense. Thanks @iMiMx

Klagio · August 20, 2021, 10:36am

Got my Coral USB device, will try it now. Is there any tutorial beside Ggithub?

tc23 · August 20, 2021, 2:33pm

Just updated. Thanks!

Klagio · August 20, 2021, 3:01pm

Ok I am doing my first configuration.

In Home Assistant I have a bunch of Hikvision cameras for which I use the generic platform, I guess those are not good for Frigate?


  - platform: generic
    name: "East one"
    still_image_url: http://admin:[email protected]/ISAPI/Streaming/channels/101/picture
    stream_source: rtsp://admin:[email protected]:554/ch1/main/av_stream

lesterleong · August 20, 2021, 3:01pm

I’m running my HA in virtualbox, my host machine is an i7 16gb ram windows 10 pc. Planning to add a m.2 coral for frigate so should I install frigate as HA addon or will it possible to run on a separate docker with Docker Desktop for Windows?

birca1987 · August 20, 2021, 4:24pm

Not sure if this is possible, never seen it on this forum. The Frigate docs are advising against virtualization, but instead run on host machine under Linux.

John_Gratis · August 20, 2021, 5:44pm

So im a bit stuck. Used frigate with cpu as a test and it worked great. Got the coral usb and I simply cannot get frigate to find it. Hoping someone has experienced this and has some tips to try.

Things I have already tried:
unplug and plug it (coral usb) back in
reboot host
change usb ports
using docker
using podman
using podman-compose
removing detector portion of config - still fails to find it, but doesnt keep restarting the container

ubuntu 20.04 x64 bit
podman (tried docker as well)

lsusb -v

Bus 002 Device 002: ID 1a6e:089a Global Unichip Corp.

podman exec -t frigate find /dev/bus
/dev/bus
/dev/bus/usb
/dev/bus/usb/001
/dev/bus/usb/001/001
/dev/bus/usb/001/002
/dev/bus/usb/001/003
/dev/bus/usb/002
/dev/bus/usb/002/001
/dev/bus/usb/002/002

config.yml:

mqtt:
  host: 192.168.1.10
  port: 1883
  user: frigate
  password: passphrase
cameras:
  front_door:
    ffmpeg:
      inputs:
        - path: rtsp://rspuser:{FRIGATE_RTSP_PASSWORD}@192.168.1.210:8554/unicast
          roles:
            - clips
            - detect
            - rtmp
    width: 1280
    height: 720
    fps: 5
    zones:
      zone_0:
        coordinates: 618,643,621,189,443,202,441,599
      zone_1:
        coordinates: 1241,720,1255,167,895,172,868,626
      zone_2:
        coordinates: 457,254,373,252,355,452,452,498
clips:
  retain:
    default: 20
    objects:
      person: 45
detectors:
  coral:
    type: edgetpu
    device: usb
  cpu:
    type: cpu

docker-compose.yml

version: "3.9"
services:
  frigate:
    container_name: frigate
    restart: unless-stopped
    image: blakeblackshear/frigate:stable-amd64
    devices:
       - /dev/bus/usb/:/dev/bus/usb/
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /home/frigate/config.yml:/config/config.yml:ro
      - /sanman/Podman/cameras:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "1935:1935"
    environment:
      FRIGATE_RTSP_PASSWORD: "12345"

podman logs frigate
 * Starting nginx nginx
   ...done.
frigate.app                    WARNING : Camera front_door has clips assigned to an input, but clips is not enabled.
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.mqtt                   INFO    : MQTT connected
detector.coral                 INFO    : Starting detection process: 35
cameras@ajalon-Rampage-Formula:~$ podman logs frigate
 * Starting nginx nginx
   ...done.
frigate.app                    WARNING : Camera front_door has clips assigned to an input, but clips is not enabled.
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.mqtt                   INFO    : MQTT connected
detector.coral                 INFO    : Starting detection process: 35
detector.cpu                   INFO    : Starting detection process: 37
frigate.app                    INFO    : Camera processor started for front_door: 40
frigate.edgetpu                INFO    : Attempting to load TPU as usb
Process detector:coral:
frigate.edgetpu                INFO    : No EdgeTPU detected.
frigate.app                    INFO    : Capture process started for front_door: 41
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 111, in __init__
    raise ValueError(capture.message)
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/edgetpu.py", line 124, in run_detector
    object_detector = LocalObjectDetector(tf_device=tf_device, num_threads=num_threads)
  File "/opt/frigate/frigate/edgetpu.py", line 63, in __init__
    edge_tpu_delegate = load_delegate('libedgetpu.so.1.0', device_config)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1.0

frigate.watchdog               INFO    : Detection appears to have stopped. Exiting frigate...
frigate.app                    INFO    : Stopping...

Minglarn · August 20, 2021, 8:24pm

So I have the issue if frigate docker is using the Quicksync option in frigate.
If I do:

cat /proc/cpuinfo  | grep "model name" | head -1
model name      : Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz

And according to Intel ARK it is enabled in the CPU.
But. I dont know if Frigate is utilizing it…
In config.yml I have this:

ffmpeg:
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p

But even if I remove these lines the CPU usage is the same… How do I know if the right ffmpeg settings are used?