Local realtime person detection for RTSP cameras

Btw I’ve solved it by changing
--device /dev/bus/usb:/dev/bus/usb
to
-v /dev/bus/usb:/dev/bus/usb

1 Like

Where can I change/train//fine-tune this:

image

image

Just curious,
Are you using substream or mainstream for detection?
frigate hasn’t detect my cats yet on substream

mainstream

My dogs show as humans. A cat would be preferred

1 Like

Just updated HA, but I’m not sure, if it is related.

1 Like

As long as you girl friend is not shown as cat. All good.

1 Like

This didn’t go down well…

notadog

7 Likes

:rofl: :rofl: :rofl:
Thank you for sharing

Ladies and Gentlemen , I have just installed Coral TPU M.2 A+E in HP Thin Client T430 and all is working fantastic. I`m running 5 cams with 7ms inference !

Running T430 with Debian, Frigate in Docker, config with MQTT, HA Integration runs great.

thank you @shukkar for your guide earlier, it helped a lot, question: Do you have HW acceleration enabled in your config? I do not and inference is 7ms.

@unrly

2 Likes

Hwaccel for ffmpeg is completely unrelated to inference times and separate from the job that the coral performs. You always want both if possible. The Coral does object detection and the hwaccel args for ffmpeg are for decoding the video stream.

Does this dog know how to cook? xD

1 Like

Thank you for letting us know about the results!

I think I may have to give it a shot soon. Edit: That T430 might be exactly what I was looking for. Knowing that I may pick one up as well.

I’m setting up Frigate on an old zbox hd11 (nvidia ION) and Coral USB but i got this error when i rise up the container after having pulled it.

i’m using docker-compose and this image:

image: blakeblackshear/frigate:stable-amd64

root@cams:~/docker-compose.d/frigate# docker-compose up


Starting frigate ... done
Attaching to frigate
frigate    |  * Starting nginx nginx
frigate    |    ...done.
frigate    | Fatal Python error: Illegal instruction
frigate    |
frigate    | Current thread 0x00007fcf4a990740 (most recent call first):
frigate    |   File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
frigate    |   File "<frozen importlib._bootstrap_external>", line 1101 in create_module
frigate    |   File "<frozen importlib._bootstrap>", line 556 in module_from_spec
frigate    |   File "<frozen importlib._bootstrap>", line 657 in _load_unlocked
frigate    |   File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
frigate    |   File "<frozen importlib._bootstrap>", line 991 in _find_and_load
frigate    |   File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
frigate    |   File "<frozen importlib._bootstrap>", line 1042 in _handle_fromlist
frigate    |   File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 36 in <module>
frigate    |   File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
frigate    |   File "<frozen importlib._bootstrap_external>", line 783 in exec_module
frigate    |   File "<frozen importlib._bootstrap>", line 671 in _load_unlocked
frigate    |   File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
frigate    |   File "<frozen importlib._bootstrap>", line 991 in _find_and_load
frigate    |   File "/opt/frigate/frigate/edgetpu.py", line 15 in <module>
frigate    |   File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
frigate    |   File "<frozen importlib._bootstrap_external>", line 783 in exec_module
frigate    |   File "<frozen importlib._bootstrap>", line 671 in _load_unlocked
frigate    |   File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
frigate    |   File "<frozen importlib._bootstrap>", line 991 in _find_and_load
frigate    |   File "/opt/frigate/frigate/app.py", line 19 in <module>
frigate    |   File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
frigate    |   File "<frozen importlib._bootstrap_external>", line 783 in exec_module
frigate    |   File "<frozen importlib._bootstrap>", line 671 in _load_unlocked
frigate    |   File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
frigate    |   File "<frozen importlib._bootstrap>", line 991 in _find_and_load
frigate    |   File "/opt/frigate/frigate/__main__.py", line 7 in <module>
frigate    |   File "/usr/lib/python3.8/runpy.py", line 87 in _run_code
frigate    |   File "/usr/lib/python3.8/runpy.py", line 194 in _run_module_as_main

Any ideas?

Thank You!

Your CPU needs to have the AVX instruction set for TensorFlow to work.

This is a “feature” of the iOS app. If you use the same URL as your internal or external URL as configured in the app, it assumes you want to open it in a web view inside the app itself. The way I got around it is by adding another subdomain for media.mydomain.com that is different than my external URL, but still points to Home Assistant.

I also created a pretty thorough blueprint you can use as well here: Frigate Mobile App Notifications

ooohhhh… thank you…
maybe i supposed that even a commodore64 with usb coral could work… :slight_smile:

thank you

Frigate is an amazing project. However, I still cannot get hardware acceleration to work with Frigate. Whenever I enable it the camera output goes green and the log shows an error ffmpeg hardware acceleration sent a broken frame. memoryview assignment: lvalue and rvalue have different structures

I have tried on a RPi 4 running HassOS and now have tried on VM running in Virtualbox on an Ubuntu 20.04 host. I am running 8 cameras - 4 Sercomm iCamera2, and 4 Wyze cams (with the RTSP firmware). I could not get hardware acceleration working on either machine with any of the cameras.

Is it just my cameras?
Has anyone had any success with hardware acceleration on a Virtualbox VM? I have enabled Nested VT-x/AMD-V in the VM.I have also installed the VA-API on the host. When I run vainfo I get the following output:

libva info: VA-API version 1.7.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_7
libva error: /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so init failed
libva info: va_openDriver() returns 1
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_6
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.7 (libva 2.6.0)
vainfo: Driver version: Intel i965 driver for Intel(R) Ivybridge Desktop - 2.4.0
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Simple            : VAEntrypointEncSlice
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileH264StereoHigh         : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc
      VAProfileJPEGBaseline           : VAEntrypointVLD

I am not sure if the errof ro the iHD driver is significant or not because the i965 driver seems to be working.

I would appreciate any suggestions.

Same error when i put the hardware acceleration on my RPI4
Config:
RPI4 (4Go)
Debian Buster 10 + Docker HomeAssistant + Addin Frigate

INFO : camera_entree: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video INFO : camera_entree: ffmpeg process is not running. exiting capture thread…
ffmpeg.camera_entree.detect ERROR : mmal: mmal_vc_shm_init: could not initialize vc shared memory service
ffmpeg.camera_entree.detect ERROR : mmal: mmal_vc_component_create: failed to initialise shm for ‘vc.ril.video_decode’ (7:EIO)
ffmpeg.camera_entree.detect ERROR : mmal: mmal_component_create_core: could not create component ‘vc.ril.video_decode’ (7)
ffmpeg.camera_entree.detect ERROR : Error while opening decoder for input stream #0:0 : Unknown error occurred
frigate.video INFO : camera_entree: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video

Blake shared the following in another post. You can access Reolink RTMP streams as follows: (Don’t forget to add the ffmeg input params in the config file - I have added them below).
Main stream:
rtmp://192.168.1.xx/bcs/channel0_main.bcs?channel=0&stream=0&user=admin&password={FRIGATE_RTSP_PASSWORD}
Sub stream:
rtmp://192.168.1.xx/bcs/channel0_sub.bcs?channel=0&stream=1&user=admin&password={FRIGATE_RTSP_PASSWORD}

ffmpeg:
  input_args: #Use for RTMP
    - -avoid_negative_ts
    - make_zero
    - -fflags
    - nobuffer
    - -flags
    - low_delay
    - -strict
    - experimental
    - -fflags
    - +genpts+discardcorrupt
    - -rw_timeout
    - '5000000'
    - -use_wallclock_as_timestamps
    - '1'
1 Like