Local realtime person detection for RTSP cameras

Sorry also forgot to say, it does not permanently change because it is wiped out at the next boot of the host.

hmm I’m not sure then. Mine stayed as Google once it identified and has remained so through several reboots.

There is nothing in frigate’s source code related to this. All of that is in the underlying tensorflow lite API provided by Google. There is nothing I can do. Why not run this on the Ubuntu host?

Ok thanks @blakeblackshear. I ran in a VM because that (or Docker - but I wanted a ‘Supervised’ installation) seems to be the recommended installation on the HA website. I also want to use the Ubuntu host to double as a NAS.

Google responded to my query about this swapping from the Global Unichip ID to the Google ID saying that it is the ‘intended behaviour’ (goodness knows why).

The workaround I have applied is to use Virtualbox because this will boot up even though a specified USB device is not present in the host. So I put both the Global Unichip and Google Inc device IDs in the USB and it seems to get sorted out automatically which one to access. It also survives a reboot of the host which is what I was after.

KVM on the other hand would not allow this because the boot of the VM failed if it finds you have attached a USB device that is not present in the host (I have not found a way to override this).

Im using ZONES… Pretty nice feature in Frigate. When a detection occurs in a ZONE, double-take sends that snapshot to compreface… So in other words, motion is detected, frigates starts to record and detects objects, but objects is in the zone is only detected when there’s actually motion.

Are you using double-take? In that case it really easy to only make compreface to “react” when motion is detectd inside the zone…

In double-take config.yml

frigate:
  url: http://192.168.1.121:5000
  image:
    height: 500
  attempts:
    latest: 10
    snapshot: 0
  cameras:
    - frigate_entren
  zones:
    - camera: frigate_entren
      zone: zone_entren

Due to random db crashes I’ve migrated my RPi4 HassOS install to HomeAssistant Supervised/Debian 10 using these instructions. Now everything works fine except I cannot enable ffmpeg acceleration in Frigate. The config worked perfectly fine under HassOS 64bit. Any help would be appreciated?

 * Starting nginx nginx
   ...done.
frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.mqtt                   INFO    : MQTT connected
frigate.app                    INFO    : Camera processor started for ipcam_pergola: 39
frigate.app                    INFO    : Camera processor started for ipcam_garden: 40
frigate.app                    INFO    : Camera processor started for ipcam_bakkie: 42
detector.coral                 INFO    : Starting detection process: 37
frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate.app                    INFO    : Capture process started for ipcam_pergola: 43
frigate.app                    INFO    : Capture process started for ipcam_garden: 45
frigate.app                    INFO    : Capture process started for ipcam_bakkie: 50
frigate.video                  INFO    : ipcam_bakkie: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : ipcam_bakkie: ffmpeg process is not running. exiting capture thread...
frigate.video                  INFO    : ipcam_garden: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : ipcam_garden: ffmpeg process is not running. exiting capture thread...
frigate.video                  INFO    : ipcam_pergola: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : ipcam_pergola: ffmpeg process is not running. exiting capture thread...
frigate.edgetpu                INFO    : TPU found
ffmpeg.ipcam_garden.detect     ERROR   : [h264_v4l2m2m @ 0xaaab05db0ea0] Could not find a valid device
ffmpeg.ipcam_garden.detect     ERROR   : [h264_v4l2m2m @ 0xaaab05db0ea0] can't configure decoder
ffmpeg.ipcam_garden.detect     ERROR   : Error while opening decoder for input stream #0:0 : Invalid argument
ffmpeg.ipcam_pergola.detect    ERROR   : [h264_v4l2m2m @ 0xaaaad5348900] Could not find a valid device
ffmpeg.ipcam_pergola.detect    ERROR   : [h264_v4l2m2m @ 0xaaaad5348900] can't configure decoder
ffmpeg.ipcam_pergola.detect    ERROR   : Error while opening decoder for input stream #0:0 : Invalid argument
ffmpeg.ipcam_bakkie.detect     ERROR   : [h264_v4l2m2m @ 0xaaaad9af5170] Could not find a valid device
ffmpeg.ipcam_bakkie.detect     ERROR   : [h264_v4l2m2m @ 0xaaaad9af5170] can't configure decoder
ffmpeg.ipcam_bakkie.detect     ERROR   : Error while opening decoder for input stream #0:0 : Invalid argument
frigate.video                  INFO    : ipcam_bakkie: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : ipcam_bakkie: ffmpeg process is not running. exiting capture thread...

On what hardware? Pi again? With what settings are you attempting? :slight_smile:

^^ something like this perhaps?

" When there is no more room to write cache files, ffmpeg crashes and the ffmpeg sent a broken frame occurs. I created #570 to improve this in the future."

Or this? Not sure if this is HassOS or Pi specific:

" 1. If you are using hardware acceleration for ffmpeg, you will need to disable “Protection mode”"

EDIT: Looks like it might be Pi specific:

"Raspberry Pi 3/4 (64-bit OS) NOTICE: If you are using the addon, ensure you turn off Protection mode for hardware acceleration.

ffmpeg:
hwaccel_args:
- -c:v
- h264_v4l2m2m"

Thanks and yes, on the same Pi 4/4GB/USB SSD. That’s what happens when I install as hass add-on under supervised. Going to try install it as docker image… I’m a docker noob though… so crossed fingers…

was using this exact same config with accel and it worked perfectly fine on the latest hassos 64 bit build. the system was generally unstable though, freeezing every few hours (as described in this thread) so i’m trying out debian/supervised.

Perhaps check the Protection Mode is disabled:

  • Supervisor
  • Frigate
  • Protection Mode, disabled
  • Restart Frigate

P.S. I started out with a Pi, migrated to a NUC-like-box (gigabyte i7 brix) with MariaDB for the backend… pretty much zero issues since.

Protection mode is disabled. I basically just restored a snapshot of a working Frigate install. Yup also on MariaDB on SSD with lots of space. Agreed I’m beginning to think its a Pi issue. I’ve tested with Frigate and the TPU disabled and the dang thing still freezes.

Major props to @Vendo232! I have the HP T430 thin client with the m.2 A+E Coral up and running in HAOS 6 (required dev build for the Coral drivers). 3 1080p Wyze v2 running ~22 inference speeds. Next step is to migrate my entire HAOS from a vm to this little guy. It’s running very smooth on it’s own, couldn’t be happier for how efficient and cheap this whole thing ended up being.

1 Like

@blakeblackshear
Good day. Once again thank you for producing/sharing this component.

I apologise if I’m oversimplifying but wanted to ask if for a future release you’d consider adding a docker variable or config.yaml entry that would permit the user increasing the webui and therefore rtpm streams fps.

As I understand it, right now it’s fixed at 5 fps.

I can currently do this ( mjpeg_url: http://192.168.1.100:5000/api/doorbell?fps=15) to create a mjpeg camera in HA with the fps I want. I understand there’s a cpu implication, but the option would likely be welcome by users that would like a more “live” view of the cameras.

Thank you

Hi!
I have a question about the frigate. I tried to find the answer for it in the topic, but i failed.

I have 8 cameras with 3 Mpx resolution. I want to integrate them with frigate just to send movement notifications and snapshots from home assistance to mobile phone (eventually person and car detection), because my actual camera software is so poor that it doesn’t even do such things… I don’t want to save records by home assistance/frigate, because i have external nvr dedicated to that stuff. Will it work on raspberry pi 4? Or do i need additional google coral accelerator to it?

Thanks in advance!

Sounds like you have created your own work around. Higher fps mjpeg streams won’t be the answer here, but a more realtime camera viewer is in the works.

There is no way a pi 4 can handle 8 cameras. Even if object detection was off, it couldn’t even decode 8 streams in real time. Also, there is no situation in which I recommend running frigate without a Coral. A Coral is always recommended.

Thank you very much for your response. This workaround does indeed address my requirement and the cpu penalty is palatable in my case.
I finally got a coral and in conjunction with a gpu it all just works really well.

Thank you again.

So how do you think, how many cameras can pi 4 handle?

This!
I’m running a locally built container from the 0.9.0-branch and there are really great features abound making this an almost feature complete NVR! The frigate interface is really useful now in relation to before where I used it if debugging / configuring something.

Further I had to do some stuff locally in order to be able to use the aforementioned msjpeg stream with the builtin nginx from any camera. Being able to get a live picture to the tv without any extra middleware except for the notifications is great!

@blakeblackshear
I thought about contributing around the jsmpeg implementation but as my time is limited and you seem to be hard at work at all things / features at once while I couldn’t follow your discussion on progress or decided features anywhere. Instead I went a hacky way to achieve it for now as you seem to have momentum in the development and 0.9.0 seems relatively imminent. Was I just supposed to make a PR and start from there?