Local realtime person detection for RTSP cameras

For anyone else wanting a cheaper Coral option - the M.2/MiniPCIe modules work just fine with frigate for around $40 shipped via mouser.

Make sure you follow the instructions here to get it installed properly.

The only issue is that I can’t get the device passed through in a virtual environment - bare metal works fine.

Which VM system have you tested?

Only ESXi and Proxmox. Not working on Proxmox even bare metal didn’t make any sense, I think it’s a debian based distro underneath?

Regardless, I’d need it to work with ESXi for my application, and it definitely refuses to work via passthrough there.

1 Like

That’s interesting. I’m using ESXi so good to know. (i was about to swap my USB version to internal one)

So is it listed in hadware section but it fails when you try to toggle the passthrough, is it grayed out or completely missing?

1 Like

It shows up as a PCIe device on ESXi, can even pass it through and the device shows up in a VM. It’s the drivers that are problematic - the device won’t initialize after installing the recommended drivers.

1 Like

Just encountered the same thing. Thanks for the workaround, @Kyle

1 Like

Thanks Blake, this did indeed fix my cheap-no-name camera that only wants to do RDP.

1 Like

Thanks for the frequency of new releases lately @blakeblackshear, it is greatly appreciated!

I just updated to the new .4 beta and I was checking out the new /debug/stats endpoint. I noticed that the json indicated that all my cameras were 1.0 or 1.13 FPS. I cam feeding a 15fps feed from each camera to frigate and I have take frame: 1 on each of my cameras in the config file. My understanding was that this meant analyze every frame? No drop frame errors in the logs so it’s not choking on anything.

0.4.0 introduced a default flag in ffmpeg that filters out frames that are nearly identical to the previous frame. When there is movement on the camera, you should see the FPS jump up to 10 dynamically. That ensures the Coral can focus on analyzing what matters. The next release will be move that motion detection into frigate so I can be even more targeted and regions will be entirely optional. All the object tracking introduced enough overhead that I am not quite happy with the way it works when you have 5+ cameras.

1 Like

Great, thanks for the explanation.

Anyone know where i can get the list of all the objects which can be detected?
I tried to add cat and frigate didn’t like it.

I am definitely using cat in mine. In fact I’m currently using frigate to look for a neighborhood cat who sprays inside my house.

Also try “dog” though - the model is not that good, especially with low light.

1 Like

@calypso Have a look in the config.example.yml there is a link to the txt file.

Could you please share your config, or what was done to end it? For whatever reason frigate throws an error whenever I add it to global.

Has anyone had any luck building the 0.4.0 beta on a Pi?

Step 6/15 : RUN pip3 install -U     opencv-python-headless     python-prctl     Flask     paho-mqtt     PyYAML     matplotlib     scipy
 ---> Running in 5f39d6e2cf9e
Collecting opencv-python-headless
  Could not find a version that satisfies the requirement opencv-python-headless (from versions: )
No matching distribution found for opencv-python-headless
The command '/bin/sh -c pip3 install -U     opencv-python-headless     python-prctl     Flask     paho-mqtt     PyYAML     matplotlib     scipy' returned a non-zero code: 1

I’ve tried a few things but no luck. I guess I can go back to building it as per the previous versions, but just wanted to check if anyone had a better solution.

Thanks :slight_smile:

Did you try removing -headless from the opencv package?

Yep, and it returned same error, just for opencv-python

It seem that frigate was stalling on autodetection of the cameras when making too many quick changes. So its resolved now.

Just need to do some fine tuning, as the cat is small, so struggling to detect.

I can’t get it to work on esxi either yet.
I can pass the pci device through to a VM and it shows up with lspci.
However my Ubuntu VM with latest apex drivers can’t initialise the coral mini pcie.
I have some errors in dmesg, I can post them here tomorrow.
What’s your error?

Apparently, the issue is this line in my dmesg:

[ 1.215014] pci 0000:00:15.5: BAR 13: no space for [io size 0x1000] [ 1.215017] pci 0000:00:15.5: BAR 13: failed to assign [io size 0x1000]

which is causing the module to be unable to access host memory. It could be a hardware issue on my end, since I’m using a NUC-like mini PC. The Google engineer did mention that he’s seen the issue with some other VM setups, so YMMV.

Regardless, he told me to contact VMware for the problem so there’s that.