Local realtime person detection for RTSP cameras

Some work is still done on the CPU, so it will increase the watts used by the NUC too.

Yes! will have to find out how much!

Just curiousā€¦ why is that important to you?

24/7/365 and it will of course add up. If I save 300 (insert currency) thatā€™s 10000 less at 3% yield I have to invest. As an example.

This is so freaking awesome. Great work!

So, my question is how much horsepower does it need if I use the new version with coral? I have a i3 6100 right now as my blue iris server, and was wondering if I can just run it on there.

I am running on an i3 NUC with 6 cameras and it is using about 1.5 cores to decode all the streams and do the processing. I have a lot of regions across those 6 cameras. It really depends on the resolution of your cameras, how many detection regions you have, and how many of your regions are being resized. That is where all the CPU usage is driven from. At this point, the stream decoding does not use the hardware acceleration built into ffmpeg (see issue). Once that is added, it will reduce CPU usage to something below 1 core for my 6 cameras. My goal is to keep the CPU usage very low so that this can be run on low power machines.

2 Likes

My worry is that blue iris hogs resources if multiple cameras are recording at once. Iā€™ll try it out when my coral gets here and go from there. This project is worth me upgrading hardware if needed.

Has anyone tried the Dev Board instead of NUC (or RasperryPi if you donā€™t need that much power) + the USB accelerator?
From budget point of view it seems to me that it makes sense to dedicate one unit for realtime object detection (1 or 2 cams, or a few regions): itā€™s more powerful than the RPI (USB2 is the bottleneck, if I understand it right), and not as overkill as an NUC (which is using half of itā€™s computing power for a lot of regions).

1 Like

Have you looked into using NVidia Jetson Nano? This might be an option to make everything self contained without the need for docker.

That was released just after the Coral. It would be another option, but would require switching to entirely different libraries again, and I like that the Coral works with any device. There is no reason I couldnā€™t run it without docker as is. In fact, I did extra work to ensure it worked in docker. I canā€™t think of a good reason to intentionally avoid it.

1 Like

Iā€™m not saying there is an issue with the direction youā€™re going. I was just thinking that docker need a host environment to run. Coral is about $80. So for $20 more you get something that is the host and processing board.

Iā€™m trying to get set up for the first time and iā€™m getting

On connect called
ERROR: Failed to retrieve TPU context.
ERROR: Node number 0 (edgetpu-custom-op) failed to prepare.

I noticed the windows host doesnā€™t have a driver for the coral. Is this the cause?

Edit: Yep, it looks like I canā€™t use a windows host. Pass USB device into a Docker Windows Container - Stack Overflow

Edit 2: I got it working finally. I ended up giving up on windows working as a host at all, even as just a pass through. Maybe someone else here can get it working and post back here.

I got it working by installing unraid (which is linux based) and using the docker you posted above like this:

It took about 10 minutes or so for the coral to fully boot. Then restart the frigate docker. Hopefully this is helpful to someone else :slight_smile:

3 Likes

Are you still actively developing this?

Yes. I get time to work on it here and there. I am actively using it in 2 homes.

2 Likes

Runs 24/7 at my house now too. I definitely owe you a beer.

One small request would be to have a ā€œlast personā€ endpoint. Sometimes best person shows me an image from the previous alert.

1 Like

That should only happen if the events were within a minute of each other and the more recent person is a lower score.

I use the motion detection from blue iris as the trigger and that state from frigate. So motion > is there a person detected > alert me. The pushover message sends me ā€œbest personā€ which is sometimes isnā€™t even from that trigger time.

I live in a vacation town with very high foot traffic, so I use blue iris to refine my trigger zones.

And yes, the latest sometimes is a much lower score because its further away from the camera.

This project looks awesome. Just ordered a coral to test it out! Thanks for all the hard work @blakeblackshear

1 Like

That could be a race condition where blue iris detects motion before frigate has a high enough scoring person. That would cause the image it sends you to be old. Why bother with the motion detection from blue iris? I ended up removing the motion detection in frigate because it seemed like a waste of CPU power to detect a bunch of false positives.

1 Like

Because I can schedule zones I care about. The trigger zone is oddly shaped so frigate wouldnā€™t be perfect on its own.

Here is an example: