Local realtime person detection for RTSP cameras

Runs 24/7 at my house now too. I definitely owe you a beer.

One small request would be to have a “last person” endpoint. Sometimes best person shows me an image from the previous alert.

1 Like

That should only happen if the events were within a minute of each other and the more recent person is a lower score.

I use the motion detection from blue iris as the trigger and that state from frigate. So motion > is there a person detected > alert me. The pushover message sends me “best person” which is sometimes isn’t even from that trigger time.

I live in a vacation town with very high foot traffic, so I use blue iris to refine my trigger zones.

And yes, the latest sometimes is a much lower score because its further away from the camera.

This project looks awesome. Just ordered a coral to test it out! Thanks for all the hard work @blakeblackshear

1 Like

That could be a race condition where blue iris detects motion before frigate has a high enough scoring person. That would cause the image it sends you to be old. Why bother with the motion detection from blue iris? I ended up removing the motion detection in frigate because it seemed like a waste of CPU power to detect a bunch of false positives.

1 Like

Because I can schedule zones I care about. The trigger zone is oddly shaped so frigate wouldn’t be perfect on its own.

Here is an example:

I implemented masking for that exact use case and it will be in the next release. I am still working on a better way to filter out false positives based on the size of the person and the location where they are standing. Then I am looking at person counting and tracking across frames. I have a 25x PTZ that I will be using to zoom in on people and track when further away.

2 Likes

I would drop BI for alerts all together when masking comes out. How are you implementing masking creation?

As a bmp in the config directory. https://raw.githubusercontent.com/blakeblackshear/frigate/person_filtering/config/back-mask.bmp

2 Likes

I’ve also been using it in production for months… Thanks Blake! Have a Coral but haven’t migrated to it yet as I’m waiting for a new mac mini to run it on instead of my laptop…

Looks like you moved to one big mask for the entire frame, is that right?? That will make things a lot easier. Do you still need to define zones under Coral or have you gone zone less yet? It would make my next config a lot easier if that was the case. Right now it’s working well enough with the CPU version that the Coral isn’t calling me too much especially as most of my machines are USB 2.0 still.

One big mask, but not zoneless yet.

This project inspired me to build an UnRAID server with Blue Iris, HassIO and Frigate. Sounds like its upgrade time :smiley:

1 Like

I agree, I removed motion detection from the original CPU version and it runs great with real-time detection.

1 Like

You can modify dockerfile from git and specify tensorflow==1.3

build that dockerfile and run containers with that.

Do you have a general idea of when you think that fancy masking technology will be available?

I can probably find time to pull out those changes and release an update this week.

2 Likes

I went ahead and released version 0.1.1 with image masking just now. I pushed the updated container to docker hub. You can see an example of the image masking in the config directory here: https://github.com/blakeblackshear/frigate/tree/master/config

3 Likes

Which is better: Google Coral or Intel computer stick?

I haven’t tried the intel compute stick, so I can’t comment. Frigate is only compatible with the Google Coral.

So the mask seems to be working for me. I’ll be testing more over the next few days. Again, thank you so much for this project, sir.

Side note; I keep seeing these in the logs

[h264 @ 0x197d340] error while decoding MB 14 70, bytestream -6

[h264 @ 0x1953340] error while decoding MB 113 47, bytestream -12

[h264 @ 0x197d340] error while decoding MB 128 55, bytestream -12

[h264 @ 0x1953340] error while decoding MB 28 45, bytestream -5

[h264 @ 0x1953340] error while decoding MB 52 58, bytestream -12

[h264 @ 0x196d600] error while decoding MB 91 77, bytestream -5

[h264 @ 0x197d340] error while decoding MB 17 41, bytestream -7

[h264 @ 0x1d5a120] error while decoding MB 141 8, bytestream -10

[h264 @ 0x1ac5580] error while decoding MB 98 85, bytestream -7

[h264 @ 0x1d5a120] error while decoding MB 65 45, bytestream -7

[h264 @ 0x197d340] error while decoding MB 48 5, bytestream -7

[h264 @ 0x1d5a120] error while decoding MB 94 71, bytestream -7

[h264 @ 0x19779c0] error while decoding MB 44 55, bytestream -6

Is the stream too fast or something?