Local realtime person detection for RTSP cameras

I implemented masking for that exact use case and it will be in the next release. I am still working on a better way to filter out false positives based on the size of the person and the location where they are standing. Then I am looking at person counting and tracking across frames. I have a 25x PTZ that I will be using to zoom in on people and track when further away.

2 Likes

I would drop BI for alerts all together when masking comes out. How are you implementing masking creation?

As a bmp in the config directory. https://raw.githubusercontent.com/blakeblackshear/frigate/person_filtering/config/back-mask.bmp

2 Likes

Iā€™ve also been using it in production for monthsā€¦ Thanks Blake! Have a Coral but havenā€™t migrated to it yet as Iā€™m waiting for a new mac mini to run it on instead of my laptopā€¦

Looks like you moved to one big mask for the entire frame, is that right?? That will make things a lot easier. Do you still need to define zones under Coral or have you gone zone less yet? It would make my next config a lot easier if that was the case. Right now itā€™s working well enough with the CPU version that the Coral isnā€™t calling me too much especially as most of my machines are USB 2.0 still.

One big mask, but not zoneless yet.

This project inspired me to build an UnRAID server with Blue Iris, HassIO and Frigate. Sounds like its upgrade time :smiley:

1 Like

I agree, I removed motion detection from the original CPU version and it runs great with real-time detection.

1 Like

You can modify dockerfile from git and specify tensorflow==1.3

build that dockerfile and run containers with that.

Do you have a general idea of when you think that fancy masking technology will be available?

I can probably find time to pull out those changes and release an update this week.

2 Likes

I went ahead and released version 0.1.1 with image masking just now. I pushed the updated container to docker hub. You can see an example of the image masking in the config directory here: https://github.com/blakeblackshear/frigate/tree/master/config

3 Likes

Which is better: Google Coral or Intel computer stick?

I havenā€™t tried the intel compute stick, so I canā€™t comment. Frigate is only compatible with the Google Coral.

So the mask seems to be working for me. Iā€™ll be testing more over the next few days. Again, thank you so much for this project, sir.

Side note; I keep seeing these in the logs

[h264 @ 0x197d340] error while decoding MB 14 70, bytestream -6

[h264 @ 0x1953340] error while decoding MB 113 47, bytestream -12

[h264 @ 0x197d340] error while decoding MB 128 55, bytestream -12

[h264 @ 0x1953340] error while decoding MB 28 45, bytestream -5

[h264 @ 0x1953340] error while decoding MB 52 58, bytestream -12

[h264 @ 0x196d600] error while decoding MB 91 77, bytestream -5

[h264 @ 0x197d340] error while decoding MB 17 41, bytestream -7

[h264 @ 0x1d5a120] error while decoding MB 141 8, bytestream -10

[h264 @ 0x1ac5580] error while decoding MB 98 85, bytestream -7

[h264 @ 0x1d5a120] error while decoding MB 65 45, bytestream -7

[h264 @ 0x197d340] error while decoding MB 48 5, bytestream -7

[h264 @ 0x1d5a120] error while decoding MB 94 71, bytestream -7

[h264 @ 0x19779c0] error while decoding MB 44 55, bytestream -6

Is the stream too fast or something?

I donā€™t think so. That looks like ffmpeg reporting occasional errors when decoding the stream. It happens more or less depending on the camera. It just skips any bad frames and moves on.

Iā€™m trying to set this up for the first time, but Iā€™m getting the following error while starting the docker container. After reading through this entire thread, and the GH page, I canā€™t seem to figure out what Iā€™m doing wrongā€¦ Thoughts? Fwiw, Iā€™m using Portainer to manage my containers.

python3: can't open file 'detect_objects.py': [Errno 2] No such file or directory

Are you using the container from docker hub or did you build it?

hmm, admittedly, Iā€™m a docker rookā€¦ From what I understand, I just pulled the image from the docker hub by entering blakeblackshear/frigate inside of portainerā€¦ but, iā€™m assuming i need to git clone the repo as well, right?

edit: got it going, but Iā€™m not entirely sure what I should be seeing for outputā€¦ hereā€™s what I currently see

On connect called
W third_party/darwinn/driver/package_registry.cc:65] Minimum runtime version required by package (5) is lower than expected (10).
Capture process for back: 27
* Serving Flask app "detect_objects" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: off
* Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)

Also, I dont see any mqtt logging, so Iā€™m assuming something is broken.

Looks like it connected to mqtt. What do you see at http://<ip of your docker host>:5000/back?

the host isnā€™t listening on port 5000, so this yields nothing. The container is in privileged mode, and its set to fwd all ports, so I imagine the container isnā€™t getting to the point of binding to the port.

edit: errr, Im dumb, the log reports its listening on 5000, so maybe my docker config is borked?