Local realtime person detection for RTSP cameras

Sure! thanks!

Thereā€™s probably a way to have both options, with I guess the ultimate goal to pull it together into a single container that acts as NVR with the object detection functionality?

Do you have a roadmap you can share? I think I could make the changes I want for my use case, but Iā€™d rather contribute back if I can.

analysis has to be on the FullHD main stream i did a lot a field tests and with SD resolution the detection is terrible. Only usable with HD. For example when someone is walking in the shade it was not detected in SD only in 1080p HD stream.

I donā€™t have a roadmap written down. I will try and find some time to think through it.

as for NVR function, just look around how the branded ones do it, if you put together a 9 cam mosaic onto a 50inch TV you never need fullHD on any of these stream, SD is perfectly enough. However the recording must be also full HD. Keep in mind if you burn your energy building an sofware NVR you will need a powerful PC/server with GPU, basically no chance to do it with just the CPU. I have branded Hikvision 4k NVRs which do hardware accelerated encoding, so you need to do the same if you build a linux server for it. I did this with i7 CPU and AMD rx580 using Vulkan for hw acceleration. Even with this you cant beat a Hikvision 4K NVR performance. So the bottom line is, you can build a s/w NVR which could give lot more freedom but keep in mind it will need a good hardware

1 Like

You donā€™t need all of those resources to save recordings to disk, even for lots of 4k cameras. Those NVRs are doing motion detection and re-encoding video, which I have no intention of doing. Just simply saving the videos as is directly to disk for viewing later if needed. Frigate will do all the heavy lifting for motion/object detection.

Was that with frigate and Coral? Iā€™m getting pretty good person detection on my 640x360 substream, even at night with the IR light on

anyone make a simple test to bundle the streams from 6-9 cams and pass it to a Google Chrome> Chromecast TV. Then your PC CPU will be dead without using GPU.

frigate + coral, SD mode detection is useless, please make such test in the shade

Not my experience - as I say, even at night itā€™s working for me. By shade, though, do you mean a strong mix of bright sunlight and shade rather than full shade? I.e. a very high contrast scene? Iā€™ve only been using it for a few days, and Iā€™m not sure UK sunlight in September gets bright enough to cause that level of contrast being dark and light areas in the same sceneā€¦

Yes, decoding 6-9 4k streams and encoding a new video output would need a GPU, but thats not what I intend to build. I just want something that can store video to disk and allow me to browse it. If I did build a grid view, I would just use the lower resolution streams that I already have decoded in frigate to build the grid and stream out a new 1080p stream. I personally donā€™t have a need to watch a grid view of all my cameras. Thatā€™s what frigate is doing for me.

yep brutal sunshine one side of the picture and shade on the other side, cam is mounted under the roof looking down the area. No detection in shade for a person. Its not a problem with frigate at all, just the SD stream is not enough.

That would work for me as well. I already have the low res streams exposed to HA, so I can get a view at a glance if I need it. Itā€™s the automated clip recording baed on person detection that I want, with 24x7 as a nice to have option for those times when I might want to view a bit more context around events.

yep, grid view is not really used by home owners, but used by shops and farms and warehouses. Up to you, if frigate wont have gridview no issue at all, Zoneminder or branded NVR products can do that.

Hello,

First of all, great work!

I have recently installed this on my NUC with Coral USB
It detects persons just fine and MQTT message is being sent. I also can view the best.jpg picture at /<camera_name>/<object_name>/best.jpg. But for some reason it doesnā€™t work for /<camera_name>/latest.jpg. I only get 404 not found error.

Any clue why?

works perfectly, i did a git clone few days ago, not sure if there was any change since then but mine works showing the latest.jpg

Thatā€™s in the release candidate. You are probably pulling stable.

1 Like

Master, i have noticed something, now i hooked up 9 cams, fullHD stream 2-4fps per cams. If a truck or car passing the image half of the image, it does not always detected. Whats the issue. Is the fps too slow or coral too slow or too many cams and thats why i am missing detections. You also meantioned having 7 cams, did you notice such? Ideally we dont want to miss a thing. I just dont know yet its due to failed object detection/model or the number of cams/coral frigate software side. Best if you do such test on your own, dont believe me :slight_smile:

Do you have a video clip from that time for testing?

yes, i have, because i have NVR and all recorded but cant remember the exact time it was today and also yesterday, i am looking at cams on 50inch TV and happened a few times and there was no detection. OK next time i will note the exact time and give you the photo from the record.