Local realtime person detection for RTSP cameras

Share your Projects!
672

argh, I always forget about the time elapsed and score averaging. Thanks for that

fps is 10 for a 720p feed w/ a single region of 460. Nuc utilization is low (hwaccel is enabled)

i moved the camera inside for some easier testing and things are working better and as expected. (possibly related to the camera/frigate restarts, see below)

I do often see errors in the logs

sometimes I see something like

frigate    | Exception in thread Thread-6:
frigate    | Traceback (most recent call last):
frigate    |   File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
frigate    |     self.run()
frigate    |   File "/opt/frigate/frigate/mqtt.py", line 37, in run
frigate    |     if not self.best_frames.best_frames[obj_name] is None:
frigate    | KeyError: 'car'
frigate    |

other times i see repeated restarts

frigate    | Creating a new capture thread...
frigate    | Starting a new capture thread...
frigate    | ffmpeg didnt return a frame. something is wrong. exiting capture thread...
frigate    | last frame is more than 5 minutes old, restarting camera capture...
frigate    | Terminating the existing ffmpeg process...
frigate    | Waiting for ffmpeg to exit gracefully...
frigate    | Waiting for the capture thread to exit...
frigate    | Creating a new ffmpeg process...

I think the ffmpeg restarts may have to do with my dafangā€™d wyze camera and the default rtsp settings. Once I get things setup in ha Iā€™ll work on stability.

Another observation, when I moved the camera I had to restart frigate for the :5000 stream to show the new camera view. The timestamp was off so likely the stream froze at some time prior, perhaps dafang related.

673

Sorry to pile on, another behavior I noticed while testing. It seems the publishing of OFF can get stuck/loop and the topic get spammed with OFFs until I trigger a person detection. Itā€™s at least 5/sec.

674

I think you found a race condition with the key error and a different issue with the OFF messages. Pushing up some fixes in a bit.

675

I pushed up a new version that fixes both of those issues. Pull again with docker pull blakeblackshear/frigate:0.3.0-beta

1 Like
676

Awesome turn around. Iā€™ll be able to test tomorrow.

Thank you!

677

Well on the plus side atleast this helped me to fix my YAML in home assistant. Now i just need to figure out where to drop the best.jpg

678

What are your plans for the best.jpg? Use in a notify or something else?

Do you still want to see someoneā€™s hwaccel config?

679

Things are looking good for those two issues.

I still have the ongoing ffmpeg restarts due to no frame in 5m. I plan on turning up ffmpeg debugs, etc. Would it be possible to add timestamps to the docker logs?

680

Timestamps would require going back and changing a lot of code at this point, and I am in the middle of a lot of changes for new features. I would change the loglevel of ffmpeg to info:

ffmpeg:
  global_args:
    - -loglevel
    - info
681

Is it possible to disable the timestamp that gets printed to the best.jpg frame in the config file? I just upgraded to 0.3 - worked great, but already have timestamps on my camera feeds. Thanks!

682

It isnā€™t possible at the moment. I removed the camera generated timestamps because ffmpeg was detecting that change and sending the frame for processing. I get better performance after removing it. If you still need yours, open an issue on github and I can add an option.

2 Likes
683

Is there a post or doc that explains more? I have camera-sourced timestamps on right now but if ffmpeg is looking at each image to see what changed to decide whether to continue processing , I can definitely imagine some improvements from only using frigate generated timestamps.

684

Here is a little more info about what ffmpeg is doing:

685

Ah ok, not a problem.

Here are my logs. https://pastebin.com/4c8WPE46 does anything jump out at you?

and holy smokes, I tried debug too, thatā€™s a lot of logs :slight_smile:

686

I donā€™t see anything obvious. It seems like ffmpeg just stops receiving frames after a while. You can copy and paste that ffmpeg command and use it directly too. There is something called ffplay that will take most of the same parameters. You can see what happens with that.

687

I found that there is a verbose level between info and debug

https://pastebin.com/1dtbMgJx

Check out near the end

No more output streams to write to, finishing.e=00:11:43.20 bitrate=221184.0kbits/s speed=0.865x ed=0.754x

688

Seems like an issue with the rtsp stream dropping out. I did increase the timeout from 10 seconds to 5 minutes in this release. I frequently have more than 10 seconds without a frame due ffmpeg filtering out duplicates, so I was getting a bunch of unnecessary restarts. Earlier versions would have detected this after 10 seconds and reconnected. I could make that configurable per camera.

689

Thanks for the links. Prior my nuc build this week I was running the initial release (no coral until a few days ago). I canā€™t say I ever watched the logs close enough to notice any restarts. Perhaps your dedupe commit was after those early builds. If the restarts are due to long periods of idle/deduping, and the restarts are seamless then they should not cause impact - but I think I do have impacts and gaps. A per-camera idle timer is interesting but how long is too long, excessive values could negate self-recovery. I will turn off dedupe as a test to see if it affects the issue.

690

I donā€™t think the deduping is the issue. 5 minutes is a very long time to go without a change and thatā€™s why I chose it as an indication that something was broken. If ffmpeg is being restarted, then something is wrong. You are saying you used the same camera without issues in the non-coral version?

691

Agreed, that is a long timeā€¦ Yes, same camera aimed at the same area. Iā€™ll also do some ffmpeg testing with output to /dev/null and see if the process dies.