So something that might be helpful, at least to me, would be a “debounce” option of sorts. With my setup sometimes a person is detected in one frame, is not detected in the next couple of frames, then is detected again less than a half a second later (of course depending on framerates, areas, thresholds, etc). What if part of the filters: option there was some sort of debounce parameter that would prevent the flurry of MQTT “ON” and “OFF” messages or that particular object? Maybe a number of frames? Does that even make sense and would that be feasible?
Frigate will wait until it hasn’t seen an object for 10 frames to determine that the object has disappeared. Does that sound consistent with what you are seeing? I could make that value configurable per camera.
Perhaps a configuration value per camera for both the detection and the disappearance?
I get the odd moth setting off an alert. They move so fast. Not sure how many frames the moth was detected for. Would be good to increase it perhaps.
That might help, and maybe I’m just misunderstanding how Frigate is doing processing in the background.
I have 2 of the cameras in question configured for 3 fps (not using take_frame). My HA sensors that pull from the debug endpoint for those cameras and for the Coral show super high detection frame rate values like 80-90 fps.
Is Frigate processing those 3 frames over and over again multiple times per second? Is that why I’m seeing the 80-90fps values for those 2 cameras?
You should see a few FPS values, but none of them should be that high. There was an issue reported with a similar problem by someone else. Frigate is very frame hungry, and it looks like some cameras cause it to get frames at a much higher rate. I added some details about the debug values that is probably helpful. https://github.com/blakeblackshear/frigate#debug-info
For each camera camera_fps = process_fps + skipped_fps.
Thanks Blake. As expected, the high framerates on those 2 cameras were because I removed the -vsync drop line from ffmpeg per your suggestion. I added -f 3 for those specific cameras and it’s back to normal.
Those 2 cameras are kinda cheap and are mjpeg feeds. My other 2 cameras that use RTSP don’t have the issue, it seems.
I’m guessing this would make my need for any kind of debouncing unnecessary now. Thanks again!
Hi Blake
Further to this, I’ve done a bit more troubleshooting and it appears it is one camera (x.x.x.x.84 / ‘pond’) that is causing the issue. I increased the log level and tweaked some ffmpeg arguments, and now have some more logs that hopefully may help.
This was from this morning. It gave the nmap error upon starting it up. The ‘pond’ camera did actually work for a moment, or at least the live feed showed at the /pond URL.
frigate051 | On connect called
frigate051 | Creating ffmpeg process...
frigate051 | ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtsp://user:[email protected]:554/h264Preview_01_sub -f rawvideo -pix_fmt rgb24 pipe:
frigate051 | Starting detection process: 24
frigate051 | Creating ffmpeg process...
frigate051 | ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtsp://user:[email protected]:554/h264Preview_01_sub -f rawvideo -pix_fmt rgb24 pipe:
frigate051 | Creating ffmpeg process...
frigate051 | ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtsp://user:[email protected]:554/h264Preview_01_sub -f rawvideo -pix_fmt rgb24 pipe:
frigate051 | Creating ffmpeg process...
frigate051 | ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtsp://user:[email protected]:554/h264Preview_01_sub -f rawvideo -pix_fmt rgb24 pipe:
frigate051 | Creating ffmpeg process...
**frigate051 | ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://x.x.x.x.84/ch0_1.h264 -f rawvideo -pix_fmt rgb24 pipe:**
frigate051 | Camera_process started for driveway: 38
frigate051 | Starting process for driveway: 38Camera_process started for side: 39
frigate051 |
frigate051 | Starting process for side: 39
frigate051 | Camera_process started for back: 40
frigate051 | Camera_process started for backside: 41
frigate051 | Starting process for back: 40
frigate051 | Starting process for backside: 41
frigate051 | Camera_process started for pond: 42
frigate051 | Starting process for pond: 42
frigate051 | * Serving Flask app "detect_objects" (lazy loading)
frigate051 | * Environment: production
frigate051 | WARNING: Do not use the development server in a production environment.
frigate051 | Use a production WSGI server instead.
frigate051 | * Debug mode: off
frigate051 | pond: ffmpeg didnt return a frame. something is wrong.
frigate051 | pond: ffmpeg process is not running. exiting capture thread...
frigate051 | Creating ffmpeg process...
frigate051 | ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://10.10.10.84/ch0_1.h264 -f rawvideo -pix_fmt rgb24 pipe:
frigate051 | ../src/plasma/client.cc:182: mmap failed
frigate051 | /usr/local/lib/python3.7/dist-packages/pyarrow/libarrow.so.17(+0x614cc0)[0xac70acc0]
frigate051 | /usr/local/lib/python3.7/dist-packages/pyarrow/libarrow.so.17(_ZN5arrow4util8ArrowLogD1Ev+0x108)[0xac70ade0]
frigate051 | /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0xd0f4)[0xabf400f4]
frigate051 | /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0x10844)[0xabf43844]
frigate051 | /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(_ZN6plasma12PlasmaClient6CreateERKNS_8UniqueIDExPKhxPSt10shared_ptrIN5arrow6BufferEEib+0x44)[0xabf43d5c]
frigate051 | /usr/local/lib/python3.7/dist-packages/pyarrow/_plasma.cpython-37m-arm-linux-gnueabihf.so(+0x1fce0)[0xabf82ce0]
frigate051 exited with code 0
frigate051 | On connect called
frigate051 | Creating ffmpeg process...
frigate051 | ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtsp://user:[email protected]:554/h264Preview_01_sub -f rawvideo -pix_fmt rgb24 pipe:
frigate051 | Starting detection process: 21
frigate051 | Creating ffmpeg process...
frigate051 | ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtsp://user:[email protected]:554/h264Preview_01_sub -f rawvideo -pix_fmt rgb24 pipe:
frigate051 | Creating ffmpeg process...
frigate051 | ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtsp://user:[email protected]:554/h264Preview_01_sub -f rawvideo -pix_fmt rgb24 pipe:
frigate051 | Creating ffmpeg process...
frigate051 | ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtsp://user:[email protected]:554/h264Preview_01_sub -f rawvideo -pix_fmt rgb24 pipe:
frigate051 | Creating ffmpeg process...
frigate051 | ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://10.10.10.84/ch0_1.h264 -f rawvideo -pix_fmt rgb24 pipe:
frigate051 | Camera_process started for driveway: 35
frigate051 | Starting process for driveway: 35
frigate051 | Camera_process started for side: 36
frigate051 | Starting process for side: 36
frigate051 | Camera_process started for back: 37
frigate051 | Starting process for back: 37
frigate051 | Camera_process started for backside: 38
frigate051 | Starting process for backside: 38
frigate051 | Camera_process started for pond: 39
frigate051 | Starting process for pond: 39
frigate051 | * Serving Flask app "detect_objects" (lazy loading)
frigate051 | * Environment: production
frigate051 | WARNING: Do not use the development server in a production environment.
frigate051 | Use a production WSGI server instead.
frigate051 | * Debug mode: off
frigate051 | No frames received from pond in 5 seconds. Exiting ffmpeg...
frigate051 | Waiting for ffmpeg to exit gracefully...
frigate051 | Exception in thread pond:
frigate051 | Traceback (most recent call last):
frigate051 | File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
frigate051 | self.run()
frigate051 | File "/opt/frigate/frigate/video.py", line 141, in run
frigate051 | frame_bytes = self.ffmpeg_process.stdout.read(self.frame_size)
frigate051 | ValueError: PyMemoryView_FromBuffer(): info->buf must not be NULL
frigate051 |
frigate051 | Creating ffmpeg process...
frigate051 | ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://10.10.10.84/ch0_1.h264 -f rawvideo -pix_fmt rgb24 pipe:
frigate051 | ../src/plasma/client.cc:182: mmap failed
frigate051 | /usr/local/lib/python3.7/dist-packages/pyarrow/libarrow.so.17(+0x614cc0)[0xac725cc0]
frigate051 | /usr/local/lib/python3.7/dist-packages/pyarrow/libarrow.so.17(_ZN5arrow4util8ArrowLogD1Ev+0x108)[0xac725de0]
frigate051 | /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0xd0f4)[0xabedb0f4]
frigate051 | /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0x10844)[0xabede844]
frigate051 | /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(_ZN6plasma12PlasmaClient6CreateERKNS_8UniqueIDExPKhxPSt10shared_ptrIN5arrow6BufferEEib+0x44)[0xabeded5c]
frigate051 | /usr/local/lib/python3.7/dist-packages/pyarrow/_plasma.cpython-37m-arm-linux-gnueabihf.so(+0x1fce0)[0xabf1dce0]
I’ve got another example that I’ll need to post separately as too many characters.
Then there is this from last week with the debug on. The camera is called frontdoor - same camera but I have moved it now). On this occasion, the frigate container actually restarted itself.
[h264 @ 0x751630] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3:00.09 bitrate=497663444.2kbits/s speed=0.000221x
2020-08-06T15:00:12.455846450Z Last message repeated 8 times
[h264 @ 0x751630] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3:00.09 bitrate=497663444.8kbits/s speed=0.000221x
2020-08-06T15:00:13.180324939Z Last message repeated 13 times
2020-08-06T15:00:13.181204976Z [h264 @ 0x751630] nal_unit_type: 7(SPS), nal_ref_idc: 3
2020-08-06T15:00:13.181899813Z [h264 @ 0x751630] nal_unit_type: 8(PPS), nal_ref_idc: 3
2020-08-06T15:00:13.183549593Z [h264 @ 0x751630] nal_unit_type: 5(IDR), nal_ref_idc: 3
2020-08-06T15:00:13.252112115Z [h264 @ 0x751630] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
[h264 @ 0x751630] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3:00.09 bitrate=497662337.2kbits/s speed=0.000221x
2020-08-06T15:00:21.503232979Z No frames received from frontdoor in 5 seconds. Exiting ffmpeg...
2020-08-06T15:00:21.504852352Z Waiting for ffmpeg to exit gracefully...
2020-08-06T15:00:23.619185784Z Last message repeated 10 times
2020-08-06T15:00:23.619314504Z frame= 8990 fps= 19 q=-0.0 Lsize= 6068250kB time=00:00:00.09 bitrate=497663446.4kbits/s speed=0.000217x
2020-08-06T15:00:23.619498648Z video:6068250kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
2020-08-06T15:00:23.619551684Z Input file #0 (rtsp://x.x.x.84/ch0_1.h264):
2020-08-06T15:00:23.620169727Z Input stream #0:0 (video): 8998 packets read (20407307 bytes); 8990 frames decoded;
2020-08-06T15:00:23.620226411Z Total: 8998 packets (20407307 bytes) demuxed
2020-08-06T15:00:23.620295539Z Output file #0 (pipe:):
2020-08-06T15:00:23.620355519Z Output stream #0:0 (video): 8990 frames encoded; 8990 packets muxed (6213888000 bytes);
2020-08-06T15:00:23.620385945Z Total: 8990 packets (6213888000 bytes) muxed
2020-08-06T15:00:23.621205390Z 8990 frames successfully decoded, 0 decoding errors
2020-08-06T15:00:23.627135229Z [AVIOContext @ 0x7661b0] Statistics: 0 seeks, 197780 writeouts
2020-08-06T15:00:23.627205043Z Exiting normally, received signal 15.
2020-08-06T15:00:23.633296582Z frontdoor: ffmpeg process is not running. exiting capture thread...
2020-08-06T15:00:33.647412282Z Creating ffmpeg process...
2020-08-06T15:00:33.647509669Z ffmpeg -hide_banner -loglevel debug -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtsp://x.x.x.84/ch0_1.h264 -f rawvideo -pix_fmt rgb24 pipe:
2020-08-06T15:00:33.911397137Z Splitting the commandline.
2020-08-06T15:00:33.911477320Z Reading option '-hide_banner' ... matched as option 'hide_banner' (do not show program banner) with argument '1'.
<SNIP>
2020-08-06T15:00:33.921578369Z matched as AVOption 'strict' with argument 'experimental'.
2020-08-06T15:00:33.923854822Z Reading option '-fflags' ... matched as AVOption 'fflags' with argument '+genpts+discardcorrupt'.
<SNIP>
2020-08-06T15:00:33.926453212Z Finished splitting the commandline.
2020-08-06T15:00:33.926480711Z Parsing a group of options: global .
2020-08-06T15:00:33.926510729Z Applying option hide_banner (do not show program banner) with argument 1.
2020-08-06T15:00:33.926534155Z Applying option loglevel (set logging level) with argument debug.
2020-08-06T15:00:33.926597394Z Applying option vsync (video sync method) with argument drop.
2020-08-06T15:00:33.926622838Z Successfully parsed a group of options.
2020-08-06T15:00:33.926643856Z Parsing a group of options: input url rtsp://x.x.x.84/ch0_1.h264.
2020-08-06T15:00:33.926665856Z Successfully parsed a group of options.
2020-08-06T15:00:33.926687596Z Opening an input file: rtsp://x.x.x.84/ch0_1.h264.
2020-08-06T15:00:33.927496913Z [tcp @ 0x1865310] No default whitelist set
2020-08-06T15:00:33.927565133Z [tcp @ 0x1865310] Original list of addresses:
2020-08-06T15:00:33.927589633Z [tcp @ 0x1865310] Address x.x.x.84 port 554
2020-08-06T15:00:33.927611762Z [tcp @ 0x1865310] Interleaved list of addresses:
2020-08-06T15:00:33.927634002Z [tcp @ 0x1865310] Address x.x.x.84 port 554
2020-08-06T15:00:33.927655631Z [tcp @ 0x1865310] Starting connection attempt to x.x.x.84 port 554
2020-08-06T15:00:40.137544842Z [tcp @ 0x1865310] Connection attempt to x.x.x.84 port 554 failed: No route to host
2020-08-06T15:00:40.137729430Z [tcp @ 0x1865310] Connection to tcp://x.x.x.84:554?timeout=0 failed: No route to host
2020-08-06T15:00:40.137763559Z rtsp://x.x.x.84/ch0_1.h264: No route to host
2020-08-06T15:00:40.149071750Z frontdoor: ffmpeg didnt return a frame. something is wrong.
2020-08-06T15:00:40.149241265Z frontdoor: ffmpeg didnt return a frame. something is wrong.
2020-08-06T15:00:40.149386318Z frontdoor: ffmpeg didnt return a frame. something is wrong.
<SNIP>
2020-08-06T15:00:40.157211878Z frontdoor: ffmpeg didnt return a frame. something is wrong.
2020-08-06T15:00:40.157661387Z frontdoor: ffmpeg process is not running. exiting capture thread...
2020-08-06T15:00:43.695186561Z Creating ffmpeg process...
2020-08-06T15:00:43.696526293Z ffmpeg -hide_banner -loglevel debug -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtsp://x.x.x.84/ch0_1.h264 -f rawvideo -pix_fmt rgb24 pipe:
2020-08-06T15:00:44.018239772Z Splitting the commandline.
2020-08-06T15:00:44.018321308Z Reading option '-hide_banner' ... matched as option 'hide_banner' (do not show program banner) with argument '1'.
<SNIP>
2020-08-06T15:00:44.030062250Z matched as AVOption 'strict' with argument 'experimental'.
2020-08-06T15:00:44.032240945Z Reading option '-fflags' ... matched as AVOption 'fflags' with argument '+genpts+discardcorrupt'.
<SNIP>
2020-08-06T15:00:44.034702209Z Finished splitting the commandline.
2020-08-06T15:00:44.034729005Z Parsing a group of options: global .
2020-08-06T15:00:44.034751727Z Applying option hide_banner (do not show program banner) with argument 1.
2020-08-06T15:00:44.034774597Z Applying option loglevel (set logging level) with argument debug.
2020-08-06T15:00:44.034797429Z Applying option vsync (video sync method) with argument drop.
2020-08-06T15:00:44.034820059Z Successfully parsed a group of options.
2020-08-06T15:00:44.034841799Z Parsing a group of options: input url rtsp://x.x.x.84/ch0_1.h264.
2020-08-06T15:00:44.034864150Z Successfully parsed a group of options.
2020-08-06T15:00:44.034886168Z Opening an input file: rtsp://x.x.x.84/ch0_1.h264.
2020-08-06T15:00:44.034911612Z [tcp @ 0x6fc310] No default whitelist set
2020-08-06T15:00:44.035182681Z [tcp @ 0x6fc310] Original list of addresses:
2020-08-06T15:00:44.035221661Z [tcp @ 0x6fc310] Address x.x.x.84 port 554
2020-08-06T15:00:44.035262698Z [tcp @ 0x6fc310] Interleaved list of addresses:
2020-08-06T15:00:44.035284271Z [tcp @ 0x6fc310] Address x.x.x.84 port 554
2020-08-06T15:00:44.035305197Z [tcp @ 0x6fc310] Starting connection attempt to x.x.x.84 port 554
2020-08-06T15:00:47.178057687Z [tcp @ 0x6fc310] Connection attempt to x.x.x.84 port 554 failed: No route to host
2020-08-06T15:00:47.178271516Z [tcp @ 0x6fc310] Connection to tcp://x.x.x.84:554?timeout=0 failed: No route to host
2020-08-06T15:00:47.178845208Z rtsp://x.x.x.84/ch0_1.h264: No route to host
2020-08-06T15:00:47.201583941Z frontdoor: ffmpeg didnt return a frame. something is wrong.
2020-08-06T15:00:47.203175686Z frontdoor: ffmpeg process is not running. exiting capture thread...
2020-08-06T15:00:53.748251551Z Creating ffmpeg process...
2020-08-06T15:00:53.748348734Z ffmpeg -hide_banner -loglevel debug -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtsp://x.x.x.84/ch0_1.h264 -f rawvideo -pix_fmt rgb24 pipe:
2020-08-06T15:00:54.052392960Z Splitting the commandline.
2020-08-06T15:00:54.052470014Z Reading option '-hide_banner' ... matched as option 'hide_banner' (do not show program banner) with argument '1'.
<SNIP>
2020-08-06T15:00:54.063361530Z matched as AVOption 'strict' with argument 'experimental'.
2020-08-06T15:00:54.065912441Z Reading option '-fflags' ... matched as AVOption 'fflags' with argument '+genpts+discardcorrupt'.
<SNIP>
2020-08-06T15:00:54.069499830Z Finished splitting the commandline.
2020-08-06T15:00:54.069825601Z Parsing a group of options: global .
2020-08-06T15:00:54.069902507Z Applying option hide_banner (do not show program banner) with argument 1.
2020-08-06T15:00:54.070091577Z Applying option loglevel (set logging level) with argument debug.
2020-08-06T15:00:54.070128206Z Applying option vsync (video sync method) with argument drop.
2020-08-06T15:00:54.070155113Z Successfully parsed a group of options.
2020-08-06T15:00:54.070526938Z Parsing a group of options: input url rtsp://x.x.x.84/ch0_1.h264.
2020-08-06T15:00:54.070580622Z Successfully parsed a group of options.
2020-08-06T15:00:54.070653954Z Opening an input file: rtsp://x.x.x.84/ch0_1.h264.
2020-08-06T15:00:54.071220813Z [tcp @ 0x642310] No default whitelist set
2020-08-06T15:00:54.071382884Z [tcp @ 0x642310] Original list of addresses:
2020-08-06T15:00:54.071484122Z [tcp @ 0x642310] Address x.x.x.84 port 554
2020-08-06T15:00:54.071938650Z [tcp @ 0x642310] Interleaved list of addresses:
2020-08-06T15:00:54.072006982Z [tcp @ 0x642310] Address x.x.x.84 port 554
2020-08-06T15:00:54.072093795Z [tcp @ 0x642310] Starting connection attempt to x.x.x.84 port 554
2020-08-06T15:00:54.075766682Z [tcp @ 0x642310] Connection attempt to x.x.x.84 port 554 failed: Connection refused
2020-08-06T15:00:54.075904661Z [tcp @ 0x642310] Connection to tcp://x.x.x.84:554?timeout=0 failed: Connection refused
2020-08-06T15:00:54.076041695Z rtsp://x.x.x.84/ch0_1.h264: Connection refused
2020-08-06T15:00:54.084935585Z frontdoor: ffmpeg didnt return a frame. something is wrong.
2020-08-06T15:00:54.085422668Z frontdoor: ffmpeg didnt return a frame. something is wrong.
2020-08-06T15:00:54.085782198Z frontdoor: ffmpeg didnt return a frame. something is wrong.
2020-08-06T15:00:54.086501590Z frontdoor: ffmpeg process is not running. exiting capture thread...
2020-08-06T15:01:03.782366936Z Creating ffmpeg process...
2020-08-06T15:01:03.782485990Z ffmpeg -hide_banner -loglevel debug -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtsp://x.x.x.84/ch0_1.h264 -f rawvideo -pix_fmt rgb24 pipe:
2020-08-06T15:01:04.046023761Z Splitting the commandline.
2020-08-06T15:01:04.046093296Z Reading option '-hide_banner' ... matched as option 'hide_banner' (do not show program banner) with argument '1'.
<SNIP>
2020-08-06T15:01:04.056241347Z matched as AVOption 'strict' with argument 'experimental'.
2020-08-06T15:01:04.058652909Z Reading option '-fflags' ... matched as AVOption 'fflags' with argument '+genpts+discardcorrupt'.
<SNIP>
2020-08-06T15:01:04.061442833Z Finished splitting the commandline.
2020-08-06T15:01:04.061474740Z Parsing a group of options: global .
2020-08-06T15:01:04.061500054Z Applying option hide_banner (do not show program banner) with argument 1.
2020-08-06T15:01:04.061524720Z Applying option loglevel (set logging level) with argument debug.
2020-08-06T15:01:04.061547294Z Applying option vsync (video sync method) with argument drop.
2020-08-06T15:01:04.061570349Z Successfully parsed a group of options.
2020-08-06T15:01:04.061628848Z Parsing a group of options: input url rtsp://x.x.x.84/ch0_1.h264.
2020-08-06T15:01:04.061653662Z Successfully parsed a group of options.
2020-08-06T15:01:04.061675773Z Opening an input file: rtsp://x.x.x.84/ch0_1.h264.
2020-08-06T15:01:04.062361648Z [tcp @ 0x127c310] No default whitelist set
2020-08-06T15:01:04.062467182Z [tcp @ 0x127c310] Original list of addresses:
2020-08-06T15:01:04.062529366Z [tcp @ 0x127c310] Address x.x.x.84 port 554
2020-08-06T15:01:04.062554255Z [tcp @ 0x127c310] Interleaved list of addresses:
2020-08-06T15:01:04.062576828Z [tcp @ 0x127c310] Address x.x.x.84 port 554
2020-08-06T15:01:04.062705640Z [tcp @ 0x127c310] Starting connection attempt to x.x.x.84 port 554
2020-08-06T15:01:04.068017291Z [tcp @ 0x127c310] Connection attempt to x.x.x.84 port 554 failed: Connection refused
2020-08-06T15:01:04.068102585Z [tcp @ 0x127c310] Connection to tcp://x.x.x.84:554?timeout=0 failed: Connection refused
2020-08-06T15:01:04.068129177Z rtsp://x.x.x.84/ch0_1.h264: Connection refused
2020-08-06T15:01:04.079474703Z frontdoor: ffmpeg didnt return a frame. something is wrong.
2020-08-06T15:01:04.080003822Z frontdoor: ffmpeg process is not running. exiting capture thread...
2020-08-06T15:01:13.810684313Z Creating ffmpeg process...
2020-08-06T15:01:13.810865439Z ffmpeg -hide_banner -loglevel debug -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -use_wallclock_as_timestamps 1 -i rtsp://x.x.x.84/ch0_1.h264 -f rawvideo -pix_fmt rgb24 pipe:
2020-08-06T15:01:14.126914494Z Splitting the commandline.
2020-08-06T15:01:14.127059491Z Reading option '-hide_banner' ... matched as option 'hide_banner' (do not show program banner) with argument '1'.
<SNIP>
2020-08-06T15:01:14.141516511Z matched as AVOption 'strict' with argument 'experimental'.
2020-08-06T15:01:14.148601532Z Reading option '-fflags' ... matched as AVOption 'fflags' with argument '+genpts+discardcorrupt'.
<SNIP>
2020-08-06T15:01:14.151344087Z Finished splitting the commandline.
2020-08-06T15:01:14.151369087Z Parsing a group of options: global .
2020-08-06T15:01:14.151393401Z Applying option hide_banner (do not show program banner) with argument 1.
2020-08-06T15:01:14.151416827Z Applying option loglevel (set logging level) with argument debug.
2020-08-06T15:01:14.151438141Z Applying option vsync (video sync method) with argument drop.
2020-08-06T15:01:14.151460289Z Successfully parsed a group of options.
2020-08-06T15:01:14.151483159Z Parsing a group of options: input url rtsp://x.x.x.84/ch0_1.h264.
2020-08-06T15:01:14.151505547Z Successfully parsed a group of options.
2020-08-06T15:01:14.151527158Z Opening an input file: rtsp://x.x.x.84/ch0_1.h264.
2020-08-06T15:01:14.151587249Z [tcp @ 0x19f8310] No default whitelist set
2020-08-06T15:01:14.151613767Z [tcp @ 0x19f8310] Original list of addresses:
2020-08-06T15:01:14.151636118Z [tcp @ 0x19f8310] Address x.x.x.84 port 554
2020-08-06T15:01:14.151657877Z [tcp @ 0x19f8310] Interleaved list of addresses:
2020-08-06T15:01:14.151679821Z [tcp @ 0x19f8310] Address x.x.x.84 port 554
2020-08-06T15:01:14.151701950Z [tcp @ 0x19f8310] Starting connection attempt to x.x.x.84 port 554
2020-08-06T15:01:14.170440178Z [tcp @ 0x19f8310] Successfully connected to x.x.x.84 port 554
2020-08-06T15:01:14.895588241Z [rtsp @ 0x19f6380] SDP:
2020-08-06T15:01:14.895755534Z v=0
2020-08-06T15:01:14.895823958Z o=- 1582065738276498 1 IN IP4 x.x.x.84
2020-08-06T15:01:14.895883957Z s=Session streamed by "rtspv4"
2020-08-06T15:01:14.895943049Z i=ch0_1.h264
2020-08-06T15:01:14.896002399Z t=0 0
2020-08-06T15:01:14.896060139Z a=tool:LIVE555 Streaming Media v2018.12.14
2020-08-06T15:01:14.896119064Z a=type:broadcast
2020-08-06T15:01:14.896175766Z a=control:*
2020-08-06T15:01:14.896232506Z a=range:npt=0-
2020-08-06T15:01:14.896288467Z a=x-qt-text-nam:Session streamed by "rtspv4"
2020-08-06T15:01:14.896343800Z a=x-qt-text-inf:ch0_1.h264
2020-08-06T15:01:14.896397243Z m=video 0 RTP/AVP 96
2020-08-06T15:01:14.896454742Z c=IN IP4 0.0.0.0
2020-08-06T15:01:14.897591330Z b=AS:500
2020-08-06T15:01:14.897657254Z a=rtpmap:96 H264/90000
2020-08-06T15:01:14.897715827Z a=fmtp:96 packetization-mode=1;profile-level-id=64001E;sprop-parameter-sets=Z2QAHq2EAQwgCGEAQwgCGEAQwgCEO1BQF/yzcBAQECA=,aO48sA==
2020-08-06T15:01:14.897851639Z a=control:track1
2020-08-06T15:01:14.897917953Z
2020-08-06T15:01:14.897976100Z Failed to parse interval end specification ''
2020-08-06T15:01:14.898033747Z [rtsp @ 0x19f6380] video codec set to: h264
2020-08-06T15:01:14.898091597Z [rtsp @ 0x19f6380] RTP Packetization Mode: 1
2020-08-06T15:01:14.898149855Z [rtsp @ 0x19f6380] RTP Profile IDC: 64 Profile IOP: 0 Level: 1e
2020-08-06T15:01:14.898208836Z [rtsp @ 0x19f6380] Extradata set to 0x19f9920 (size: 44)
2020-08-06T15:01:14.898273705Z [rtp @ 0x19f9aa0] No default whitelist set
2020-08-06T15:01:14.900729784Z [udp @ 0x19f9f90] No default whitelist set
2020-08-06T15:01:14.900871744Z [udp @ 0x19f9f90] end receive buffer size reported is 131072
2020-08-06T15:01:14.900942224Z [udp @ 0x1a0a2e0] No default whitelist set
2020-08-06T15:01:14.901008926Z [udp @ 0x1a0a2e0] end receive buffer size reported is 131072
2020-08-06T15:01:14.922403452Z [rtsp @ 0x19f6380] setting jitter buffer size to 500
2020-08-06T15:01:14.922559764Z [rtsp @ 0x19f6380] hello state=0
2020-08-06T15:01:14.967850819Z Failed to parse interval end specification ''
2020-08-06T15:01:14.969817723Z [h264 @ 0x19f9490] nal_unit_type: 7(SPS), nal_ref_idc: 3
2020-08-06T15:01:14.969885296Z [h264 @ 0x19f9490] nal_unit_type: 8(PPS), nal_ref_idc: 3
2020-08-06T15:01:14.970537060Z [rtsp @ 0x19f6380] CSeq 4 expected, 0 received.
2020-08-06T15:01:16.686355740Z [h264 @ 0x19f9490] nal_unit_type: 7(SPS), nal_ref_idc: 3
2020-08-06T15:01:16.687135687Z [h264 @ 0x19f9490] nal_unit_type: 8(PPS), nal_ref_idc: 3
2020-08-06T15:01:17.095422237Z [h264 @ 0x19f9490] nal_unit_type: 7(SPS), nal_ref_idc: 3
2020-08-06T15:01:17.095537679Z [h264 @ 0x19f9490] nal_unit_type: 8(PPS), nal_ref_idc: 3
2020-08-06T15:01:17.095880801Z [h264 @ 0x19f9490] nal_unit_type: 5(IDR), nal_ref_idc: 3
2020-08-06T15:01:17.096724654Z [h264 @ 0x19f9490] Format yuvj420p chosen by get_format().
2020-08-06T15:01:17.096859837Z [h264 @ 0x19f9490] Reinit context to 640x368, pix_fmt: yuvj420p
2020-08-06T15:01:17.126945611Z [h264 @ 0x19f9490] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
2020-08-06T15:01:17.967998345Z Last message repeated 5 times
2020-08-06T15:01:17.968168934Z [rtsp @ 0x19f6380] All info found
2020-08-06T15:01:17.968213822Z [rtsp @ 0x19f6380] rfps: 23.416667 0.018526
2020-08-06T15:01:17.968246914Z [rtsp @ 0x19f6380] rfps: 23.500000 0.015363
2020-08-06T15:01:17.969198154Z [rtsp @ 0x19f6380] rfps: 23.583333 0.013172
2020-08-06T15:01:17.969268393Z Last message repeated 1 times
2020-08-06T15:01:17.969300411Z [rtsp @ 0x19f6380] rfps: 23.666667 0.011954
2020-08-06T15:01:17.969324151Z Last message repeated 1 times
2020-08-06T15:01:17.969347503Z [rtsp @ 0x19f6380] rfps: 23.750000 0.011708
2020-08-06T15:01:17.969418390Z [rtsp @ 0x19f6380] rfps: 23.833333 0.012435
2020-08-06T15:01:17.969443538Z [rtsp @ 0x19f6380] rfps: 23.916667 0.014134
2020-08-06T15:01:17.969466482Z [rtsp @ 0x19f6380] rfps: 24.000000 0.016806
2020-08-06T15:01:17.969490296Z [rtsp @ 0x19f6380] rfps: 23.976024 0.015937
2020-08-06T15:01:17.969512962Z Input #0, rtsp, from 'rtsp://x.x.x.84/ch0_1.h264':
2020-08-06T15:01:17.969536314Z Metadata:
2020-08-06T15:01:17.969559146Z title : Session streamed by "rtspv4"
2020-08-06T15:01:17.969582146Z comment : ch0_1.h264
2020-08-06T15:01:17.969605460Z Duration: N/A, start: 1596726077.093333, bitrate: N/A
2020-08-06T15:01:17.969629997Z Stream #0:0, 22, 1/90000: Video: h264 (High), 1 reference frame, yuvj420p(pc, bt709, progressive, left), 640x360 (640x368), 0/1, 90k tbr, 90k tbn, 180k tbc
2020-08-06T15:01:17.969716884Z Successfully opened the file.
2020-08-06T15:01:17.969759883Z Parsing a group of options: output url pipe:.
2020-08-06T15:01:17.969784846Z Applying option f (force format) with argument rawvideo.
2020-08-06T15:01:17.969808493Z Applying option pix_fmt (set pixel format) with argument rgb24.
2020-08-06T15:01:17.969831641Z Successfully parsed a group of options.
2020-08-06T15:01:17.969853715Z Opening an output file: pipe:.
2020-08-06T15:01:17.970884453Z [pipe @ 0x1a34130] Setting default whitelist 'crypto'
2020-08-06T15:01:17.970942026Z Successfully opened the file.
2020-08-06T15:01:17.970975358Z detected 4 logical cores
2020-08-06T15:01:17.978275987Z [h264 @ 0x1a5b540] nal_unit_type: 7(SPS), nal_ref_idc: 3
2020-08-06T15:01:17.978344375Z [h264 @ 0x1a5b540] nal_unit_type: 8(PPS), nal_ref_idc: 3
2020-08-06T15:01:17.978374855Z Stream mapping:
2020-08-06T15:01:17.978442632Z Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
2020-08-06T15:01:17.978489612Z Press [q] to stop, [?] for help
2020-08-06T15:01:17.978516408Z cur_dts is invalid (this is harmless if it occurs once at the start per stream)
2020-08-06T15:01:17.978922585Z [h264 @ 0x1a5b540] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
2020-08-06T15:01:17.979019083Z [h264 @ 0x1a5b540] Format yuvj420p chosen by get_format().
2020-08-06T15:01:17.979047360Z [h264 @ 0x1a5b540] Reinit context to 640x368, pix_fmt: yuvj420p
2020-08-06T15:01:17.979246615Z [h264 @ 0x1a5b540] Frame num gap 22 20
2020-08-06T15:01:17.983916853Z cur_dts is invalid (this is harmless if it occurs once at the start per stream)
2020-08-06T15:01:17.984009833Z [h264 @ 0x1a5b540] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
2020-08-06T15:01:17.986753703Z cur_dts is invalid (this is harmless if it occurs once at the start per stream)
<SNIP, more of the same>
2020-08-06T15:01:18.701361580Z [h264 @ 0x1a5b540] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
2020-08-06T15:01:18.703086767Z cur_dts is invalid (this is harmless if it occurs once at the start per stream)
2020-08-06T15:01:18.880519757Z [h264 @ 0x1a5b540] nal_unit_type: 7(SPS), nal_ref_idc: 3
2020-08-06T15:01:18.880626403Z [h264 @ 0x1a5b540] nal_unit_type: 8(PPS), nal_ref_idc: 3
2020-08-06T15:01:18.880657235Z [h264 @ 0x1a5b540] nal_unit_type: 5(IDR), nal_ref_idc: 3
2020-08-06T15:01:18.896667223Z [graph 0 input from stream 0:0 @ 0x1c33d60] Setting 'video_size' to value '640x360'
2020-08-06T15:01:18.896750870Z [graph 0 input from stream 0:0 @ 0x1c33d60] Setting 'pix_fmt' to value '12'
2020-08-06T15:01:18.896921181Z [graph 0 input from stream 0:0 @ 0x1c33d60] Setting 'time_base' to value '1/90000'
2020-08-06T15:01:18.896969125Z [graph 0 input from stream 0:0 @ 0x1c33d60] Setting 'pixel_aspect' to value '0/1'
2020-08-06T15:01:18.897042679Z [graph 0 input from stream 0:0 @ 0x1c33d60] Setting 'sws_param' to value 'flags=2'
2020-08-06T15:01:18.897068271Z [graph 0 input from stream 0:0 @ 0x1c33d60] Setting 'frame_rate' to value '90000/1'
2020-08-06T15:01:18.897097103Z [graph 0 input from stream 0:0 @ 0x1c33d60] w:640 h:360 pixfmt:yuvj420p tb:1/90000 fr:90000/1 sar:0/1 sws_param:flags=2
2020-08-06T15:01:18.897121640Z [format @ 0x1c34550] Setting 'pix_fmts' to value 'rgb24'
2020-08-06T15:01:18.897643611Z [auto_scaler_0 @ 0x1a689a0] Setting 'flags' to value 'bicubic'
2020-08-06T15:01:18.897697795Z [auto_scaler_0 @ 0x1a689a0] w:iw h:ih flags:'bicubic' interl:0
2020-08-06T15:01:18.897722405Z [format @ 0x1c34550] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_null_0' and the filter 'format'
2020-08-06T15:01:18.897746164Z [AVFilterGraph @ 0x1a68880] query_formats: 4 queried, 2 merged, 1 already done, 0 delayed
2020-08-06T15:01:18.897769442Z [swscaler @ 0x1b158e0] deprecated pixel format used, make sure you did set range correctly
2020-08-06T15:01:18.898800995Z [swscaler @ 0x1b158e0] No accelerated colorspace conversion found from yuv420p to rgb24.
2020-08-06T15:01:18.898948417Z [auto_scaler_0 @ 0x1a689a0] w:640 h:360 fmt:yuvj420p sar:0/1 -> w:640 h:360 fmt:rgb24 sar:0/1 flags:0x4
2020-08-06T15:01:18.902122205Z Output #0, rawvideo, to 'pipe:':
2020-08-06T15:01:18.902192092Z Metadata:
2020-08-06T15:01:18.902219036Z title : Session streamed by "rtspv4"
2020-08-06T15:01:18.902245221Z comment : ch0_1.h264
2020-08-06T15:01:18.902275979Z encoder : Lavf58.20.100
2020-08-06T15:01:18.902349422Z Stream #0:0, 0, 1/90000: Video: rawvideo, 1 reference frame (RGB[24] / 0x18424752), rgb24(left), 640x360, 0/1, q=2-31, 497664000 kb/s, 90k fps, 90k tbn, 90k tbc
2020-08-06T15:01:18.902376107Z Metadata:
2020-08-06T15:01:18.903067333Z encoder : Lavc58.35.100 rawvideo
2020-08-06T15:01:18.904345752Z [rawvideo @ 0x1a34a70] Encoder did not produce proper pts, making some up.
2020-08-06T15:01:18.997378705Z [h264 @ 0x1a5b540] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
2020-08-06T15:01:19.028475163Z ../src/plasma/client.cc:182: mmap failed
2020-08-06T15:01:19.029652361Z /usr/local/lib/python3.7/dist-packages/pyarrow/libarrow.so.17(+0x614cc0)[0xac721cc0]
2020-08-06T15:01:19.030347791Z /usr/local/lib/python3.7/dist-packages/pyarrow/libarrow.so.17(_ZN5arrow4util8ArrowLogD1Ev+0x108)[0xac721de0]
2020-08-06T15:01:19.030421049Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0xd0f4)[0xabed70f4]
2020-08-06T15:01:19.030447882Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(+0x10844)[0xabeda844]
2020-08-06T15:01:19.030472122Z /usr/local/lib/python3.7/dist-packages/pyarrow/libplasma.so.17(_ZN6plasma12PlasmaClient6CreateERKNS_8UniqueIDExPKhxPSt10shared_ptrIN5arrow6BufferEEib+0x44)[0xabedad5c]
2020-08-06T15:01:19.030496288Z /usr/local/lib/python3.7/dist-packages/pyarrow/_plasma.cpython-37m-arm-linux-gnueabihf.so(+0x1fce0)[0xabf19ce0]
<container restarted>
Thanks again for your work and dedication 
So I’m now running Frigate with 6 Hikvision cameras:
- 3 processing 4fps 640x480 (2mpx cameras’ 2nd sub-stream)
- 3 processing 2fps 720p (slightly more expensive 2mpx cameras’ 3rd sub-stream)
I’m running it on a 6th gen Core i5, with the Coral connected via USB 3 (around 10 - 11ms inference) and seeing around 35% CPU usage. I’ve also enabled clips on all cameras as well, so every now and then there’s a brief spike in CPU usage which I assume is related to writing out these.
Overall very happy. Planning on adding one more camera running at 640x480 over the weekend. Looking fairly stable so far.
Does 0.6 have any improvements on nightime detection? On 0.5 i detect people during the day without issues, but at night I get nothing. (0.4 didn’t have the same issues)
Hi,
This is far better than previous solutions I have tried. Thanks, it’s great job, and so easy to configure!
My setup comprise 4 Ankle 4k cameras using the low-res stream plus a cheap RTSP camera. The frigate container is placed beside Home Assistant and Motion containers on the same PC. I’m using 0.6.0-rc1 release.
Therefore, night detection isn’t good. I understand that the model may haven’t been trained with black and white poor contrasted IR images…
@calypso told the night detection used to be better with version 0.4.
The diagram on Blake’s readme suggests that the system is waiting motion detection before sending the frame to the object detection. May be, images aren’t submitted because motion was not detected because of low contrast?
Thanks for your work!
I think you are quite correct. In night mode i think its far harder for motion to be detected. In 0.4 the frame was being scanned regardless of motion.
That is my suspicion as well. If you have a video clip you can send me, I can work on refining the sensitivity at night.
1 Like
I’ll do it tonight.
Which resolution do you prefer?
How to transmit files efficiently?
Ideally, it would the exact same video feed you analyze with frigate. Dropbox, Google Drive are all fine. Most people have sent me a link in a PM.
With 0.6.0rc1 I’m getting really high CPU use (over 80% when usually around 25%) and I think it’s to do with ffmpeg trying to process more frames than there are:
On 0.5.0 I have these (the actual frames set directly on the cameras)
Happy to help debug if I’m pointed in the right direction!
Thanks
The -vsync drop parameter was removed on 0.6.0 in order to make the video clips work. Apparently, ffmpeg isn’t able to get a proper frame rate for some cameras, so you need to specify a framerate on your input params: https://github.com/blakeblackshear/frigate/issues/176
My guess is that this was a problem before 0.6.0, but the -vsync drop parameter just discarded the extra frames.
Definitely related to motion detection sensitivity. I had the guards now walk around with flashlights at night, and it detects them no issues. No touch, the guard just fades into a sea of grey when using IR, no enough of a change to initiate frigate to start scanning.
Thanks, this works much better.
anyway, I have still some detection miss. Contrast of the image should matter.
Is it a way to feed the coral device directly with a video stream or file ?
++ Simulot