Local realtime person detection for RTSP cameras

blakeblackshear · March 1, 2020, 5:52pm

No other errors before that message? Are you using a Coral?

Krapstar · March 1, 2020, 6:05pm

Yep, I do have a usb Coral and the logs dont complain about TPU not being there so I am assuming it is being used. I also obeserved that this only happens with Unifi RTSP stream(1024x576 H264) cameras but not with Hikvision substream(640x480 H264H). I can try reducing the aspect ratio…may be that will help. Will post update in a bit.

Update: Tried lowering the resolution… didnt help. Process got stuck the moment there was motion but the Hikvision camera stream detection seems to work fine.

blakeblackshear · March 1, 2020, 6:24pm

So it works if you only setup your hikvision cameras and stops if you add your unifi cameras?

Krapstar · March 1, 2020, 6:27pm

Correct. Btw, all cameras work great with v0.3

Edit: To clarify - All cameras startup fine…I can see the debug stream working fine. I get the error mentioned above about detection appearing to be stuck only from unifi cameras when there is motion.

Edit2: Confirmed…works fine v0.3 and near instant detection too.

Edit3: Hikvision camera stream seems to be getting stuck after a bit as well.

blakeblackshear · March 1, 2020, 6:51pm

It should restart the detection subprocess and keep going. What happens after it says it is stuck?

Krapstar · March 1, 2020, 6:52pm

Like you said it restarts the process but gets stuck again. The debug stream “dies” as in I can see the timestamp frozen.

blakeblackshear · March 1, 2020, 8:42pm

And 0.4.0 does not work either right? Trying to narrow down where the change could have happened.

Krapstar · March 1, 2020, 11:09pm

Double checked - it looks like 0.4.0-beta works. Seems to be post 0.5.0-rc1.

Edit: Here is the version breakdown and status

0.3.0 - Everything works great and near instant
0.4.0-beta - Everything works but choppy/slow
0.5.0-rc1 - I get the following error when I try to pull up the debug frame:

Error on request:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/werkzeug/serving.py", line 323, in run_wsgi
    execute(self.server.app)
  File "/usr/local/lib/python3.7/dist-packages/werkzeug/serving.py", line 314, in execute
    for data in application_iter:
  File "/usr/local/lib/python3.7/dist-packages/werkzeug/wsgi.py", line 506, in __next__
    return self._next()
  File "/usr/local/lib/python3.7/dist-packages/werkzeug/wrappers/base_response.py", line 45, in _iter_encoded
    for item in iterable:
  File "/opt/frigate/detect_objects.py", line 224, in imagestream
    frame = object_processor.get_current_frame(camera_name)
  File "/opt/frigate/frigate/object_processing.py", line 47, in get_current_frame
    return self.camera_data[camera]['current_frame']
KeyError: 'current_frame'

0.5.0-rc2 - Same error as above
0.5.0-rc3 - I get black screen in the debug stream with the following error…

/arrow/cpp/src/plasma/store.cc:738: Disconnecting client on fd 8

0.5.0-rc4 - Same black screen as above

driveway: ffmpeg_process exited unexpectedly with 1
driveway: exiting subprocess
/arrow/cpp/src/plasma/store.cc:738: Disconnecting client on fd 8

0.5.0-rc5 - Debug stream works fine but the error mentioned above happens.

Detection appears to be stuck. Restarting detection process

Hope this helps and let me know if you need any additional info

Thanks

surge919 · March 2, 2020, 12:16am

@blakeblackshear
Unless I’m missing something, there’s no easy way to view the logs in a container on Synology.
There is a logs tabs in the UI but I can’t figure out how to simply view the logs in a terminal.
ie: tail -f logfile

Can you add an option to direct the logs to a regular log file?

I’ve had a few instances where 1 of my 2 feeds stopped getting processed by frigate.
when this happens, going to the 5555/ url would show the last time the feed was working.

This happened twice.

A third time, the feed stopped for both and nothing was showing in the Terminal view.

In a previous post you mentioned the restart is not required in the Run the container with section of your docs. Does that mean restarts are enabled by default or if I don’t have that, they are not?

I’m using grafana for better graphing and this shows where the drops are.
The first 2 areas show where one camera feed just stopped getting processed
The 3rd area shows where frigate just died.

blakeblackshear · March 2, 2020, 12:31am

Viewing logs like this doesn’t work? https://www.synology.com/en-us/knowledgebase/DSM/tutorial/Virtualization/How_can_I_debug_and_fix_common_Docker_container_issues

blakeblackshear · March 2, 2020, 12:35am

Frigate manages all the processes itself, and very rarely crashes in a way that would cause the container to stop. Docker only restarts the container if it exits. Adding the restart unless-stopped parameter would only have an effect if the container actually exits or the server restarts.

blakeblackshear · March 2, 2020, 12:41am

Just realized that I never call the function to restart the detection process when it gets stuck. I just log a message and keep going. Going to fix and push up rc6 tonight.

ManuelPerrot · March 2, 2020, 12:47am

rc3 rc4 rc5 rc6 and rc7:
I’m randomly having black screens on the debug url.

Killing and recreating the container with the exact same config sometime solve the problem.

It seems to not start the ffmpeg process?
I don’t have this line in the logs when the issue happens:

frigate    | {'streams': [{'index': 0, 'codec_name': 'h264', 'codec_long_name': 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10', 'profile': 'High', 'codec_type': 'video', 'codec_time_base': etc...

/debug/stats

{"camera1":{"detection_fps":0.0,"fps":5.0,"skipped_fps":0.0},"coral":{"detection_queue":0,"detection_start":0.0,"fps":0.0,"inference_speed":10.0},"plasma_store_rc":null,"tracked_objects_queue":0}

blakeblackshear · March 2, 2020, 12:55am

Ok. Here is rc6.

Changes:

Update docs
Handle errors better in detection process
Actually call the restart function when detection hangs

Docker image is available with docker pull blakeblackshear/frigate:0.5.0-rc6

surge919 · March 2, 2020, 1:16am

it works but it’s not as good as tail -f logfile

the first Log view they mention just shows basic “container started, stopped, deleted, etc… no actual frigate info”

the second Log view they mention isn’t as interactive and when it Exports it reverses the order so it’s not a share friendly.

for example… here’s the latest output (there’s nothing to check, i’m just showing the output)

Frigate-0.5.0-rc6

date stream content

2020-03-02 01:04:13 stderr [rawvideo @ 0x5568d9575680] Encoder did not produce proper pts, making some up.

2020-03-02 01:04:13 stderr encoder : Lavc57.107.100 rawvideo

2020-03-02 01:04:13 stderr Metadata:

2020-03-02 01:04:13 stderr Stream #0:0: Video: rawvideo, 1 reference frame (RGB[24] / 0x18424752), rgb24(left), 1920x1080, q=2-31, 746496 kb/s, 15 fps, 15 tbn, 15 tbc

2020-03-02 01:04:13 stderr encoder : Lavf57.83.100

2020-03-02 01:04:13 stderr comment : live

2020-03-02 01:04:13 stderr title : Session streamed by “wyze”

2020-03-02 01:04:13 stderr Metadata:

2020-03-02 01:04:13 stderr Output #0, rawvideo, to ‘pipe:’:

2020-03-02 01:04:13 stderr [auto_scaler_0 @ 0x5568d962b760] w:1920 h:1080 fmt:yuv420p sar:0/1 → w:1920 h:1080 fmt:rgb24 sar:0/1 flags:0x4

2020-03-02 01:04:13 stderr [format @ 0x5568d95a2220] auto-inserting filter ‘auto_scaler_0’ between the filter ‘Parsed_null_0’ and the filter ‘format’

2020-03-02 01:04:13 stderr [auto_scaler_0 @ 0x5568d962b760] w:iw h:ih flags:‘bicubic’ interl:0

2020-03-02 01:04:12 stderr frame= 0 fps=0.0 q=0.0 size= 0kB time=-577014:32:22.77 bitrate= -0.0kbits/s speed=N/A frame= 23 fps= 11 q=-0.0 size= 201204kB time=00:00:01.15 bitrate=1433272.3kbits/s speed=0.554x [graph 0 input from stream 0:0 @ 0x5568d95a1380] w:1920 h:1080 pixfmt:yuv420p tb:1/90000 fr:15/1 sar:0/1 sws_param:flags=2

2020-03-02 01:04:11 stderr frame= 12 fps=7.8 q=-0.0 size= 104976kB time=00:00:00.60 bitrate=1433272.3kbits/s speed=0.39x [h264 @ 0x5568d95590c0] Reinit context to 1920x1088, pix_fmt: yuv420p

2020-03-02 01:04:11 stderr Press [q] to stop, [?] for help

2020-03-02 01:04:11 stderr Stream #0:0 → #0:0 (h264 (native) → rawvideo (native))

2020-03-02 01:04:11 stderr Stream mapping:

2020-03-02 01:04:11 stderr Stream #0:1: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s

2020-03-02 01:04:11 stderr Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(progressive, left), 1920x1080 (1920x1088), 15 fps, 15 tbr, 90k tbn, 30 tbc

2020-03-02 01:04:11 stderr Duration: N/A, start: 1583111049.299267, bitrate: N/A

2020-03-02 01:04:11 stderr comment : live

2020-03-02 01:04:11 stderr title : Session streamed by “wyze”

2020-03-02 01:04:11 stderr Metadata:

2020-03-02 01:04:11 stderr Input #0, rtsp, from ‘rtsp://g6ev8v43:[email protected]/live’:

2020-03-02 01:04:11 stderr frame= 1 fps=1.0 q=-0.0 size= 8748kB time=00:00:00.05 bitrate=1433272.3kbits/s speed=0.0494x Guessed Channel Layout for Input Stream #0.1 : mono

2020-03-02 01:04:10 stderr [rawvideo @ 0x5567e0378400] Encoder did not produce proper pts, making some up.

2020-03-02 01:04:10 stderr encoder : Lavc57.107.100 rawvideo

2020-03-02 01:04:10 stderr Metadata:

2020-03-02 01:04:10 stderr Stream #0:0: Video: rawvideo, 1 reference frame (RGB[24] / 0x18424752), rgb24(left), 2304x1296 [SAR 1:1 DAR 16:9], q=2-31, 1433272 kb/s, 20 fps, 20 tbn, 20 tbc

2020-03-02 01:04:10 stderr encoder : Lavf57.83.100

2020-03-02 01:04:10 stderr title : Media Server

2020-03-02 01:04:10 stderr Metadata:

2020-03-02 01:04:10 stderr Output #0, rawvideo, to ‘pipe:’:

2020-03-02 01:04:10 stderr [auto_scaler_0 @ 0x5567e03aa320] w:2304 h:1296 fmt:yuvj420p sar:1/1 → w:2304 h:1296 fmt:rgb24 sar:1/1 flags:0x4

2020-03-02 01:04:10 stderr [swscaler @ 0x5567e042a9e0] deprecated pixel format used, make sure you did set range correctly

2020-03-02 01:04:10 stderr [format @ 0x5567e03a77e0] auto-inserting filter ‘auto_scaler_0’ between the filter ‘Parsed_null_0’ and the filter ‘format’

2020-03-02 01:04:10 stderr [auto_scaler_0 @ 0x5567e03aa320] w:iw h:ih flags:‘bicubic’ interl:0

2020-03-02 01:04:10 stderr frame= 0 fps=0.0 q=0.0 size= 0kB time=-577014:32:22.77 bitrate= -0.0kbits/s speed=N/A [graph 0 input from stream 0:0 @ 0x5567e03af1e0] w:2304 h:1296 pixfmt:yuvj420p tb:1/90000 fr:20/1 sar:1/1 sws_param:flags=2

2020-03-02 01:04:09 stderr [h264 @ 0x5567e03fbac0] Reinit context to 2304x1296, pix_fmt: yuvj420p

2020-03-02 01:04:09 stderr Press [q] to stop, [?] for help

Krapstar · March 2, 2020, 1:30am

Similar error with 0.5.0-rc6

Detection appears to be stuck. Restarting detection process
Waiting for detection process to exit gracefully...
/arrow/cpp/src/plasma/store.cc:738: Disconnecting client on fd 6
Starting detection process: 124

blakeblackshear · March 2, 2020, 2:21am

At least it is actually restarting it now. Does it restart over and over?

Krapstar · March 2, 2020, 2:23am

Yep. It just keeps restarting but the debug screen remains frozen.

blakeblackshear · March 2, 2020, 2:24am

Can you post the output of /debug/stats?

Krapstar · March 2, 2020, 2:27am

I am primarily testing Backpatio atm.

Detection appears to be stuck. Restarting detection process
Waiting for detection process to exit gracefully...
/arrow/cpp/src/plasma/store.cc:738: Disconnecting client on fd 6
Starting detection process: 126

date	stream	content
2020-03-02 01:04:13	stderr	[rawvideo @ 0x5568d9575680] Encoder did not produce proper pts, making some up.
2020-03-02 01:04:13	stderr	encoder : Lavc57.107.100 rawvideo
2020-03-02 01:04:13	stderr	Metadata:
2020-03-02 01:04:13	stderr	Stream #0:0: Video: rawvideo, 1 reference frame (RGB[24] / 0x18424752), rgb24(left), 1920x1080, q=2-31, 746496 kb/s, 15 fps, 15 tbn, 15 tbc
2020-03-02 01:04:13	stderr	encoder : Lavf57.83.100
2020-03-02 01:04:13	stderr	comment : live
2020-03-02 01:04:13	stderr	title : Session streamed by “wyze”
2020-03-02 01:04:13	stderr	Metadata:
2020-03-02 01:04:13	stderr	Output #0, rawvideo, to ‘pipe:’:
2020-03-02 01:04:13	stderr	[auto_scaler_0 @ 0x5568d962b760] w:1920 h:1080 fmt:yuv420p sar:0/1 → w:1920 h:1080 fmt:rgb24 sar:0/1 flags:0x4
2020-03-02 01:04:13	stderr	[format @ 0x5568d95a2220] auto-inserting filter ‘auto_scaler_0’ between the filter ‘Parsed_null_0’ and the filter ‘format’
2020-03-02 01:04:13	stderr	[auto_scaler_0 @ 0x5568d962b760] w:iw h:ih flags:‘bicubic’ interl:0
2020-03-02 01:04:12	stderr	frame= 0 fps=0.0 q=0.0 size= 0kB time=-577014:32:22.77 bitrate= -0.0kbits/s speed=N/A frame= 23 fps= 11 q=-0.0 size= 201204kB time=00:00:01.15 bitrate=1433272.3kbits/s speed=0.554x [graph 0 input from stream 0:0 @ 0x5568d95a1380] w:1920 h:1080 pixfmt:yuv420p tb:1/90000 fr:15/1 sar:0/1 sws_param:flags=2
2020-03-02 01:04:11	stderr	frame= 12 fps=7.8 q=-0.0 size= 104976kB time=00:00:00.60 bitrate=1433272.3kbits/s speed=0.39x [h264 @ 0x5568d95590c0] Reinit context to 1920x1088, pix_fmt: yuv420p
2020-03-02 01:04:11	stderr	Press [q] to stop, [?] for help
2020-03-02 01:04:11	stderr	Stream #0:0 → #0:0 (h264 (native) → rawvideo (native))
2020-03-02 01:04:11	stderr	Stream mapping:
2020-03-02 01:04:11	stderr	Stream #0:1: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s
2020-03-02 01:04:11	stderr	Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(progressive, left), 1920x1080 (1920x1088), 15 fps, 15 tbr, 90k tbn, 30 tbc
2020-03-02 01:04:11	stderr	Duration: N/A, start: 1583111049.299267, bitrate: N/A
2020-03-02 01:04:11	stderr	comment : live
2020-03-02 01:04:11	stderr	title : Session streamed by “wyze”
2020-03-02 01:04:11	stderr	Metadata:
2020-03-02 01:04:11	stderr	Input #0, rtsp, from ‘rtsp://g6ev8v43:[email protected]/live’:
2020-03-02 01:04:11	stderr	frame= 1 fps=1.0 q=-0.0 size= 8748kB time=00:00:00.05 bitrate=1433272.3kbits/s speed=0.0494x Guessed Channel Layout for Input Stream #0.1 : mono
2020-03-02 01:04:10	stderr	[rawvideo @ 0x5567e0378400] Encoder did not produce proper pts, making some up.
2020-03-02 01:04:10	stderr	encoder : Lavc57.107.100 rawvideo
2020-03-02 01:04:10	stderr	Metadata:
2020-03-02 01:04:10	stderr	Stream #0:0: Video: rawvideo, 1 reference frame (RGB[24] / 0x18424752), rgb24(left), 2304x1296 [SAR 1:1 DAR 16:9], q=2-31, 1433272 kb/s, 20 fps, 20 tbn, 20 tbc
2020-03-02 01:04:10	stderr	encoder : Lavf57.83.100
2020-03-02 01:04:10	stderr	title : Media Server
2020-03-02 01:04:10	stderr	Metadata:
2020-03-02 01:04:10	stderr	Output #0, rawvideo, to ‘pipe:’:
2020-03-02 01:04:10	stderr	[auto_scaler_0 @ 0x5567e03aa320] w:2304 h:1296 fmt:yuvj420p sar:1/1 → w:2304 h:1296 fmt:rgb24 sar:1/1 flags:0x4
2020-03-02 01:04:10	stderr	[swscaler @ 0x5567e042a9e0] deprecated pixel format used, make sure you did set range correctly
2020-03-02 01:04:10	stderr	[format @ 0x5567e03a77e0] auto-inserting filter ‘auto_scaler_0’ between the filter ‘Parsed_null_0’ and the filter ‘format’
2020-03-02 01:04:10	stderr	[auto_scaler_0 @ 0x5567e03aa320] w:iw h:ih flags:‘bicubic’ interl:0
2020-03-02 01:04:10	stderr	frame= 0 fps=0.0 q=0.0 size= 0kB time=-577014:32:22.77 bitrate= -0.0kbits/s speed=N/A [graph 0 input from stream 0:0 @ 0x5567e03af1e0] w:2304 h:1296 pixfmt:yuvj420p tb:1/90000 fr:20/1 sar:1/1 sws_param:flags=2
2020-03-02 01:04:09	stderr	[h264 @ 0x5567e03fbac0] Reinit context to 2304x1296, pix_fmt: yuvj420p
2020-03-02 01:04:09	stderr	Press [q] to stop, [?] for help