Local realtime person detection for RTSP cameras

Correction : after about 20 minutes, it did recover after the last occurrence.

It’s working really hard to process 100fps and look for motion. That is strange. I haven’t seen anything like that before. What cameras are you using?

It’s similar to the pattern I saw (here)

My cameras are set to deliver a fixed 8 fps but the stats show them as 35-50 fps from the time the ‘not enough space’ messages started until I restarted.

I am using 7 fps sub stream from 2 reolink POE wired cameras. Now the 2nd camera is skipping frames even though it’s not processing many frames and I am got several telegram notification with same image (same timestamp) of parked car. Log shows repeat of

/arrow/cpp/src/plasma/eviction_policy.cc:134: There is not enough space to create this object, so evicting 87 objects to free up 80246016 bytes. The number of bytes in use (before this eviction) is 399656192.

garage: ffmpeg didnt return a frame. something is wrong.

garage: ffmpeg process is not running. exiting capture thread…

eating ffmpeg process…

ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://user:[email protected]:554//h264Preview_01_sub -f rawvideo -pix_fmt rgb24 pipe:

/arrow/cpp/src/plasma/eviction_policy.cc:134: There is not enough space to create this object, so evicting 87 objects to free up 80246016 bytes. The number of bytes in use (before this eviction) is 399656192.

Hi,

Firstly thanks for an awesome project… kudos.

There are a couple of things I’m struggling with though.

1: After defining my region boxes, when i load up http://myip:myport/cameraname I dont actually see the boxes to see if they are in the right place, is there another url for checking this?

2: Is there more documentaion for this project other than the readme in the github repo? It would be great if there was a full list or properties and arguments for the docker compose for us newbies.

again, great project :slight_smile:

Regions are no longer needed and have been removed from the example config in Github. The Readme has an example of a docker compose entry.

So how do I make it so only certain areas of my view trigger people? for example I don’t want it triggering for people walking past on the pavement.

just stopped by to look for the same answer to this question

You have to do that with a mask. Defining areas for specific object types and alerts is coming in a future release. Alternatively, you can use 0.3.0 with regions, but you lose object tracking and a few other things.

Thanks for the reply. Could you point me to the documentation for using masks? I didn’t see any details in the readme. Probably couldn’t see the wood for the trees right enough.

Take a look at the comment in the example config and the example mask. https://github.com/blakeblackshear/frigate/tree/master/config

Nice, thanks for the quick help!

Quick question, I pulled up my rtsp stream in vlc to make sure I make the right size .bmp. Its a unifi protect cam, thats using 16:9 / 1024x576. From your example image I just need to black out what I want to ignore, but does the white part need to be transparent or does a white background work?

Just a little confused by the instructions. On one of my cameras, I want it to focus on my driveway and walkway, my driveway is black while the walkway is cement.

Thanks again.

>     ## The mask works by looking at the bottom center of the bounding box for the detected
>     ## person in the image. If that pixel in the mask is a black pixel, it ignores it as a
>     ## false positive. In my mask, the grass and driveway visible from my backdoor camera 
>     ## are white. The garage doors, sky, and trees (anywhere it would be impossible for a 
>     ## person to stand) are black.

I have 2 cameras.
The Wyze camera is my driveway camera and an 2K Amcrest camera for the frontdoor.

I was noticing the response times for my frontdoor camera was getting very slow (in the upper hundred ms range instead of the usual under 10)
I pointed frigate to the substream which is 640 and it had been working for about 22 hours. ping response times remained under 10 ms.

There have been no restarts of the frontdoor camera process but there were a few successful restarts of the driveway camera process. However it then it got into a loop of restarting with the following pattern

[h264 @ 0x562dd384fe40] error while 43992400kB time=22:17:01.80 bitrate=147456.0kbits/s speed=   1x    147456.0kbits/s speed=   1x
2020-03-19T17:47:09.209953381Z decoding MB 110 25, bytestream -5
No more output streams to write to, finishing.e=00:00:11.06 bitrate=746496.0kbits/s speed=0.15x        147456.0kbits/s speed=   1x
2020-03-19T17:47:14.331309523Z frame=  166 fps=2.3 q=-0.0 Lsize= 1008450kB time=00:00:11.06 bitrate=746496.0kbits/s speed=0.15x
2020-03-19T17:47:14.331475431Z video:1008450kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
2020-03-19T17:47:14.340192298Z driveway: ffmpeg didnt return a frame. something is wrong.
2020-03-19T17:47:14.340439010Z driveway: ffmpeg process is not running. exiting capture thread...
2020-03-19T17:47:19.059355623Z Creating ffmpeg process...

This shows how the restarts progressed.

So I created mask files where the part I wanted to exclude from detection is black and the target area is white but unfortunately still getting notifications when people are detected in the black area. My rtsp is 1920x1080 as are my BMP file. Not really sure where to go from here…

Yesterday morning I switched to 0.5.1-rc2 (from 0.5.0) and see that in the afternoon I started getting the There is not enough space to create this object errors and Frigate essentially stopped working. I don’t recall seeing this before, so going to switch back to 0.5.0 and make sure this was ‘new’ behavior.

I had 0.5.1-rc2 exit on me due to a crash inside the container (ExitCode 135 when viewed with docker inspect). It ran for 2 days before crashing.

Can’t view debug / process info since it exited the container completely (I don’t currently have docker restart the container automatically).

This was the final part of the log:

discardcorrupt -vsync drop -rtsp_transport udp -stimeout 10000000 -use_wallclock_as_timestamps 1 -i rtsp://172.16.16.192:554/onvif1 -f rawvideo -pix_fmt rgb24 pipe:
2020-03-20T11:44:50.598942668Z cat_cam: ffmpeg didnt return a frame. something is wrong.
2020-03-20T11:44:50.601420830Z cat_cam: ffmpeg process is not running. exiting capture thread...
2020-03-20T11:44:57.481011090Z Creating ffmpeg process...
2020-03-20T11:44:57.481110957Z ffmpeg -hide_banner -loglevel panic -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport udp -stimeout 10000000 -use_wallclock_as_timestamps 1 -i rtsp://<redacted>:554/onvif1 -f rawvideo -pix_fmt rgb24 pipe:
2020-03-20T11:45:00.646988792Z cat_cam: ffmpeg didnt return a frame. something is wrong.
2020-03-20T11:45:00.647058574Z cat_cam: ffmpeg didnt return a frame. something is wrong.
2020-03-20T11:45:00.649640708Z cat_cam: ffmpeg process is not running. exiting capture thread...
2020-03-20T11:45:07.509543918Z Creating ffmpeg process...
2020-03-20T11:45:07.509617084Z ffmpeg -hide_banner -loglevel panic -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport udp -stimeout 10000000 -use_wallclock_as_timestamps 1 -i rtsp://172.16.16.192:554/onvif1 -f rawvideo -pix_fmt rgb24 pipe:
2020-03-20T13:26:20.432644347Z /arrow/cpp/src/plasma/store.cc:738: Disconnecting client on fd 2113
2020-03-20T13:26:20.432689543Z /arrow/cpp/src/plasma/store.cc:738: Disconnecting client on fd 2112
2020-03-20T13:26:20.432699180Z /arrow/cpp/src/plasma/store.cc:738: Disconnecting client on fd 2111
2020-03-20T13:26:20.432707012Z /arrow/cpp/src/plasma/store.cc:738: Disconnecting client on fd 2110
2020-03-20T13:26:20.432729280Z /arrow/cpp/src/plasma/store.cc:738: Disconnecting client on fd 2109
2020-03-20T13:26:20.432805429Z /arrow/cpp/src/plasma/store.cc:738: Disconnecting client on fd 2108
2020-03-20T13:26:20.432817177Z /arrow/cpp/src/plasma/store.cc:738: Disconnecting client on fd 2107
2020-03-20T13:26:20.432844047Z /arrow/cpp/src/plasma/store.cc:738: Disconnecting client on fd 2106

Exited again :frowning:

Same exit code - 135.

2020-03-20T17:38:29.330288192Z Creating ffmpeg process...
2020-03-20T17:38:29.330307900Z ffmpeg -hide_banner -loglevel panic -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport tcp -stimeout 10000000 -use_wallclock_as_timestamps 1 -i rtsp://<redacted>:554/Streaming/Channels/2/preview -f rawvideo -pix_fmt rgb24 pipe:
2020-03-20T17:38:29.341194049Z ffprobe -v panic -show_error -show_streams -of json "rtsp://172.16.16.192:554/onvif1"
2020-03-20T17:38:29.475568053Z WARNING: Invalid RefPicListX[] entry!!! It is not included in DPB
2020-03-20T17:38:32.181313629Z {'streams': [{'index': 0, 'codec_name': 'h264', 'codec_long_name': 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10', 'profile': 'Baseline', 'codec_type': 'video', 'codec_time_base': '0/2', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'width': 1280, 'height': 720, 'coded_width': 1280, 'coded_height': 720, 'has_b_frames': 0, 'sample_aspect_ratio': '0:1', 'display_aspect_ratio': '0:1', 'pix_fmt': 'yuv420p', 'level': 10, 'chroma_location': 'left', 'field_order': 'progressive', 'refs': 1, 'is_avc': 'false', 'nal_length_size': '0', 'r_frame_rate': '359/12', 'avg_frame_rate': '0/0', 'time_base': '1/90000', 'start_pts': 9000, 'start_time': '0.100000', 'bits_per_raw_sample': '8', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}, {'index': 1, 'codec_name': 'pcm_alaw', 'codec_long_name': 'PCM A-law / G.711 A-law', 'codec_type': 'audio', 'codec_time_base': '1/8000', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'sample_fmt': 's16', 'sample_rate': '8000', 'channels': 1, 'bits_per_sample': 8, 'r_frame_rate': '0/0', 'avg_frame_rate': '0/0', 'time_base': '1/8000', 'start_pts': 0, 'start_time': '0.000000', 'bit_rate': '64000', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}]}
2020-03-20T17:38:32.181406030Z Creating ffmpeg process...
2020-03-20T17:38:32.181416195Z ffmpeg -hide_banner -loglevel panic -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport udp -stimeout 10000000 -use_wallclock_as_timestamps 1 -i rtsp://<redacted>:554/onvif1 -f rawvideo -pix_fmt rgb24 pipe:
2020-03-20T17:38:32.192435840Z Camera_process started for front: 48
2020-03-20T17:38:32.195353737Z Starting process for front: 48
2020-03-20T17:38:32.196224662Z Camera_process started for cars: 49
2020-03-20T17:38:32.198595008Z Camera_process started for cat_cam: 50
2020-03-20T17:38:32.198647983Z Starting process for cars: 49
2020-03-20T17:38:32.201254847Z Starting process for cat_cam: 50
2020-03-20T17:38:32.257994927Z  * Serving Flask app "detect_objects" (lazy loading)
2020-03-20T17:38:32.258276943Z  * Environment: production
2020-03-20T17:38:32.258407311Z    WARNING: This is a development server. Do not use it in a production deployment.
2020-03-20T17:38:32.258448492Z    Use a production WSGI server instead.
2020-03-20T17:38:32.258517781Z  * Debug mode: off
2020-03-20T17:38:35.949369286Z WARNING: Invalid RefPicListX[] entry!!! It is not included in DPB
2020-03-20T20:32:41.781249467Z Detection appears to be stuck. Restarting detection process
2020-03-20T20:32:41.873991707Z Waiting for detection process to exit gracefully...
2020-03-20T20:32:42.188380630Z /arrow/cpp/src/plasma/store.cc:738: Disconnecting client on fd 6
2020-03-20T20:32:42.266854286Z Starting detection process: 415

Will set up a restart policy again.

I redone my masks and they appear to be working. Didn’t do anything different so not sure what happened.

I am keep seeing the “here is not enough space to create this object” messages and often it stops the detection (doesn’t always recover).

I see the same thing from time to time.

@blakeblackshear. I have automation that reports when items are low… for example when the coral fps drops below a value for X minutes. Could you add in something that allows us to trigger a restart for the parent process as a fail safe?