Local realtime person detection for RTSP cameras

wyze beta firmware

when it hangs in frigate, the stream still works in vlc player

I get pretty regular ffmpeg restarts since 3.x due to no frame received. Its been a while since I looked at it but assumed it was my wyze and the dafang firmware.

i would like to use a raspberry pi 3+ with coral usb, can i follow your steps? do you use coral?

@surge919 try running the blakeblackshear/frigate:dev image. I added a 10s delay before attempting to reconnect. Letā€™s see if that works.

doneā€¦ Iā€™ll let you know how it goes.
thanks

I docker exec into the container and killed the processā€¦ it restarted after 10 seconds but eventually had the issue again and started itā€™s loop of failures and restarts.

Iā€™ll leave it alone and see if it dies naturally.

Strange. Technically there shouldnā€™t be any difference between the way frigate restarts the ffmpeg process and the way restarting the container would restart the process. It just takes longer. I didnā€™t see any zombie processes in your ps output, so I am not sure what else to try other than adding an even longer delay.

@blakeblackshear do the regions still work with the latest 0.5.0 release of Frigate? I donā€™t want it to scan all of the image, I only want to scan parts of the image. Currently I have this as my config for 0.3.0:

cameras:
 cam1:
   ffmpeg:
     input: rtsp://camera.ip

   take_frame: 1

   objects:
     person:
       min_area: 5000
       max_area: 100000
       threshold: 0.5

   regions:
     - size: 500
       x_offset: 750
       y_offset: 20
       objects:
         person:
           min_area: 2000

     - size: 500
       x_offset: 750
       y_offset: 530
       objects:
         person:
           min_area: 2000

What should I change in order to get it working with 0.5.0, while roughly keeping the areas the same?

It died on itā€™s own.

what is restarting the process?
Iā€™d like it to not restart so I can go into the container and try different timeout values and possibly some other ffmpeg args.

2020-03-04T23:41:43.181310365Z driveway: ffmpeg_process is still running but didnt return any bytes
2020-03-04T23:41:43.181454781Z driveway: ffmpeg_process is still running but didnt return any bytes
2020-03-04T23:41:43.181618330Z driveway: ffmpeg_process is still running but didnt return any bytes
2020-03-04T23:41:43.181752984Z driveway: ffmpeg_process is still running but didnt return any bytes
2020-03-04T23:41:43.181916848Z driveway: ffmpeg_process is still running but didnt return any bytes
2020-03-04T23:41:43.182031243Z driveway: ffmpeg_process exited unexpectedly with 0
2020-03-04T23:41:43.182163480Z Letting driveway rest for 10 seconds before reconnectingā€¦
2020-03-04T23:41:53.168781553Z Terminating the existing ffmpeg processā€¦
2020-03-04T23:41:53.169488304Z Waiting for ffmpeg to exit gracefullyā€¦
2020-03-04T23:41:53.169853647Z Creating ffmpeg processā€¦
2020-03-04T23:41:53.170069058Z ffmpeg -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://username:[email protected]/live -f rawvideo -pix_fmt rgb24 pipe:
2020-03-04T23:43:08.554563645Z driveway: ffmpeg_process exited unexpectedly with 0
2020-03-04T23:43:08.555301805Z Letting driveway rest for 10 seconds before reconnectingā€¦
2020-03-04T23:43:18.563785362Z Terminating the existing ffmpeg processā€¦
2020-03-04T23:43:18.564535656Z Waiting for ffmpeg to exit gracefullyā€¦
2020-03-04T23:43:18.564751182Z Creating ffmpeg processā€¦
2020-03-04T23:43:18.564856569Z ffmpeg -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://username:[email protected]/live -f rawvideo -pix_fmt rgb24 pipe:

You need to use a mask to prevent it from looking everywhere now. You can remove regions in your config. Check the breaking changes on the release and look at the updated example config.

Frigate starts and manages the ffmpeg process itself. There isnā€™t a way to prevent it from restarting the process, as frigate will just exit if there is nothing to process. The best way to play with ffmpeg args is with the config file. You can also run the same ffmpeg command from the logs from within the container. Just pipe all the output to /dev/null.

ffmpeg must be exiting because it either loses connection to your camera or encounters an error. Frigate sees that the ffmpeg process stopped and tries to restart it in order to start processing frames again. In your case, the newly started ffmpeg process is also exiting before sending any frames. Frigate is just going to try and keep restarting it.

Can you make the restart time a variable I can pass through the config file so I can experiment with longer wait times?

Just to share what Iā€™m seeing in the logs with my wyze+dafang hacks on rc7

https://pastebin.com/RjF0T7La

I added a restart_delay parameter that should be set under the ffmpeg config section just under input. Just set it to the # of seconds you want to wait between restarts. Just pull down the latest dev image.

Another thing you could try is commenting out the driveway camera from your frigate config and running this command within the container manually:

ffmpeg -loglevel info -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://username:[email protected]/live -f rawvideo -pix_fmt rgb24 pipe: > /dev/null

That should simulate exactly what frigate is doing and you should be able to see why ffmpeg is exiting.

I pulled the latest blakeblackshear/frigate:dev but it didnā€™t seem to pull down anything newer.

Is this how I should add the delay?

cameras:
  driveway:
    ffmpeg:
      input: rtsp://username:[email protected]/live
      restart_delay: 25
      height: 1920
      width: 1080

logs are still showing 10 seconds

2020-03-05T02:54:27.374722572Z Letting driveway rest for 10 seconds before reconnecting...
2020-03-05T02:54:37.383850982Z Terminating the existing ffmpeg process...

You are specifying it in the config correctly. I see the image updated in docker hub. Maybe try again.

Also, your width and height need to be indented up one level to be inline with ffmpeg, and shouldnā€™t the width and height values be switched?

I was deleting the container and not the image. I deleted both and now have the correct dev version and the delay is changing as expected.
I didnā€™t even notice the width / height entriesā€¦ I simply went with your example in the doc. Itā€™s fixed now.

I changed the transport to UDP and the stream came back after I killed the ffmpeg process but the image was very jumpy. The fps detection fps was very erraticā€¦ probably due to the jumpy image.
This was evident with both cameras.

Iā€™ll keep digging but it seems like restarting the container is cutting the TCP connection which is why it starts working.

Shouldnā€™t :

      height: 1920
      width: 1080

be:

      height: 1080
      width: 1920
1 Like

Iā€™m not sure why I didnā€™t try this before, but I killed the ffmpeg process of the other ā€˜frontdoorā€™ camera and it reconnects fine.
So this may have something to do with Wyzeā€™s implementation of rtsp.

I think the udp option is the way to go if we can figure out the constant distortion of the stream over UDP. The stream shows up fine in VLC player so hopefully itā€™s a matter of finding the correct ffmpeg combination.

I mage a ā€˜frigateā€™ grafana dashboard using the influxdb-persisted metrics via home-assistant and am happy to share the dashboard json for anyone interested in doing the same:

2 Likes