Local realtime person detection for RTSP cameras

Share your Projects!
1698

Hi,

I was hoping someone could help me install this great add-on. I’m getting stuck installing frigate on a rPi4 running Hassio with a Coral USB.

I installed Hassio and the portainer add-on. Then I added a stack with the docker compose example:


frigate:
  container_name: frigate
  restart: unless-stopped
  privileged: true
  shm_size: '1g' # should work for 5-7 cameras
  image: blakeblackshear/frigate:stable
  volumes:
    - /dev/bus/usb:/dev/bus/usb
    - /etc/localtime:/etc/localtime:ro
    - /mnt/data/supervisor/homeassistant:/config
  ports:
    - "5000:5000"
  environment:
    FRIGATE_RTSP_PASSWORD: "password"

image
image2048×1536 228 KB

The container seems to ‘start’

image
image2048×1536 205 KB

But this is the log:

image
image2048×1536 309 KB

I read a comment earlier that the container needs to be in host not bridge, but I don’t understand how to change that or if my setting is incorrect. I am new to Docker and Portainer. Any help would be appreciated.

1699

At the moment, the container does not run for RPi4. There is a modified version that works here. I am planning to add support as soon as the tensorflow lite runtime supports python 3.8 and I can use the new shared memory features of python.

1 Like
1700

Hi again,

I managed to get Frigate running, thanks a lot for your help. And thanks Blake for developing it, is really great.

I have two questions…

Has anyone managed to get the Coral working properly in a Virtualbox? Would changing to another vm software make a difference?

I am having trouble with the automatomation, could someone share their configurtion? Are there any dependencies, feels like I am missing something…

Thanks!

1701

@Carlos_Lara

What issues are you having? I can get it to run for a while but it is unstable and ends up freezing my ubuntu VM requiring a reboot of the VM after a short time.

I am running a Win10 host, with an Ubuntu VM using VirtualBox. I would really like it to work, because I don’t want to have to setup a separate linux machine just for this, but I am looking at an Atomic Pi to run it separately if I can’t get the VM thing working properly.

1702

Hi Ryan,

I have exactly the same configuration and issues. When frigate uses the coral, the inference time sinks dramatically but is not stable and freezes after a while.

How are you running frigate? I am using the add-on… I could imagine ditching windows and installing ubuntu but would like to keep HA in a vm, it is simply very convenient. And being a windows guy would mean that there is another variable to juggle if something goes wrong.

1703

@Carlos_Lara
Ok, I am glad someone else is having the same issues as me, so its not just me and my config. I have the portainer addon which I used to start the docker file. I also don’t want to ditch windows as I have Blue Iris for video recording running in Windows.

I been also wondering if the mini PCIe version of coral would have different results, its also much cheaper at $35.

I wouldn’t have any problems changing to a different VM software if that would make a difference. I wonder if anyone has had any luck with something other than VirtualBox?

1704

So far the detection has been amazing with one exception. I have a shed that is pretty far away… around 20-30M. And I just can’t get detection to work well at that range even during the day, let alone night when it switches to B/W.

Is there anyway to crop the high resolution rtsp stream to only use say the top left corner of an image (where the shed is) before it is shrunk down for detection? I am wondering if this would help with detection for this area.

1705

It already does that. It doesn’t run detection on the entire frame. It uses motion to select a custom area to look for objects and then resizes just that area to 300x300. Frigate may run object detection many times on the same frame as it tries to analyze areas of motion and zoom in on objects it detects.

1706

Hello,

It’s such a great project and thank you so much. It’s working fine with below setup running in non VM. Tried to enable passthrough to a Ubuntu 19.40 VM for the Coral accelerator and it shows in the system but frigate can’t pick it up. Without the coral, it’s working fine with CPU detection. Any idea on how to get it working with ESXI VM? thanks in advance.

CPU: Intel E3-1260L
using Google Coral

1707

It should be possible right away but it’s not so probably the easiest way is to use an USB3.0 expansion PCIe card and enable passthrought for it.

And install “driver” from Coral website.

1708

more info below, looks like was able to get it working in VM, the message in log file "
No EdgeTPU detected. Falling back to CPU." not showing(but not stable, appear again after reboot). Below is the debug info from web

{"backyard_01":{"camera_fps":16.0,"detection_fps":0.9,"ffmpeg_pid":35,"frame_info":{"detect":1595211828.242963,"process":1595211827.010773,"read":1595211830.256317},"pid":53,"process_fps":0.9,"read_start":0.0,"skipped_fps":15.2},"coral":{"detection_start":1595211829.778067,"fps":1.8,"inference_speed":567.8,"pid":27},"frontdoor_01":{"camera_fps":16.0,"detection_fps":0.9,"ffmpeg_pid":51,"frame_info":{"detect":1595211827.601838,"process":1595211824.083892,"read":1595211830.208835},"pid":54,"process_fps":0.3,"read_start":0.0,"skipped_fps":15.7},"plasma_store_rc":null}
1709

So I have been able to get daytime to work good now. Seems I had the noise filter off. So at that distance there was too much digital noise and it would not detect.

However nighttime I get no detection’s at that distance. Here is an example, recorded from the mjpeg stream from frigate. (so quality isn’t the best when using this stream for viewing) I am able to walk down the side of the yard without issue. Will only detect if I walk into the middle of the frame.

https://youtu.be/kR6HOu7Rdxc (starts at :59)

So perhaps similar to this https://github.com/blakeblackshear/frigate/issues/67
But the reverse. With distance no detections.

1710

Do you have a min size defined? That could do it. What’s the resolution of your camera?

1711

The stream it is looking at is 1280x720p

          min_area: 500
          max_area: 100000
          threshold: 0.2

I shouldn’t say it never see’s me. I will get the odd hit. But it is rare. Here is one capture…

unnamed
unnamed1280×720 45.8 KB

1712

Night is usually more difficult. If you bought some external IR illuminators it would help, I do pretty well at detecting with mine, it brings up the contrast a lot so that it can make out objects better.

It’s strange that it took the upper left hand corner, you don’t have fixed detection boxes defined do you? They are no longer needed and would make problems for this case over the way it’s done now.

1713

Nope, no fixed boxes. I have only installed the version from a week or two ago. So just getting started. It does detect throughout the entire frame. Just the left hand side is the worst. And of course the most important area. :slight_smile:

Didn’t really want to put any extra hardware on the house. Originally I have always used outdoor PIR’s. But they tend to look ugly with so many things mounted on walls. This is a new house and was hopeing to have a single camera and be done. Nice and clean. Might have to look at PIR’s or external IR.

1714

It would be great if you could get me a complete mp4 file of a missed positive. I need about 30 seconds worth of video before the event to allow motion detection to settle. You can use ffmpeg to copy the video feed into an mp4 without decoding/encoding. Missed positives like this are difficult to capture.

1715

I can do that. I am actually recording the full stream 24/7. So can send you the 15 minutes around this example. There is actually two walkthroughs. So you can see both.

1 Like
1716

I use PCIe coral and works perfect.

1717

@Carlos_Lara
Just to follow up with this. I ended up migrating my sever to Unraid. So now I run Windows in a VM, HASSOS in a VM and Frigate in a docker container, so far everything seems to be working well.