Local realtime person detection for RTSP cameras

Those 2 things are unrelated. I’m not sure why adding hwaccel parameters for ffmpeg would increase CPU usage. The newer processors don’t seem to benefit as much, but I haven’t seen an increase.

Same thing happened to me on my 10th gen Intel using the recommended params with all env vars and everything

I don’t have a 10th Gen processor, so I haven’t been able to troubleshoot. I’m sure someone has figured out how to get hwaccel working in ffmpeg with the newer processors.

Had some time today to test out 0.8.0-beta1, and overall it’s super solid and the HA integration makes life way easier. I had not been saving clips on older versions, only using it for detection. Great job @blakeblackshear!

However, I noticed that when I initially start the container, CPU usage is extremely low. I see the ffmpeg processes and some python3 processes running, hovering around 5-6% CPU.

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND    
 8131 root      20   0  159160  28004  20152 S   5.3  0.2   0:00.26 ffmpeg     
 8135 root      20   0  162988  30348  20224 S   5.3  0.2   0:00.34 ffmpeg     
 8111 root      20   0  976688  65612   5456 S   5.0  0.4   0:00.19 python3    
 8112 root      20   0  976688  65628   5456 S   5.0  0.4   0:00.27 python3    
 7934 root      20   0 1184076  95436  37580 S   3.0  0.6   0:00.99 python3    

When an object (in this case, a car) moves through one of my zones (call it “street”), the car parked in my driveway then begins to be detected as well. Several python3 processes then take more CPU and even after the car on the street moves out of the frame, the parked car continues to be detected. The CPU usage increases to 15-25% for each of those python3 processes, and never drops from that point.

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND    
30687 root      20   0  845644  86448  19964 S  25.6  0.5   1:34.02 python3    
30691 root      20   0  964812  74632  14832 S  17.6  0.5   1:11.90 python3    
30689 root      20   0  964808  74948  14832 S  11.6  0.5   0:47.82 python3    
30726 root      20   0  165224  31984  20132 S   5.0  0.2   0:32.62 ffmpeg     
30695 root      20   0  976688  65284   5056 S   4.7  0.4   0:32.42 python3    
30697 root      20   0  976688  65224   5000 S   4.3  0.4   0:32.14 python3 

From past posts I remember seeing that continued detection of the parked car is expected behavior at this point. Do you have any plans to implement something to prevent non-moving objects from going through the detection pipeline? I’m guessing my CPU goes higher because these cameras are the cheap ones that I can’t reduce the framerate or resolution on (1280x720, 25fps), and I do remember you saying a while ago that Frigate is very “frame-hungry”. I did set fps to 3 in my config for those particular cameras.

Would appreciate any suggestions!

@blakeblackshear I am getting database is locked error on 0.8 beta with basic config.

* Starting nginx nginx


   ...done.


frigate.app                    ERROR   : database is locked
mqtt:
  host: 192.168.0.5

logger:
  default: info  

cameras:
  back:
    ffmpeg:
      inputs:
        #- path: rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@10.0.10.10:554/cam/realmonitor?channel=1&subtype=2
        - path: rtmp://192.168.0.130:1935/bcs/channel0_sub.bcs?channel=0&stream=1&user=user&password=pass
          roles:
            - detect
            - rtmp
    height: 720
    width: 1280
    fps: 5

As long as there are objects to track, it will keep tracking. That’s why the CPU usage stays higher. I have an issue for discontinuing detection when no motion is occurring within the bounding box, but I haven’t gotten to it yet.

1 Like

Can you tell me more about the volume you have mounted for clips storage?

I guess I’m confused on how all this fits together. I have Frigate installed on a NUC via Debian. I’m using Docker-Compose to start it. I setup a simple config file that connects to my camera and I can verify its connected by using the web server.

On my Proxmox server I install HassOS in a VM and Mosquito as the MQTT via an add-on. Now I don’t know where to go from here. How does all this mesh together? Am I supposed to add the camera to HA? Is there a tutorial anywhere that picks up from here as I’m not sure how to proceed from here. Is Frigate just for motion detection? I mean, is it not designed to feed the live video feed anywhere except with motion?

Frigate setup definitely isn’t for the faint of heart.

You need to install the Frigate Home Assistant Addon, and everything will be available in HA under Integrations. It’s way easier with 0.8.0-beta than with previous versions.

The readme gives you the details: https://github.com/blakeblackshear/frigate/tree/release-0.8.0#installing

It is particularly confusing at the moment because I just released a beta version a few days ago with several new parts and there are many new configuration options.

@blakeblackshear Just wanted to say how great Frigate is. You’ve clearly put in a ton of work and the amount of help you’re giving out in this thread is amazing. When do you sleep?

Thanks for such a great alternative to motioneye/zoneminder! I’m very happily supporting you on GitHub now.

In particular, Frigate has really reduced the number of false positives from the camera at the front of my house – to zero. And it’s actually capturing more motion than zoneminder, with object detection, was previously. I am now keeping track of the neighborhood cats that come by and bribing them with treats so that they love me (and using the camera to determine which cats like which treats best).

5 Likes

What is this thing you call sleep? Seriously though, thanks for the kind words and support.

3 Likes

I’ve already installed Frigate on my dedicated NUC via Docker. Am I supposed to install it again in the HA VM??

No. There is an integration for Homeassistant that you need to install. When you install the integration, it will prompt you for the host address of your frigate install. It does not have to be on the same machine as homeassistant, but it has to be accessible from homeassistant. The integration will add all the sensors and camera entities from frigate for you.

You CAN run frigate as an addon in HassOS, but you can also run it anywhere else.

I’m working on cutting a subscription or two that way I can free up funds to start supporting you on GitHub. This project and home assistant have had such a huge impact on everyone in my family’s lives. Thank you so much for sharing.

1 Like

may i know what score value u use for cat detection? and r u running wide angle camera? whats is the resolution?. my Hikvision 2.8 wide 1080p currently cannot detect any cat around my house. tq

I don’t have anything special set for cats other than a min_area filter on one of my zones:

cameras:
  front:
    # ...
    zones:
      front_patio:
        coordinats: 0,...
        filters:
          cat:
            min_area: 20000
    save_clips:
      enabled: true
      pre_capture: 2
      object:
        - person
        - dog
        - cat
objects:
  track:
    - person
    - dog
    - cat

The filters will be specific to your camera’s resolution and how large a cat will appear on the camera’s field of view

i’ll play with zones and min area. tq

Ok, so I’ve added the Frigate add-on to HA in the repo and installed the non-beta version. I’m not being prompted for the Frigate host address however. Where is that address input at?

You need the beta version for the HA integration