Frigate is using more than 90% CPU

Third party integrations
1

Hi

i setup frigate with 7 cams but cpu load is all time more than 90%
is this normal? or do someone knows how to get it down. i connected 3 reolink cams and 4 berghoch cam. im using a coral tpu.

Bild 1
Bild 1977Ă—548 40.7 KB

mqtt:
  host: 192.168.1.110
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: admin
  password: password
  stats_interval: 60


detectors:
  coral:
    type: edgetpu
    device: usb

database:
  path: /media/frigate/frigate.db


# Optional: logger verbosity settings
logger:
  # Optional: Default log verbosity (default: shown below)
  default: debug
  # Optional: Component specific logger overrides
  logs:
    frigate.event: debug



birdseye:
  enabled: True
  restream: False
  width: 1920
  height: 1080
  quality: 8
  mode: motion

ffmpeg:
  global_args: -hide_banner -loglevel warning -threads 2
  hwaccel_args: preset-rpi-64-h264
  input_args: [] #preset-rtsp-generic
  output_args: []


cameras:
####################################################################################
##                                     CAM1                                       ##
####################################################################################
  cam1:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam1
          roles:
            - detect
            - record
            - rtmp
    best_image_timeout: 60
    detect:
      # width: 1920
      # height: 1080
      fps: 5
    zones:
      heizraum:
        coordinates: 524,1080,391,710,337,417,506,412,803,1080
        objects:
          - person
      bierbauch:
        coordinates: 1375,929,1297,657,1920,243,1920,0,910,0,1195,934
        objects:
          - person
    objects:
      track:
        - person
        - car
        - truck
        - cat
        - dog
      filters:
        person:
          #min_area: 5000
          #max_area: 100000
          #min_ratio: 0.5
          max_ratio: 2.0
          min_score: 0.5
          threshold: 0.7
    record:
      enabled: True
      expire_interval: 60
      retain:
        days: 0
        mode: all
      events:
        pre_capture: 5
        post_capture: 5
        objects:
          - person
          - car
          - truck
          - cat
          - dog
        required_zones: []
        retain:
          default: 10
          mode: active_objects
          objects:
            person: 15

    snapshots:
      enabled: True
      clean_copy: True
      timestamp: False
      bounding_box: False
      crop: False
      height: 175
      required_zones: []
      retain:
        default: 10
        objects:
          person: 15
    motion:
      threshold: 25
      #contour_area: 30
      #delta_alpha: 0.2
      #frame_alpha: 0.2
      #frame_height: 50
      #mask: 0,900,1080,900,1080,1920,0,1920
      improve_contrast: False
      #mqtt_off_delay: 30

####################################################################################
##                                     CAM2                                       ##
####################################################################################
  cam2:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam2
          roles:
            - detect
            - record
            - rtmp
    best_image_timeout: 60
    detect:
      # width: 1920
      # height: 1080
      fps: 5
    # zones:
    objects:
      track:
        - person
        - car
        - truck
        - cat
        - dog
      filters:
        person:
          #min_area: 5000
          #max_area: 100000
          #min_ratio: 0.5
          max_ratio: 2.0
          min_score: 0.5
          threshold: 0.7
    record:
      enabled: True
      expire_interval: 60
      retain:
        days: 0
        mode: all
      events:
        pre_capture: 5
        post_capture: 5
        objects:
          - person
          - car
          - truck
          - cat
          - dog
        required_zones: []
        retain:
          default: 10
          mode: active_objects
          objects:
            person: 15

    snapshots:
      enabled: True
      clean_copy: True
      timestamp: False
      bounding_box: False
      crop: False
      height: 175
      required_zones: []
      retain:
        default: 10
        objects:
          person: 15
    motion:
      threshold: 25
      #contour_area: 30
      #delta_alpha: 0.2
      #frame_alpha: 0.2
      #frame_height: 50
      #mask: 0,900,1080,900,1080,1920,0,1920
      improve_contrast: False
      #mqtt_off_delay: 30
####################################################################################
##                                     CAM3                                       ##
####################################################################################
  cam3:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam3
          roles:
            - detect
            - record
            - rtmp
    best_image_timeout: 60
    detect:
      # width: 1920
      # height: 1080
      fps: 5
    # zones:
    objects:
      track:
        - person
        - car
        - truck
        - cat
        - dog
      filters:
        person:
          #min_area: 5000
          #max_area: 100000
          #min_ratio: 0.5
          max_ratio: 2.0
          min_score: 0.5
          threshold: 0.7
    record:
      enabled: True
      expire_interval: 60
      retain:
        days: 0
        mode: all
      events:
        pre_capture: 5
        post_capture: 5
        objects:
          - person
          - car
          - truck
          - cat
          - dog
        required_zones: []
        retain:
          default: 10
          mode: active_objects
          objects:
            person: 15

    snapshots:
      enabled: True
      clean_copy: True
      timestamp: False
      bounding_box: False
      crop: False
      height: 175
      required_zones: []
      retain:
        default: 10
        objects:
          person: 15
    motion:
      threshold: 25
      #contour_area: 30
      #delta_alpha: 0.2
      #frame_alpha: 0.2
      #frame_height: 50
      #mask: 0,900,1080,900,1080,1920,0,1920
      improve_contrast: False
      #mqtt_off_delay: 30
####################################################################################
##                                     CAM4                                       ##
####################################################################################
  cam4:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam4
          roles:
            - detect
            - record
            - rtmp
    best_image_timeout: 60
    detect:
      # width: 1920
      # height: 1080
      fps: 5
    # zones:
    objects:
      track:
        - person
        - car
        - truck
        - cat
        - dog
      filters:
        person:
          #min_area: 5000
          #max_area: 100000
          #min_ratio: 0.5
          max_ratio: 2.0
          min_score: 0.5
          threshold: 0.7
    record:
      enabled: True
      expire_interval: 60
      retain:
        days: 0
        mode: all
      events:
        pre_capture: 5
        post_capture: 5
        objects:
          - person
          - car
          - truck
          - cat
          - dog
        required_zones: []
        retain:
          default: 10
          mode: active_objects
          objects:
            person: 15

    snapshots:
      enabled: True
      clean_copy: True
      timestamp: False
      bounding_box: False
      crop: False
      height: 175
      required_zones: []
      retain:
        default: 10
        objects:
          person: 15
    motion:
      threshold: 25
      #contour_area: 30
      #delta_alpha: 0.2
      #frame_alpha: 0.2
      #frame_height: 50
      #mask: 0,900,1080,900,1080,1920,0,1920
      improve_contrast: False
      #mqtt_off_delay: 30
####################################################################################
##                                     CAM5                                       ##
####################################################################################
  cam5:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam5
          roles:
            - detect
            - record
            - rtmp
    best_image_timeout: 60
    detect:
      width: 1920
      height: 1080
      fps: 5
    # zones:
    objects:
      track:
        - person
        - car
        - truck
        - cat
        - dog
      filters:
        person:
          #min_area: 5000
          #max_area: 100000
          #min_ratio: 0.5
          max_ratio: 2.0
          min_score: 0.5
          threshold: 0.7
    record:
      enabled: True
      expire_interval: 60
      retain:
        days: 0
        mode: all
      events:
        pre_capture: 5
        post_capture: 5
        objects:
          - person
          - car
          - truck
          - cat
          - dog
        required_zones: []
        retain:
          default: 10
          mode: active_objects
          objects:
            person: 15

    snapshots:
      enabled: True
      clean_copy: True
      timestamp: False
      bounding_box: False
      crop: False
      height: 175
      required_zones: []
      retain:
        default: 10
        objects:
          person: 15
    motion:
      threshold: 25
      #contour_area: 30
      #delta_alpha: 0.2
      #frame_alpha: 0.2
      #frame_height: 50
      #mask: 0,900,1080,900,1080,1920,0,1920
      improve_contrast: False
      #mqtt_off_delay: 30
####################################################################################
##                                     CAM6                                       ##
####################################################################################
  cam6:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam6
          roles:
            - detect
            - record
            - rtmp
    best_image_timeout: 60
    detect:
      width: 1920
      height: 1080
      fps: 5
    # zones:
    objects:
      track:
        - person
        - car
        - truck
        - cat
        - dog
      filters:
        person:
          #min_area: 5000
          #max_area: 100000
          #min_ratio: 0.5
          max_ratio: 2.0
          min_score: 0.5
          threshold: 0.7
    record:
      enabled: True
      expire_interval: 60
      retain:
        days: 0
        mode: all
      events:
        pre_capture: 5
        post_capture: 5
        objects:
          - person
          - car
          - truck
          - cat
          - dog
        required_zones: []
        retain:
          default: 10
          mode: active_objects
          objects:
            person: 15

    snapshots:
      enabled: True
      clean_copy: True
      timestamp: False
      bounding_box: False
      crop: False
      height: 175
      required_zones: []
      retain:
        default: 10
        objects:
          person: 15
    motion:
      threshold: 25
      #contour_area: 30
      #delta_alpha: 0.2
      #frame_alpha: 0.2
      #frame_height: 50
      #mask: 0,900,1080,900,1080,1920,0,1920
      improve_contrast: False
      #mqtt_off_delay: 30
####################################################################################
##                                     CAM7                                       ##
####################################################################################
  cam7:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam7
          roles:
            - detect
            - record
            - rtmp
    best_image_timeout: 60
    detect:
      width: 1920
      height: 1080
      fps: 5
    # zones:
    objects:
      track:
        - person
        - car
        - truck
        - cat
        - dog
      filters:
        person:
          #min_area: 5000
          #max_area: 100000
          #min_ratio: 0.5
          max_ratio: 2.0
          min_score: 0.5
          threshold: 0.7
    record:
      enabled: True
      expire_interval: 60
      retain:
        days: 0
        mode: all
      events:
        pre_capture: 5
        post_capture: 5
        objects:
          - person
          - car
          - truck
          - cat
          - dog
        required_zones: []
        retain:
          default: 10
          mode: active_objects
          objects:
            person: 15

    snapshots:
      enabled: True
      clean_copy: True
      timestamp: False
      bounding_box: False
      crop: False
      height: 175
      required_zones: []
      retain:
        default: 10
        objects:
          person: 15
    motion:
      threshold: 25
      #contour_area: 30
      #delta_alpha: 0.2
      #frame_alpha: 0.2
      #frame_height: 50
      #mask: 0,900,1080,900,1080,1920,0,1920
      improve_contrast: False
      #mqtt_off_delay: 30

go2rtc:
  streams:
    cam1:
      - rtsp://admin:[email protected]:554/channel=1&stream=1.sdp?
    cam2:
      - rtsp://admin:[email protected]:554/channel=1&stream=1.sdp?
    cam3:
      - rtsp://admin:[email protected]:554/channel=1&stream=1.sdp?
    cam4:
      - rtsp://admin:[email protected]:554/channel=1&stream=1.sdp?
    cam5:
      - rtsp://admin:[email protected]:554/h264Preview_01_main
    cam6:
      - rtsp://admin:[email protected]:554/h264Preview_01_main
    cam7:
      - rtsp://admin:[email protected]:554/h264Preview_01_main
2

Yes, it is a hungry beast.
What are you running it on?

3

This is main reason I switched to docker. You can’t do basically nothing with your installation type.
If you use docker than you will be able to limit frigate cpu usage, make it use more gpu or find other settings that suit your needs.
The only thing that you might try is using high and low streams. It depends on your cameras, google it, but you can use high and low definition streams and that can, maybe help, with your cpu usage.

4

thanks

i tried it know with the reolink sub stream. it seem to make it a bit better. but far away from good.

i m running it on an raspberry pi with SSD

5

I think you should forget pi and go with something stronger like pc. I have i7 but unfortunately 1st gen that doesn’t support avx cpu instructions. But as it seams avx is needed for frigate and especially if you plan to use some ai software like codeproject.ai.
But there is a good news.
You can buy motherboards kits with cpu, cooler and ddr3 ram from Chinese for approximately 100 €
So I will suggest find some older comp. buy a motherboard with cpu, memory and cooler that supports avx and you will probably get things going for as little money as possible.

6

Have you actually gone to the system tab inside the frigate webui? What does it say here:

image
image937Ă—526 15.6 KB

Because mine barely touches the CPU as long as the Coral is properly working.

1 Like
7

Mine also barley touching anything but lack of avx cpu instructions got my server to crash. Not by using frigate, but using ai.

Screenshot from 2024-01-02 20-41-21
Screenshot from 2024-01-02 20-41-21680Ă—720 22.8 KB

8

You need a gaming PC to run that.
You could probably have a cluster 10 pi5’s. and it would still peg the CPU…

9

The Pi5 has ArmV8.2 Mat/Mul vector instructions and for ML is approx x6 faster than a Pi4, but why people run detection at high FPS is curious as being told you have detected a human, dog or whatever 30 times in a second makes no difference to knowing once over even longer time frames.
The OrangePi5 Rk3588s is approx same price but near 2x as efficient and the hardware video decode where I have seen x16 1080p concurrent is much stronger than the RPI 5.
Even gaming GPU’s have video decode limits, but for Gflops/watt the RK3588s is in another league beating current Apple Arm Silicon and would not need so many but clustering smaller efficient SBC negates decode limits and extends detect FPS.
With Yolov8 running on a single core @ 320 image size yolov8n_full_integer_quant.tflite
Gives
Speed: 6.7ms preprocess, 33.4ms inference, 6.7ms postprocess per image at shape (1, 3, 320, 320)
Which is approx 21 FPS and on x4 cores 84FPS and there really is no reason for a plethora of detections for the same object, so that FPS is pretty scalable, especially as you add more to a cluster.

2 Likes
10

i just checked the system panel.
ffmpeg is using a lot of cpu, round 15-18% and i m using 7 cameras

Bildschirmfoto 2024-01-03 um 10.45.36
Bildschirmfoto 2024-01-03 um 10.45.361916Ă—840 50.9 KB

11

Go for a Rockchip device or a PC

+1 for stuartiannaylor#s comments regarding architecture and fps

1 Like
12

It would look like you are not using hardware video decode and using the CPU.
You will have to google / check the docs / blakeblackshear/frigate · Discussions · GitHub and get the correct ffmpeg setting.

PS:
Frigate is good but the fixed detectors are quite constraining in use, especially with fixed params.
I have never really found the perfect NVR because the target, location and type of cam.
Say with face ID verification the detector process could be something like Blazeface to get optimal frame of a face boundingbox to send to one of the models in GitHub - serengil/deepface: A Lightweight Face Recognition and Facial Attribute Analysis (Age, Gender, Emotion and Race) Library for Python.
A nature cam might use a standard centroid motion tracker and counter on a smaller subframe and cut the optimal viewport from the larger main stream to get the best input picture.
Vehicle & licence plate recognition… Same again as very likely totally different models and params

GitHub - AlexxIT/go2rtc: Ultimate camera streaming application with support RTSP, RTMP, HTTP-FLV, WebRTC, MSE, HLS, MP4, MJPEG, HomeKit, FFmpeg, etc. is an amazing camera streaming application that once inplace it will proxy a cam so you can pull multiple streams whilst only ever loading the source cam for a single instance of its streams.
The rest of Frigate I am not too sure about, because of its ridgid nature as with frameworks such as Export - Ultralytics YOLOv8 Docs there are a lot of really flexable options that fixed models exclude.

Apols for going off at a bit of a tangent, but does anyone have a Pi5 and one the new Hat AI! for Raspberry Pi 5 – Pineberry Pi so you can use M.2 Accelerator A+E key | Coral as would be really interested if the ultralytics Yolov8 model exported to TF Edge TPU format manages in FPS and likely would be interesting for the community.

1 Like
13

The first thing you need to do is stop detecting on the full resolution

The online manual clearly recommends using a substream with small resolution for detection.

You can use the full resolution for recording.

Example. My Ubiquiti cameras are 1920x1080 and set to 15 fps. And they have two substreams. The smallest is 640x360 and runs at 5 fps

What difference does that makes to the detector? Well it is the number of pixels per second that counts. That is whay loads the CPU. 1920x1080x15 = 31104000 pixels per second. And the substream 640x360x5 = 1152000 pixels per second. The ratio is 27 times. The CPU has to work 27 times harder on the full stream at full speed.

But the detection does not really get any better unless you need to detect people very very far away from the camera. You need to use substream directly from the camera. Do not use ffmpeg to downscale. That also cost CPU. Use the camera low resolution substream.

Here is a snip from my config

  camera4:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/camera4_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://192.168.1.56:554/s0
          roles:
            - record
    detect:
      enabled: True

Note that the detect uses the go2rtc local substream which again is defined as

go2rtc:
  streams:
    camera4_sub:
      - rtsp://192.168.1.56:554/s1

The record role fetches directly from the camera as it only runs when I detect something.

I did not set the detection size as the substream is already a reasonable size so it does not need to resize it before detection.

You say you tried but I think you did not try the right way because it has to make a huge difference. Most of the CPU power is the detection.

1 Like
14

You might want to try disabling Birdseye unless you really need it as that tends to be very ffmpeg/CPU intensive. That and the suggestion from the other person to check your ffmpeg configuration to make it use the GPU are likely to make the biggest difference. I realized i had a similar issue after getting a coral.

15

that’s not accurate. Many users run frigate on intel celeron n100 with 8-10 cameras without issues and plenty of CPU headroom

others run on older J4125 celeron with 4-6 cameras and have no issues as well. It just needs to be properly setup according to the docs

Frigate has a recommended hardware section in the docs Recommended hardware | Frigate

16

I run Frigate on an N200 (marginally faster than N100). I have 6 cameras. Same server also runs my website (Perl based webapp Foswiki). I have plenty of headroom. I use a USB Coral TPU and the detection on the low resolution substreams as I describes a few post above. I have enabled the birdsview for my 6 cameras. You do not need a gamer PC. You need the TPU and to use low res substreams for detection and you should be OK.
You can still record in full resolution. See above

1 Like
17

I bought an N100 miniPC off Amazon (a Kamrui AK2 Plus, N100, 16gb, 1tb ssd). Looking on the frigate website it mentioned Beelink PCs which I saw at the time but the Kamrui was much cheaper with a $100 off coupon when I bought it.

I assume it refers more to the chip (N100) rather than the Beelink device specifically?

I just set this up running HASSIO on the PC with 3 Waze cam V3 (I’ve done the old school rtsp on SD card method to pull the streams).

I was the same, it showed 97% CPU usage and my HA became painfully slow. The streams were choppy as (it’s windy out so I can easily see movement). When I stopped frigate and looked at the streams on the Generic Cam add on, they were smooth agin.

My config

mqtt:
host: xxxxx-emqx
user: -----
password: -----

detectors:
cpu1:
type: cpu
cpu2:
type: cpu

cameras:
driveway:
ffmpeg:
inputs:
- path: rtsp://------

Should I be trying to get my hands on a coral TPU? Its so expensive…

18

you are using CPU for detection when you should be using openvino Object Detectors | Frigate

19

Thanks very much, I’ve made that change now.
The CPU use in the add on page (add on cpu usage) has gone down but should I be looking at this or the CPU usage in system in the actual frigate page?

Regarding the hardware I’ve got, will this be sufficient? The ultimate goal is to have all 4 wyze cams running (3 x v3 and 1xv2) with the V3 recording to an NVR and the V2 recording low res detection (it sits in a garden shed)

I wondered whether the wifi (Deco M9, 2 out of 3 wired ethernet) was the issue but the streams on the Wyze app are pretty smooth and likewise the RTSP stream in VLC is very smooth.

20

I’ve set up my Frigate to use the Edge TPU but I am still experiencing slow speeds with my HA especially getting dashboard to load. I’m seeing 70% CPU Usage, which is definitely from Frigate as I go back to 2% when I stop the Frigate Addon. I’m pulling the Cameras in HA from a Swann DVR using this RTSP Method - Swann DVR CCTV systems and HA

I can see that the method I then use to get my cameras into Frigate “FFMPEG” is what is chewing up my CPU Usage. Is there a better way to do this? I’ve included my frigate code and some screenshots.

For refence, I’m using a HP t520 ThinClient which has a AMD GX-212JC (dual core) 1.2GHz Integrated CPU. HA reports it as running as 1.4GHz.

image
image1713Ă—702 47.1 KB

image
image590Ă—767 65.3 KB

image
image743Ă—777 44.1 KB

 mqtt:
  enabled: False

detectors:
  coral:
    type: edgetpu
    device: pci
cameras:
  dummy_camera: # <--- this will be changed to your actual camera later
    enabled: False
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:554/rtsp
          roles:
            - detect
  Front_Door:
    enabled: True
    detect:
     width: 1280
     height: 720
     fps: 5
    ffmpeg:
      inputs:
        - path: rtsp://xxxx:[email protected]:554/ch01/0
          roles:
            - detect ```