Frigate FFMPEG issues on hw acceleration

Third party integrations
1

Hi Folks,

I’m struggling heavily on the ffmpeg hwaccel_args:

The Goal:
Running Frigate as responsive as it can be within HA as an Add-on.

My setup
I’m Running Proxmox 8.1.10 on a HP Elitedesk 800 G3 Micro tower with a Intel Core I5-6500T CPU
HA (latest) is running in a VM with 2vcpu (host) and 8gb of RAM within HA I created a frigate network mapping to my Synology NAS.
Frigate is installed by Blake Blackshears official documentation (adding his repo to the add-on repo list in HA.
The Firgate version I installed is the Frigate (Full) stable version and I turned off protection mode for this add-on in HA.

The Story
I replaced my rPI 4b with the Elitedesk after I discovered the rPI was heavily underpowered for running Frigate.
Now ffmpeg isnt working at all. The thing is running without any “hwaccel_args:” set, but then my CPU load raises significantly and remains constantly between 97% and 99,8% which makes my whole HA setup irresponsive.

The issue
At first ffmpeg wasnt even starting. With these “hwaccel_args:” settings:
hwaccel_args : preset-vaapi

So after digging the web I discovered that the actual best practise settings compatible with my CPU was as follows:
image
with this added to my camera config:
image
this partly solved my issue as the FFMPEG was starting at first, but now ffmpeg is stopping automatically because my cam’s stream wasn’t retrieved within 20 seconds (regarding to the logs):

image
image721×28 1.79 KB

After digging around on the web I found this thread in which other strugglers say this solved the same issue at their side by adding this to their camera config:
image
So I did this but the issue remains.

As a last resort I’m posting this to this community, I bet this thread will help other strugglers as well when solved.

Below my full config (yes Ill buy the coral TPU as soon as I got this stable without it :slight_smile: )

mqtt:
  enabled: true
  host: HAs_IP
  port: 1883
  user: mqttuser
  password: mqttpwd

ffmpeg:
  hwaccel_args:
    - -hwaccel
    - qsv
    - -qsv_device
    - /dev/dri/renderD128

detectors:
  detector_name:
    type: cpu

record:
  enabled: true
  retain:
    days: 30
    mode: motion
  events:
    retain:
      default: 45
      mode: motion

snapshots:
  enabled: true
  retain:
    default: 3

cameras:
  CAM2_2k:
    detect:
      width: 2304
      height: 1536
      #fps: 25
    ffmpeg:
      input_args: -rtsp_transport tcp
      inputs:
        - path: rtsp://Username:[email protected]:88/videoMain
          roles:
          - detect
      hwaccel_args:
        - -c:v
        - h264_qsv

I was thinking my camera might be taking more time than 20 secs to provide a stream but I stopwatched the stream using VLC and I ended up getting a stream in around 15 seconds (15.2/15.3 and 15.1 seconds) so this can’t be the cause

2

You shouldn’t be using manual ffmpeg args. I’d suggest using preset-vaapi

2 Likes
3

Hi Nicolas,

Thnx 4 your input.
Unfortunately with the preset-vaapi ffmpeg isn’t even starting at all.

I agree that it should work with the i5-6500t but it doesn’t fit some reason…
Perhaps it isn’t the hardware acceleration settings that bugs me and maybe I should search for the root cause with Vaapi enabled.

I’ll keep that in mind during my trials on errors :+1:

Edit:
Your reply made me realize, I could try frigate in an lxc container within Proxmox and see what happens…if it works with the Vaapi preset…then the issue isn’t the hardware acceleration settings as it’s the same hardware/cpu.
Then the issue should be found in one of the differences between frigate as lxc container in proxmox vs frigate as a (docker) container within home assistant.
I’ll post the results asap.

1 Like
4

then it would be good to show logs

5

Good one I’ll post them too asap

6

A few things come to mind here…

You have HA (and Frigate) running with only 2 VCPUs. Have you tried allocating more CPU cores to the VM?

Additionally, Frigate is running in a docker container in a VM. It might be better to have Frigate running in docker on a bare metal instance (like on the rPi). I kinda think that ffmpeg is trying to allocate resources and can’t because of the lack of CPUs and threads within the CPUs. It could also be something odd like how Proxmox is exposing the vCPUs to the guest OS. Hosting Frigate by itself in a container directly on the rPi might yeild better results. Yes, your CPU may be high, but it might still be better than a container within a VM. I’ve run Frigate on a rPi4b with 6 4K cameras and never maxed out the CPU on it.

Finally, I’d let go2rtc process the stream rather than Frigate itself. All my 4K cameras are processed by go2rtc and then Frigate just accesses that stream without having to worry about processing much.

7

Hi Bill,

Thanks for your input…I’ll definitely try to allocate More vcpu’s and test…
The reason why I would like to run frigate within HA, is because frigate itself doesn’t support any form of user accounts.

A tread of another user complaining about this ended in a comment of Blake Blacksheer Himself suggesting to run frigate via HA to connect to it using a user account.
The frigate app for mobile also requires to fill out a user account.
This way I ended up with frigate within HA.

I’ll post my results on adding more cpu here… if in the end I cannot manage to get this to work…ill run it via a lxc container. And have to accept the lack of user account functionality.

1 Like
8

Yeah, I get that. This is one of those situations where security trumps performance. With that said, I’d still try increasing the resources to your HA VM and also try go2rtc for your ffmpeg encoding.

However, I have a feeling you’re going to have better results with the lxc container route.

9

I have no doubts the lxc will perform better.

As I’m currently unable to troubleshoot…I’ll continue to research more things to try as soon I’m able to test things out.

I slightly get a feeling it’s the VM type in proxmox causing the issues as i got ha running in a VM type: i440fx which is a type that didn’t support pci-e passthrough. So prolly I need to migrate (again to a Q35 type vm).

Results will come soon :+1:

1 Like
10

yesterday evening I’ve found some sparetime, and I got this running now for 15hours so I’m pretty confident when I say it really works.

At first I’ve tried all your suggestions but without success. Then I took the most rigorous approach by migrating to a Q35 type VM. And added the HD Graphics 530 PCI device to this VM.

Unfortunately this wasnt enough because Proxmox was complaining about IOMMU being disabled.

After some further digging I realized that I had to add “intel_iommu=on” to my Grub file.
This is almost perfectly described in this manual. Almost because the author replaced the underscore_ with a space in it’s code-view so remember this need to remain a underscore!
And don’t forget to run update-grub after you’ve edited the grub otherwise your changes will not be taken into the running config.

After this Frigate runs fine but with a CPU load between 50% and 70%. Now it comes to tweaking and optimizing.
In fact I got currently around 30% CPU power left which is a lot more for running HA than my previous setup with an RPi 4b :slight_smile:

Thanks all for your input in the end this discussion pointed me the right direction and I hope someone else’s also helped with this.

2 Likes
11

Hmm 30% of cpu power is sill high. Mine is running in container that use 5 - 6 % of cpu. I saw somewhere in frigate doc not to use 25 fps, so I didn’t set up fps value in my frigate config. I have coral tpu and nvidia gpu.

12

Yeah same reason why I out commented the 25fps line in my config :wink: Actually I’ve read the fps should be set to 5fps depending on how fast your detections are needed, but in general 5fps should be sufficient for detecting persons etc.

I believe my cpu has a higher rate mainly because the GPU is a buildin in the i5 CPU I have, and I’m using CPU detector which significantly increases CPU usage. But the coral is highly desired by me, it will come for sure.

I got some pretty decent VGA’s lying around here, but I’m affraid of the energy consumption :slight_smile: Do you have a dedicated VGA or a CPU embedded? If dedicated, can you tell me your model and do you know how much the VGA increased the energy consumption of your setup?

13

I bought used nvidia quadro p1000 for 70 euro and that much i payed coral tpu new, online.
I have i3 second generation comp with ddr3 ram. So it is old computer. I cant say how much exactly is power consumption because I have more than one device connected to one power outlet. But in the day I have only comp on that outlet it was around 0.8 kw per day. Now is aorund 1.6 to 1.7 kw per day. But I have comp, network printer and two routers on that outlet.

14

you can use openvino, it will use far less cpu Object Detectors | Frigate

15

Thanks…I see it is supported from the 6th gen Intel CPU’s and on :+1:
So glad I decided to buy 6th gen :joy:
I’ll try that one for sure

16

I understand…I had gen 2’s and 3’s also laying around but decided to go for a newer gen. My philosophy: newer hardware provides more performance with the same amount of energy.

Thanks for your reply…taking the other devices in count I think your energy consumption did raise not as significantly as I would expected.

Bottom line I shouldn’t complain about energy consumption as I’ve shut down my Dell poweredge R620 with a dual Xeon CPU when the energy prices raised sky-high. So everything after that was a significant improvement in energy consumption :joy: