PSA How to configure Proxmox for lower power usage

Hello all,
I want to share something I recently discovered while running Home Assistant in a Proxmox 6.4-12.
You can tune a Proxmox installation to consume less power with a simple command.

In a default Proxmox configuration the OS is tuned to “performance”. In my case it means the CPU is running 100% all the time. It’s possible to configure a different profile so the CPU is “sleeping” more. And a more idle CPU means less power usage. These CPU states are called CPU governor states.
The amount of power saving depends on:

  • your CPU model
  • your load inside Proxmox

So your mileage may vary.

Take a look as follows:

  • go to your Proxmox console
  • click on a node
  • open a shell for that node
  • in the shell, at the command prompt enter
    cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors
    to see the list of available CPU governor states
    (I have ‘powersave’ and ‘performance’)
  • enter
    echo "powersave" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
    to enable a more power efficient state
  • measure the power usage of your computer

I got the following results by switching from “performance” (default setting) to “powersave”:

  • Intel NUC i5 Boxnuc8I5Beh2, Intel® Core™ i5-8259U Processor (6M Cache, up to 3.80 GHz), Home Assistant 2021.7.3, mqtt broker, NodeRed, InfluxDB, Grafana
    power consumption went from constantly between 14-23 Watts to 8 Watts with spikes to 15 Watts.
  • HP T630 thin client, quad core AMD GX-420GI 2.2 Ghz processor, running Home Assistant 2021.7.3 only
    idle power usage went from 22 Watts average to 10 Watts average

The command to set the power state needs to be given after every Proxmox restart.
You can automate this by having a ‘@reboot’ entry in your crontab.
If you are getting emails after crontab executes, try
echo "powersave" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor >/dev/null 2>&1

In my country the Netherlands using 10 Watts for 365 days / 24 hours per day means an additional electricity bill charge of about 20 euros (based on a kWh charge of 24 eurocents per hour)

I am not noticing longer reponse times or instability since I switched over to the “powersave” profile. So I will continue using the “powersave” profile and enjoy lower CPU temperatures and a lower power bill.

Please share your experiences!

References:

29 Likes

Appears I’m seeing anywhere from 20 to 30 watt reduction in power as well as a CPU utilization decrease. This is a desktop machine (i9-9900) running PVE 7.0-10.

I’ll continue to monitor this but it appears to be quite an improvement in both power consumption as well as resource utilization.

Thanks for the tip!

In the graphs below the command was entered at approx 10:45. Top is watts read from the utility meter, bottom is CPU utilization.

1 Like

There’s also powertop, that can help with lowering the power usage.

How would you use powertop inside a Proxmox environment?

in configuration.xml you can add the current gov as a sensor in order to verify the current cov used.
There is also a switch defined which lets you toggle gov from performance to powersave.

sensor:    
  - platform: command_line
    name: CPU Governor
    command: "cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor"

switch:        
  - platform: command_line    
    switches:
      cpu_governor_performance:
        friendly_name: CPU Governor (performance)
        command_on: "echo \"performance\" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor"
        command_off: "echo \"powersave\" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor"
4 Likes

Thanks for posting this, I don’t actually run Home Assistant on my Proxmox server currently, but I’m gonna be doing a little experimentation to find a good balance on my server. “Conservative” or “On Demand” governors might be a little less impactful on performance but I’ve got some testing to do to be sure.

If @mchangsp hadn’t posted this and @gibman hadn’t posted that sensor template, I would not have ever even noticed my machine is running both it’s CPUs at peak turbo when I’m realistically using about 1-3% of my CPUs probably 99+% of the time, so thank you both!

There’s a bit of discussion of this on the proxmox forums as well, including a few different ways people get their selection to be persistent without just dropping in a crontab line, too: [TUTORIAL] Fix always high CPU frequency in proxmox host.

1 Like

There’s a Proxmox CPU Scaling Governor helper at https://tteck.github.io/Proxmox/

9 Likes

Thats a nice collection of setups there, props!

I’m not even running Proxmox but instead an intel N4500 based system (gigabyte brix).

Found a few more tweaks.

Active-State Power Management (aspm w/ the powersupersave policy)

also if you have no use of bluetooth and/or audio then disable those in the BIOS.
you can also tell the kernel to skip those by blacklisting them.

In my case I am running HASS OS and for I need ssh access on port 22222
:new:[add-on] HassOS SSH port 22222 Configurator - Home Assistant OS - Home Assistant Community (home-assistant.io)

once connected you can add the blacklist lines as such

vi /mnt/boot/config.txt

module_blacklist=snd_hda_intel
module_blacklist=snd_hda_codec_realtek
module_blacklist=snd_hda_intel
module_blacklist=snd_hda_codec
module_blacklist=bluetooth
pcie_aspm=force pcie_aspm.policy=powersupersave

hit ESC, then
:wq
then
enter key
=>saved

You can also try and mess around with ‘Aggressive Link Power Management’

  - platform: command_line    
    switches:
      alpm:
        friendly_name: Aggressive Link Power Management
        command_on: "echo SATA_ALPM_ENABLE=true | sudo tee /etc/pm/config.d/sata_alpm"
        command_off: "echo SATA_ALPM_ENABLE=false | sudo tee /etc/pm/config.d/sata_alpm"

Have added a few more CPU related sensors that could be useful.
In my case its a 2 cores setup.

sensor:  
  - platform: command_line
    name: CPU temp
    command: "cat /sys/devices/virtual/thermal/thermal_zone0/temp"
    value_template: '{{ value | multiply(0.001) | round(2) }}'
    unit_of_measurement: '°C'

  - platform: command_line
    name: CPU freq 0
    command: "cat /sys/devices/system/cpu/cpufreq/policy0/scaling_cur_freq"
    value_template: '{{ value | multiply(0.000001) | round(2) }}'
    unit_of_measurement: 'GHz'
    
  - platform: command_line
    name: CPU freq 1
    command: "cat /sys/devices/system/cpu/cpufreq/policy1/scaling_cur_freq"
    value_template: '{{ value | multiply(0.000001) | round(2) }}'
    unit_of_measurement: 'GHz'
    
  - platform: command_line
    name: CPU Governor
    command: "cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor"
                  
  
  - platform: command_line
    name: CPU min. freq
    command: "cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq"
    value_template: '{{ value | multiply(0.001) | round(2) }}'
    unit_of_measurement: 'MHz'
    
  - platform: command_line
    name: CPU max. freq
    command: "cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq"
    value_template: '{{ value | multiply(0.001) | round(2) }}'
    unit_of_measurement: 'MHz'
    
  - platform: command_line
    name: Bios version
    command: "cat /sys/class/dmi/id/bios_version"

Also tried playing around with powertop which basically just told me that it was unable to obtain the needed stats.

4 Likes

i’ve tried your sensors but if you are in a vm, the host system is a different one, how you can have it?
thanks

I use Proxmox, which is running on top of a Linux install. You can read the sensors from that host Linux instance to get the actual values of the hardware sensors.

Im interested in changing the governer from HA guest to proxmox host.
The host runs a mythtv backend as well
Id like to run the host in ‘performance’ when any tv is in use but ‘powersave’ at all other times.
I have followed the guidance above but cant see how a guest can control the host in that way, I assume i need to run the command on the host?

Im going on a power saving diet and even thought its a modern host its uses a lot more power in perforance than it does in power save, but powersave causes issues with video playback and performance doesnt.

Thanks

The ‘guest’ can control the Proxmox host if the ‘guest’ can send a command to the Proxmox host.

Suggestion: put some shell scripts on the Proxmox host first.
One script puts the Proxmox host in performance mode, the other script puts the host in power save mode
From the ‘guest’ you execute the scripts remotely at the appropriate moment
Examples how to do that:

You will probably need to use sshpass so that the ‘guest’ can send the password to the Proxmox host to authenticate the ssh session.

But that script doesn’t change it permanently. Does anyone know how to automatically set the governor at boot?

don’t recommend, but
https://www.reddit.com/r/homelab/comments/bltm26/proxmox_power_usagemanagement_still_no_cpu_scaling/emul6ek/

Found another tip to lower a VM’s CPU usage in case you don’t use a GUI in the VM: disable ‘Use pointer for tablet’.
See https://www.reddit.com/r/homelab/comments/q1m383/a_small_but_useful_tip_for_proxmox_users/

4 Likes

For my HA running on proxmox, I added this switch which updates based on the status. Ofc, you have to add your HA authorized keys to proxmox host to skip password check (guide).

switch:
    - platform: command_line
      switches:
        proxmox_turbo:
          friendly_name: Proxmox Turbo
          unique_id: proxmox_turbo_switch
          command_on: ssh -i /config/.ssh/id_rsa root@PROXMOX_HOST 'echo "performance" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor'
          command_off: ssh -i /config/.ssh/id_rsa root@PROXMOX_HOST 'echo "powersave" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor'
          command_state: ssh -i /config/.ssh/id_rsa root@PROXMOX_HOST 'cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor | grep -oc "performance"'
          icon_template: >-
            {% if states('switch.proxmox_turbo') == on %}
              mdi:speedometer
            {% else %}
              mdi:speedometer-slow
            {% endif %}
7 Likes

Very useful info! Just FYI, from what I read the “powersave” governor locks your CPU at its min frequency. A better choice for most might be the “conservative” governor, which throttles back the CPU when possible but allows clock speeds to return to normal when the CPU is loaded.

It’s also worth mentioning that you may have to set the governor for each core. My /sys/devices/system/cpu/cpufreq/ directory contained four subdirectories - policy0, policy1, policy2, policy3 - and I had to change the scaling_governor file in each. Actually, this could open some interesting possibilities about setting different governors for each core based on your different VMs’ needs.

With the performance governor, my homelab setup (AMD FX-4100 PC, PoE switch, WAP, and small router) was running at ~145W with all four cores pinned at 3.6GHz. Switching the “powersave” reduced power use to ~104W, and all four cores were pinned at 1.4GHz. With “conservative”, my power use seems to have stabilized around 106W. I’m seeing cores constantly fluctuating between 1.4GHz and 3.6GHz, with the occasional spike to ~4GHz. Odd, I didn’t think my CPU turbo boosted. Perhaps an artifact?

Clock frequencies were observed with watch -n.1 "grep \"^[c]pu MHz\" /proc/cpuinfo".

I’m wondering if further tuning is possible to get consumption down. Not sure if I can mess with governor settings within each VM? Or perhaps messing with CPU affinity?

2 Likes

How cool is this!
Thanx for sharing information!!!

Just curious:
I recently switched to proxmox on a Dell OptiPlex 7050:

  • 3.4 GHz Intel Core i5-7500 Quad-Core
  • 32GB of 2400 MHz DDR4 RAM
  • 512GB SSD NVMe PCIe
  • no spinning disks

baseline of my power consumption: 28W (thus without any VM running)

Is this OK? Some people suggest that it is rather high, but when I run

echo "powersave" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

there is no change in power consumption.

kind regards,
Bart

on my old I5 7200u and proxmox, Powersave definitely does not lock freq to the minimum. It scales as needed and expected.