Higher CPU Usage KVM/QEMU on host vs virtual machine

I switched to KVM/QEMU vs docker supervised. I’ve noticed that the host machine shows 20-30% cpu usage by qemu-system-x86_64, but the virtual machine is uses less than 5%. I found online that making these changes to the config can help, but they did not help.

  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>

  <!-- after: this config drops to about 3% of a host CPU core -->
  <clock offset='utc'>
    <timer name='hpet' present='yes'/>
    <timer name='hypervclock' present='yes'/>
  </clock>

Home Assistant runs fine, but I’d like to reduce the host OS cpu. This is the only virtual machine I’m running.

What can I provide to help?

3 Likes

Got the same problem running homeassistant-ova on debian 11 in kvm/qemu/libvirt.

Any update suggestions on this?

1 Like

I see the same thing too… I’ve tried to assign 8GB of RAM, and 4 VCPUs, but Home Assistant continues to suck down RAM…

I tried the clock suggestion from @patmann03 … But it’s still so slow and so laggy…

Python…

I see similar behavior :frowning: hassio shows idle time 99% but host cpu shows 8-12% usage for this VM. I have other VMs they don’t show same CPU usage. I did disable all addons and removed passthru USB devices from host and hassio kvm cpu usage on host is contantly 7-12% (at ‘top’).

Hello,

I confirm. I have the same issue. 24% CPU usage by qemu on the Host, but close to 0 inside the VM.
Is there really such a big overhead due to virtualization?

Best regards,
Hendrik

Hey, I know that this thread is a little bit old, but you can find it through google when you look for the specific problem of running Home Assistant in Qemu/KVM and encounter higher cpu load on the host than on the guest.

I was running the system with 3 GB of RAM and typically had 30 % CPU on the virtualization host, although inside the load was way lower at some percent. I tried to change the tick settings, but that did not change anything.

After setting RAM to 3,5 GB I saw a decrease in CPU load, maybe related to context switches. Further changing to 4 GB did not change anything. This is what Telegraf gives me:

There is a visible decrease in CPU load. Also context switches and CPU load does usually does not change over the weeks, so I think this is not an effect from the system restart. Interestingly, Power consumption of the Intel J4205 stayed the same.

Hello @crbble ,

thanks for sharing.
I have increased the memory to 4GB and I see no improvement.
In the HA Webinterface I see ~10% CPU load. On the Host, I see 40%.

Do you have a link at hand, that describes how to get the graph of context switches?

Best regards,
Hendrik

Hi,

same problem with my installation of HAOS as a Qemu VM managed by libvirt. The last two devices I added were a Conbee II Zigbee USB stick and a Z-Wave USB stick along with ZWave integration, ZWave JS add-on, and Zigbee2Mqtt add-on. I added the USB sticks as usb-serial devices to the VM via

<serial type="dev">
  <source path="/dev/serial/by-id/usb-...-if00"/>
  <target type="usb-serial" port="2">
    <model name="usb-serial"/>
  </target>
  <alias name="serial2"/>
  <address type="usb" bus="0" port="5"/>
</serial>

HAOS is using constantly almost one CPU core on the host, but the guest is idling at a few percent.

I installed the VM with this command at first:

  virt-install --import --connect qemu:///system --name haos --memory 2048 --vcpus=2 \
    --disk haos_ova-10.5.qcow2,format=qcow2,bus=virtio  \
    --osinfo detect=on,require=off  \
    --network=bridge:br0,model=virtio \
    --boot uefi

Other values for --os-info did not work.

The above command configured a USB2 controller. To get a more ‘virtualization-friendly’ XHCI USB3 controller (USB emulation — QEMU documentation), I re-installed the VM with

  virt-install --import --connect qemu:///system --name haos --memory 2048 --vcpus=2 \
    --disk haos_ova-10.5.qcow2,format=qcow2,bus=virtio  \
    --controller usb3 \
    --osinfo detect=on,require=off  \
    --network=bridge:br0,model=virtio \
    --boot uefi

but that didn’t change the load issue on the host.

I researched for a while and found a comparable issue description here: KVM / qemu – high CPU load but low CPU usage – and thus goes by another day

I followed the same steps as described in the post, i.e. hooking up to the qemu-system-x86_64 process with strace -p <process ID>. That results in many lines like

ppoll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN}, {fd=6, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}, {fd=10, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=0}, {fd=21, events=POLLIN}, {fd=23, events=POLLIN}, {fd=36, events=POLLIN}, {fd=37, events=POLLIN}, {fd=43, events=POLLIN}, {fd=44, events=POLLIN}, {fd=46, events=POLLIN}, {fd=47, events=POLLIN}, {fd=48, events=POLLIN}, {fd=49, events=POLLIN}, {fd=50, events=POLLIN}, {fd=51, events=POLLIN}, {fd=52, events=POLLIN}, {fd=53, events=POLLIN}, {fd=54, events=POLLIN}, {fd=55, events=POLLIN}, {fd=56, events=POLLIN}, {fd=57, events=POLLIN}, {fd=58, events=POLLIN}, {fd=59, events=POLLIN}, {fd=60, events=POLLIN}, {fd=61, events=POLLIN}, {fd=62, events=POLLIN}, ...], 77, {tv_sec=0, tv_nsec=487819}, NULL, 8) = 0 (Timeout)

I checked the file descriptors with

ls -la /proc/<process ID>/fd/

but I couldn’t find a real connection to the usb-serial devices.

The article linked above reports a solution to the problem with aio=native, I tried

<disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none' io='native'/>

in the libvirt qemu XML configuration file for the VM, but: no difference in host load.

Try to use host-CPU instead of kvm64/qemu64.

(qemu-system-x86_64 -cpu host)

In libvirtd’s qemu configuration I have

<cpu mode='host-passthrough' check='none' migratable='on'/>

a check with ps aux resulted in:

qemu-system-x86_64 -cpu host,migratable=on

I tried

<cpu mode='host-passthrough' check='none' migratable='off'/>

ps xau resulted in:

qemu-system-x86_64 -cpu host

I tried

<cpu mode='host-model' />

which gave

qemu-system-x86_64 -cpu Cooperlake,ss=on,vmx=on ...

All had the same load.

Over time, I’ve seen a couple of threads on this subject, and it seems almost all who post see this as well so you are not alone. I too have around 20+% CPU load on the Host and just a few% on HAOS. I use Host pass-through as well.

It is a pitty… The virtualisation overhead should be smaller…

Ok, I looked into this thread because I had the same issue.
I could finally solve because I compared an old installation on an old server with my new installation and figured out, that the new defaults in QEMU when setting the whole thing up in virt-manager added a problamatic hardware to the VM.

in the xml it was looking like:

<sound model="ich9">
  <address type="pci" domain="0x0000" bus="0x00" slot="0x1b" function="0x0"/>
</sound>

In the virt-manager it was just listed as “Sound ich9”.
This is the one to be removed. Dropped from ~12% to 1-2% cpu utilization.

3 Likes

Interesting find… I tried this and it reduced my CPU load from around 20% to around 15%.

docker stats before removing “Sound ich6” device was 2-5%. Now it’s showing 0.00%. Very nice find! host PC top hassio does not standout from other VMs.

I know this post is somewhat older, but seems to come up with recent google search.
So here is what I did and worked for me.
My home assistant is installed as a VM not CT.
Under options I disabled “Use tablet for pointer”

2 Likes

I followed both suggestions from @nielssiebert & @George80 and removed the sound device and the tablet from my VM. I also removed the CD-ROM drive. The CPU usage dropped a bit and the overall power consumption of my host server is now about 2 watt less than before - now 14 watt average compared to 16 watt.

Thank you both for the tips.

Perhaps we can remove even more “devices” from the VM to make it slimmer?

Forgive my stupidity here, but is HASSos a cut down Linux distro with HASS running as a docker image ?

With a bunch of docker images running.

Ok, makes sense to me then, as the host would include the iotasks/translaction in its cpu usage display, where as the guest (and docker inside the guest) wouldn’t know that the host was having to translate instructions and then schedule them.

Tho I might have miss understood the original statement here.