Python3 high CPU Usage

bdraco · August 17, 2020, 7:10pm

If you are on python 3.8, https://github.com/home-assistant/core/pull/38821 could fix the executor being overloaded (in 0.114.1).

azogue · August 17, 2020, 7:59pm

Saw it yesterday, I think. It is one of the customizations, first with 100, last runs with the selected value of 64 from the PR. No apparent change in behaviour

bdraco · August 17, 2020, 8:12pm

Mostly in reference to pyhap:

Also to analyze the py-spys, look at the file and line numbers and if its a select()/sleep() like operation, exclude this from the analysis as you can assume its using no cpu and just blocking.

bedson · August 18, 2020, 4:45pm

As requested @bdraco I am sending py-spy records (120, 360s and dump). After two days, CPU usage jumped to 30%. Is there anything else I can do to help fix this problem? Thanks for your help

bdraco · August 18, 2020, 4:57pm

Thank you. telegram is the only thing that stands out. There is quite a bit of time in zeroconf. Let’s see if 0.114.3 solves the issue for you as it has the zeroconf fix.

bdraco · August 19, 2020, 3:02am

I realized I only posted this in https://community.home-assistant.io/t/high-cpu-usage-after-0-113.

Here is the current status all the known cpu related issues that I’m aware of:

bdraco · August 20, 2020, 1:31pm

0.114.3 has been published. If you still see cpu creep, please post a new py-spy

bedson · August 22, 2020, 12:53pm

Thank you for your help. After the update cpu usage doesn’t increase

bdraco · September 7, 2020, 4:07am

The remaining known performance concerns that I’ve been tracking have now all been addressed in 0.115dev which is scheduled for beta tomorrow.

javiges · September 18, 2020, 4:29pm

I have home assistant core 0.115.1 in docker in a nuc, with version 0.114.4 and earlier the processor usage is 6% but now it does not drop below 25% and I don’t know why, help please

bdraco · September 18, 2020, 5:28pm

Please post a py-spy using the instructions above

Also debug logging may show the issue as well.

javiges · September 19, 2020, 2:50pm

bdraco · September 19, 2020, 4:02pm

It looks like you have a template that is firing very frequently. If you turn on debug logging for homeassistant.event you should be able to see which one it is

javiges · September 19, 2020, 4:15pm

How do I have to put the log?

javiges · September 19, 2020, 4:50pm

that’s OK?

  default: error
  logs:
    homeassistant.event: debug```

bdraco · September 19, 2020, 5:56pm

# Example configuration.yaml entry
logger:
  default: warning
  logs:
    homeassistant.event: debug

Something like this should work ^^

javiges · September 21, 2020, 11:34am

I found the problem, I have sensors with templates and when removing the entity_id they always made a request for data, what I did was include the entities of each sensor in the template and thus lower the use of cpu to 6%

osiangwynedd · September 21, 2020, 2:33pm

I have HA running in a VirtualBox VM, image obtained from the web site 3 weeks ago and running completely upt-o-date versions of HA, OS and Supervisor etc. I can’t get py-spy installed. I’ve installed the Terminal/SSH add-on and logged into HA via the Terminal that it adds to the side bar. Then this:

apk add cargo
cargo install py-spy

It ran through the build process for about 10 minutes and failed with:

Compiling addr2line v8.11.0
error: could not find native static library 'unwind', perhaps an -L flag is missing?
error: aborting due to previous error
error: failed to compile 'py-spy v0.3.3' intermediate artifacts can be found at '/tmp/cargo-installatomdZrk'
Caused by:
  could not compile 'remoteprocess'

bdraco · September 21, 2020, 2:35pm

There are pre-built binaries here https://github.com/benfred/py-spy/releases in case one of them works for your OS

osiangwynedd · September 21, 2020, 3:04pm

Thank you, but it’s the HASSOS running in a VM. Do you know off-hand which of those is relevant?