Python3 high CPU Usage

Hi, recently…i believe with the 103 update i have an ever increasing cou load on my homeassistant container. Checking with Glances, it turns out to be the pyhton3 process. When i restart, it lies within normal 3-9% but over the course of 12h it goes up to more than 100%. This is on a Intel NUC.
It seems not to be any of the usual culprits like mqtt or mariadb.
Is there a way to easily check what is using python3 so much?

2 Likes

So what i did so far:

Turn off ALL Custom Components :white_check_mark:
Disable almost all integrations: :white_check_mark:
Disable ZWave: :white_check_mark:

None of that helped in isolating the problem or identifying it.

Some help in the Discord Chat pointed me to analyze the running python 3 process. I used a tool called Py-Spy for that. https://github.com/benfred/py-spy This allows you to profile a running process and even works with Python 3.6 and above.

Now I have a sample output from a fresh restart, and a sample output from several hours later when my process monitor actually shows the high CPU Usage.

However, I still don’t understand what processes are actually calling the different functions in python. Thought I’m sharing the results, maybe somebody has an idea on what is using so much of my CPU and how to fix it.

Freshly restarted:


image

Running for several hours:


image

A big difference seems to be the initialize connection (socket_client.py). Any Idea what that is?

I suffer from the same problem. I initially had HA running on RPi3+B (HassOS), but it was crashing over night. Then I moved it to ESXi (HassOS as well) and same thing was happening. As of today I have assigned to it 2 CPUs (Xeons) and 2GB of RAM with vm hosted on SSD. At least it is not crashing anymore, but the CPU runs high shortly after reboot. I have very straight forward setup with few addons (mosquito, zigbee2mqtt, ssh, samba, configurator and node red) I have disabled all of them, but no change. I’m really getting sick and tired of not being able to look in historical logs. This is my second shot at HA in two years and it seems it always fails in the middle of the project. I’m really close on giving up on it :frowning:

For me it resolved itself after updating to 105 and redoing the integrations like netatmo, spotify, etc the new way.

Unfortunately with 108 it came back…

1 Like

Running HA on docker on my Rpi3. homeassistant/raspberrypi3-homeassistant:stable

Python is running 100% After about 1 day…
docker restart home-assistant and it’s fixed for about 1 day…

I just updated to 0.108.8, when the problems persists, i will check in again with more details.

Something definitely started in 108 but it seems to have been resolved in 108.6 or 108.8 (I skipped 7).

I still see more usage of the Python 3 Process in 108.8 than with previous versions.

Have the same problem on Intel-NUC and i´m very unhappy about it.
I had bought it to ditch the PI but it is even worse whe python3 is using so much of its power :frowning:
Definitively hope it will be fixed somehow

I have the same issue. Does anyone know how can we find the cause? I have activated debug and can’t find anything in the logs

I have updated from 0.108.8 to 0.109.3 and it is running much better, but I need to look again in 7 or so hours

At this point I’m going desperate.
Do you think that this might be an indication of a bad SD card?

Doubtful, for me its running on an SSD. I saw on Github that a lot of people had problems with the “Brother” printer component. But that is not the problem in my case. For now im leaning on doing a complete reinstall as im sure i still have some old legacy stuff in there that i took over from very old version of homeassistant.

i was leaning too, after trying for days of not being able to figure it out, i blew it away…

Did it actually help?

ya, it did…but now i have to reconfigure the years of work that i had in. i first tried reinstalling, then restoring, but i had the same issue. Now i’m trying to configure everything i can in the UI first, then copying what i can out of my old config, automations, scripts etc.

i actually came across your post yesterday, and saw that you were thinking about it. that’s when i made up my mind…

I just had tried something new.
I removed the default_config: in my configuration.yaml.

Previously i tried already disabling all my components one by one, but wasnt able to narrow it down.

However, since removing default_config: my CPU Usage is MUCH lower.
HomeAssistant process is around 20% instead of 100%.
I do suspect the ssdp: but i will try to activate one by one all of the components in the default_config: and see where this leads me.

EDIT - NEW UPDATE

Found it… its the zeroconf:

When i add this one back to my config, it immediately uses a lot of CPU power.
Without it everything is fine. Now to find out why.

2 Likes

In my case I removed Z-wave configuration from configuration.yaml and added as a component and I have been running Home Assistant for 3 days without any issues.

that seems to solve it for me too! Running on docker inside an Ubuntu Server on ESXI.

Should have been fixed by 110.4 update. However, i also found that if I add my System to more than one VLAN HomeAssistant will use a lot more CPU Power. Zeroconf on or off.