I’ve been noticing some high CPU for my HA VM. I consoled into the VM and ran top. This was the result.
That seems like a lot. Is it? If so, how do I troubleshoot this?
If I see this right docker is using around 125 % of cpu.
Yes, but I assumed that was a rollup of all the containers, no? Either way, I don’t think journald should be using that much CPU.
No it should not. But the docker should not use this amount of cpu either. I don’t know where the problem lies.
I can tell you that I have i3 from 10 years ago. I bought used nvidia p1000 gpu and google coral for my doorbell camera. My average cpu usage is around 14% and I have 26 running containers.
Probably you miss configured something or you have some hardware issue.
Maybe the issue is with your installation type. I don’t know, I’m just guessing.
…and I figured it out. Thanks for the validation that what I was seeing wasn’t normal.
It seems that during some testing recently, I turned on debug logging. I’m really surprised that it would pig up the system that badly, but it sure did! Once I downloaded the full log and saw all those DEBUG messages, I knew what had happened.
Oops. As usual, my own worst enemy.
Okay, I stand corrected. I’m still having high CPU from my HA instance and now I think I know why. I’m seeing a ton of these in my HA Host log:
J
an 16 15:39:47 homeassistant systemd-journald[216]: Suppressed 3129790 messages from docker.service
Jan 16 15:40:17 homeassistant systemd-journald[216]: Suppressed 3089976 messages from docker.service
Jan 16 15:40:17 homeassistant systemd-journald[216]: Data hash table of /var/log/journal/cf80bdcfdfd62dbcb0182f54ccbfde9c/system.journal has a fill level at 75.0 (85335 of 113777 items, 25165824 file size, 294 bytes per hash table item), suggesting rotation.
Jan 16 15:40:17 homeassistant systemd-journald[216]: /var/log/journal/cf80bdcfdfd62dbcb0182f54ccbfde9c/system.journal: Journal header limits reached or header out-of-date, rotating.
Jan 16 15:40:47 homeassistant systemd-journald[216]: Suppressed 3003347 messages from docker.service
Jan 16 15:41:17 homeassistant systemd-journald[216]: Suppressed 3113272 messages from docker.service
Jan 16 15:41:47 homeassistant systemd-journald[216]: Suppressed 3027035 messages from docker.service
Jan 16 15:41:47 homeassistant systemd-journald[216]: Data hash table of /var/log/journal/cf80bdcfdfd62dbcb0182f54ccbfde9c/system.journal has a fill level at 75.0 (85335 of 113777 items, 25165824 file size, 294 bytes per hash table item), suggesting rotation.
Jan 16 15:41:47 homeassistant systemd-journald[216]: /var/log/journal/cf80bdcfdfd62dbcb0182f54ccbfde9c/system.journal: Journal header limits reached or header out-of-date, rotating.
Jan 16 15:42:17 homeassistant systemd-journald[216]: Suppressed 3168883 messages from docker.service
Jan 16 15:42:17 homeassistant systemd-journald[216]: Data hash table of /var/log/journal/cf80bdcfdfd62dbcb0182f54ccbfde9c/system.journal has a fill level at 75.0 (85333 of 113777 items, 25165824 file size, 294 bytes per hash table item), suggesting rotation.
Jan 16 15:42:17 homeassistant systemd-journald[216]: /var/log/journal/cf80bdcfdfd62dbcb0182f54ccbfde9c/system.journal: Journal header limits reached or header out-of-date, rotating.
Jan 16 15:42:47 homeassistant systemd-journald[216]: Suppressed 3195389 messages from docker.service
Jan 16 15:43:17 homeassistant systemd-journald[216]: Suppressed 3090354 messages from docker.service
Jan 16 15:43:47 homeassistant systemd-journald[216]: Suppressed 3087393 messages from docker.service
What is going on here?
Okay, now I think I’ve solved it for real this time. Looks like it was this:
pychromecast seems to FREAK THE FUCK OUT when it can’t connect to a device. Hundreds of thousands of log messages.