100% CPU usage

Configuration
1

My HA core has been maxing out my CPU frequently over the last week. It started under v2024.5 but I’ve now upgraded to v2024.6 and it’s the same. It’s running in a Docker container on a Lubuntu 20.04 ESXi VM.

If I run htop it shows this as the prime consumer of CPU:

python3 -m homeassistant --config /config

I’ve installed the Profiler integration but haven’t got to grips with how to use it yet, and when I try to start the profiler it times out due to HA running at100% CPU usage.

There are demuxing errors in the HA log for the doorbell camera, but I don’t know know whether this is cause or effect of the CPU usage:

2024-06-23 13:00:37.614 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 13:07:33.341 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error opening stream (ERRORTYPE_110, Operation timed out) rtsp://****:****@192.168.1.94:554
2024-06-23 13:14:39.225 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error opening stream (ERRORTYPE_110, Operation timed out) rtsp://****:****@192.168.1.94:554
2024-06-23 13:21:55.006 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error opening stream (ERRORTYPE_110, Operation timed out) rtsp://****:****@192.168.1.94:554
2024-06-23 13:30:00.906 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 13:39:27.310 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 13:48:27.912 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Stream ended; no additional packets
2024-06-23 13:56:23.195 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error opening stream (ERRORTYPE_110, Operation timed out) rtsp://****:****@192.168.1.94:554
2024-06-23 14:04:28.510 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error opening stream (ERRORTYPE_110, Operation timed out) rtsp://****:****@192.168.1.94:554
2024-06-23 14:13:41.822 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 14:22:47.887 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 14:31:35.735 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 14:40:21.332 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error opening stream (ERRORTYPE_110, Operation timed out) rtsp://****:****@192.168.1.94:554
2024-06-23 14:49:50.426 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 14:58:56.041 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error opening stream (ERRORTYPE_110, Operation timed out) rtsp://****:****@192.168.1.94:554
2024-06-23 15:08:34.568 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 15:22:12.760 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 15:34:10.822 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 15:43:56.525 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error opening stream (ERRORTYPE_110, Operation timed out) rtsp://****:****@192.168.1.94:554
2024-06-23 15:56:13.521 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 16:06:57.388 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 16:17:19.533 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream while finding first packet: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 16:27:44.958 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error opening stream (ERRORTYPE_110, Operation timed out) rtsp://****:****@192.168.1.94:554
2024-06-23 16:39:46.178 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 16:50:43.987 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 17:05:24.941 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 17:17:06.805 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'
2024-06-23 17:28:22.322 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error opening stream (ERRORTYPE_110, Operation timed out) rtsp://****:****@192.168.1.94:554
2024-06-23 17:41:33.602 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.db1c_f50065133] Error from stream worker: Error demuxing stream: [Errno 110] Operation timed out: 'rtsp://userid:[email protected]:554'

I also can’t remember how to disable the camera in the HA config!

Any tips for how to diagnose this further?

2

It’s not clear from that if the camera is causing that or the camera is failing because of that. I would start by disabling add-ons that are high usage but non essential like frigate, kodi, plex, etc. Then the same for custom integrations until it calms down.
There is also this tool:
2024.5+: Tracking down instability issues caused by integrations.

3

Many thanks for the advice. It looks like the camera was indeed the problem. I managed to track down where I had configured it - in the Generic Camera integration - and disabled it. It’s been 18 hours now and the problem hasn’t returned.

Fingers crossed…

4

Oops…spoke too soon…it’s happening again…looks like the same camera is generating the same demuxing errors in the log, even though I’ve disabled it in the Generic Camera integration.
I have HA debug enabled, but I don’t see anything obvious in the logs that’s triggering the CPU usage.
I’ve tried running the Profiler integration, and it seems to have generated a callgrind.out and profile file, but they’re from 20 hours ago (long before the problem recurred). The Profiler service seems designed to be run only for short periods- is there any reason that I couldn’t leave it running for 24 hours??

5

Do you have default_config in your yaml?

6

Yes I have an empty default_config element at the top of my configuration.yaml