Small CPU spikings after os 14.0 and core 2024.12.01

vukisz · December 8, 2024, 6:58pm

Hello. Did a OS update to 14.0 and core update to 2024.12.01.
All went well and everything is working. But during a day I noticed that my Intel NUC’s (running HA as VM on proxmox) fan is kicking in for short bursts. This is unusual.
Looked at proxmox CPU usage history. Normally it’s around 1.5-2% usage for HA VM but got recurring spikes to 3.0-3.5%. I guess that’s when fan kicks in.
Added CPU sensor to HA and also noticing those spikes.

Anyone else experiencing something similar after latest core and OS updates?

bdraco · December 8, 2024, 7:08pm

Check the section Sluggish performance or increased CPU usage

vukisz · December 9, 2024, 7:56am

Thanks @bdraco . But I cannot find any suspects there

The only integration I could recognize was a roborock integraton (with quite low execution times). Disabled it but issue still persists.
And now I see a pattern CPU usage increases every 5 minutes starting at 5th minute 22th second and ending on 37 second.

EDIT: Did all integrations disable and enable one by one and tracked, that when shelly was disabled there were no issues.
Found out this fresh issue on HA core issues: Statistics component causes significant slowdown · Issue #132597 · home-assistant/core · GitHub
And indeed I am using shelly sensors helpers for shelly sensor entities… Most likely it’s related.

bdraco · December 9, 2024, 11:07am

It also looks like you have some rather expensive templates

vukisz · December 9, 2024, 11:27am

Wow you can see it only from that screenshot?
Well there are some helper templates calculating energy consumed from grid+energy consumed from solar. And similar ones.
Another one is calculating max max heating time in hours from 5 climate entities.
Any way I can see which one has has hardest toll?

bdraco · December 9, 2024, 11:44am

I’d look at one that have regex searches in them.

vukisz · December 9, 2024, 1:04pm

Thanks, but non of them have regexp searches. Maybe 8 sonoff nspanel blueprint automations have something which looks like expensive templates.
I also have 8 heavy SQL queries (SQL integration), but reworked that to run every hour, or when climate entity changes between heat/idle

bdraco · December 9, 2024, 2:54pm

Its also possible you have one that is executing a lot more than expected. You can check with Finding a run-away template on that same link above

vukisz · December 10, 2024, 6:33am

Thanks for additional tips. This reports those simple templates:

RenderInfo object in memory: <RenderInfo Template<template=({{ (states('sensor.solar_energy_consumed_by_home_today') | float(default=0)) +(states('sensor.energy_total_today_from_grid') | float(default=0)) }}) renders=1508> all_states=False all_states_lifecycle=False domains=frozenset() domains_lifecycle=frozenset() entities=frozenset({'sensor.energy_total_today_from_grid', 'sensor.solar_energy_consumed_by_home_today'}) rate_limit=None has_time=False exception=None is_static=False>
RenderInfo object in memory: <RenderInfo Template<template=({{ (states('sensor.shellypro3em_ec6260977bf0_phase_a_active_power') | float(default=0)) +(states('sensor.solax_srzjcxmcpy_grid_1_power') | float(default=0)) }}) renders=30198> all_states=False all_states_lifecycle=False domains=frozenset() domains_lifecycle=frozenset() entities=frozenset({'sensor.shellypro3em_ec6260977bf0_phase_a_active_power', 'sensor.solax_srzjcxmcpy_grid_1_power'}) rate_limit=None has_time=False exception=None is_static=False>
RenderInfo object in memory: <RenderInfo Template<template=({{ (states('sensor.shellypro3em_ec6260977bf0_phase_b_active_power') | float(default=0)) +(states('sensor.solax_srzjcxmcpy_grid_2_power') | float(default=0)) }}) renders=30398> all_states=False all_states_lifecycle=False domains=frozenset() domains_lifecycle=frozenset() entities=frozenset({'sensor.shellypro3em_ec6260977bf0_phase_b_active_power', 'sensor.solax_srzjcxmcpy_grid_2_power'}) rate_limit=None has_time=False exception=None is_static=False>
RenderInfo object in memory: <RenderInfo Template<template=({{ (states('sensor.shellypro3em_ec6260977bf0_phase_c_active_power') | float(default=0)) +(states('sensor.solax_srzjcxmcpy_grid_2_power') | float(default=0)) }}) renders=30320> all_states=False all_states_lifecycle=False domains=frozenset() domains_lifecycle=frozenset() entities=frozenset({'sensor.shellypro3em_ec6260977bf0_phase_c_active_power', 'sensor.solax_srzjcxmcpy_grid_2_power'}) rate_limit=None has_time=False exception=None is_static=False>
RenderInfo object in memory: <RenderInfo Template<template=({{ (states('sensor.shellypro3em_ec6260977bf0_total_active_power') | float(default=0)) + (states('sensor.solax_all_pv_power') | float(default=0)) }}) renders=30266> all_states=False all_states_lifecycle=False domains=frozenset() domains_lifecycle=frozenset() entities=frozenset({'sensor.shellypro3em_ec6260977bf0_total_active_power', 'sensor.solax_all_pv_power'}) rate_limit=None has_time=False exception=None is_static=False>

EDIT: I could also provide my 60 sec. profiler dumps when CPU is spiking and another one when it’s under normal usage if this would help?

banalac · December 10, 2024, 2:32pm

Hi,

I have the same issue since I updated HA Core from 2024.12.0b1 to 2024.12.0b2.
Today I downgraded HA Core to 2024.11.3 and the issue is gone.

banalac · December 10, 2024, 2:38pm

vukisz · December 11, 2024, 7:26am

@bdraco based on @banalac findings seems that these CPU spiking issues introduced from 2024.12.0b2. Maybe “Statistics component causes significant slowdown” github issue is only a red herring.
Based on 2024.12.0b2 release notes one PR seems to match: #131702 Improve recorder history queries. Can this PR cause those CPU spikings?

EDIT: disabling all template helpers does not resolve issue. Disabling shelly integration (with noisy sensors) still solves this issue. Is there a way to prevent shelly sensor (gen 2) being so noisy ?

vukisz · December 18, 2024, 5:55am

Solved in 2024.12.4 most likely #131702 Revert "Improve recorder history queries.