0.117.0 continual memory increase

Well, you are correct in some regard. Something in 0.117.x is causing the problem. Perhaps a way it is logging, or there is a new library it is using or there’s a loop in some code somewhere that wasn’t there before when it interacts with something.

I’m just trying to find more specifically what the problem is with my set-up, sharing it here, and then hopefully they can fix whatever it is when we find it.

Something in 0.117.x is interacting with our current Integrations in a way that it wasn’t before.

They’ll find it and fix it. I’m confident in the developers and their abilities. The main thing is not losing any data, and so far I have not.

I have the issue, memory leak, but ok after upgrading to 0.117.2

The onvif integration had a memory leak, so if you have onvif camera’s that might be an explanation.

Another memory leak is coming from the Brother integration, probably only if you have the integration and your printer is turned of, more information and a solution to test can be found in the following bug reports:


I’ve seen a memory leak in 0.117.x as well and do have the Brother printer integration. Interestingly my printer had been turned off after a power outage and was such unreachable. Looks like a fix may go into 0.117.4 (https://github.com/home-assistant/core/pull/42833).

1 Like

Dropped by to say I think I’ve narrowed mine down to the brother integration, looks like I’m 4 hours late to the party! Great work guys, installing 0.117.4 as we speak :slightly_smiling_face:

1 Like

In 0.117.3 I removed Brother Integration and I resolved the problem.

So from what we’ve investigated here and the conversations on GitHub, the issue was:

Brother Printer Integration

This appears to have been fixed in 0.117.4 and I await a steady memory graph on my system after upgrading.

Well done everyone who checked their own systems and contributed to the conversation here.

2 Likes

Interesting. I’m using the Brother integration but not experiencing the memory leak. Is it because the printer is never off but in sleep mode?

Yes, only with the printer off I had the memory leak

It happened exactly when you restart Home Assistant with the printer off. If you turn it off after Home Assistant is on, and you don’t restart it, all is good.

Any news/update for this problem.
This is after 24 hours (after reboot host)

No brother, no ring intergration.

In the Home Assistant issue, there are links to a modified code of the new Profiler that is expected for version 0.118.0 but you can install as custom_component and can profile the memory. Maybe it does not show nothing, but maybe can help. Here is the link with the information: https://github.com/home-assistant/core/issues/42752#issuecomment-721153813

Is very easy to use. Configure it to execute for 1, 2 or 3 hours (it depends how fast it grows in your system) and post the file that it produces.

I think I found the problem.

I deleted DSMR intergration.
For my power and gas consumption.

Now. 117.4 is stable and 4% core ram for the last couple of hours.

Looking good.

Hmm, no. The Brother fix did it for me. The ONVIF fix sorted things for others. That does look like things are still ongoing for you. Damnit.

Yes i also continute to have the same memory issue, less then before but continue reboot 1 / 24h :frowning:

I’m sure it’s the DSMR intergration.
After 24 hours. everything is still good.
So in my case dsmr is the problem.

.

1 Like

just to confirm, I see an issue with DSMR as well.

after deleting the dsmr integration from UI I see this. black line indicates the momemt I did that and rebooted some times… (after reboot the integration is back again… )

I am trying to figure if it is related to grafana>influxDB or something, together with dsmr. I am seeing a lot of IOwaits since 117.0

in my VM, I am seeing an effect of both RFXCOM and DSMR integrations.

What changed in 0.117 for DSMR is that force_update property is set True for the entities. This means that every time data is received from the meter, it generates an event even when data does not change. This was changed because integrations like influxdb listen to state changed events. Without the property set, it can take hours for some data to change and nothing is reported to influxdb even though data is reported.

The downside is that smart meters connected through DSMR report data fast (every 5 seconds or so). There are 30 entities in this integration, so HA needs to handle 30 events per 5 seconds. Apparently it’s very hardware dependent whether this amount can be handled.

I’ve added a kind of rate limiter in this PR:


You can copy-paste the code in custom_components folder to see if it makes a difference. It’s set by default there to not report faster than 30 seconds.
1 Like

Hi Robbie,
thank you for digging into this one. !

I am not able to find the correct path where to put the sensor.py file.
The file I got by doing these steps, clicking:
your link > on the bottom of the page click ‘Fix pylint’ aeaa98a > homeassistant/components/dsmr/sensor.py > 3dots > view file > Raw > copy all > past in new txt file. rename new txt file to sensor.py

I have put this sensor.py file directly in config/custom_components, I dont see a dsmr folder in there.
I see there only my other integrations, just hacs/smart_sensor/unifigatway/wiz_light/xiaomi_map_extractor

I have put back dsmr sensor info in configuration.yaml, and added the integration again.
Result after 5 minutes it took to write this, still 9% cpu usage, where it was just befor that 2%.

My first guess I am either taking the wrong file, or putting it somewhere incorrectly.

I feel maybe the need for a word of advice :slight_smile:

image

You need to make a folder dsmr inside custom_components and copy the entire contents from here:


To this new folder. It will then override the built in integration. You should be able to see a warning in the logs that it’s doing that.