HA Memory Leak

Hi folks, I’m new to Home Assistant, but not new to embedded devices. I have been experiencing an obvious memory leak in Home Assistant and was wondering how I can track down the cause.

This is my current memory free state, as you can see I loose about 15% in a day.
Annotation 2019-12-26 083833
I have made an automation that reboots HA in the early morning when the memory usage is high, otherwise HA will crash and reboot itself whenever it wants.

My setup is Hass.io running on a Pi 3B+. I have the following Integrations turned on: UniFi Controller, Plex, Google Cast, Met.no, and MQTT. I have a couple other Hass.io addons, like Mosquitto Broker, but those don’t seem to cause the issue as I can reboot just HA and the leak goes away. I have UniFi Controller Integration turned on, but my UniFi controller is running on a separate dedicated Pi (I’ve had UniFi for years and love it).

Here’s my configuration.yaml:

# Configure a default setup of Home Assistant (frontend, api, etc)
default_config:

# Uncomment this if you are using SSL/TLS, running in Docker container, etc.
# http:
#   base_url: example.duckdns.org:8123

# Text to speech
tts:
  - platform: google_translate

group: !include groups.yaml
automation: !include automations.yaml
script: !include scripts.yaml

sensor:
  - platform: systemmonitor
    resources:
      - type: disk_use_percent
        arg: /
      - type: memory_use_percent
      - type: processor_use
      - type: last_boot
  - platform: qnap
    host: CEREBRAL.localdomain
    username: !secret qnap_user
    password: !secret qnap_pass
    ssl: false
    verify_ssl: false
    monitored_conditions:
      - status
      - volume_percentage_used
      - system_temp
  - platform: qnap
    host: 192.168.2.10 #someday get mDNS forwarding working on axonbridge
    username: !secret qnap_user
    password: !secret qnap_pass
    ssl: false
    verify_ssl: false
    monitored_conditions:
      - status
      - volume_percentage_used
      - system_temp
  - platform: cups
    host: 192.168.1.5 #printerserver.localdomain
    printers:
      - Samsung_ML-191x_252x_Series
      - HP_Deskjet_F2200_series
climate:
  - platform: honeywell
    username: !secret honeywell_user
    password: !secret honeywell_pass
    region: us
media_player:
  - platform: webostv
    host: LGWebostv.localdomain
cover:
  - platform: myq
    username: !secret myq_user
    password: !secret myq_pass
octoprint:
  host: octopi.localdomain
  api_key: !secret octoprint_key
  number_of_tools: 1
  bed: true
camera:
  - platform: mjpeg
    name: OctoPrint
    still_image_url: http://octopi.localdomain/webcam/?action=snapshot
    mjpeg_url: http://octopi.localdomain/webcam/?action=stream

Is there any known issues with stuff I’m using? Narrowing this down could take quite awhile and I appreciate any help trying to figure this out.

It’s not fun, but the easiest way would be to disable integrations one by one until the leak disappears.

I finally broke down and started turning off integrations. I have determined the cups integration is the source of the issue. In fact, I believe the cups integration was also causing high memory usage on my printer server as well. I’ll start looking in to creating an issue for it, but given that the cups integration was just a nice-to-have and of no practical use for me, I’m tempted to just keep it disabled and move on.

Yes, sadly something has happened to CUPS sensor. Had to disable it as it rendered HA almost unusable:


As You can see memory usage depends on polling rate. Two things jump to my mind:

  1. Added a printer with underscores in name (like HP_Laserjet_XXXX) and
  2. custom printer drivers names are several lines long.

HA runs in venv on RPI3+.
Any ideas how to fix it- some automations rely on CUPS :frowning:
TIA, JR

Does anyone know if this has been raised as an issue yet (couldn’t see anything with a quick search on Github)? If not I might see about raising it. I use the CUPS sensor to control the power to printers so they are only powered on if there is a job waiting for them.

Sadly, I’m am similar issues with memory leaking and am not using CUPS. I am however using the ubiquity client. I’ve no particular reason for thinking that the Ubiquity client is to blame other than I have it in common with the original author of this thread. The impact is to me that my automated light switches (and everything, of course), which rely on mqtt become non-functional until I reboot.

Having read other threads I’m thinking I may need to bail on Hassio and learn about and move to docker. Even better, I’m hoping I can keep Hassio and progressively offload components over to docker until I find where the issue is and Hassio becomes stables for me again.

I’d be really interested in hearing from anybody with experience doing this or with thoughts on alternate approaches - especially those that don’t require me to teach myself new technologies.

Filed a bug/issue report on HA Github …
Fingers crossed. JR

I have added to your issue, as my config is different it should help devs narrow it down to not being a specific install type etc.

Linking issue to make it easier for others to find and add details from their installs.

@fabaff I notice this has been assigned to you on github, I was just wondering what sort of priority it had in your work stack.

Hi!
I am running o a Pi 4 with 4GB of memory. In just over a day it drains the memory. This morning I could not even access Lovelace and via ssh I got into the Pi and saw I only had 25MB memory left.

I am starting to suspect it has to do with Mosquitto and reason is that I added more things to my setup (Bluetooth presence) and since then it just escalates.

Another thing I noticed as well is the time it takes to reboot the Pi…it is 30 min from the time I issue reboot in the shell till it is back up and running.

I am also using Mosquitto MQTT for my camera running on Blue Iris. But once I added more to the MQTT bus it got a lot worse. Now it is unusable and none of the automation’s work when it suffers of no memory.

Any help on how to start finding out what is eating my memory?

I’m using a 3 B+ and ever since upgrading to 108.x I’m getting the same effects on my Pi. After each reboot the memory usage just goes on upward trajectory

I rebooted the pi last night and then at 6am today

Hi!

My problem also started after upgrading to 108.x. iowait is shooting to the roof as well and stays at 98% and up to 100% constantly.
Memory just keeps going up till it runs out. Reboots takes 45min before it becomes responsive.
Think there is something strange going on here…

This is my situation with my Raspberry Pi 3b + with 1gb RAM:

Cattura1

I’ve also 12 shelly pm installed and 2 miflora sensor.

Is it normal?
Is there a possibility to decrease memory around 50% or i should necessary buy a pi4 with 4gb ram?
I’ve ssd installed on my pi 3b+, is it compatible with pi4?
Thanks for any advice.
Regards,

Alessandro

Commenting here to let everyone know I have this issue too and to supply some data to hopefully find a common denominator.

Unfortunately, I can not narrow it down to a Home Assistant version. I generally update my HA and add-ons on a raspberry pi 3b+ every week. Over the last few weeks, I noticed that lovelace locks up occasionally but typically would come back up at some point (seems to restart by itself). It has progressed to the point that other add-ons are effected (my NginxProxyManager will go down and some times SSH too).

After some digging, I noticed the memory usage steadily increases until the system becomes unstable (usually after 36-48 hours). I moved NginxProxyManager to my new server thinking that might help but it did not. My temporary solution right now is a daily restart automation at 4am so the system stays stable.

# Restart HA because memory leak
- alias: Restart HA
  trigger: 
    platform: time
    at: "04:00:00"  
  condition:
    condition: time
    weekday:
      - mon
      - tue
      - wed
      - thu
      - fri
      - sat
      - sun
  action:
    - service: hassio.host_reboot

Below is a picture of my add-ons.


I currently DO NOT use the following add-ons:

  • ADB - Android Debug Bridge
  • APC UPS Daemon (No USB)
  • Check Home Assistant configuration
  • Let’s Encrypt
  • NGINX Home Assistant SSL proxy
  • Nginx Proxy Manager
  • Network UPS Tools
  • Portainer
  • Remote API proxy
  • ozwcp
  • phpMyAdmin

I currently use the following add-ons:

  • APC UPS Daemon
  • AirSonos
  • Backup Hassio to Google Drive
  • File editor
  • MariaDB
  • Mosquitto broker
  • Node-RED
  • SONOS AudioClip TTS
  • Samba share
  • Tautulli
  • Terminal & SSH

I read something in here about the cups integration. I used to use this weeks ago but it has been commented out of my configuration.yaml after trying it out for like an hour. The only integrations I can think of adding since this has been an issue is the Brother Printer Integration and the new Internet Printing Protocol (IPP) integration. Unfortunately, I think these were even added after the issue was present but I can’t remember. If someone here wants a list of my integrations I can try to provide those too.

are you guys using the androidtv integration?
I noticed that the memory load increases quite a bit (from ~50% to ~75%) when my androidtv is on.
I believe this is related to the new feature in the media_player: a screenshot of the tv picture is transfererd every few seconds.

Another thing I noticed: Adguard addon seems to have a memory leak. I disabled it for now.

Seems the CUPS integration memory leak issue has been closed on git hub with no action so obviously no interest from the code owner to even look never mind fix that one :frowning:
If anyone sees this that has the ability to re-open it and nudge someone to take a look that would be helpful…