Home Assistant is very slow and get stuck once a couple of days

Hello,

Here goes. Since a week+ is our home assistant front end slow, get stuck sometimes (have to restart by removing power) and it is hard for me to figure out why. Before, it was al running fast. This is what I see:

  • It takes a 10 to 15 seconds to load the website when opening.
  • It takes very long before the Hass.io screen is loaded…
  • I have Node-Red installed as add-on. This one also loads slow.
  • Logbook page is ‘taking ages’ to load (60 to 120 seconds?) and when you then click on a entity it keeps loading forever
  • History is taking long but within what I don’t mind.

This is mostly my resources of my rpi3+
image

What can I do?

Log:

starting version 3.2.8
19-08-18 11:18:53 INFO (MainThread) [__main__] Initialize Hass.io setup
19-08-18 11:18:54 INFO (SyncWorker_0) [hassio.docker.supervisor] Attach to Supervisor homeassistant/armhf-hassio-supervisor with version 180
19-08-18 11:18:54 INFO (SyncWorker_0) [hassio.docker.supervisor] Connect Supervisor to Hass.io Network
19-08-18 11:18:55 INFO (SyncWorker_0) [hassio.docker.interface] Cleanup images: ['homeassistant/armhf-hassio-supervisor:179']
19-08-18 11:18:56 INFO (MainThread) [__main__] Setup HassIO
19-08-18 11:18:56 INFO (SyncWorker_1) [hassio.docker.interface] Attach to homeassistant/armhf-hassio-dns with version 1
19-08-18 11:18:56 INFO (MainThread) [hassio.dns] Update resolv from Supervisor
19-08-18 11:18:56 INFO (MainThread) [hassio.utils.gdbus] Introspect org.freedesktop.systemd1 on /org/freedesktop/systemd1
19-08-18 11:18:56 INFO (MainThread) [hassio.misc.forwarder] Start DNS port forwarding to 172.30.32.3
19-08-18 11:18:56 ERROR (MainThread) [hassio.utils.gdbus] DBus return error: b'Error connecting: Could not connect: No such file or directory\n'
19-08-18 11:18:56 WARNING (MainThread) [hassio.dbus.systemd] Can't connect to systemd
19-08-18 11:18:56 INFO (MainThread) [hassio.utils.gdbus] Introspect org.freedesktop.hostname1 on /org/freedesktop/hostname1
19-08-18 11:18:56 ERROR (MainThread) [hassio.utils.gdbus] DBus return error: b'Error connecting: Could not connect: No such file or directory\n'
19-08-18 11:18:56 WARNING (MainThread) [hassio.dbus.hostname] Can't connect to hostname
19-08-18 11:18:56 INFO (MainThread) [hassio.utils.gdbus] Introspect de.pengutronix.rauc on /
19-08-18 11:18:56 ERROR (MainThread) [hassio.utils.gdbus] DBus return error: b'Error connecting: Could not connect: No such file or directory\n'
19-08-18 11:18:56 WARNING (MainThread) [hassio.dbus.rauc] Can't connect to rauc
19-08-18 11:18:56 INFO (MainThread) [hassio.host.apparmor] Load AppArmor Profiles: {'hassio-supervisor'}
19-08-18 11:18:56 INFO (MainThread) [hassio.host.apparmor] AppArmor is not enabled on host
19-08-18 11:18:56 INFO (SyncWorker_1) [hassio.docker.interface] Attach to homeassistant/raspberrypi3-homeassistant with version 0.97.2
19-08-18 11:18:57 INFO (MainThread) [hassio.store.git] Load add-on /data/addons/git/a0d7b954 repository
19-08-18 11:18:57 INFO (MainThread) [hassio.store.git] Load add-on /data/addons/core repository
19-08-18 11:18:57 INFO (MainThread) [hassio.store.git] Load add-on /data/addons/git/15ef4d2f repository
19-08-18 11:18:57 INFO (MainThread) [hassio.store] Load add-ons from store: 62 all - 62 new - 0 remove
19-08-18 11:18:57 INFO (MainThread) [hassio.addons] Found 5 installed add-ons
19-08-18 11:18:57 INFO (SyncWorker_0) [hassio.docker.interface] Attach to hassioaddons/node-red-armv7 with version v4.0.7
19-08-18 11:18:57 INFO (SyncWorker_2) [hassio.docker.interface] Attach to homeassistant/armv7-addon-check_config with version 2.2
19-08-18 11:18:58 INFO (SyncWorker_8) [hassio.docker.interface] Attach to homeassistant/armv7-addon-samba with version 8.1
19-08-18 11:18:58 INFO (SyncWorker_3) [hassio.docker.interface] Attach to homeassistant/armv7-addon-configurator with version 3.4
19-08-18 11:18:58 INFO (SyncWorker_1) [hassio.docker.interface] Attach to esphome/esphome-hassio-armv7 with version 1.13.6
19-08-18 11:18:58 INFO (MainThread) [hassio.updater] Fetch update data from https://version.home-assistant.io/stable.json
19-08-18 11:18:58 INFO (MainThread) [hassio.snapshots] Found 2 snapshot files
19-08-18 11:18:58 INFO (MainThread) [hassio.discovery] Load 0 messages
19-08-18 11:18:58 INFO (MainThread) [hassio.ingress] Load 2 ingress session
19-08-18 11:18:58 INFO (MainThread) [__main__] Run Hass.io
19-08-18 11:18:58 INFO (MainThread) [hassio.api] Start API on 172.30.32.2
19-08-18 11:18:58 INFO (MainThread) [hassio.addons] Phase 'initialize' start 0 add-ons
19-08-18 11:18:58 INFO (MainThread) [hassio.core] Hass.io reboot detected
19-08-18 11:18:58 INFO (MainThread) [hassio.tasks] All core tasks are scheduled
19-08-18 11:18:58 INFO (MainThread) [hassio.core] Hass.io is up and running
19-08-18 11:19:00 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
19-08-18 11:19:00 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_nodered
19-08-18 11:19:00 INFO (MainThread) [hassio.homeassistant] Updated Home Assistant API token
19-08-18 11:19:00 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request running

How old is your SD card?

Maybe that is dying / has died?

How big is your database? That usually is the culprit…

Good question, it is about 1,5 a 2 years now. It is a SanDisk Ultra though… How can I verify what the errors are on the SD card?

I would actually suspect a failing SD card and generally errors will appear in the log file if that is the case. From a login prompt type “sudo dmesg” or “sudo cat /var/log/syslog”

SD cards really are not meant to be used for non-stop I/O storage (unlike SSD) as they go bad after so many write cycles. I know when my Raspberry PI would give up the ghost in the past is was because the SD card had failed.

Holy cow, it is 1,32 gb… :sweat_smile:

Is this (too) big btw for home assistant?

Ok… I first have to enable ssh… =]

Okey, I checked dmesg. /var/log/syslog did not exists. So,in dmesg I get tons of:

[246197.063214] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[246197.063293] brcmfmac: power management disabled
[246513.051995] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[246513.052065] brcmfmac: power management disabled
[246829.050634] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[246829.050720] brcmfmac: power management disabled
[247145.038762] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[247145.038910] brcmfmac: power management disabled
[247461.097799] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[247461.097857] brcmfmac: power management disabled

Anything other I am looking for? As far as I scroll through the log, I don’t see any storage related errors…

Those messages just mean you don’t have wi-fi enabled.

I assume you meant 1.32 gb and not 132gb, that latter would be a problem, 1.32gb is fine. I assume you have something like the following in your configuration.yaml

# Database
recorder:
  purge_keep_days: 3

As for no syslog entry, guess is depends on what distro you’re using. What files are in /var/log? “ls /var/log” you could also try “cat /var/log/kern.log” or “cat /var/log/messages”

1 Like

I had similar problems until I tuned HA on my RaspberryPi to reduce or offload “write activity” . I have a beefer system that I use as a NAS and other function so I use this external server for a lot. Here’s what I’ve done.

  1. Reduce or disable logging
logger:
  default: error
  1. Migrate Database to external MariaDB instance
recorder:
  purge_keep_days: 7
  db_url: mysql://username:[email protected]/hass?charset=utf8
  1. Migrate Mosquito Server (MQTT) to external system
mqtt:
  broker: 192.168.0.xxx
  port: 1883
  client_id: home-assistant-1
  username: username
  password: password
  1. Disable Zwave logging in options.xml under homeassistant configuration directory.
  <Option name="logging" value="false" />
  1. Disable local system logging (Send to external syslog server) /etc/rsyslog.conf comment out all log paths and add:

#*.*;auth,authpriv.none		-/var/log/syslog
#cron.*				/var/log/cron.log
#daemon.*			-/var/log/daemon.log
#kern.*				-/var/log/kern.log
#lpr.*				-/var/log/lpr.log
#mail.*				-/var/log/mail.log

#user.*				-/var/log/user.log
#mail.info			-/var/log/mail.info
#mail.warn			-/var/log/mail.warn
#mail.err			/var/log/mail.err
*.* @@192.168.0.xxx:514


5 Likes

My biggest one was I believe 31 gigabyte, so that shouldn’t be much of a problem. I migrated from a Pi to a NUC to speed up things like history graphs etc.

Sdcard and log size will do it, although getting off the pi and going to a board with kick solved this for me. Asus tinder board, had the same size as the rpi3 with cpu and 16GB of onboard eMMC storage to run hass, added sdcard for the logs db. Just make sure to run the Armbian distro, tinderOS is dated.

Either a Freudian slip ,
Or pun-level 1000.

2 Likes

I assume you meant 1.32 gb and not 132gb, that latter would be a problem

yeah cultural difference. It is indd 1.32gb :grin:

I had purge_keep_days on 5 and interval on 2. Yesterday I set it on 3. As for deleting the database…
Yesterday, when I deleted the database and restart home assistant I can open the logbook and history very fast. Today I tried the same and it was very very slow again… I checked the database, it is 180 megabytes. I just deleted the database again and the logbook and history are fast again.

As for no syslog entry, guess is depends on what distro you’re using. What files are in /var/log? “ls /var/log” you could also try “cat /var/log/kern.log” or “cat /var/log/messages”=

I am using hassio image. when I execute ls on /var/log nothing is shown…

Thnx for your reply, this will be my last resort as I first want to know what is going on :slight_smile:

As for loading the website, I have some more details:

  • The website loads in approximately 6 to 7 seconds because it takes 6 to 7 seconds to retrieve the manifest.json from the rpi3+ and it looks that the browser is waiting for this file.
  • When I click on the hassio menu item it takes arround 22 seconds to load the page! With Chrome I did a performance profile and these are the results:
    • It took 9 seconds to get custom-panel.e12a8623.js
    • It took 3 seconds to get custom-elements-es5-adapter.js
    • It took 9 and 6 seconds (in parallel) to get two chunck.[…].js files

Clearly the db is causing your slowdown, as is sooo often the case.
You should consider configuring recorder to only include those sensors that are of actual importance to you.
A simplified example :

recorder:
  purge_keep_days: 3
  purge_interval: 1
  include:
    entities:
      - sensor.nas_hdused

This will only store values for that 1 sensor into your db, greatly reducing size & file access.

Agree, also you can exclude stuff as well that will help quite a bit with db size:

 exclude:
    entities:
      - sun.sun # Don't record sun data
      - binary_sensor.living_room_sensor_motion
      - weather.openweathermap
      - sensor.yr_symbol

I’m still not a big fan of SD cards when you’re using something with lots of disk I/O. You can also do a file system check on reboot see https://www.tecmint.com/fsck-repair-file-system-errors-in-linux/. Good to have a monitor hooked up at the time so you can watch screen output.

It still seems like you’re SD card might be going bad, slowness over time is a sign of that.

Getting rid of the spammy sensors like sun and weather is a good start.
Getting rid of everything that you don’t need is the next level.
Getting rid of the Pi is where you’ll end up.

2 Likes

This is what I ultimately did.

NUCS can be had on eBay anywhere from $50 to $100. I got a Celeron NUC with 8gigs of ram and a 32gig ssd for $50. Solved these similar issues for me.