I was looking for usable ways to reduce the amount of IO-traffic home assistant is creating. I came accross the topic https://community.home-assistant.io/t/steps-to-reduce-write-cycles-and-extend-sd-ssd-life-expectancy/291718. However, although applying as much as possible of the pointers given in that topic, I remained with a IO-traffic to my SSD over 10GB/day (with about 90 devices) resulting in increase of TBW over 40GB/dg (which was also in RAID1-mode using BTRFS, so actually altogether 80GB/day).
So I decided to look for alternatives, and I came accross the ‘anything sync daemon’, which I’m being using for a month or two for now. And I want to share my experiences with it, because I think it turned out to be quite a success.
The anything-sync-daemon actually makes use of temporary file systems as an overlay over the real file system, that is being synced backed tot the real file system with an adjustable time interval.
Tested setup:
- Lenovo ThinkCentre M70q i3-10100T (with 2TB Kingston NV2 SSD / 1TB Crucial MX500 SSD, 48GB SDDR4 RAM).
- KVM via Libvirt with 2x40GB raw partitions (partition passthrough) and 12GB RAM.
- Home assistant supervised with Adguard-Home, File editor, Nginx reverse proxy + Let’s encrypt, MQTT-broker + Zigbee2MQTT, Node-red, Samba back-up and Samba share add-ons.
Manual (can be used as a script, run as root or add sudo):
# Install packages tar, awk, pv, zstd, wget, unzip, make and rsync:
apt install -y tar mawk pv zstd wget unzip make rsync
# Clone the anything-sync-daemon github (I did it with downloading using wget, you can also use git for it):
cd /tmp/
wget "https://github.com/graysky2/anything-sync-daemon/archive/refs/heads/master.zip"
# Unzip the archive:
unzip master.zip`
# Install the anything-sync-daemon:
cd anything-sync-daemon-master
make install-systemd-all
Synchronization interval
In /lib/systemd/system/asd-resync.timer you can edit the the sync interval from the temporary file system to the real file system:
[Unit]
Description=Timer for Anything-sync-daemon - 1Hour
PartOf=asd-resync.service asd.service
[Timer]
OnUnitActiveSec=1h
I actually increased it to 12h.
Configuration
In /etc/asd.conf are the settings. I adapted this to:
#
# /etc/asd.conf
#
# For documentation, refer to the asd man page
## WARNING Do NOT edit anything in this file while asd is running!
## To protect data from corruption, in the event that you do make an edit
## while asd is active, any changes made will be applied the next time
## you start-up asd.
# Define the target(s) directories in the WHATTOSYNC array.
# Do NOT define a file! These MUST be directories with an absolute path.
#
# Note that the target DIRECTORIES and all subdirs under them will be included.
# In other words, this is recursive.
#
# Below is an example to whet your appetite.
#WHATTOSYNC=('/srv/http' '/var/lib/monitorix' '/foo/bar')
WHATTOSYNC=(
'/var/log'
'/var/spool'
'/var/cache'
'/usr/share/hassio'
'/var/lib/ntp'
'/var/lib/NetworkManager'
'/var/lib/docker/containers'
)
# Define where data will reside in tmpfs.
# This location must be mounted to tmpfs and MUST be writable and executable.
#
# If using bleachbit, do NOT invoke it with the '--clean system.tmp' switch or
# you will remove a key dot file (.foo) from /tmp that asd needs to keep track
# of sync status.
#
# Note that using a value of '/dev/shm' can cause problems with systemd's
# NAMESPACE spawning only when users enable the overlayfs option.
#
#VOLATILE="/tmp"
# ASD can break hardlinks present in the system, it has a safety check to ensure
# that the synced directories don't have the presence of any hardlinks in them
# by default, incase you want the directory to work standalone
# and not affect any other hardlinks of the files present in the synced director y,
# you can disable this safety check
ENABLE_HARDLINK_SAFETY_CHECK=1
# Uncomment and set to yes to use an overlayfs instead of a full copy to reduce
# the memory costs and to improve sync/unsync operations.
#
# You must modprobe either the 'overlayfs' or 'overlay' module prior to running asd if
# you enable this option. Distros running the linux kernel version >=3.18.0 are likely
# using the 'overlay' module while some distros shipping older kernels, notably Ubuntu
# provide the older version of this technology which is provided in the 'overlay fs'
# module not 'overlay' module.
USE_OVERLAYFS="yes"
# Uncomment and set to no to completely disable the crash recovery feature of as d.
#
# The default is to create crash recovery backups if the system is ungracefully
# powered-down due to a kernel panic, hitting the reset switch, battery going
# dead, etc. Some users keep very diligent backups and don't care to have this
# feature enabled.
USE_BACKUPS="no"
# Uncomment and set to an integer that is the maximum number of crash recovery
# snapshots to keep (the oldest ones are delete first).
#
# The default is to save the most recent 5 crash recovery snapshots.
#BACKUP_LIMIT=5
Make sure not to include mountpoints or sockets, because those aren’t being taken into account (so those file trees seem to be empty).
Enable anything-sync-daemon (asd) service
systemctl enable asd.service
reboot
Check service
root@HASSIO-server:~# asd p
Waiting for lock...
flock: getting lock took 0.000002 seconds
flock: executing /usr/bin/asd
Anything-sync-daemon v6.0.0 on Debian GNU/Linux 11 (bullseye)
Systemd service is currently active.
Systemd resync service is currently active.
Overlayfs v23 is currently active.
Asd will manage the following per /run/asd.conf settings:
owner/group id: root/0
target to manage: /var/log
sync target: /var/.log-backup_asd
tmpfs target: /tmp/asd-root/var/log
dir size: 44M
overlayfs size: 924K
recovery dirs: none
owner/group id: root/0
target to manage: /var/spool
sync target: /var/.spool-backup_asd
tmpfs target: /tmp/asd-root/var/spool
dir size: 8,0K
overlayfs size: 0
recovery dirs: none
owner/group id: root/0
target to manage: /var/cache
sync target: /var/.cache-backup_asd
tmpfs target: /tmp/asd-root/var/cache
dir size: 208M
overlayfs size: 0
recovery dirs: none
owner/group id: root/0
target to manage: /usr/share/hassio
sync target: /usr/share/.hassio-backup_asd
tmpfs target: /tmp/asd-root/usr/share/hassio
dir size: 2,6G
overlayfs size: 192M
recovery dirs: none
owner/group id: ntp/112
target to manage: /var/lib/ntp
sync target: /var/lib/.ntp-backup_asd
tmpfs target: /tmp/asd-ntp/var/lib/ntp
dir size: 4,0K
overlayfs size: 4,0K
recovery dirs: none
owner/group id: root/0
target to manage: /var/lib/NetworkManager
sync target: /var/lib/.NetworkManager-backup_asd
tmpfs target: /tmp/asd-root/var/lib/NetworkManager
dir size: 24K
overlayfs size: 8,0K
recovery dirs: none
owner/group id: root/0
target to manage: /var/lib/docker/containers
sync target: /var/lib/docker/.containers-backup_asd
tmpfs target: /tmp/asd-root/var/lib/docker/containers
dir size: 452K
overlayfs size: 436K
recovery dirs: none
Result
- In my setup, the IO decreased from 10GB (with 40GB increase of TBW) a day, to less than 2GB
- CPU use decreased from 65% tot 55%.
- RAM use increased from 2.2GB to 2.5GB on average.
- A really lot snappier system (probably partially due to the fact the most used files are in the RAM, especially the qlite-database).
- Rock solid service, without any drop-outs attributable to anything-sync-daemon for over a month now.
Downsides:
- Probably not suitable for al setups. In my case an earlier try-out with MariaDB resulted in failure of the database service (but had included /var/lib/docker as a whole at the time, instead of only the container-subfolder).
- More RAM usage
- Data loss is possible in case of sudden / unexpected power loss
- Due to the closed nature of HAOS, it’s only possible in the supervised installation method or circumvention of the way HAOS works (for example mounting the drive in another system and make the adaptations).
Ways to check IO
# Modified files during last 60 minutes (including my output)
root@HASSIO-server:~# find / -xdev -mmin -60 -type f
/var/lib/docker/overlay2/072669f3e55cf1028d3fdb41a89f825acd5cb97a329c65709ffb5f664526906d/diff/var/cache/samba/browse.dat
/var/lib/docker/overlay2/3bc6b33e03f5e2f65a3cd85b26ff3adc91e4af9e5a2a04d4a28db3703f1550dc/diff/var/log/nginx/access.log
# TBW-check (requires smartmontools installed):
root@HASSIO-server:~# smartctl -a /dev/sdX
Further tools like built-in virsh IO-tools (virt-top) or iotop.