since few months ago my pi3b+ is getting very slow and HA ends up not responding. Restart fixes it and then after 1 week or so again the same thing.
I’ve added few checks and realized that it may be related with memory… once swap hits close to 100% and RAM 90% this happens.
I do not have that many addons and it was working ok before… there are few discussion on potential memory leaks but anyway it is not the purpose of my topic.
I’m running an SSD with my pi and because of that I think that raising the swap size could reduce or even solve some of these issues. The default setting is 25% of the real device memory (in this case 256Mb) and I think that it is probably having in SDcards in mind.
I’ve managed to change it using the commands below and so far so good… the problem is that this is not persistent (and after restart goes back to original swap size). As far as I understood I would need to change the /usr/libexec/hassos-zram or /usr/lib/systemd/system/zram-swap.service but the filesystem is read-only. Is there any other way to do this?
Just a quick update to mention that increasing the swap size is working well so far. Now the RAM is stable 65% - 70% and the swap is between 30% - 35% so not that much from the forced 25% (the problem is that this configuration is lost when restarted)
PS - doing it with SD card is not the best option since will just kill it faster.
I know this is an ancient topic, but still the only topic asking this question, and today I found an extremely hacky workaround to make this apply automatically on boot, effectively allowing one to run commands on the host ssh session on boot automatically.
I felt like it would be worth documenting even if it means reviving a 2 year old topic.
If you have the community ssh addon, you can grant it passwordless ssh access to the debug ssh on port 22222 and then run ssh commands on addon start. Which you can in turn set to autostart on boot.
This is extremely hacky and potentially has severe security implications, so if you’re unsure of what these security implications are, turn back here.
If misused it can also break your hassio installation.
Basically:
Generate an ssh key on the community ssh addon with ssh-keygen. Leave the password blank.
Enable debug ssh (host ssh, port 22222) with the newly generated public ssh key (~/.ssh/id_rsa.pub by default). Doing this will give anyone and anything with access to the community ssh addon full control over your system, including the ability to backdoor it without you ever knowing! Only do this if you know what you’re doing!
You can now ssh into [email protected] -p22222 without password from within the community ssh addon. You can also directly run commands, like ssh [email protected] -p22222 zramctl.
The community addon lets you specify commands to run on start. You can put ssh commands in here. For instance, for my pi4B 2GB, I change the grossly misunderstood default zram swap configuration to 2GB with zstd compression, like this: init_commands: ["ssh [email protected] -p 22222 'swapoff /dev/zram0; sleep 10; zramctl --reset /dev/zram0; zramctl -a zstd -s 2G --find; mkswap /dev/zram0; swapon /dev/zram0; echo 1 > /proc/sys/vm/swappiness'"]
This is, as mentioned, a terrible hack, but it does the job and is a significant improvement over the defaults, especially when you’re running a 50GB ltss timescale database on 2GB RAM.
Finally, in regards to the original post, while wholly outdated by now, zram swap is not disk swap. zram swap is compressed in-memory swap that never touches your SD card or any other storage. You’re basically just using system RAM more efficiently instead of having to resort to disk swap.
HassIO, at least on the Pi4B, does not use disk swap at all, and only uses a tiny pool of 25% zram swap.
Setting it to 100%/2G like I did does not mean it uses up all memory as zswap, btw. It only uses memory when needed, and when you’re hitting swap, well, it is needed. Setting zram to 100% of your memory is safe and only offers benefits over not having it in almost every situation. You could even push it beyond 100% if you’re feeling daring, as the size you set is the uncompressed size, compressed the size will be smaller.
The default 25% for zram is a lot of wasted potential, and Home Assistant might want to consider setting this to >80% if not 100%, so hacks like this are not necessary. Also when you’re memory constrained, the relatively heavy zstd as compression algo, even on weak systems, beats running out of swap or falling back to disk swap by miles.
I just want to say thank you.
The ssh workaround to run commands is a great idea.
The more I look at HAOS the more I realize that the system is built on a purely “my way or the highway” logic.
There is already a whole overlay system in there, but instead of making it work and allowing the user to change anything, it’s all locked down to hell.
HAOS is not a general purpose OS. It has been created as an appliance to solely serve the needs of HA. You can, however, run HAOS in Debian or any other Linux distro which supports KVM, or any non-Linux OS using a type-2 hypervisor.
You can also run HA in a container setup, in any OS of your choosing.
Sure. It’s just the philosophy.
In 10.0 the support for Raspberry Pi 3B+ and all other 1G devices was quietly dropped by removing zram altogether.
Without zram these devices do not have enough RAM to run HA for any serious amount of time. With zram, especially with larger ZRAM, such as 1-2G they can run HA all day long.
It is unfortunate then, that such a breaking change was made and it took me so much digging to find out why my HA grinds to a halt and barely functions after upgrading hassos from 9.5 to 10.2.
Oh and, I did create an issue on GitHub. The devs don’t care about devices with 1GB of RAM. But at least then the main website should probably be updated that 10.0+ does not support them anymore.