Journal fills disk and crashes HAOS

My journal files apparently filled their partition and really crashed hard. It’s a VM so I expanded disk capacity to get the system up, and then I manually cleared most of the journal log files, but I’d really like to make this a permanent solution, similar to some of the suggestions at Steps to reduce Write Cycles and extend SD/SSD life expectancy, but no matter what I do, I can’t seem to get read/write access to /etc/systemd/journald.conf - I’ve tried remounting RW but to no avail.

I can post all my steps in here, but it’s a pretty standard HAOS install.

P.S. Journal files grow quickly… they’re going up several MB every couple minutes.

I restored from a backup and it blew up almost instantly.

This journal fill really broke things. I’m considering a complete rebuild now.
image

Home Assistant OS is largely a read-only operating system. This means that you can’t really change settings on the OS level.

That said, the systemd-journald should automatically shrink the size of the journal database according to the free space.

Reading the messages it seems there I/O errors. This is most likely a communication issue or disk issue with your external SSD. If you are using Raspberry Pi, quite often this is related to power (make sure to have powered your Pi with a suitable power supply or use a USB hub separately powered). Another issue which happens quite often is that with some disks the advanced USB mass storage protocol UAS makes troubles. In that case, you have to add an exception to cmdline.txt for your disk.