Raspberry Pi / filesystem stability

Hardware
16

I’ve seen similar problems, already two cards burned by Hass in about 1.5 year.
From what I’ve seen, the root cause is not the write speed of the SD card, but rather the use of sqlite as a database. Sqlite does a filesystem flush after every change to the database, which not only alters the blocks of the database on disk, but also the filesystem metadata of the sqlite database. This prevents the Linux kernel from grouping and optimizing disk writes, but on the other hand it makes sure the on-disk database will always be consistent and up-to-date.

Now the weak point of flash media is not in the write cycles, but in the erase cycles: flash blocks wear from erase cycles. Flash media controllers try to even out the wear by shuffling logical blocks around, but there’s only so little wear leveling an SD card can do. Given the amount of writes (and syncs) you get with a Hass system with many events, you can imagine the amount of wear the SD card gets is beyond the normal casual use the SD card vendors expect.

I have been thinking how to solve this for quite some time, and I’ve come up with some possible directions for solution. Here they are in random order:

  1. Create a tmpfs to put the sqlite database on it. Works great, but the database will not be persistent over crashes or reboots.
  2. As 1, and have a cron job copy the database back to disk every an hour. At boot, copy the database from disk to tmpfs, start hass, and off you go.
  3. Put the sqlite database on NFS storage backed by something that’s not flash, or by a lying NFS server telling the client “yes, I have synced your blocks to disk”, while at the same time finding a good time to actually group that bunch of writes to disk.
  4. Tell Hass to use an in-memory Sqlite database. Same as option 1, less fiddling, but not evern constistent over Hass restarts.

Configure like this:

recorder:
  db_url: sqlite://

So far the easy options that don’t require any changes to hass.

A solution that needs coding is to tell Hass to put the sqlite database in memory (a RPi 3 has sufficient memory for that), and have Hass backup the database to disk once every hour (or on whatever configurable time interval). This is very easy to do in C, the Sqlite documentation even has example code. Unfortunately I haven’t found a way in SQLAlchemy to get to the underlying Sqlite connection, and even if I got, the pysqlite library doesn’t support the backup functions the C library does.

Regards,
Erik