Automated partial snapshot help

Configuration
1

I have the following automation that creates a full snapshot every night:

- id: daily_backup
  alias: Daily Backup at 1 AM
  trigger:
    platform: time
    at: '01:15:00'
  action:
  - service: hassio.snapshot_full
    data_template:
      name: Automated Backup {{ now().strftime('%Y-%m-%d') }}
  - service: notify.ios_iphonex
    data_template:
      message: Creating backup named "Automated Backup {{ now().strftime('%Y-%m-%d') }}"

However lately I have been getting a timeout warning in the logs 5 minutes after the service is called:

Timeout on /snapshots/new/full request

Good news is that the snapshots do still appear to be working correctly. Bad news is that it appears to be eating up swapfile use that is only released after a restart.

I reckon it is my esphomeyaml directory that is causing the issue. The stats for this directory are mad. Possibly due to the massive number of tiny files. I have 12 sensor boards defined but each one has a full copy of what appears to be all the compile libraries.

1024×663 624 KB

Is there a way to exclude this from the backup?

I could not find any documentation on the snapshot service, specifically how to define a partial snapshot.

2

Actually it does look like the swapfile is dropping back to 50% of the disk space.

Still, I would like to exclude this directory from the backup. It’s using a lot of resources unnecessarily.

3

Out of curiosity I checked my esphomeyaml folder as I’m running Hassio on a 32Gb card and wondered how much of that was being gobbled up by the addon.
While the file count was large at nearly 13k, the size was a respectable 160Mb.
I also run about 12 esp’s with various sensors and attachments.

I’m not sure why your folder size is so large. Are you running stable or dev ?

4

Have a closer look. Check the “size on disk” attribute of your directory full of files.

The actual size of my files is similar to yours (~160MB) but as they are mostly small files and there are a lot of them (11 thousand) and there is a finite disk allocation unit size they take up a large amount of disk space. Size on disk = 11GB. This is because the default allocation unit size for NTFS = 4096 bytes which is the smallest amount of space a file can occupy. If you have ten 1 byte files they take up 41KBytes of space on the disk, not 10Bytes (this is an over simplification but you get my point).

Now this may just be a windows reporting problem as the file system is not actually NTFS on the card. Just had a read and found that EXt4 uses the same default unit allocation size.

Either way I don’t need to be adding 160MB to my backup. I only need to make copy of the esphomeyaml directory manually when the configs - change, which is rarely compared to the HA config. So I would like to be able to define a partial backup to exclude them.

5

Not sure how many esphome devices you have but I’ve noticed that the .esphome directory inside config/esphome contains stale files sometimes when I rename devices. You should probably go in there and check it out if you haven’t already. Also, you might want to try doing a “clean build” before you snapshot to clear out the build files (it’ll take longer next time you do a build but should reduce your 11GB)

For example:
image

6

Thanks but I’ve already done that. Also Otto has done some optimisation since that post was written.