Tips for shell_command and auto-deleting old snapshots

Tags: #<Tag:0x00007f7398c75cf0>

I was having an issue trying to delete old snapshots using shell_command. After a year of being annoyed about it, I finally got it working today. Along the way I figured out a few important things I feel are worth sharing to the community. Fully functional script at the end.

  • shell_command runs in the homeassistant container if you are running HASSIO. I’m not sure about other installation types, but probably consistent. The mounted drives are different in this container vs. the host system that you can access via the developer SSH on port 22222. Keep that in mind when testing via SSH. You’ll want to run a shell inside the homeassistant container to test from or else the paths won’t be right if you’re on the host.

  • The backups are not mounted by default in the homeassistant container, so you have to mount them in your script. In my case, with full HASSIO, I mounted /dev/sda8 on /mnt/data and the snapshots are at /mnt/data/supervisor/backup. You can mount wherever you want. I chose /mnt/data to be consistent with where the host system mounts that device so that I don’t have to remember different locations based on what I’m logged into.

  • Although the backup folder wasn’t mounted for us in the homeassistant container, /config, /share and /ssl are available. The addons folder is not mounted either, but mounting the drive ourselves gives us that too, if you need it for some reason.

  • Note that all of those folders are technically subfolders on /dev/sda8, so when we mounted that drive to /mnt/data they are also accessible through that mount. For example, /config and /mnt/data/supervisor/homeassistant are the same folder on the disk. The other folders are also under the supervisor folder, but their names are consistent (share, ssl, backup, addons). I find it odd that the config folder is called homeassistant on the disk, but whatever.

  • A limitation I ran into with shell_command is that it doesn’t like to run multiple commands, nor does it handle for loops. I think it’s a limitation of the compact shell that runs the command (BusyBox). I worked around this limitation by making a bash script to do all the work and the shell_command just calls my script. I created my bash script in my config folder so that it’s part of my backups. Using a separate bash script file also lets me make changes without having to restart homeassistant, whereas editing the shell_command settings in configuration.yaml would require restarting homeassistant.

  • My bash script automates creating a temporary mount folder, mounting the drive, looping the available *.tar files over 7 days old and deciding which ones to keep or delete, then unmounting and removing the temporary mount folder. Aside from age, the decision to keep or delete requires examining the friendly name of the snapshot to determine if it’s one of the automated backups because I want to retain all manual backups regardless of age. I found a way to get the friendly name from the snapshot.json file inside the tar file without having to write it to the filesystem, so it’s nice and tidy.

My script file /config/tools/clean_snapshots.sh:

#!/bin/bash

echo "Cleaning up automated snapshots older than 7 days. Retaining manual backups of any age."
mkdir /mnt/data
mount /dev/sda8 /mnt/data
for i in `find /mnt/data/supervisor/backup/*.tar -mtime +7`
do
  NAME=`tar xfO $i ./snapshot.json | jq .name`
  AGE="$((($(date +%s) - $(date +%s -r $i))/86400))"
  if [[ $NAME == *Automated* ]]; then
    echo "  - removing $(basename $i) from $AGE days ago ($NAME)"
    rm $i
  else
    echo "  + keeping  $(basename $i) from $AGE days ago ($NAME)"
  fi
done
umount /mnt/data
rmdir /mnt/data

To call it, I have this in my configuration.yaml:

shell_command:
  delete_old_backups: '/config/tools/clean_snapshots.sh'

My next project is mounting a SAMBA share on my network and sync the snapshots so they are properly backed up on another computer.

3 Likes

Hi Keith,

thanks for the Instruction.
Unfortunately I could not get your exact script working on my NUC Hass.IO Image.

Config.yaml:

shell_command:
  clean_snapshots: config/shell_scripts/clean_snapshots.sh

Error:

Error running command: config/shell_scripts/clean_snapshots.sh, return code: 127

I even tried something very basic but it does not work, too:

mkdir /mnt/data
mount /dev/sda8 /mnt/data
find /mnt/data/supervisor/backup/* -type f -name '*.tar' -mtime +4 -exec rm {} \;
umount /mnt/data
rmdir /mnt/data

It could be possible that I have to mount it with a different name.
Could you tell me how you figured out /dev/sda8 or do you see any other mistake I made?

It seems I didn’t get notified of your post.

It’s been a while since I tinkered with this, but I have it running on a NUC with Hass.IO as well. Maybe you can get more info from the logs if you set to debug level for shell command:

logger:
  default: warning
  logs:
    homeassistant.components.shell_command: debug
1 Like

Hey Keith,

No problem!
I added the logger:

ran the shell-command via Developer-Tools:


2020-10-02 16:05:31 DEBUG (MainThread) [homeassistant.components.shell_command] Stderr of command: `config/shell_scripts/clean_snapshots.sh`, return code: 127:
b'/bin/sh: config/shell_scripts/clean_snapshots.sh: not found\n'
2020-10-02 16:05:31 ERROR (MainThread) [homeassistant.components.shell_command] Error running command: `config/shell_scripts/clean_snapshots.sh`, return code: 127
NoneType: None

That looks like a path problem. It’s currently relative to the current directory, which probably isn’t what you expect when it runs in the automation. Try giving it a full path starting from the root of the filesystem (start with / and include your mounted drive path).

Hi Keith,
Just a word of caution on how Home Assistant runs shell commands. You use shell_command platform in your yaml, I have a bad experience with command_line one.
Depending on the way you invoke the Home Assistant, the shell commands can be executed with either bash or sh. The last one is severely limited, of course, and often leads to errors and problems.
I discovered this with my tiny CPUtemperature script that worked fine until I … implemented the systemd startup script. See the screenshot:
20201018-ha-cpu_temp
I documented this behavior in the original thread Autostart using systemd and I’m still trying to troubleshoot it further.
Similar phenomena could contribute to the problem described here.

thanks for the advice. Unfortunately I dont know how to get the full drive path. Could you make an example or tell me what to do / where to look to get the paths needed? In my previous post I also asked you how you managed to get the paths …
Thanks in advance!

It looks like shell_command isn’t finding your script file. Have you tried a slash at the start of your config entry?

For example, you have this:

shell_command:
  clean_snapshots: config/shell_scripts/clean_snapshots.sh

but I think you need:

shell_command:
  clean_snapshots: /config/shell_scripts/clean_snapshots.sh

Edited it into:

shell_command:
  clean_snapshots: /config/shell_scripts/clean_snapshots.sh  

Getting this error unfortunately:
2020-10-29 09_53_20-Configuration - Home Assistant

I think the mounting of the path is not working… how did you find out /dev/sda8 though?