Home assistant backup folder repeat itself taking up a lot of space

Tags: #<Tag:0x00007f739442dcb8>

Hey all!!

I am running HA on Docker, I configure it to backup daily and recently I notice that it is taking up 3GB for just 1 day.

After i SAMBA into it and take a look, it looks like this

image

and I can just keep clicking into the backup folder, it just seems to go on and on, like this

image

so… is it my configuration issue or it is suppose to look like that?
I just removed some old snapshot to free up some space, (7 days of snapshot was 22GB)

Appreciate someone can shed me some light!!

Updated with more information below:

HA Info:
HA Version: 0.110.3
OS: Ubuntu
Supervisor version: 224
Autobackup version by @jcwillox: 0.5.2

Configuration.yaml:

recorder:
  purge_keep_days: 1
  purge_interval: 1

auto_backup:
  auto_purge: true

Automation.yaml

- alias: Perform Daily Backup
  trigger:
    platform: time
    at: 05:00:00
  action:
    service: auto_backup.snapshot_full
    data_template:
      name: 'DailyBackup: {{ now().strftime(''%A, %B %-d, %Y'') }}'
      keep_days: 7

what size is you database file

home-assistant_v2.db

mite have to do some reading on the recorder

my last 7 days

image

about 8gb

1 Like

That’s not the issue. This is the issue:

9740b6b5412edab3b2bd7cfc5bbf4a468fe92818

Each one of those folders contains a copy of the backup. Which is 90MB.

1 Like

Hi Myle,

My database size it 827.15 MB, i use autobackup to keep track of the snapshots.

Your screenshot of snapshot seems like how it should work instead of mine,
which the backup folder another backup folder which loop for a ridiculous amount of time ( as shown below) and eat up a large amount of storage

Yep, this is the issue. thanks for clarification, it can actually still go on and on until it took up 3GB for 1 day

It’s definitely not normal.

1 Like

worst thing is I have no idea where to start the troubleshooting… thats why im here to seek for some advises

bugger
I use the google one

image

dum qusetion
what your config options

1 Like

my google drive running out of space sadly… haha

Here are my configs

Configuration.yaml:

recorder:
  purge_keep_days: 1
  purge_interval: 1

auto_backup:
  auto_purge: true

Automation.yaml

- alias: Perform Daily Backup
  trigger:
    platform: time
    at: 05:00:00
  action:
    service: auto_backup.snapshot_full
    data_template:
      name: 'DailyBackup: {{ now().strftime(''%A, %B %-d, %Y'') }}'
      keep_days: 7

That’s pretty broken, not sure how you got all those subdirectories, but there should only be *.tar files in the root /backup directory.
I don’t see any issue with your configuration.

I think it’s quite weird that every subdirectory is the same and that it appears to go on forever, so my guess is actually that somehow a directory junction (or hard link) has been used to create a loop. That is that /backup/backup actually links to /backup but the OS doesn’t perceive it that way and so it appears as infinite subdirectories.

To check this open the windows command prompt in the /backup directory and run the command dir, you looking for something saying <JUNCTION>.

You can try deleting anything that is not a *.tar file in the root /backup directory, and see if the problem persists (i.e. it’s still creating subfolders). It’s also worth checking the home assistant logs and supervisor logs around the time you take a snapshot. Are you running any other backup utilities that may be conflicting?

1 Like

Hi Josh,

Thanks for responding!
I’m not using any other backup utilities.

I tried to DIR and look for junction but didnt find anything

image

Although, I found that there’s a link after
docker exec -it hassio_supervisor bash

image

hmm this seems to be the issue!
After rm -rf it, and re-initiate another snapshot, the problem is gone!

To all:
Appreciate for everyone’s tips!

Kudos!

Great! so it was a circular link on the linux side, it’s weird that was even created, it couldn’t have been home assistant or any of its components, as they don’t have access to the /backup folder, so maybe the supervisor or an addon caused it.

Hi @jcwillox ,

Now I have another issue, after deleting the backup folders,
Supervisor > Snapshot still show there are 3 snapshots but in backup directory there’s only 1

image

and Autobackup shows even more
image

I tried to click on Supervisor>snapshot>delete, nothing happened.

Would appreciate if anyone can advice on how do I delete these records?

Auto Backup shows a snapshot as monitored until it expires even if it’s deleted externally, you could manually edit/delete the file under /config/.storage/auto_backup.snapshot_expiry and restart.
As for the supervisor it creates the list of snapshots by reading from the /backup directory. There should be a reload button in the top right corner, otherwise a restart should fix it.

Hi Josh,

Awesome, it solved all the problem after deleting the auto_backup.snapshot_expiry and restart!

Million thanks!
kudos!

1 Like

@kavierkoo I had a similar problem recently but it was with my config folder. I only discovered it after the Visual Studio Code plugin started pegging my i5 NUC CPU useage at 100%. Turned out that it was disappearing down the infinite plughole of the config folder enumerating files as it went! I couldn’t get rm -rf to remove the link, so I exported a backup and used 7-zip to delete it. Once I did a wipe & restore, it was fine.

However, on checking today I find that my backup folder has the same problem as you, but this time the rm -rf worked, but it is a strange problem!

@sota yea lol… When I run hassio directly on Ubuntu, I never have this issue,
It only happen after I dockerize it…
Are you running on linux? or Docker?

Let’s see if we have any similarity here.
I’m running it on Docker, and Addons are:

  • AdGuard Home
  • SSH & Web Terminal
  • Samba Share
  • HACS
  • Auto Backup
  • and few of frontend cards.

I use automation to snapshot daily 5 AM and keep for 7 days, and uses auto backup more insight info.

I’m running HA Supervised under Ubuntu Server 18.04 along with about 10 or 12 other containers under docker compose.
I noticed the problem because the NUCs fan was running flat-out after installing VS Code. Other than that, I have:
Appdaemon 4
ESPHome
Google Drive Backup which runs every night
Let’s Encrypt
MariaDB
Mosquitto Broker
Node-Red
SSH & Web Terminal
Samba
Zigbee2mqtt

I suspect that file corruption may be at the root of the problem. However, it’s almost impossible to prove so I guess I’ll just keep an eye on it for now. I am planning to move to Proxmox, so this may force me to get on with it.

I see, I noticed mine after seeing the HDD just keep increasing on lovelace dashboard,

Hmm, the common thing I guess was we both running HA Supervised with Docker on the same OS and version too.

and yea, we can only monitor it for now. Good info about Proxmox… but I think the dashboard not as elegant compare to HA (just my opinion)… I do use Netdata and customize my own small dashboard to monitor my webserver tho.


(Still having trouble with Netdata on docker not able to collect logs from HDDTEMP)