Backup - how to exclude media?

Configuration
1

I deleted some old frigate clips a few weeks ago and today I discovered that my nightly backups have not been working.

Error from “Create Backup”:
Failed to to call /backups/new/full - Unknown error, see supervisor

From the Supervisor log:

Supervisor
    await self.sys_run_in_executor(_folder_save, folder)
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/supervisor/supervisor/backups/backup.py", line 442, in _folder_save
    atomic_contents_add(
  File "/usr/local/lib/python3.11/site-packages/securetar/__init__.py", line 188, in atomic_contents_add
    atomic_contents_add(tar_file, directory_item, excludes, arcpath)
  File "/usr/local/lib/python3.11/site-packages/securetar/__init__.py", line 188, in atomic_contents_add
    atomic_contents_add(tar_file, directory_item, excludes, arcpath)
  File "/usr/local/lib/python3.11/site-packages/securetar/__init__.py", line 191, in atomic_contents_add
    tar_file.add(directory_item.as_posix(), arcname=arcpath, recursive=False)
  File "/usr/local/lib/python3.11/tarfile.py", line 2157, in add
    tarinfo = self.gettarinfo(name, arcname)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 2030, in gettarinfo
    statres = os.lstat(name)
              ^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/data/media/frigate/clips/driveway-1691343383.672016-p6n15h-clean.png'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/src/supervisor/supervisor/backups/manager.py", line 210, in _do_backup
    await backup.store_folders(folder_list)
  File "/usr/src/supervisor/supervisor/backups/backup.py", line 462, in store_folders
    raise BackupError(
supervisor.exceptions.BackupError: Can't backup folder media: [Errno 2] No such file or directory: '/data/media/frigate/clips/driveway-1691343383.672016-p6n15h-clean.png'

So, it looks like the problem is the deleted Frigate clips. Is there a databse in Frigate that I may have corrupted by deleting old clips?

I added this to my configuration.yaml:

backup:
  folder: /home/user/homeassistant/config/backup
  excludes:
    - media

I can now make a backup, but it is 18Gb, so the media folder is still being backed up? A partial backup of everything except media is only 587Mb.

So, how do I automatically back up everything except the media folder?

========== UPDATE ==========

Well, my backup: lines are wrong, but I don’t know why?

invalid

2

Not sure about this, but what happens if you specify /data/media/ instead of -media?

3

It seems that anything I put there gives me a configuration error.
Ironically, running the check configuration gives me a green check, but restarting Home Assistant gives me a configuration error.

4

No answer? How do I exclude the media folder from backups?

5

I can’t find anything related to excluding folders in the documention, seems it’s not supported.

6

I’m not entirely sure, what you’re doing here. :slight_smile:

As it seems, you’re running HA-OS or at least supervised (hence the supervisor log you stated). If so, the backup: integration is already part of the system, so no need to get it into configuration.yaml. But if you need to set it (because of another OS), you can only set backup: with no additional info, it’s just that one word, nothing else.

7

So, I am stuck with 30GB backups because of Frigate? Huge backups are not a problem because I have a 500Gb SSD on an Intel NUC, but it takes 30 minutes to make it.

When it works. But I can’t even do a full backup now because of this error:

supervisor.exceptions.BackupError: Can't backup folder media: [Errno 2] No such file or directory: '/data/media/frigate/clips/driveway-1691343383.672016-p6n15h-clean.png'

If I do a partial backup excluding media, it works fine, except I can’t do an automated partial backup. Yes, there is a service.partial_backup but I have to list every folder and add-on in the yaml file. If I install a new add-on, I have to manually go back to the automation to add it to the add-on list. (Plus I have to figure out the yaml syntax for each add-on).

Maybe I am chasing the wrong issue.
Why is backup even looking for specific files? Isn’t backup just doing a tar -czf of my Home Assistant folders?

What’s wrong with Supervisor?

8

I am still looking for where I saw this in the documentation. I do see that someone asked for this years ago in features request.

9

Nothing, you just don’t need to set backup: in your configuration.yaml as it is automatically included. :slight_smile: And I’m not sure, how the system reacts, if you set it despite it is already included. :wink:

How do you even do the backups? I’m using the GoogleDriveBackup Add-on (very good btw.), but if I do a manual backup via the UI, I can select “partial backup” and just unmark the Media folder. I guess that’s not what you’re doing here, so is it an automation?

10

I get a configuration error. But only after a restart, not from the check configuration service call.

I do a partial Samba Backup every 1:30AM, but for unknown reasons I discovered that there were no backups for two weeks. Samba backup seems to be working again, and I haven’t changed anything.

I also have an automation that calls the full backup service call at 03:00AM. This is the one that is failing.

I don’t like having single points of failure so I do a snapshot backup before changing anything. Full backups fail, while partial without the Media folder works.

WHY should “missing files” in the Media folder be causing the failure of a full backup?

11

Yes, but you get the configuration error, because you’re trying to set a configuration, that is not valid (only backup: - nothing else). I’m not sure if you should set a correct (only backup:) configuration, despite this integration is already set by HA as default.

This is wrong:

backup:
  something

This is valid configuration, but already included by the system, so not sure if this is good:

backup:

Not sure about the files though, that seems to be a totally different thing. My first steps would be to check the space, where the backup should be safed. File system can handle bigger files (eg. not FAT formatted)? Is enough space on the partition where HA is installed. As far as I know, the backup is made on the HA drive and is than copied over. So on both systems/partions should be enough space. Try a different time for the automation than 03:00, and check the automation trace, maybe there is something pointing into one direction. The other logs as well. And while at it, enable the debug for the backup integration.

logger:
  logs:
    homeassistant.components.backup: debug

Why it is searching for files that are not there seems odd. Might want to post the automation? :slight_smile:

Sorry, no concrete help here, but maybe some more light in the dark. :wink:

12

I think this is the root of all my issues.
The automation s a simple at a time trigger call service.full_backup
When I go to:
Settings, System, Backups, Create Backup, Full
I get this error (after fifteen or twenty minutes):
unknown error

When I go to the Supervisor logs, I see:

23-08-09 14:33:27 ERROR (MainThread) [supervisor.backups.backup] Can't backup folder media: [Errno 2] No such file or directory: '/data/media/frigate/clips/porch-1690741710.696236-n1z3pz-clean.png'
23-08-09 14:33:28 ERROR (MainThread) [supervisor.backups.manager] Backup a25deeba error

If I do a partial backup of everything but media, the backup works fine.

13

I use duplicati for backup. I run it as system service not a container. This works just fine. It’s not a ha backup but you just set it up and forget that it’s there.

14

I’m not using Frigate, so someone else, who is actually using it, might be better to help.

Anyway, if you have deleted some pictures, I’m quite sure the database from Frigate must be synced/updated. As I said, not a user, but the config options from the Add-on clearly set a database path and name, so it seems safe to assume, that if pics get deleted, the DB should know.

I have no idea how to solve this, but check, if the Add-on is offering some “clean up” options. And, as well just guessing, as the media folder is organized by HA, there might be another failure point there.

Nonetheless, what I’d do: disable the integration, stop the Add-on and restart HA. Then re-enable and see, if the database gets synced. :wink:

EDIT: For future reference, seems this topic is going along in another topic: