I would like to propose the possibility of making incremental backups. I personally use an SD card in my Raspberry Pi and this type of backups can be of great use in this type of systems.
Thinking more on it, I don’t know if it is possible, but incremental backups in real time for regular files and daily backup for database can be a good strategy.
Thank you for reading.
But wouldn’t that mean that you would need all your backup files in order to restore?
So backup 1 is 1 GB with everything.
Backup 2 is 2 MB
Backup 3 is 10 MB
In order to restore to previous state you will need all three files?
Not sure that is going to be good.
Someone might delete the “old backup” because I don’t need that anymore and it’s just taking up a lot of space.
I think it depends on how is built in front and backend, also on user knowledge. I’m not an expert on this, but in my experience I find this type of backup strategy to be faster than standard backup.
My backup is round 100Mb and it takes several minutes to finish, but I’m afraid with Gb information to take much much longer.
Probably a solution also is to give this as an option to choose instead of turning it in the standard solution.
100MB is pretty low. Main part of backup in history database - mine has over 2GB. It’s size depends on number of entites and history days (default 10), so my backups were well over 2GB big. But, i have my HA in VM on Syno, so space or SD wear-out is not a problem, but time is… when HA updated it took ages for backup to finish. So i moved my history to MariaDB on Synology, so now my backups are under 100MB.
So, the main question is if it would work at all… as said, the biggest part of backup is history database and i doubt that this file can be backed-up partly only as a difference between two. And, since even in an incremental backup this file would be always copied (since it’s ALWAYS different from previous one) the whole thing looses it’s sense.
I don’t see the point of incremental backups considering the ridiculous volume of data (even with large HA systems) that is backuped ! All the more incremental backups imply a strict policy to restore and qreater risk of mistakes when restored !
I don’t quite understand you… by “ridiculous volume of data” - do you mean ridiculously low or ridiculously high? As said, full backup can be quite large. Even in my VM on DS920+ Syno it took over 15 minutes to finish - that’s why i moved my history DB out of HA → to eliminate it from backup. I can’t even imagine how long it takes on low-end hardware like Pi3…
I guess i voted too soon. I’m not a fan of incremental backup, i would only like to see a smaller backups. Maybe a better solution would be to choose either DB is included in backup or not, or even exclude it by default. Restoring history DB if file is more or less useless anyway, since it won’t reflect last histroy, but old one.
What’s important is configuration, not history. Default DB is very vunerable anyway. It’s enough that you connect to DB with another SQL program and it’s considered corrupt…
I agree backup strategy deserves a good discussion, and is a possible area for improvement.
Incremental backups are a common solution, and certainly one option. Generally, full backups are for disaster recovery or archival use, while incrementals are for restoring things which were damaged or inadvertently deleted. I think in HA the focus has been on DR (I bricked my system) rather than restoring individual files or components. Hence no incremental option.
The problem of course is that the majority of users are running a RPi, and many of those use an SD card. Obviously power users, like developers, don’t. So although it’s presented as the beginner solution, this environment doesn’t get the same love from developers. Doing many large, full backups on an SD card is a bad idea.
My solution has been to use a file sync app on my laptop to copy just the modified files from the config directory (which includes the database) occasionally. This way I can do fewer full backups (saving space and I/O on my SD card) and still be able to restore, say, the latest config file after restoring a full backup.
As was pointed out above, the database is probably useless after a crash anyway. Because of this, I keep the retention short (4 days), exclude most data from Recorder and store the long-term data I need in text files, which get saved elsewhere when I sync. I suppose you could say those are my incremental backups. (I’ve said it before, but to me Recorder needs a serious re-think and re-write.)
I’ll briefly mention archival backup, which is another possible reason for incrementals, but HA doesn’t typically run in an environment with records retention requirements, so I’ll leave that option alone.
I use THIS plugin for storing backups to another location (in my case it’s NAS). Backup on SD card can be useless - if SD card goes dead or corrupt, then you won’t be able to get backup file from it anyway.
I agree with Tom that these backups are more for DR than anything else and that’s why it’s definitely no need for to database to be included.
Ridiculously low 
and backup taking so much time to backup so little data volume should interrogate you on avalaible ressources of your instance Side note: a local backup made on an SD Card is not a backup, it’s just playing loto there 
Yep you are right the most important for most users is backup of configuration to be able to restore system in working condition ! For DB you can backup it easily either using proper SQL commands on it or dump of it (done in proper conditions or it won’t be restorable !).
If when you connect at db you corrup it, you don’t use the proper tool or way to do it !
For backup I use here since months a great plugin hassio-nextcloud-backup/README.md at master · Sebclem/hassio-nextcloud-backup · GitHub that automatically uploads every night either a full or incremental backup to my Nextcloud instance (so offsite backup and data still safe as backuped in my personal storage system).