Lost history after updating to 2024.10

I use purge_keep_days in configuration file to store data for custom mqtt sensors for up to 10 years and after update I have lost data from anything before February 2024

I also have a full backup from about a week back, will restoring it bring back my data? As I tried the backup that automatically gets created from updating and it did not restore my data…

Hi John,
Not sure what happened, but if your data is truly not there after a full host reboot, I suggest going to your backup and reloading it.

As an additional point, saving 10 years worth of data in a database that is intended to be used live for quick processing of events is not a good idea.
If you cannot use the LTS service and set these numbers up with a state_class, then I encourage you to look into influxDB and put a copy of your live data there, then use graphana to graph it there. Leave the local HA database to the default settings of 10 days with LTS after that and you will have fewer problems dealing with a giant unwieldy database file that is always in risk of being lost like this.

1 Like

If I create a backup now can I load it after I have restored a previous backup from a week ago?

Sure, but it will restore the empty database again.
It won’t merge anything.

Data hadn’t been restored after restoring back up so I think that something else is the issue now but I would prefer not to use grafana as I don’t want the hassle of setting it all up and home assistant worked just fine until recently

I also noticed my installation is around 35gb which is excluding the backups and the full backup it creates is only 6gb?

Was thinking the backup might only hold so much history?

No, it just copies the entire home-assistant_v2.db file, along with the rest.

You’ve run into a design limitation of HA. Both the “live” and “archival” data are stored in the same database. There are a number of benefits to this, but also a number of drawbacks. One is that there are occasional database schema changes performed during version updates. These can get quite CPU and I/O intensive, slowing down the update or even causing failures, if your database is very large.

I don’t know if that’s the case this time, but I did notice this in the change notice blog:

The statistics developer tools have been improved. They now have the new-style data tables…