Recorder not working, Sqlite not closed properly, no access to any historical data

After updating to the latest core HA, the system would not start, except in safe mode.

Managed to get the system running with a backup of one week ago.

All seems to work, except that the recorder does not work, with a message in the logs:
''' The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly '''

At present no access to the history, no access to energy data, no access to any historical records.

If someone has an idea - pls let me know ...

If you take a peak in /config, you most likely find a homeassistant.db.CORRUPT

Your "History" is basically History , and the Recorder start from scratch ( Im sure it's working )

Thanks for responding.

I did not find a .CORRUPT file, just the normal database stuff.

Or do you mean it IS CORRUPT (rather than being named ".CORRUPT")?

/config ls -l home-assistant*.*
-rw-r--r--    1 root     root     5124157440 Jun 23 08:55 home-assistant_v2.db
-rw-r--r--    1 root     root         98304 Jun 23 08:58 home-assistant_v2.db-shm
-rw-r--r--    1 root     root       4194192 Jun 23 08:58 home-assistant_v2.db-wal

And it does not seem to store anything. If this is because the recorder is or is-not working I do not know. All I get is present live data. No logging, or no access to logging, nor history

>> Any way to recover from this ?? The years of history are worth a bunch ....

And if NOT:

>> restart from scratch with new dB, and then restore backup ?

No, we actually mean a file named "*corrupt"

With a 5GB database, that one is definitely not a brand new one.
Stating the obvious, but you won't have history for the past week, i.e. since the backup.

That's a warning you can ignore. It just means the Db was not closed cleanly.

Just notice the current Size, and take a peak before you go to bed, Daily Purge Occur around 4 am ( Not sure the exact time ) , so check in again in the morning ... i guess that's the easiest way, another is using i.e sqlite-browser

( And if you have scripts and templates etc. , requesting datas ( i.e. from past week or 2, they will not find any, and causing errors and warnings )

PS: Default Purge is 10 Days, and as ChrisB says those 2-3 week "statistics" are gone and start from scratch ( or built up upon the 2-3 week old LT-DB-Datas "leaving a gap" )

I believe you, thou i have also no idea here, as you don't show anything, im sure you are not just telling a story, or :slight_smile:

OOH Wait , i just notice

Does that means when you try to access System/Settings/Logs it's just shows a "Spinning thingy" ?
In that case i had same issue few releases ago :frowning: , i believed it was do to me not following my "procedures" as usual ...i.e checking logs before Updating, and fix and perform a restart, before upgrading ( Lesson Learned )

So before you go any further, post your About# System Ver. screenshot ( Also from which core etc You upgraded from )
And check your Core-Log, prior to your first update attempt

Your Lesson would be ( Backup Before Update ) Then you'll only "loose" a few hours of datas

All,

I must admit (with embarrassment), that I somehow put an "include" statement for the recorder in the configuration.yaml, with very few sensors. :frowning:

As I was having trouble with HA restarts, I had not done any restart for a month, and somehow "forgot" what I had done... thus when there was a restart, the result was no Recording on 99.9 % of the variables. Removing the "Include" statement returned everything normal.

The problem was entirely my own doing... Thanks for your willingness to help...

Bernard

PS: As I am not good at databases, and equally did not want to loose the history of the past years, I sought the help of ChatGPT. Must say quite enlightening. Totally assisted this NOOB to troubleshoot and narrow down the issue...

great that you figured it out, and yes as "a 61" im 65, and have "to much" in my head, so i can relate to "forget&forgot" :laughing: