DB image malformed. How to fix it?

Installation Home Assistant OS
21

I went with an external mysql database instead of the sqlite one. Is a bit snappier (history,…) and no so to prone to errors.

1 Like
22

Thanks, saved my logs after a restore as well. This took about 3-5 minutes on a Raspberry Pi 4 with a 1GB Sqlite DB. Resulting DB was 22 MB, so I guess it threw a lot of corrupted data out.

For those struggling with the “install sqlite3 in the homeassistant container” - no need on Hass.io (or whatever it’s being called now.)

If you enable the root login (don’t bother with the SSH addon, it doesn’t get you access to the low level OS the way the real root login does), you can do the fixes from the main shell of the system.

The config directory is available in /mnt/data/supervisor/homeassistant

See https://developers.home-assistant.io/docs/operating-system/debugging#ssh-access-to-the-host for instructions on how to enable SSH root logins.

On a recent release, here’s how I did the recovery.

Prerequisites:

Enable SSH access to the host per procedure linked above.

Step 1: Login to the host

ssh -l root -p 22222 hassio

It will bring up the hassio-cli interpreter.

Step 2: Stop Hassio

Type core stop and wait for it to complete.

Step 3: Change directories to the data dir

  • Type login to get the root prompt.
  • Then, run
    cd /mnt/data/supervisor/homeassistant/

Step 4: Recover the SQLite database

Run the following, then grab a cup of coffee/tea/beer.

sqlite3 ./home-assistant_v2.db ".dump" | sqlite3 ./home-assistant_v2.db.fix
mv ./home-assistant_v2.db ./home-assistant_v2.db.broken
mv ./home-assistant_v2.db.fix ./home-assistant_v2.db

Step 5: Get back into the hassio-cli
Type hassio-cli to re-enter the interpreter

Step 6: Restart Home Assistant

core start

Validation:

Exit out of the hassio-cli by typing exit, then watch the homeassistant container’s logs:

docker logs --follow homeassistant
9 Likes
23

Same here, just an empty file as an output :frowning_face:

24

lost all the data, but was fine for me! thanks!

25

tried that too - empty file as an output :frowning:
Tried to copy the DB manually from 2nd install - I’m migrating now - but that ended up with completely messed UP HA. Entered into kind-of safe mode… Had to restore from snapshot again…

26

Same issue and fix was an empty file. Had to toss history and start from scratch. Is there a way to recover data from the corrupted database using JupyterLab. I have not tried it, but maybe will give it a shot.

27

As I’m in the process of migrating from Hassio installed directly on external SSD (which is not recommended as I found out) to Raspbian installation with Hassio on docker I tried following thing.
So, now I have old setup on SSD and I’m setting up new one on Raspbian on SD card. I did follwoing:

  • booted old SSD setup
  • created another snapshot and downloaded it to my PC
  • booted new sd card setup
  • restored it with “restore selected” option (no wipe)

Now, I my history is there. I think recently I did the “wipe and restore” option… I think. But honestly - I dont know why now its here :slight_smile:

28

@eddriesen thanks for this! Unfortunately like some others, I was getting a 0 byte fixed database with this method. Using the newer sqlite3 (v3.29.0+) .recover command though, I managed to retrieve a sizable amount. Leaving this here for others (and myself inevitably) in the future:

sqlite3 ./home-assistant_v2.db ".recover" | sqlite3 ./home-assistant_v2.db.fix
mv ./home-assistant_v2.db ./home-assistant_v2.db.broken
mv ./home-assistant_v2.db.fix ./home-assistant_v2.db
14 Likes
29

Hello
Try this command that worked well for me :

sqlite3 ./home-assistant_v2_old.db “.dump” | sed -e ‘s|^ROLLBACK;( – due to errors)*$|COMMIT;|g’ | sqlite3 ./home-assistant_v2.db

30

This works for me too, when getting a zero size result with the dump method.

Unfortunately, in my case, the database is always corrupted after restoring a snapshot.

31

this gives me a “line xy database or filesystem full” error.
However, filesystem got 10g free space and db size is 1,2g so there should be plenty of space.
any idea?

I came to this error after moving from supervised installation to hassos and restoring from a snapshot. its a bit sad, if i need to expect this after each restore.

32

@Filoni Thanks for the info. Working like a charm.
I had to compile sqlite3.30 (since I’m on Buster on my RaspberryPi3) from source but after that initial effort, I could go from a 1.2GB (broken) to a 500MB (working fine) DB.

33

Great!! Working like a charm!!

34

“.recover” worked well for me! However, I had to copy it down to my Windows machine and do it there, because on my RPi:
“.dump” threw these errors (despite having 6.2GB free):
Error: near line 1877906: database or disk is full
and “.recover” said this:
SQL error: no such table: sqlite_dbpage

Thx again!!

2 Likes
35

Hi Filoni, thanks for the hint. Worked for me after restoring a snapshot gave me tons of errors in the logs.

36

Thanks, I referenced your post on Fix corrupted / malformed SQLite database (home-assistant_v2.db) :+1:

37

Hi! Just saw this in my logs too…
Is it safe to just delete the db?

The db is just for storing values right? No settings and other stuff get delete?

38

Same here! After restoring a snapshot it works initially until you reboot.

39

If your history is sooooooo important, why don’t you use what you need to keep to an influxdb?? I have lost count of the number of times I have had to delete my database for one reason or another… It’s just history anyway…

40

:rofl:

Some care, some don´t :man_shrugging: