HASS.IO on Rpi3b+ Crashed - trying to recover

Friends,

My HASS.IO crashed. On the console I just get repeated messages about docker0 port 1 disabled state and the like. Searching the interwebz leads me to believe that my SD card may be to blame. However, I am trying to see if I can get any config off of that SD card by mounting it on a centos box.

Disk /dev/mmcblk0: 32.0 GB, 32010928128 bytes, 62521344 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x136a9aa4

        Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1   *        2048       67583       32768    c  W95 FAT32 (LBA)
/dev/mmcblk0p2               1        2047        1023+  ee  GPT

I can mount the p1 and p2, but neither seem to be actual useful data:

root-> ls -l
total 3306
-rwxr-xr-x 1 root root   25311 Nov 21 06:14 bcm2710-rpi-3-b.dtb
-rwxr-xr-x 1 root root   25574 Nov 21 06:14 bcm2710-rpi-3-b-plus.dtb
-rwxr-xr-x 1 root root   24087 Nov 21 06:14 bcm2710-rpi-cm3.dtb
-rwxr-xr-x 1 root root   52116 Nov 21 06:14 bootcode.bin
-rwxr-xr-x 1 root root    1847 Nov 21 06:14 boot.scr
-rwxr-xr-x 1 root root      34 Nov 21 06:14 cmdline.txt
-rwxr-xr-x 1 root root      52 Nov 21 06:14 config.txt
-rwxr-xr-x 1 root root    6666 Nov 21 06:14 fixup.dat
drwxr-xr-x 2 root root   14336 Nov 21 06:14 overlays
-rwxr-xr-x 1 root root 2857060 Nov 21 06:14 start.elf
drwxr-xr-x 2 root root    2048 Dec 16 11:00 System Volume Information
-rwxr-xr-x 1 root root  364060 Nov 21 06:14 u-boot.bin

Any ideas where the actual data resides? I’d love to pull off the configs if not the entire recent snapshots… I got lazy and didn’t backup the recent snaps. /facepalm

Any hints?

TIA

Just a guess that is a HassOs image. You could either try navigating down the System Volume directory or use the Linux find command to look for configuration.yaml.

I have the same problem. Hassio has already crashed 2 times tonight what i wanted to say the memory card is brand new

What type of card?

It’s a Samsung 32GB MicroSDHC EVO Select

Thanks for the suggestion but no joy. That directory contains:

-rwxr-xr-x 1 root root    76 Apr 10 16:15 IndexerVolumeGuid
-rwxr-xr-x 1 root root    12 Dec 16 11:00 WPSettings.dat

Meanwhile /dev/mmcblk0p2 contains:

drwx------ 2 root root   12288 Nov 21 06:14 lost+found
-rw-r--r-- 1 root root 6713280 Nov 21 06:14 zImage

I’m guessing that the HASS.IO image is some sort of compressed image? Any idea how to delve deeper?

Likely a compressed image.
lost+found is where Unix based systems put “broken” files (lost inodes) in case you can extract any useful information from them.

I don’t work in a docker. I have HASS installed in a venv using Raspbian Stretch. What I do periodically is take an image of the SD card particularly before an update. In the event of hosing HA I just reimage the SD card and I’m back in business.

Also I have a second SD card with identical setup located close to the Raspberry Pi. Again if the active system crashes I just pop out the offending SD card and pop in the replacement.

I’m wondering if it’s the physical SD card at fault or just your file structure. You could try taking an image of the relevant SD card and restore that image to a new SD card. Not sure it would work but it may be worth a try. As I said I don’t have docker experience so it’s just a suggestion

If the Pi lost power it could very well be a corrupt system. It there are files in lost+found I could almost guarantee filesystem corruption.

lost+found is empty.

Anyone familiar with these images? Is there a way to extract zImage? I tried several suggested things with no success. Not that the image is failing, its more like nothing recognizes that as any sort of extractable image.

Actually, I’m now not convinced I’m looking at the entire flash.

/dev/mmcblk0p1            32M  3.7M   29M  12% /mnt
/dev/mmcblk0p2            23M  6.8M   14M  34% /mnt2

Those partitions are tiny, so maybe I just have a missing/failed partition

What dies a disk partitioning program say?

OK, so there are a lot more partitions on there than dmesg listed. An lsblk gave me a lot more. The largest one (mmcblk0p8) being 29g in size. Inside of that is:

drwxr-xr-x   6 root root  4096 Jan 28  2018 .
dr-xr-xr-x. 18 root root   274 Apr 10 23:13 ..
drwxr-xr-x   2 root root  4096 Jan 28  2018 cli
drwx--x--x  14 root root  4096 Apr 10 22:11 docker
-rw-r--r--   1 root root   393 Nov 21 06:14 hassos.json
drwx------   2 root root 16384 Nov 21 06:13 lost+found
drwxr-xr-x   9 root root  4096 Dec 18 09:16 supervisor

With the docker directory holding 4.3g of ~stuff~. I’m a total docker newbie so I’m now digging into a way to open those docker images with another system to see if I can extract something useful.

And the good news: the “supervisor” directory had all my config (/supervisor/homeassistant) and a backup image (/supervisor/backup).

I’ll now attempt to rebuild the system and restore from backup.

Thanks for all the tips and pointers so far!

1 Like

I think the docker containers should not have any useful non-standard information.In fact, an upgrade replaces those containers.

Thanks bosborne, that is helpful.

Good news is that I was able to restore from a recent backup and I lost very little. Now, to get another SD card and start doing some better backup routines.

Also, I do see a lot of complaints about corrupted SD cards… is this a known flaw? Is there any way to avoid it other than just choosing a different platform to run hassio on?

I have had a good experience following the Pi group’s recommendation and using the SD Formatter when reusing a card. Some here had it work on cards they thought were bad.

Did you have a power interruption? If not shut down cleanly, any Unix-like system can have filesystem corruption. It is a filesystem design, not an SD Card issue. The solution would be to have a UPS that signals the Pi to shut down cleanly. I do not believe that can easily be added to Hassio though.

This looks promising for Raspbian.

https://wiki.debian.org/apcupsd