Home Assistant OS won't boot anymore

I’ve been running HA OS for a while and suddenly I lost connectivity. It’s running on a RPi 4b, the green and red LED show activity, the Rpi side ethernet port LEDs are dark but I see a link on the router side. When I connect a HDMI cable, I don’t see anything, monitor doesn’t even say it’s connected to anything (caveat, I have never used an external monitor on that installation so it might just not be set up). I tried 2 different RPi 4b with the same SD card.

I put the SD card in another linux machine but my debugging skills are still limited.
When I run fsck I get the following output:

xxxx@xxxx:~$ sudo fsck /dev/sde
fsck from util-linux 2.37.2
e2fsck 1.46.5 (30-Dec-2021)
ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/sde

The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
 or
    e2fsck -b 32768 <device>

Found a gpt partition table in /dev/sde

Here’s what parted says:

Number  Start   End     Size    File system  Name              Flags
 1      1049kB  34.6MB  33.6MB  fat16        hassos-boot       msftres
 2      34.6MB  59.8MB  25.2MB               hassos-kernel0
 3      59.8MB  328MB   268MB                hassos-system0
 4      328MB   353MB   25.2MB               hassos-kernel1
 5      353MB   622MB   268MB                hassos-system1
 6      622MB   630MB   8389kB               hassos-bootstate
 7      630MB   731MB   101MB   ext4         hassos-overlay
 8      731MB   63.9GB  63.1GB  ext4         hassos-data

Trying to find how much free space I got I tried doing this which does show some partitions are at 100%:

xxxx@xxxx:~$ sudo mount /dev/sde1 tmp/ && df -h tmp/ && sudo umount tmp
Filesystem      Size  Used Avail Use% Mounted on
/dev/sde1        32M  7.5M   25M  24% /home/xxxx/tmp
xxxx@xxxx:~$ sudo mount /dev/sde2 tmp/ && df -h tmp/ && sudo umount tmp
Filesystem      Size  Used Avail Use% Mounted on
/dev/sde2       9.7M  9.7M     0 100% /home/xxxx/tmp
xxxx@xxxx:~$ sudo mount /dev/sde3 tmp/ && df -h tmp/ && sudo umount tmp
Filesystem      Size  Used Avail Use% Mounted on
/dev/sde3       132M  132M     0 100% /home/xxxx/tmp
xxxx@xxxx:~$ sudo mount /dev/sde4 tmp/ && df -h tmp/ && sudo umount tmp
Filesystem      Size  Used Avail Use% Mounted on
/dev/sde4       9.8M  9.8M     0 100% /home/xxxx/tmp
xxxx@xxxx:~$ sudo mount /dev/sde5 tmp/ && df -h tmp/ && sudo umount tmp
Filesystem      Size  Used Avail Use% Mounted on
/dev/sde5       132M  132M     0 100% /home/xxxx/tmp
xxxx@xxxx:~$ sudo mount /dev/sde6 tmp/ && df -h tmp/ && sudo umount tmp
mount: /home/xxxx/tmp: wrong fs type, bad option, bad superblock on /dev/sde6, missing codepage or helper program, or other error.
xxxx@xxxx:~$ sudo mount /dev/sde7 tmp/ && df -h tmp/ && sudo umount tmp
Filesystem      Size  Used Avail Use% Mounted on
/dev/sde7        85M  359K   78M   1% /home/xxxx/tmp
xxxx@xxxx:~$ sudo mount /dev/sde8 tmp/ && df -h tmp/ && sudo umount tmp
Filesystem      Size  Used Avail Use% Mounted on
/dev/sde8        58G   17G   40G  30% /home/xxxx/tmp
xxxx@xxxx:~$

According to Partitioning | Home Assistant Developer Docs, /dev/sde6 should be the Bootstate partition, so that could be a reason why I can’t mount it. Some of the partitions do show 100% use but I’m not sure if that’s a red herring or actually the root cause, e.g. some might be swap partitions for which 100% use might make sense.

Does anyone know what this means for the state/health of the file system or how I could find out more?
I appreciate any help, thanks!

The RPI must detect the hdmi connection in order to use it, so connect the cable and then reboot the RPI.

Good suggestion :+1:, still no signal though. Both RPi.

No HDMI signal from your device

Your SD card is hosed. Re flash Home Assistant then restore yesterday’s backup.

This can happen if the power is removed from the Pi without a proper shutdown of the OS. It can be fixed using SuperGrub, but the easiest is to simply reflash the SD card and restore your latest backup.