Corrupt GPT partition table, can't read superblock

Hi!

After having my Raspberry Pi with home assistant plugged out of the powerlines for two weeks, I have trouble getting it to boot back up again. Before unplugging it, I had no issues.

When connecting the sdcard to a Linux laptop, and using fdisk -l, I get the following:

The backup GPT table is corrupt, but the primary appears OK, so that will be used.


Disk /dev/mmcblk0: 29.72 GiB, 31914983424 bytes, 62333952 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 1ED17572-FEA5-4C23-82C3-57C2F576F6B3

Device           Start      End  Sectors  Size Type
/dev/mmcblk0p1    2048    67583    65536   32M Microsoft reserved
/dev/mmcblk0p2   67584   116735    49152   24M Linux filesystem
/dev/mmcblk0p3  116736   641023   524288  256M Linux filesystem
/dev/mmcblk0p4  641024   690175    49152   24M Linux filesystem
/dev/mmcblk0p5  690176  1214463   524288  256M Linux filesystem
/dev/mmcblk0p6 1214464  1230847    16384    8M Linux filesystem
/dev/mmcblk0p7 1230848  1427455   196608   96M Linux filesystem
/dev/mmcblk0p8 1427456 62333918 60906463   29G Linux filesystem

Also, I can only mount the first (fat) partition, and not any of the other 7 (p2 to p8). It throws me an error about the superblocks:

ubuntu@ubuntu:~$ sudo mount /dev/mmcblk0p2 /media/sdcard
mount: /media/sdcard: can't read superblock on /dev/mmcblk0p2.
ubuntu@ubuntu:~$ sudo mount /dev/mmcblk0p8 /media/sdcard
mount: /media/sdcard: wrong fs type, bad option, bad superblock on /dev/mmcblk0p8, missing codepage or helper program, or other error.

After some Googling I came across gdisk, so I ran sudo gdisk /dev/mmcblk0 and got the message that the backup header and backup partition table have an error. Therefore, I tried a standard validate (-v) and write (-w) to fix it:

ubuntu@ubuntu:~$ sudo gdisk /dev/mmcblk0
GPT fdisk (gdisk) version 1.0.8

Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
on the recovery & transformation menu to examine the two tables.

Warning! One or more CRCs don't match. You should repair the disk!
Main header: OK
Backup header: ERROR
Main partition table: OK
Backup partition table: ERROR

Partition table scan:
  MBR: hybrid
  BSD: not present
  APM: not present
  GPT: damaged

Found valid MBR and corrupt GPT. Which do you want to use? (Using the
GPT MAY permit recovery of GPT data.)
 1 - MBR
 2 - GPT
 3 - Create blank GPT

Your answer: 2

Command (? for help): p
Disk /dev/mmcblk0: 62333952 sectors, 29.7 GiB
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 1ED17572-FEA5-4C23-82C3-57C2F576F6B3
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 62333918
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048           67583   32.0 MiB    0C01  hassos-boot
   2           67584          116735   24.0 MiB    8300  hassos-kernel0
   3          116736          641023   256.0 MiB   8300  hassos-system0
   4          641024          690175   24.0 MiB    8300  hassos-kernel1
   5          690176         1214463   256.0 MiB   8300  hassos-system1
   6         1214464         1230847   8.0 MiB     8300  hassos-bootstate
   7         1230848         1427455   96.0 MiB    8300  hassos-overlay
   8         1427456        62333918   29.0 GiB    8300  hassos-data

Command (? for help): v

Caution: The CRC for the backup partition table is invalid. This table may
be corrupt. This program will automatically create a new backup partition
table when you save your partitions.

Identified 1 problems!

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/mmcblk0.
The operation has completed successfully.

Although there are no errors in the update process, rerunning gdisk gives exactly the same result as before. Therefore, I tried to rebuild the backup partition table and the backup header via the recovery menu of gdisk, If I then verify (before writing to disk), gdisk tells me all is good. So I write to disk:

ubuntu@ubuntu:~$ sudo gdisk /dev/mmcblk0
GPT fdisk (gdisk) version 1.0.8

Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
on the recovery & transformation menu to examine the two tables.

Warning! One or more CRCs don't match. You should repair the disk!
Main header: OK
Backup header: ERROR
Main partition table: OK
Backup partition table: ERROR

Partition table scan:
  MBR: hybrid
  BSD: not present
  APM: not present
  GPT: damaged

Found valid MBR and corrupt GPT. Which do you want to use? (Using the
GPT MAY permit recovery of GPT data.)
 1 - MBR
 2 - GPT
 3 - Create blank GPT

Your answer: 2

Command (? for help): r

Recovery/transformation command (? for help): ?
b	use backup GPT header (rebuilding main)
c	load backup partition table from disk (rebuilding main)
d	use main GPT header (rebuilding backup)
e	load main partition table from disk (rebuilding backup)
f	load MBR and build fresh GPT from it
g	convert GPT into MBR and exit
h	make hybrid MBR
i	show detailed information on a partition
l	load partition data from a backup file
m	return to main menu
o	print protective MBR data
p	print the partition table
q	quit without saving changes
t	transform BSD disklabel partition
v	verify disk
w	write table to disk and exit
x	extra functionality (experts only)
?	print this menu

Recovery/transformation command (? for help): d

Recovery/transformation command (? for help): e
Warning! This will probably do weird things if you've converted an MBR to
GPT form and haven't yet saved the GPT! Proceed? (Y/N): y

Recovery/transformation command (? for help): v

No problems found. 2014 free sectors (1007.0 KiB) available in 1
segments, the largest of which is 2014 (1007.0 KiB) in size.

Recovery/transformation command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/mmcblk0.
The operation has completed successfully.

Unfortunately, this does not seem to alter anything. If I run gdisk again, I get the same warnings about broken backup header and partition table. Also, fdisk -l and trying to mount it give the same warnings and errors as before. To me it seems as if gdisk is not actually writing to the disk, even though it does not throw errors.

I am in the dark now. It seems as If I have found the problem (corrupted backup partition table and backup header of GPT) and the solution (copying the main header and partition table to the backup via gdisk), which then seems to have effect (gdisk’s verify option gives no more errors), but does not actually do anything (restarting the gdisk process gives exactly the same errors).

Is there anyone who has an idea what I can do to further debug, and hopefully fix, the problem?

Also, I did try to save the superblock for the different partitions. For partition 2, this seemed to succeed, for all other partitions it did not.

ubuntu@ubuntu:~$ sudo dumpe2fs /dev/mmcblk0p2
dumpe2fs 1.46.5 (30-Dec-2021)
Filesystem volume name:   hassos-kernel
Last mounted on:          /mnt/data
Filesystem UUID:          e8f2f30a-5a3d-4d0c-8958-f97749e5c260
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
Filesystem flags:         signed_directory_hash 
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              6144
Block count:              24576
Reserved block count:     1228
Free blocks:              14067
Free inodes:              6132
First block:              1
Block size:               1024
Fragment size:            1024
Reserved GDT blocks:      95
Blocks per group:         8192
Fragments per group:      8192
Inodes per group:         2048
Inode blocks per group:   256
Flex block group size:    16
Filesystem created:       Wed Jul 28 22:27:28 2021
Last mount time:          Wed Jul 28 22:27:28 2021
Last write time:          Wed Jul 28 22:27:28 2021
Mount count:              1
Maximum mount count:      -1
Last checked:             Wed Jul 28 22:27:28 2021
Check interval:           0 (<none>)
Lifetime writes:          9 MB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:	          128
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      f6d5c97e-b90e-468c-a98b-a31f2d49e8d9
Journal backup:           inode blocks
Checksum type:            crc32c
Checksum:                 0xfa3f0370
Journal features:         journal_checksum_v3
Total journal size:       1024k
Total journal blocks:     1024
Max transaction length:   1024
Fast commit length:       0
Journal sequence:         0x00000004
Journal start:            0
Journal checksum type:    crc32c
Journal checksum:         0x2fdf00b5


Group 0: (Blocks 1-8192) csum 0x02ee [ITABLE_ZEROED]
  Primary superblock at 1, Group descriptors at 2-2
  Reserved GDT blocks at 3-97
  Block bitmap at 98 (+97), csum 0x0000a577
  Inode bitmap at 101 (+100), csum 0x0000725f
  Inode table at 104-359 (+103)
  6278 free blocks, 2036 free inodes, 2 directories, 2036 unused inodes
  Free blocks: 1915-8192
  Free inodes: 13-2048
Group 1: (Blocks 8193-16384) csum 0x1282 [INODE_UNINIT, ITABLE_ZEROED]
  Backup superblock at 8193, Group descriptors at 8194-8194
  Reserved GDT blocks at 8195-8289
  Block bitmap at 99 (bg #0 + 98), csum 0x0000b6fd
  Inode bitmap at 102 (bg #0 + 101), csum 0x00000000
  Inode table at 360-615 (bg #0 + 359)
  1916 free blocks, 2048 free inodes, 0 directories, 2048 unused inodes
  Free blocks: 8325-10240
  Free inodes: 2049-4096
Group 2: (Blocks 16385-24575) csum 0x843c [INODE_UNINIT, ITABLE_ZEROED]
  Block bitmap at 100 (bg #0 + 99), csum 0x0000145a
  Inode bitmap at 103 (bg #0 + 102), csum 0x00000000
  Inode table at 616-871 (bg #0 + 615)
  5873 free blocks, 2048 free inodes, 0 directories, 2048 unused inodes
  Free blocks: 18703-24575
  Free inodes: 4097-6144