It does sound like a supervised install of HA would better suit you. Create a full snapshot and copy it off the card.
Reinstall using this method: Installing Home Assistant Supervised on Raspberry Pi OS
Restore your snapshot.
You may have to delete the home assistant database and restart if this is corrupted.
You should then be back where you started but now have access to the underlying pi OS.