How to debug HassOS going offline overnight?

Hi everyone.

I have a standard HassOS on Raspberry Pi 4 installation which seems to be going offline overnight. When I check it in the morning, it’s not pingable, and the only way to get access to the system is to power-cycle the Raspberry (which I’m aware is not good).

My question is, how should I debug this? I’ve checked all the logs in the supervisor tab, but they only seem to log up to the point of the last reset. Is there any way of checking what happened before the Raspberry was turned off, like a persistent log file?

Thanks in advance.

No, unfortunately the logs are not persistent across restarts.

Before restarting, you can put the SD card in a Linux PC to read the logs, or a Windows PC running DiskInternals Linux Reader (free).

1 Like

I see, I will try that. Thank you!

I had to wait for HA to break in the same way (it worked fine for a few days) to try this. Last night it finally did, so I tried this approach.

The home-assistant.log file shows nothing of interest. I couldn’t read the host logs - I found a bunch of .journal files, but they seem to be binary. They cannot be read without journalctl, am I right?

One thing that has been happening was that at random times while using the UI was that I would get a “read-only file system” error, which seems to be a symptom of a dying microSD. That might be at fault for the overnight crashes as well. I’m going to replace the card (since this one was a random one that I had around, and is probably not best suited for the job) and see if that fixes the issue.

Thank you for your help!

check this threads HA stops work every Monday

I believe the problem lies on your microSD corrupted and HA will check it every Monday night or something like that

1 Like

Thank you! Yes, I would say that the card has something to do with it. I will order a better card, and in the meanwhile, check if the crashes are exactly one week apart.

Better yet change it to USB SSD or if you want the best then get your self ArgonOne M2 <- so far is the best casing for RPI4.

1 Like

@rousveiga in the thread suggested by @kingrichard you will find a number of instances where Raspberry PI’s crash sunday night after midnight. In this github you can find more info.
HA stops work every Monday · Issue #47928 · home-assistant/core (github.com)

After replacing the SD card with a new one the crashes have stopped (for now?).

Thing is, we still don’t know what process triggers the crash. Something is happening at sunday night just after midnight that somehow uses the SD card to it’s limits (and over).