Hi all, my PI went bananas again last night. I did hook up a monitor to my PI and it did show error messages from 01:52 onwards.
I have added my findings to the github entry as the person involved there (Michael) suggested I do that.
In short the following error messages seem to be the interesting ones:
[#####.#####] mmc0: card never left busy state
[#####.#####] mmc0: error -110 whilst initialising SD card
Looking up these messages in google show remarks on bad SD cards.
But I am not entirely convinced here:
The error only happens every sunday at around 02:00
Rebooting any other time (hard or the proper way) does not cause this to happen
One remark did trigger a new line of thought though: it might be that my card is a fake 32GB.
The card tells the system it is indeed 32GB but in reality it is only 16GB.
At 02:00 the PI is doing some health check including the state of the SD card and tries to access the SD card beyond the 16GB and goes into error mode.
So, I am going to buy a new 32GB card and try to copy my HASS.IO setup to this new card and look what happens. If no error occurs then I will check the old card to see if my assumption is correct.
I rebuilt my instance onto a new SD card last week - and also deleted the database after recovering from backup.
This morning - all still working for the first time in weeks (might be months by now)
Not sure which, if either, of the above changes made the difference. This is starting to look like an SD card issue as mentioned by Harmpert above.
Yes, fair point! My database does seem to be growing quite quickly these days.
I just used the standard backup/recover method.
Took a snapshot before shutting down, downloaded to pc. Created the new disk/image of HA. Installed Samba, uploaded the snapshot to HA and recovered from that.
You do not even have to install samba. Allready during the onbording you can choose to select and upload a snapshot from your computer. See this post.
I remember it didn’t work flawlessly in my case, but it end it worked. (can’t full remember what went wrong and how I solved it. I think the progress window didn’t show, but I just waited)
@Plevuus although I am going to try the card route I am still not convinced that the problem we are facing is necessarily a card issue. I have seen some more people having instability problems that were only temporarily solved by using a new card. Having said that, the card might be part of the problem.
@Harmpert, You can use the free DiskInternals Linux-Reader to read the content of your Home-Assistant SD-card on a windows PC.
C’T magazine has a nice SD-card test on their site (in Dutch).
Good to hear!
Indeed, you do have a point, It still is a good idea to identify which process is running on sunday night that triggers a (seemingly) problematic SD-card to act up.
I started with HA in nov. 2020. And from the beginning (I think) my RPI4 crashes every Monday during the night. And I’m not doing any automation at that time.
My HA crashed at 3h04 this night (summer time GMT+1, Brussels) . Some previous crashes happened at (winter time) +/- 2h00, 1h30, 2h30
I have always the same behavior:
No automation was done this morning
But I can logon from my PC to my PI. In the overview menu, I can even manually set some lights.
But I cannot open any other menu than “overview”. I receive the error message “Unable to load the panel source: /api/hassio/app/entrypoint.js” when trying to open other menus, like the logging menu, Red-one menu, supervisor menu.
Not possible to connect to my RPI with Samba
I always have to power off/on my RPI
After power off/on -> some screennshots indicating that memory use, disk use, processor temp are very acceptable.
@hdehaseleer, You write 64GB SSD, but I assume you mean a 64GB SD, right? What is the brand and model of the SD card and what is it’s speed-class?
Please have a look at the linked Github-issue. Multiple people got rid of the issue by upgrading their SD cards. (actually all people who tried)
Yes, i’m using a SD card (not SSD). (Kingston Canvas Select plus / 10 I U1 A1 100MB/s)
I didn’t believe that a hardware problem could arrive each Monday moring 3h00.(and only on Monday)
But indeed, perhaps the OS is doing some cleanup Monday +/- 3h00 which the SD card cannot follow.
I’ll buy a new one and give it a try.
Indeed, changing the SD card solved the problems. It’s for the second time that my RPI didn’t crash on Monday. This after several months.
I have now a Samsung Evo Plus MicroSDXC 64GB UHS Class I | Class 10 | U3 |.
So it is a U3 that writes at 60MB/s read 100MB/s
Before I had a Kingston Canvas Select Plus microSDXC 64 GB UHS Class I | Class 10 | U1. So a U1
For information: I bought two identical SD cards at the time I bought my RPI model4. So I tried first with the second identical Kingston card. But this second SD card had the same problem of the “Monday crash”
@Antoine_Evertze do try a new SD card with proper capabilities. Somehow a process runs on Sunday night that causes an issue with some SDcards. As you can see in all cases listed here and in the github the problem does not occur with a proper SDCard . mine has been running for two months now without any issue.
As I said, I changed my SD card with a Samsung Evo and this solved the problem. I changed nothing else. It’s already my 4th Monday without a crash.
Somebody of the development team should look at the (housekeeping?) application that runs every Monday morning around 02h. It cannot be hard to correct this problem by ex. limiting the writing speed or the CPU utilization of this application. This will reduce the frustration of a lot of persons and increase the reputation of HA.
Who can forward this problem to the development team?
It’s been a while sinds the initial problem. In my case the problem was solved with the july 2021 update.
But now the last two weeks the problem is back. Already tried a different sd card (evo). Didn’t help.
Any suggestions?
March '21 installed a sandisk extreme 32GB card and haven’t had any crash issues since then. My raspberry Pi ran unattended for like 5 months (had a bit of a break on the home automation stuff). In the fall picked up and have been actively using the stuff again without any crash issue.
Keeping my fingers crossed…
My database does seem to be growing quite quickly these days.