After updating to the latest version of Home Assistant, I’ve encountered some unexpected issues that I wanted to share in case anyone else is experiencing something similar.
What’s Happening:
Re-enabled Automations: Automations that I had disabled prior to the update suddenly became re-enabled. I’m certain they were turned off, but upon checking after the update, they were back on.
Custom Entities Reset: All of my custom entities, such as input booleans and input times, were reset to their default values.
Broken Trigger in Automation: I also found one automation that has a broken trigger. It uses a light group as a trigger and it says it has the wrong name. The automation says that it ends in an ‘s’, but that name doesn’t exist and the correct one is non-plural.
I haven’t had a chance to fully investigate the scope of the changes, but I’ll update with more details once I have more information.
Database Comparison:
I run nightly backups, so I compared the database from before and after the update to try and pinpoint the issue. Here’s what I found:
Before the Update (Sept 26): I had 160 automations enabled and 22 disabled.
Night After the Update (Sept 27): The number of enabled automations jumped to 220 and 109 disabled.
Following Night (Sept 28): This number increased further to 308 automations but still 109 disabled, and it has remained at that level as of the latest backup.
Something clearly happened during or after the update that caused a significant number of automations to reactivate or duplicate, leading to what I can only describe as rogue automations.
System Versions:
Here are the details of my system for reference:
Core: 2024.9.3
Supervisor: 2024.09.1
Operating System: 13.1
Frontend: 20240909.1
Questions:
Has anyone else experienced something similar after updating?
Are there any specific areas I should check for potential issues, or other things I may need to restore manually?
Wow, that is quite something. I am on this forum daily, but have not heard anything similar. What version were you updating from? A recent one from the last month or so, or a much older one?
First thing I would check if the recorder database has corrupted. If it has ( and you’re using the default recorder database) you should see the broken database in your config folder. I’m not sure the states of automations are helpers are restored from that though.
Have you checked the logs to see if something weird happened during startup?
I wasn’t that far behind, 2024.8.1.
I havn’t been able to parse out the log just yet. There is a lot in there because of all the old automations running and failing.
I am not sure what the recorder db is or what it does, I’ll have to look into that. I’m looking at the recorder_runs table and see in past backups there was only one entry, but on the day after the update I see two entries, the day after that I see 4, as of today I see 6. How would you interpret that?
Something else I am noticing, not sure if this is normal, but for example a disabled automation with the metadata_id 26416 shows up in the after update db, but before the update it doesn’t exist in the states_meta table at all. In the current table it has four entries where it was off, then on.
I did a count of the missing automations from the pre-update table and there are 150. So my guess is that there are 150 automations that I had disabled that are now back on.
The recorder database is what stores entity state history, so that would likely be the place where a restart would look to restore previous state. I think you are already looking inside it The default is called home-assistant_v2.db and if there are multiple files in your config folder with similar names that might indicate it got corrupted.
The logs would be the first place to look for abnormal termination of the startup sequence. The one from the previous boot is always in the config folder, so next time check that first. If you rebooted several times, you need to look from the Home Assistant UI. I assume that too comes from the recorder db, so if that was corrupted you might miss older data there too.
If you have proper backups, you might be able to recover a recorder db from before the corruption (if that was the cause). But you’d lose all recorded entity informtion from the backup till now. But step one would be to figure out if your recorder db got reset due to corruption.
Another cause could be the automations.yml file contains something bad. That should show up in the logs every time it gets read. It also might be why ot all automations get loaded.
I cannot seem to find anything in the log yet, but I will keep looking. I wish the backup had the log files so I could reference them.
The only db files I see are home-assistant_v2.db and home-assistant_v2.db-wal. That seems to be in all the backups so I would think its normal.
I compared the old and new db’s and isolated the differences with the automations and plan on fixing them, unless you think it would be better to restore from the back up?
I’m okay losing a couple days of history.
It does. The backup is a zip file, you can unzip it. the log files for the current boot and the one before are in your config folder, so pick the right day. But is does depend how many times you booted after the problem if the backup that night had something useful.
Automations are in automations.yaml if you created them from the gui, these should also be there. But anyway, the ones you make yourself in yaml should also be in the backups, so extracting them and doing a file compare should be informative.
Personally I use github for all my yaml except secrets.yaml, that allows for even better comparison and history. And it serves as yet another backup.
Does doing it that way tell you whether or not the automations are enabled? That is pretty much where this all fell apart. Had the automations.yaml had the enabled flag then it would have been as simple as swapping the files with the backup, but alas it’s not lol. I’m not even sure if how I’m checking the db is accurately telling me which ones were active or not.
The default is called home-assistant_v2.db and if there are multiple files in your config folder with similar names that might indicate it got corrupted.