PSA: 2024.7 recorder problems

Configuration
52

There’s 2 separate issues. Which one do you have?

This thread was created to address issues caused by custom integrations that @tom_l has highlighted.

Separately, there’s an influx of people who have a separate (older) issue related to purge that will be addressed in 2024.8. That issue was not created by this release, it’s just more apparent with this release.

If you have issues related to purge, please check out the issue linked above. Otherwise, this is not the correct thread.

4 Likes
53

Thanks for clarifying @petro

54

Same with me!

55

Just for clarity, why is the issue more apparent with this release? I have two installs, one old one that has been upgraded for years which has the DB issue, but the other (newer Install) has not.

Is it just more apparent because the history / recorder stalls in this release? What was the issue before?

Sorry for nagging. I just want to understand.

3 Likes
56

Yes this is a good workaround, thank you.

57

There are some HA which start 2 or 3 years ago where the schema of the DB is involving. Like mine, starting in 2021 with schema version 23 where we are now in version 43. There was no issue till an update coming with 2024.7 of Alpine version that cause sqlite dependencies and locking of database.
It’s not core HA code change or sql code purge change.
For those old db, even with new schema, they still have some difference in table (index, keys) . I’m not a DBA but it’s what I understood.

So now migration script will have to recreate some tables to remove those parts which is no more used but remains. It’s a risky process and sqlite have define some steps (12) for that where thanks to @bdraco try to resolve.

I hope this is more clear for everybody :wink:

details here : Purge causes recorder to stop writing to the DB until HA is restarted (Auto purge happens at 4:12am) · Issue #117263 · home-assistant/core · GitHub

Solution (you need to consider this for advanced users/own risk - backup and check to have enough free disk space of course !
It’s solve the issue on mine.

Link: Purge causes recorder to stop writing to the DB until HA is restarted (Auto purge happens at 4:12am) · Issue #117263 · home-assistant/core · GitHub

3 Likes
58

Hi,
I encountered (probably the 1st issue).
Is there a way to write an automation to get a notification (that part I know how to do) when the recorder stops working? (that part I don’t)…
D.

59

yes, possible via a check on CPU usage and memory increasing.

2 Likes
60

It’s more apparent because more people are hitting it. The bug existed long before this release but there wasn’t enough people/information to figure out what the problem was. Just take a look at the linked issue and you’ll get a history dating back before this release.

1 Like
61

Thanks, I see the issue is connected with the SQLite version combined with old DB schemas and that some install methods had an earlier upgrade of SQLite, and now “we all got it” because of Alpine updates.

62

Have the same recorder issue and added “auto_purge: false” (without result).
But now I notice a higher power consumption, is that also related?
image
When recorder freezes, after restart, the CPU load falls to its normal value.

Read before ask could have helped me…

63

Opened up a pull request to block icloud3 from breaking the Home Assistant recorder:

4 Likes
64

Opened up a pull request to block older versions of places from breaking the Home Assistant recorder:

3 Likes
65

same thing happening here

66

I have the same problem with Home Assistant 2024.7.1

The recorder stops at 4:11 in the morning.
I have none of the mentioned integrations above installed.
Restarting Home Assistant solves the problem until the next morning.

Please InfluxDB is still recording and the data isn‘t lost using Grafana.

Please solve the problem as soon as possible.
I don‘t like loosing all data from my energy dashboard :flushed:

67

So you have the first issue. For which a temporary solution is available in the first post.

1 Like
68

Opened up a pull request to block older versions of hass-variables from breaking the Home Assistant recorder:

2 Likes
69

I think those three custom integrations cover the second issue set out in the OP.

The first issue has been fixed on dev, and will be shipped in 2024.7.2.

My goal is to ship it today.

…/Frenck

11 Likes
71

Will it (it being the SQLite DB rewrite) really be merged in 2024.7.2? The github issue still refers it to be a fix for 2024.8. Smart fix for the custom integrations :slight_smile:

72

Thanx for pointing out the temporary solution.
I added the lines to my configuration.yaml

Now I have seen that Frenck is working on a 2027.2 update.

I wonder if that solves the problem.
Do I have to delete the lines in my configuration.yaml after updating to 2027.2?