I’ve been having an issue lately where my history data no longer updates. Which means my temp sensor graphs don’t work, my switch history doesnt work, nothing works in case of history. The sensors still state the right data, but the graphs dont update.
Here you can see the graphs all flattening out, but the sensor data is still right (and not the same as the graph)
Here you can see my bedside lamp saying it’s been off for the last 24 hours, but it hasnt been (it’s on right now even)
Any idea whats going on?
I’ve checked the basics of memory use etc, and seems ok
Turning it on/off has mixed results. It sometimes fixes it briefly, but i will def lose all the data. If i leave it, it will often ‘come right’ and somehow get all the lost data back (which doesnt happen if i reboot, it’s all lost)
currently running 0.92.2
if i go to hass.io->system->system log, it’s all empty and wont load anything
I’ve thought in the past that maybe this is happening as it kinda explains it, but makes little sense otherwise, and that is:
Sensors history goes to RAM, then it writes it to SD card.
Sensor history graphs is always read from SD card.
Hassio cant write to SD card for whatever reason, so data stays in RAM so history graphs dont update as there is no new data on SD card
Eventually HASSIO is able to write the RAM data to SD card, and the history graphs ‘come right’
thats my (completely uneducated) theory anyway
Anyone have any ideas? It’s really frustrating… happens every day or 2? usually only blanks for an hour or 2 and comes right
Hi, thanks. Happy to try that, but how does that explain previous times where it has magically come right? surely if it’s corrupt, it wont self heal?
Im happy to do those things if that fixes it, but i dont think thats the fix… there has to be something else going on?
to make matters even worse, it has now changed. I looked at my graphs to see if they had self healed, but the data is different again
compare that to the one from 2 hours ago
it now looks like the data is all there, just behind by nearly a full day?
Is there a weird time offset thing or time zone that could be messing with this? Im in New Zealand (GMT + 12) so maybe that has somethign weird to do with it? some weird system clock being set to US or something and it messing with graphs and not updating or something?
You wanna hear an even crazier story? The dat is slowly coming back… I’ve been checking it over time, and the history graph went to nearly 24 hours behind as i showed above, but over the last 2 days, it has slowly been showing me more history
e.g. 2 days ago it had ~20 hours flat lining,
next morning it had 16 hours flat lining
evening 12 hours
next day 6 hours
now 3.5 hours flatlining
The data is clearly not going missing, as it’s showing up later… It just has a weird flatlining thing where the graphs just dont display data
I would have to agree with @tom_l it’s time to delete your database file. I’d definitely be curious to know how large it is… you will probably want to add a database file size sensor so you can see how quickly it grows and that will help you determine what purge interval to use in your settings.
Thanks guys. I thought the database would purge itself and only kept a few days data already, but turns out that file is 4.5GB!
So recorder will let me set a purge thing and keep my db file at a sensible size? regarless of if i use Maria or the default SQLite? Im new to dealing with these database things, sorry everyone!
and apologies, i am indeed using a RPi 3B.
Strange failure mode though with the data not displaying at the time, but clearly being there later and doing all sorts of weird stuff?
Yes use recorder to set the purge interval. Note that the purge interval for the DB will be reset every time you restart. So if you are still configuring HA and restarting often the DB may never get purged.
Using the recorder exclude and include options to stop recording things you don’t care about will also reduce the DB size.
You can also prevent these displaying using exclude and include options for history and logbook.
@neilvangeffen
Very cool sensor cards! It is awesome how it shows minimum and maximum temperatures with the respective times of occurrence.
I searched for a similar native-card but could not find it. Is yours a custom card? In that case, could you please share it with the community?
Thanks
Cheers!
I have to jump on this topic. I’m experiencing the same issue on my HASSOS Raspi4. History graphs work for a while (this time about two weeks) and now I´m getting flatlines again. I’m using mariadb, purge every 2 days.
I know I could delete the DB and that worked in the past, too. But I don´t like deleting my database all the time. There must be an issue that creates this problem. I´ve googled it for quite a while now and people are reporting it every now and then but I never came across a solution other than “delete the db”. Any other ideas ?
I have been fighting this issue as well.
I am running hassio on a dedicated VM on one of my servers. This issue is prevalent enough that I have not been able to go more than 24 hours before my graphs flat line.
I ran my system in debug mode and pulled off almost a 1GB log file but didnt see any errors in the log around the time where the flat line occurred.
If I reboot the system / restart home assistant then it recovers and I start getting data again.
I added a sensor to track the database file size but only saw it in the 2GB range when the issue happened.
I even tried adding a automation that would purge the database every 4 hours in hopes that would help but no such luck.
I can re-produce this issue pretty easily so if there is anything that can be done to provide more information just let me know and I will try to capture it.
I am running:
MariaDB 1.3
System Ubuntu 18.04.3 LTS
So I let my system stay in the “broken” mode for over 24 hours and noticed something funny.
As @neilvangeffen mentioned previously the data is “coming back”
I took screen shots about 12 hours after the issue ocured and it showed the loss of data happened around 530AM ( 1/31 ), so I waited and checked it the next day around 9AM (2/1), and it showed that I had data all the way up to 11:30 ( 1/31).
So looks like there might be something wrong with the lovelace default graphs?
Does anyone know where this should be reported? Or if there is a way to configure the default timeline of data that is used by those graphs?
Bump. I’ve started down the HA around the beginning of the year… certainly love the community! Having the same issue however.
RP4 (4GB) running on Hassio, not a huge installation, I wouldn’t think I’m getting overloaded by sensor data as I’m still adding components and learning the various technologies. My DB file, never flushed (just learned about that from this thread), is almost 2GB… though I’m going to look into both flushing and installing MariaDB.
I don’t know if I’ve ever waited long enough to notice it start to try to “catch up”, so maybe I’ll just try and ignore the frustrating flat-line on my graphs so I can replicate the experiences others have noted. Still doing a lot of research and troubleshooting in general though, so it’s difficult to say where one problem ends and another begins! I’ll recount my experiences if I make any major breakthroughs. Thanks!
Ok, I’ve made some significant progress, though I can’t call it a total win. I’m a little frustrated by the implications tbh, but I’ll ignore that for now.
TL;DR: I moved my DB entirely off the Hassio unit and onto a laptop connected via ethernet and setup the recorder to purge weekly.
During my initial troubleshooting I noticed the the load on my system monitor sensor was regularly peaking above 4. I don’t feel that I had an extensively data hungry configuration, as I’m still assembling the pieces and exploring the integration options. I was sure to purchase a top-tier 32GB card… so while It’s altogether likely that I might have had a particularly hungry add-in or integration that was doing a lot of R/W operations, it is a bit disappointing to find that the RP4 4GB Hassio couldn’t handle the load.
To address this I decided to spin up a Mariadb container on a decently powerful laptop that I had already setup as an external MQTT broker, Kodi front-end, and Zoneminder server. It’s a little bit more load on that laptop than I really like, but it ended up solving my problem for the short-term. The load on my HA instance is back to an avg of less than 1, and is consequently MUCH more stable and consistent in the historical data.
It took me a few days to find this thread, but I said I’d share my findings
a complete history wasn’t loading after transfer to a new faster SD card. may have corrupted the DB file. thanks to the comments on this post i just went and deleted the old db file and config and restarted. now history is showing up.
Similar happen to me once I moved from PI3 to PI4 via restore feature on first logon screen. All my graphs went flat while the db had ~600MB. I removed the db file, rebooted and it started collecting back. Nice!
