I have noticed that when I go into the history and try to choose “3 Days” or “1 Week” and display it, the CPU usage goes through the roof. It seems that “1 Day” works, somewhat, it still consumes a lot of CPU but it’s navigable. When I just click on “History” and nothing else from the front page, I can watch the duration and date input fields get enabled and disabled without any input. It’s almost like a javascript but is stuck in an infinite loop, I believe this is causing the high CPU load. Switching to “3 Days” or “1 Week” (and choosing a day in the past so I see more data) causes a runaway-like condition and the CPU doesn’t seem to ever recover. I had to restart hass to retain control of the cores.
I have home assistant running on an Orange Pi Plus 2E. It’s 1.6GHz quad core and isn’t beefy but historically this wasn’t a problem. I’m running Home Assistant version 0.83.2. Any ideas?
Mine is currently 647MB, it only stores data for 14 days. I left it running for about 5-10 minutes and it was still railing the CPU so I restarted hass. What kind of CPU/system are you running and what size is your DB?
I use the following recorder block to limit the history to 14 days.
# Keep two weeks of data
recorder:
# Delete events and states older than 2 weeks
purge_keep_days: 14
Mine is currently 647MB, it only stores data for 14 days. I left it running for about 5-10 minutes and it was still railing the CPU so I restarted hass. What kind of CPU/system are you running and what size is your DB?
My CPU is probably slower than yours, I’m using a Raspberry Pi 3 with no overclocking.
I also find CPU goes to 100% when clicking history, but it does load eventually, and the CPU usage goes back to normal. My DB is hosted on another system running MariaDB 10. My DB is around 200Mb and I’ve got purge_keep_days set to 5 days.
What I did find to speed things up it to not record every entity from HA. I’m using the include and exclude properties to not log weather entities, for example.
Another thing you can do if you are having issues with the database is to just wipe it completely and start from scratch.
I don’t think your DB is really that big, some people have a DB over 1 GB, so I’m guessing there could be some corruption?
5-10 mins is way too long to view history, try looking at memory usage also, if memory usage is high that would explain why it’s taking so long or freezing.
Thanks for the response. I noticed that I put the excludes list in my history: section instead of in the recorder: section. If I understand it, that means that I was recording everything but only displaying a subset of the sensors.
I’ve moved the excludes, deleted the db file and restarted home assistant. I’ll let it go a few weeks and try it again…
So I’ve been watching the DB file and it’s much smaller than it used to be. So far 255MB. Anyway, after deleting the DB and letting it get recreated I have noticed that the problem of the javascript being stuck in a loop has not disappeared. I’ve captured a couple animated gifs in hopes that it shows more clearly what I’m talking about.
You’ll see in the gif below, when I click on History the CPU usage on the system jumps (as expected). Then the header area of HASS seems to keep submitting keeping the CPU crunching. This problem is exacerbated when looking at a larger amount of time, presumably because each time the fields gray out the data has to be fetched.
I also got a capture of the browser with the debug console enabled so we can see the network requests. You can see a new one get queued up every time the “Show entries for” and “Period” fields get grayed out. I am not doing anything with the keyboard when this happens by the way.
Same “refresh loop” issue here, running 0.84.2. Cannot really narrow down which version brought that strange “behavior”, but I didn’t have this issue with 0.76.2.
I have done a gigantic amount of exclusion from recorder/history, but the problem didn’t disappear. Now that I see your post, I guess many people could experience the same issue.
I tried this out and I do see that %iowait bumps up for a single instance seemingly every time the “Showing entries for” and “Period” fields get disabled but the values are less than 1 usually:
07:19:47 AM CPU %user %nice %system %iowait %steal %idle
07:19:48 AM all 3.26 0.00 1.75 0.00 0.00 94.99
07:19:49 AM all 9.57 0.00 1.26 0.00 0.00 89.17
07:19:50 AM all 25.19 0.00 0.76 0.00 0.00 74.06
07:19:51 AM all 9.80 0.00 1.01 0.00 0.00 89.20
07:19:52 AM all 14.65 0.00 2.53 0.51 0.00 82.32
07:19:53 AM all 23.43 0.00 1.76 0.00 0.00 74.81
07:19:54 AM all 33.59 0.00 11.36 0.25 0.00 54.80
07:19:55 AM all 37.94 0.00 6.53 0.00 0.00 55.53
07:19:56 AM all 30.15 0.00 5.53 0.00 0.00 64.32
07:19:57 AM all 24.56 0.00 3.29 0.00 0.00 72.15
07:19:58 AM all 23.81 0.00 1.50 0.00 0.00 74.69
07:19:59 AM all 20.85 0.00 5.28 0.00 0.00 73.87
07:20:00 AM all 18.56 0.00 8.42 0.00 0.00 73.02
07:20:01 AM all 25.25 0.00 6.19 0.25 0.00 68.32
07:20:02 AM all 25.06 0.00 2.51 0.25 0.00 72.18
07:20:03 AM all 24.87 0.00 1.27 0.00 0.00 73.86
07:20:04 AM all 23.99 0.00 2.78 1.01 0.00 72.22
07:20:05 AM all 18.64 0.00 7.30 0.00 0.00 74.06
07:20:06 AM all 20.35 0.00 5.03 0.00 0.00 74.62
07:20:07 AM all 22.28 0.00 3.04 0.00 0.00 74.68
07:20:08 AM all 23.62 0.00 1.76 0.00 0.00 74.62
07:20:09 AM all 23.87 0.00 1.51 0.00 0.00 74.62
07:20:10 AM all 19.65 0.00 6.05 0.00 0.00 74.31
07:20:11 AM all 21.72 0.00 5.05 0.00 0.00 73.23
07:20:12 AM all 24.37 0.00 1.26 0.00 0.00 74.37
07:20:13 AM all 24.37 0.00 1.76 0.00 0.00 73.87
07:20:14 AM all 27.57 0.00 3.26 0.00 0.00 69.17
07:20:15 AM all 20.20 0.00 5.30 0.00 0.00 74.49
07:20:16 AM all 21.46 0.00 5.81 0.00 0.00 72.73
07:20:17 AM all 24.74 0.00 2.30 0.77 0.00 72.19
07:20:18 AM all 25.32 0.00 1.77 0.00 0.00 72.91
07:20:19 AM all 26.77 0.00 1.52 0.00 0.00 71.72
07:20:20 AM all 45.57 0.00 5.32 0.00 0.00 49.11
07:20:21 AM all 44.30 0.00 6.58 0.00 0.00 49.11
07:20:22 AM all 47.72 0.00 3.05 0.00 0.00 49.24
Not that I am an expert here, but the %iowait value changes don’t seem to indicate a problem with the SD Card or wherever the Orange Pi stores the db.
Interesting, though, that the idle% value stays sign above 50% in most cases.
But I can’t really provide any further recommendations here, sorry.
Maybe somebody else sees something I don’t.
Hate to bring up a thread that’s gone two months with no responses, especially if all i have to offer is a “Me too!” comment.
Docker on Synology:
HA 0.85.1 container
recorder includes/excludes configured
recorder logging/keeping 28 days
database ~2.8GB constant
MariaDB container hosting recorder database
When I try to view the history tab it will load after about 30 seconds but then the date selection and time range will gray out randomly. The data displayed will not change if I select a different date. The memory and CPU usage of the HA container easily spikes to 3GB and about 70% CPU usage. If I leave the history tab and go back to a states page the memory usage will very slowly start to come down and the CPU usage goes back to normal (ie. < 5%) immediately. I usually have to restart the HA container though to get memory back under control. Normally my HA container sits at about 250MB memory usage.
The recorder and history page used to be really fast as in loaded in under 10 seconds and you could select dates and date ranges and they would populate the data quickly. I don’t have an exact estimate when it stopped working properly but I’d put it around 0.80.
Hello squirtbrnr, about a month ago I migrated my home assistant from an ARM version of Ubuntu to hass.io running on a raspberry pi. I no longer see this issue with the refreshing history/logbook pages. I’m not sure why but it was constantly a problem on the other installation. This morning after seeing your post I opened up the history page and let it sit for 5 minutes, no reloads (greying out of the date and period fields). I also did this with the logbook, also no reloads. I’m not sure where this leaves you but it’s another observation. If you have a raspberry pi, you may want to try hass.io. I like it better than manually installing and running home assistant because of it’s portableness - I converted to hass.io around 0.87 and now I’m running 0.89. I haven’t experienced this issue yet using hass.io.
thanks for the update. I’m in the process of updating my HA install from 0.85.1 to at least 0.87.1 and hopefully higher. I held off on the switch to Lovelace so I’m a few versions behind and maybe one of those updates improves the history/recorder functionality. Unfortunately I don’t have a spare RPi at this time. My Synology is an Intel model and plenty fast, so I don’t think it’s a performance issue. I may try blowing away the recorder database as a troubleshooting step too.
EDIT: Just updated to Lovelace and 0.87.1 for starters and my problems almost went away. History loads much quicker and doesn’t grey out the date or range selection. Memory usage still increases a lot while viewing history, but not nearly as bad. I’m calling this “fixed” for me.
