Loading state history graph, so so so slow zzzzz

Hi,
I store only 9 mqtt sensors updated every 5 secondes.
In configuration.yaml, recorder: set to only include these sensors, keeps them 5 days.
Result, after only 2 days, loading state so so … so long to get data!
I like a lot HA, it’s an awesome software, but this behavior makes this program almost unusable :roll_eyes:

I have a Raspberry Pi 3 Model B with a 64Gb San disk ultra, can I expect a better result with that ?
Or is this a structural issue of HA ?
Thank you

Actually on Synology DS716+
On this page drawing history for only 24h for 4 sensors, need at least 30 secondes.

Data sent by mqtt from an EspEasy wemos mini D1 with 3x PZEM-004T V3.0 (modebus-rtu)

In view of the problem of storing data over a long period with HA, for now as workaround at the same time I store them in a CSV file. This from a Nodered outside HA

Probably not. Your bottleneck is most likely the SD card I/O rate and, to a lesser extend, the CPU speed to generate the graph.

Moving the database to another storage medium, like a NAS, will already help a lot. And it will also preserve the health of your SD card. If network storage is not an option, an SSD can help. Also, replacing the Pi3 with a Pi4 is going to help significantly on SD card i/o speed, network speed and CPU performance. It’s also pretty straightforward to add an SSD to the Pi4 now.

Others may suggest using a NUC instead of the Pi, but that’s overkill if you ask me.

1 Like

@HeyImAlex thank you for your precise analysis. I agree a NUC is overkill.
Today more than one minute to display graphs…
it’s just incredible to rely on high-performance hardware to catch up with a structural problem.
It has only about 80k-160k of data to read to display 24h, that should not be a problem today ! Sorry but this ruins this awesome program

I tried it, with a raspberry pi 3B 64bit, and a old ssd Samsung 840.
Yes it’s much faster :grinning: 10s vs 1 minute with the NAS.
I think my NAS has enough work with security camera.
And now:
Healthy Yes
Supported Yes

Grafana is the key :grinning:

I have had the same problem with unresponsive HA and slow retrieval of history, even to the extent of needing to restart HA without understanding why. I figured out in the end that the problems were caused because of high frequency updates and recording of real-time energy data and the dependent template sensors. It takes a long time for the front end to retrieve the data when it has been recorded every second for a whole week. The database didn’t seem to be an issue in my case, but data thinning through use of the throttle filter helped.

You can read how I solved this in my project post here: