Energy dashboard: long-term statistics offset by one hour

[justinh998]

Hello everyone,

Note: Since I am unfortunately only allowed to include two media elements as a new user, I have inserted all images of the Energy Dashboard as Base64 strings.

I’m not sure whether I’m making a mistake here or whether this is actually a bug.

After a certain period of time, I can only view the states of entities in the history every hour instead of every minute or second, depending on how often the entity has been updated.

I can currently look back up to 12 days with detailed data. The statuses of entities that are older than 13 days are only displayed in hourly resolution. This is intentional so that the database doesn’t take on astronomical proportions after a few years.

I’ve been using the Energy Dashboard for about a year now, and today I noticed something: If the data of entities that I have defined for different areas (battery, solar, power grid …) are only available as long-term statistics (older than 12 days), then they are shifted by one hour in the Energy Dashboard.

A small example of this:

• On 02 May 2025 (a day with detailed status data), the Ch2 YieldDay entity has increased by 61 Wh according to the history from 08:00 to 09:00. The Energy Dashboard shows a value of 0.06 kWh for the same period. So it fits.
  

  
  
  
  

• On 16 April 2025 (a day with only long-term statistics data), the Ch2 YieldDay entity has increased by 85 Wh according to the history from 08:00 to 09:00, but the Energy Dashboard only shows 0.04 kWh for the same period, which does not fit, as 85 Wh would actually be 0.085 kWh (rounded up to 0.09 kWh).
  

  
  
  
  

Where does the 0.04 come from?

• If you look at the history again, you can see that the Ch2 Yield Day entity increased by 41 Wh from 07:00 to 08:00 - which corresponds to exactly 0.04 kWh .

The 0.085 kWh (0.09 rounded up) then only appears from 09:00 to 10:00 in the Energy Dashboard.

So for some reason, the data is always shifted forward by one hour when it comes from the long-term statistics.

Am I just making a mistake or is the data in the history / energy dashboard actually wrong?

If the data is actually falsified by the transformation to long-term data, it wouldn’t be the end of the world because it would be relatively easy to correct with an SQL query in the database (actually, all you have to do is ‘turn back’ all the Unix timestamps by one hour), but I would consider it a relatively big mistake.

[Biscuit (Geoff)]

You are correct in that there is a discrepancy in the display of hourly figures in the Energy Dashboard, when looking between the period with available entity state data, and the period with only long-term statistics data.

We can easily see entity state values using the History display, and here is my solar PV entity, with the value at 09:00 and the value at 10:00 local time this morning.

This is displaying correctly, as the entity state line stops at ‘now’ and is at ‘now’ the value of the entity state.

The state value at 09:00 was 10895.16, and at 10:00 was 10896.77. These figures are taken at the very first state change following the exact hour, and the difference between them is 1.61.

If we look at the Energy Dashboard, then the solar production column for 09:00 to 10:00, which appears just after 10:00 (ie at the end of the hour period described) has a value of 1.61.

So far, so good. The entity state is correct, the history is correct, and the energy dashboard is correct.

You are also correct in that Home Assistant keeps statistical values (as long as the statistics integration is turned on and the individual entity has the correct settings). These are both a five-minute summary and a one-hour summary, which is generated every hour immediately following the exact hour. Yes it is true that the Energy Dashboard switches to using the long-term statistics after the entity data has been purged at the default 10 day point, but the statistics data is already there and can be accessed using the Statistics Graph card.

Here is the plot for the exact same sensor, with values captured only at the hour.

You will see that at 08:00 the state value is 10895.15, and at 09:00 the value is 10896.75

These are correct.

The slight difference (10895.16 and 10895.15) is because the Statistics Integration captures the values at the exact hour, and I took my value at the first state value after the hour. What I should have done is take the last state value of the preceding hour. The difference is negligible, hence the solar graph still works.

The big difference is that the long-term state value is recorded for 08:00. Hence the ‘apparent’ time shift of one hour.

In slow motion:

The entity state was 10895.15 at 09:00, and the entity state was 10896.76 at 10:00 (exact time). The difference is 1.61.
The Energy Display uses the difference in the state value between 10:00 and 09:00 and plots this at 09:00 for the period 09:00 to 10:00.

The Statistics Integration holds both short and long-term tables:

The statistics_short_term table stores 5-minute aggregates of the data from the states table. The statistics table contains the long-term statistics data. It stores hourly aggregates of the data from the statistics_short_term table. For both tables, Depending on the entity type, different data is tracked.

The long-term entry for an hour period contains, where possible, mean, min, max, sum, and state values. Most importantly, for the state value:

• state - the sensor’s state at the end of the hour

hence for the hour period 08:00 to 09:00, the recorded long-term values are held at 08:00 for the hour, including the ‘state’ value which was the value at the end of the hour, not at the start of the hour.

This means that, when the Energy Display comes to work with the Long-Term statistics values (after 10 days) the period for 08:00 to 09:00 uses the LTS state value at 09:00 (which was the real state value at 10:00) and subtracts the LTS state value at 08:00 (which was the real state value at 09:00), and plots this on the graph at 08:00. Hence 08:00 to 09:00 as plotted is actually 09:00 to 10:00 as actually happened.

So it is all quite correct, just that one data set uses the state value at the start of the hour, and other the state value at the end of the hour.

@justinh998

If the data is actually falsified by the transformation to long-term data, it wouldn’t be the end of the world because it would be relatively easy to correct with an SQL query in the database (actually, all you have to do is ‘turn back’ all the Unix timestamps by one hour), but I would consider it a relatively big mistake.

The data is not falsified, it is just that LTS keeps the state value for the end of the hour, not the start of the hour (which is quite sensible), and the Energy Display does not shift the computation method to account for this. I guess you could raise this as a change request, or dive in and fix the code with a pull request. Yes, it probably is just a case of performing the SQL call on the LTS table one hour ahead.

For those that can code this stuff, it is probably harder to explain the problem than it is to actually fix it…

[justinh998]

Thank you for the detailed explanation

@Biscuit

The data is not falsified, it is just that LTS keeps the state value for the end of the hour, not the start of the hour (which is quite sensible)

Why exactly does this make more sense than saving the values at the beginning of the hour? Ultimately, it would not matter whether the value from the beginning of the hour or from the end of the hour is saved, in either case it is possible to trace what was consumed in the respective hour.

@Biscuit

Yes, it probably is just a case of performing the SQL call on the LTS table one hour ahead.

However, this would not be a good solution because the data would then be processed again unnecessarily, if only the values from the beginning of the hour (and not from the end) are saved, this step becomes obsolete.

@Biscuit

I guess you could raise this as a change request, or dive in and fix the code with a pull request. Yes, it probably is just a case of performing the SQL call on the LTS table one hour ahead.

That’s actually not a bad idea.
I would just have to familiarise myself with the Home Assistant code first.

[tom_l]

@justinh998

Why exactly does this make more sense than saving the values at the beginning of the hour?

Because the statistics are generated from the past hour’s data. At the end of the hour the stats reflect what just happened in that hour. Moving that to the beginning of the hour would look like HA could predict future use. It can’t. Nothing can (reliably).