Correcting Energy States in Home Assistant with SQL

Yesterday it snowed all day, and my photovoltaic panels were completely covered with snow. Nevertheless, Home Assistant showed a PV yield of 4kwh around 07:30. After some hours of troubleshooting I was able to find the error this morning. For whatever reason, my power meter had reset the info interface to the default, which no longer provides the value in the desired resolution.

That was quickly resolved. But now there were incorrect state values in Home Assistant that needed to be corrected. And I want to share these steps with you how I corrected the values.

I’m assuming SQLite Web is installed and configured to access the home assistant database.

Since I knew the timing and entity of the error, finding the wrong values was quite easy. What I want to correct is the state and statistics, and we start the with the state:

The ID I was looking for is 1058. The next step is to look for the wrong counter readings. The actual counter value is 40917.2

As I scroll further, I suddenly found the outliers.

For the next step, the upper and lower State_IDs are needed:

  • 63274444
  • 62767537
    All entries in between should be overwritten. In this case, that’s about 11 records.

After the update, a quick check of the states:

This looks good.

Now I apply the same process to the statistics – the SQL Statements are listed here for reference:


SELECT * FROM "statistics_meta" where statistic_id = "sensor.esp_heizungunten_pv_ertrag_gesamt"

SELECT * FROM "statistics" where metadata_id = 138 order by id desc

UPDATE "statistics"
set state = 40917.2
where
metadata_id = 138 and id > 927664 and id < 931627

SELECT * FROM "statistics_short_term" where metadata_id = 138 order by id desc

UPDATE "statistics_short_term"
set state = 40917.2
where 
metadata_id = 138 and id > 11124473 and id < 11169297

And here is the result:

I hope this helps others in understanding how to update values in the Home Assistant Database.

Cheers

3 Likes

After looking at the energy dashboard I recognized that there was still the unwanted 4kwh peek :frowning:

Looking at the database it was clear - the Energy Dashboard uses the sum flield:

So what I did now was the same method I used before, but with manual correction.

For example - for all state values with 40917.2 I set the same sum of 3502.29999…

For the next state of 40917.3 I set the sum to 3502.4 and so on. This needs to be corrected in statistics and statistics_short_term.

Wow… I’ve had to edit statistics using the UI way too often to fix the same outliers. I know it’s a problem with the energy monitor I’m using reporting, but I can’t fix that so am left to trying to detect the statistics outliers and then manually changing them. It sucks… I really wish there was a better workflow for this.

Why not?

You can filter all sorts of issues through template sensors and utility meters.

Or you could wait 10 days (default) for the state data to purge.

The long term statistics that the energy dashboard uses can be fixed a lot easier than messing about with the database by going to Developer Tools → Statistics.

Yeah, I suppose you’re right that I could set up manual filters for all the sensors that bug out. I’ve avoided that option with equal parts laziness and just feeling like it was too hacky. Whenever I start hard-coding outlier limits I’m concerned I’ll set them at the wrong spot and lose data in a different way. But either way, thanks for the advice!