Fixing increasing sensor data

Configuration
1

I have some increasing utility meter sensors which have some bad data and I am unable to fix them from the developer → statistics tab, I change the wrong data but their value always bounces back

image
image1033×254 25.5 KB

image

I tried manually fixing all bad data in statistics and statistics_short_term tables as well without luck.
Where can I go to fix this data?

2

I am not sure if this is the solution and I would normally say it is not the way to go …but it worked for me. I experienced similar issues with my gas and electricity meters and this seemingly originated from the short_term, at some point it resets and for some reason it sometimes (!!?) creates values no–clue-why.
My solution is to

  • set statistics to the correct value, this I did manually as just a few records (I noticed this rather quickly)
  • delete all short_term for this sensor, await the first record (5-10mins) and set the sum to the same value of the newest record in statistics
3

For this

  • Set statistics to the correct value, both state and sum values?
    image
    image1155×233 42.2 KB
  • delete all short_term for this sensor, I have about 2200 rows of this sensor inside the statistics_short_term table, delete all of them?
4

Hi,
Statistics: .I only did the sum part…the state-difference plus previous sum should be the new sum, it should then update correctly based on new short_term
And I indeed removed all short_term for that specific sensor, waiting for the first record and use the 381.6 (in above case) to update its sum… after that, all OK again.
You may want to try this out on a separate instance…and for sure do a backup before :slight_smile:

I had issues with cost so I had to rebuild the whole cost sensor history too…that is a long and different story

5

Ok so in statistics I have these latest entries:

image
image1164×483 86.6 KB

They are too high by about 26.8, the 63.8 should actually be 37, I usually do an update sum = sum - 26.8 from the time I see the bad data, will that do it?
And once I delete the short term data, will it grab this latest value from statistics?

6

OK on sum repair, this would also show your energy dashboard as-it-should-be
And no, from that I have seen, short_term will not grab from stats…it seems to be the other way round…you can easily test this out by putting short term on low values and see them represented in 1-2 hours in the stats table.
So, after the sum-repair you have to make the last short-term record map the stats record.
IMO there is not much risk if you keep an eye out for 1-2hrs and then manually adjust where you see it go wrong…phpmyadmin allows manual record updates without sql as you already know

8

Don’t think that worked:

image
image1140×173 31.3 KB

Keeps pulling bad value back after I set it manually
The current state makes no sense, it keeps adding it to the short term table
image
Where can it be changed?

9

image
Wasn’t even aware of this but that is what fixes that value :slight_smile:

10

Well… I indeed did not think about source being wring, having seen many issues with the short_term.