Bad Data in Energy Dashboard

Something happened yesterday afternoon that has caused a problem with my energy dashboard. It seems that the sensor.grid_export_energy was at 13,790.76 kWh at 3:50 pm then it rolled over to zero at about 4:15 pm. I’m getting data via the Enphase Envoy integration, but it’s unclear to me how it gets into the grid_export_energy entity. It isn’t listed in the Envoy entities, but I did fine a sensor.grid_export_power in my config file that I must have put in on the advice of a tutorial at some point.

So my questions are, how do I go about debugging this to learn what happened, how do I keep it from happening again, and how do I fix the data currently in my database? I realize the last one might be a big challenge for someone of my limited skills, but this is currently overwhelming my annual energy plots, which is sort of the whole point to using HA for me.

You could try and go to dev tools > statistics and edit the entity and change it to zero at 3:50pm

Well that was easier than I expected. That seems to have fixed everything in the energy dashboard. There are still some other plots that I’ve made that are showing bad data, but they are much less important. Thanks for the help.

Any advice on debugging this so it doesn’t happen in the future?

No sorry, don’t know anything about that integration

Hi, every so often I get a spike on my water usage and it suggests I’ve used 45 billion litres of water! I have to go in and delete the entries for cost, hourly and daily usage.

Your sensors should not report zero when they are unavailable. They should report unavailable.

Hi Tom, is that a reply to my issue?

It is for both of you.

What is the origin of your sensor ? Is it an esp ? You could add a filter to prevent any 0 value to get in HA.
Phil

So, in my case, it’s an Node MCU running ESPEasy, I publish the count from a water meter and a gas meter on the same device using MQTT, home assistant is used as the broker and then the values are used as sensors for the energy dashboard. I don’t see how there can ever be a situation when the sensor can become unavailable all the while the broker is running. Or am I missing something?

Do the messages have the retained flag set?

No, they do not.

Ok so the sensors will be unavailable after a home assistant restart or reload, until a new value occurs.

Now the million dollar question, do you use a value_template in your mqtt sensor configs?

For my case, the issue was not a rogue zero, it was a rollover from a high value to zero. image

If your sensor state class is total_increasing:

Total → 0 → Total = add the whole total to the state

Total → unavailable → Total = do nothing.

I have the same problem. The value for the generated energy suddenly dropped to 0 and since then only confusing values ​​are displayed on the dashboard.

I’m having a similar issue. On June 5 I have what appears to be a momentary data point sometime between 2:00 and 4:00 pm. (I can only share one image per post, but it looks virtually identical to frankenberrypi’s image).

The bad data point reads ‘3:00 PM - 4:00 PM, Consumed solar: 9,900.9 kWh, Daily to Grid: 9,900.9 kWh’.

I understand that I could go and correct this value in my statistics, however, when I look at the underlying sensor data for each in Developer Tools → Statistics, the values for all sensors are already zero for that period.

My sensors started reading correctly today, but there is the issue of that bad data point skewing all of my historical data. Any advice or suggestions on how to purge this bad data point on my Energy Dashboard?

Update: solved, potentially user error on my part. I went to enter an offsetting correction to the data, but had another look at the 3:00-4:00PM period from that date and found the bad data point. It was easy to correct from there. Not sure how I could have missed that the first time, but thanks for the support anyway. :slight_smile:

I was not able to sort out the underlying problem for my setup. I suspect I made a mistake configuring one of the sensors for the energy dashboard. I think I had put a sensor as always increasing when in fact it rolls over every few months. The repair effort of editing the statistics did get the graphs back to normal.

Look here:

This could be the issue. Including a fix (work in progress)