Energy Dashboard got wrong data, no way to clear it

My Tesla Powerwall disconnected from my wifi network. When it reconnected my energy dashboard numbers went crazy high.

image

A bit sad because this will screw up my monthly, yearly statistics I was looking forward to. I cannot find any way to correct this bad data or to flag it as bad. It seems that there should be a mechanism for this.

Matt

I have also this problem but with my meters:
I calculate the Values for getting energy and delivering energy to/from the net in node red and it works fine. But today i made the update to 2021.9.x and now it shows me not the values of today but the total values
so instead of 10kWh i see 736kwh consumtion today an so on

1 Like

I got exactly the same issue. Insanely high solar production reported on several days, while the source from Solarman states normal values. I see no option to fix it. Would love a feature for this.

the only way to fix this kind of issues is by recalculating the statistics in your database manually.

Most of the time, the issue is a result from the following scenario:

The sensor is increasing constantly.
With every new statistic calculation, it will recalculate the last state with the new one - and create a sum.

for example:
10:00 - Value: 130 kWh - Sum: 0 kWh
11:00 - Value: 145 kWh - Sum: 15 kWh
12:00 - Value: 150 kWh - Sum: 20 kWh

During a Disconnect, it might be, that the Sensor becomes unavailable - or drop to 0 (depending on the integration or template)

continuing with the example above:

12:00 - Value: 150 kWh - Sum: 20 kWh
13:00 - Value: 0 kWh - Sum: 20 kWh (Disconnection timeframe)
14:00 - Value: 0 kWh - Sum: 20 kWh (Disconnection continues)

Now, the connection comes back - and the sensor value will be restored:

14:00 - Value: 0 kWh - Sum: 20 kWh (Disconnection continues)
15:00 - Value: 150 kWh - Sum: 170 kWh (Connection restored, sensor will be reset to its previous value)

Unfortunately, this is an issue which can happen for most integrations - and manual sensors which are added to the Energy Dashboard - and it is nearly impossible to cover those scenarios.
The best option would be to catch the situation that the sensor will be reset to 0 when it will become unavailable.
This needs to be done by either the integration - or within your template.

2 Likes

I also have this problem with the huawei_solar plugin.
Manually recalculating the values and updating the database seems the only solution, but its a lot more effort than I can put in right now, and i’m pretty sure I’d screw everything up.

My other post here: Energy dashboard showing wrong values · Issue #54946 · home-assistant/core · GitHub

yes, there’s a high chance of messing up the database.
Whenever I had such an issue, I’m ended up in dropping the database and creating a new one in the end.

Unfortunately, the issue is on how the energy dashboard is designed - AND additionally, how the integration is designed and handles the sensor became unavailable.
In some integrations, the sensor just fall to 0.0 instead of getting unavailable - and then, this will cause issues in the statistic-recordings.

So you should raise the ticket with the integration, rather than with the energy dashboard.

my integration (for my utility meter) is a rest/template sensor which happens to deliver a zero every now and then - so no point in raising a ticket with the integration i guess :upside_down_face:

but maybe this thread would be a good place for a guide on how to create a template which replaces such zero value with ‘unavailable’ (which is what would be needed to not throw off the statistics afaict).

since i’m struggling with this right now anyways. as said it’s a REST sensor i’m getting the data from:

sensor:
  - platform: rest
    name: evn_sensors
    resource: http://to.rest.api
    json_attributes:
      - 1.8.0
      - 2.8.0
    value_template: "OK"

which i’m pulling the values from like this:

template:
  - sensor:
      - name: "EVN import"
        unit_of_measurement: "Wh"
        device_class: energy
        state_class: total_increasing
        state: >-
          {{ state_attr('sensor.evn_sensors', '1.8.0') | int }}
      - name: "EVN export"
        unit_of_measurement: "Wh"
        device_class: energy
        state_class: total_increasing
        state: >-
          {{ state_attr('sensor.evn_sensors', '2.8.0') | int }}

so i’d need to get an if statement in there to check if it’s 0 and replace it with ‘unavailable’ - but i just can’t get it right :pleading_face:

anyone able to lend me a coding hand?

tia.

As someone brand new to home assistant and who takes the long term storage of my irreplaceable data seriously i’m horrified by what i’ve been seeing. The first thing’s i’m digging into is how to create proper, reliable backups of all of my data and all i’m running into are countless threads of some game breaking bug making it either impossible to properly backup data properly, or for it to create backups that are silently corrupt with no way of knowing until it’s too late. Or this issue, where a seemingly trivial problem requires a database expert to solve. How do they expect people to depend on a system like this when the foundation it’s built upon is seemingly rotten?

It’s looking like the safest course of action is to do nightly shutdowns of the virtual machine this is running on followed by complete snapshot backups of the entire OS. This is the only true reliable option for backups. Not an option for many.

nice to meet you.

i guess your misconception here is that nobody is expecting anybody to depend on anything without contributing to it - it’s a free and open DIY solution after all. so if something isn’t working as you would hope to it’s your own decision to either do (D) something about it (I) yourself (Y) - or go find a better solution. there are plenty of commercial products out there which offer the luxury of working out of the box like advertised.

my2c.

by using template sensors this can be solved by adding an availabilty template. it essentially adds a condition which defines when the sensor should return ‘unavailable’. this could also be set so it never returns less than the previous value - which is what most others have issues with when dealing with busted energy statistics.

since mine either works correctly or sends ‘0’ i just check for this. still gotta wait and see if it really works, but at least i got the config right. it’s a bit confusing as this changed from ‘availabilty_template’ in the legacy configuration format to ‘availability’ in the current one.

template:
  - sensor:
      - name: "EVN Import"
        unit_of_measurement: "Wh"
        device_class: energy
        state_class: total_increasing
        state: >-
          {{ state_attr('sensor.evn_sensors', '1.8.0') | int }}
        availability: >-
          {{ state_attr('sensor.evn_sensors', '1.8.0') | int > 0 }}