Energy Statistics are completely off

Energy
1

Everything was working properly until mid of October 2024 (guess I installed a new update at this time).

So I have a SmartMeterReader via Tasmota which is sending the data to HA. While statistics seems to work fine, I now see totally ridiculous values a couple of times per day.

Bildschirmfoto 2024-11-06 um 13.52.02
Bildschirmfoto 2024-11-06 um 13.52.021526×706 55.2 KB

Bildschirmfoto 2024-11-06 um 13.52.13
Bildschirmfoto 2024-11-06 um 13.52.131526×706 40.4 KB

So of course I can go into the statistics and adjust the values:

Bildschirmfoto 2024-11-06 um 13.53.04
Bildschirmfoto 2024-11-06 um 13.53.04573×821 40.1 KB

But this is a pain in the butt as this now happens multiple times per day.
The bad thing, nothing has changed on the Tasmota SmartReader as well as on the MQTT server, so the issue must be inside HA.

A couple of log entries I could find for this statistics entity are here:

Entity sensor.stromzaehler_verbrauch_aktuell from integration template has state class total_increasing, but its state is not strictly increasing. Triggered by state 559.0 (570.0) with last_updated set to 2024-11-05T12:45:28.664900+00:00. Please create a bug report at Issues · home-assistant/core · GitHub
07:50:10 – (WARNING) Sensor - message first occurred at November 5, 2024 at 13:51:10 and shows up 5 times
Error while dispatching event for sensor.stromzaehler_energie_verbrauch to <Job track state_changed event [‘sensor.stromzaehler_energie_verbrauch’] HassJobType.Callback <bound method EnergyCostSensor._async_state_changed_listener of >>
06:51:09 – (ERROR) helpers/event.py
Entity sensor.stromzaehler_energie_verbrauch from integration template has state class total_increasing, but its state is negative. Triggered by state 51486.293 with last_updated set to 2024-11-06T05:50:59.588715+00:00. Please create a bug report at Issues · home-assistant/core · GitHub
06:51:09 – (WARNING) Sensor

The sensor itself is created via a template sensor:

- sensor:
  - name: "Stromzaehler Energie Verbrauch"
    unique_id: "stromzaehler_verbrauch"
    unit_of_measurement: 'kWh'
    device_class: "energy"
    state_class: "total_increasing"
    state: >
        {{ states('sensor.bitshake_smartmeterreader_mt681_total_in') | float }}

So any idea what is happening here? And…even more interesting…how to prevent this…

I was hoping for a HA update that might fix this…but to far…nothing new…

Bildschirmfoto 2024-11-06 um 14.13.26
Bildschirmfoto 2024-11-06 um 14.13.26697×344 12.9 KB

Thanks for your help…

2

Looks like your SmartMeterReader sensor is giving big negative numbers periodically, and then when it goes back to ‘normal’ this is interpreted as a large value increase (since the state class is total_increasing, it ignores decreases and only uses increases). You can confirm this by looking at the history for ‘sensor.bitshake_smartmeterreader_mt681_total_in’. As to the fix, that’s going to likely be dependent on the integration providing that sensor (or the firmware on the device).

3

Thanks for the hint. You are right, the sensor is sometimes delivering bogus readings. No idea what causes this…

{"Time":"2024-11-07T09:04:59","Uptime":"19T16:04:28","UptimeSec":1699468,"Heap":171,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":3,"Berry":{"HeapUsed":4,"Objects":39},"Wifi":{"AP":1,"SSId":"eismondIOT","BSSId":"F0:9F:C2:DD:1E:2E","Channel":1,"Mode":"11n","RSSI":100,"Signal":-37,"LinkCount":6,"Downtime":"0T00:03:45"}}
{"Time":"2024-11-07T09:04:59","MT681":{"Total_in":51506.579,"Total_t1":51506.579,"Total_t2":0.000,"Power_cur":698,"Total_out":0.000,"Meter_id":"090149534b0004300c6a"}}
{"StatusSNS":{"Time":"2024-11-07T09:05:00","MT681":{**"Total_in":3602879753402.976,**"Total_t1":103130721.683,"Total_t2":0.000,"Power_cur":724,"Total_out":0.000,"Meter_id":"090149534b0004300c6a"}}}
{"Time":"2024-11-07T09:05:09","Uptime":"19T16:04:38","UptimeSec":1699478,"Heap":171,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":3,"Berry":{"HeapUsed":4,"Objects":39},"Wifi":{"AP":1,"SSId":"eismondIOT","BSSId":"F0:9F:C2:DD:1E:2E","Channel":1,"Mode":"11n","RSSI":100,"Signal":-37,"LinkCount":6,"Downtime":"0T00:03:45"}}

I’ve seen on a different post that there seems to be a “filter-function” to ignore those way-off values, but I can’t get this to work.

Here is what I tried for my template sensor:

- sensor:
  - name: "Stromzaehler Energie Verbrauch"
    unique_id: "stromzaehler_verbrauch"
    unit_of_measurement: 'kWh'
    device_class: "energy"
    state_class: "total_increasing"
    filters:
        - filter: outlier
          window_size: 10
          radius: 5
    state: >
        {{ states('sensor.bitshake_smartmeterreader_mt681_total_in') | float }}

But this is giving me an error. And I assume it’s better to add this to the “sensor.bitshake_smartmeterreader_mt681_total_in” entitey?

So what’s the correct approach here?

Thanks…again…

4

Just one addition: The sensor.bitshake_smartmeterreader_mt681_total_in sensor has been automatically created via the Tasmota integration.

I assume that it would be the best to add the filter here, but I don’t know how. There is no definition for the device in any of the config files.