Energy Dashboard showing bogus values

[Zarox666]

It happend again today:

Yes this happened at 8 pm… I live in the southern part of germany. We had overcast the entire day and the PV stopped producing any power before 5pm

Other Examples:

Shelly 3EM 3-phases Energy sensor

Thanks for the hint @maxym Good Point! @fridayAr I think I got the same that you had today. The Engery Dashboard is just a buggy piece of junk. According to it my 350 VA UPS (listed as a Battery system in the Dashboard) consumed 12kWh between 13:00 and 14:00 today. What a nonsense… Even better: I have two Shellys measuring what goes into the Battery (UPS and Battery Charger. Values get combined into “UPS Energy to Battery”) and I have one Shelly that measures what comes out of the batter…

Shelly 3EM 3-phases Energy sensor

I love this solution and it worked like a charm until today. Today I noticed an error with this and I just don’t know where i have the error: [Bildschirm­foto 2022-12-12 um 11.48.40] I only produced 0.8kw but it tells me that I exported 3.5kw. the solar production value is correct. Did I make a mistake with the Riemann sum? # Sensor for Riemann sum of energy consumption (W -> Wh) sensor: - platform: integration source: sensor.Netz_Total_Power_Consumed name: energy_imported un…

Energy integration and utility meters

Hello everyone I’ve been playing around with the new built-in Energy integration. I have set everything up, found the documentation for utility meters to select different tariffs. The problem now with utility meters is that a cycle reset is required so the counter goes back to 0 every hour, month, day, year, etc… The problem is that when I do a reset, let’s say daily, the energy integration substracts the accumulated value and the next reading I get a negative value. I’ve been checking this d…

What is happening there?
Why is the Dashboard showing exorbitantly high or low values for a single hour, while the corresponding Sensors feeding the Dashboard dont show this increase at all…

And yes, all Values that showed Bogus values so far, do have state_class: total_increasing

[tom_l]

Show the history graph for the sensor(s) for that hour.

[maxym]

exactly, please show power and energy graphs.
Recently I faced impossible jumps in energy calcukated by Riemann sum integration configured to trapezoidal mode. I’ve been adviced to change to „left” mode. It was a solution in my case.

btw change of the mentioned mode is not possible in helper configuration for unknown reason. you have to delete the helper and create again (possibly with the same name to maintain data continuity)

[Zarox666]

Here you go:

so exaclty the same as in the incident in my first link

@maxym
All my Riemann Sums are “left” mode.
But I dont think it matters.

with @tom_l pointing me again at the sensors…

look what I found:

Let me explain:
“UPS Energy to Battery” is the sum of the Energy reported by the Shelly Plug S for the Battery Charger plus the Energy reported by th Shelly Plug S that sits on the input site of the UPS.
Why the Dashboard is correlating that with the produces Solar Enegery during that hour I dont know… thats probably an independant Bug.

Here is its definition:

# ---- UPS Energy to Battery ---- #
- sensor:
    - name: "UPS Energy to Battery"
      unit_of_measurement: "kWh"
      device_class: energy
      state_class: total_increasing
      state: >
        {{ [ float(states('sensor.ups_charger_energy')), 
             float(states('sensor.ups_input_energy')) ]
             | sum }}
      availability: >
        {{ not 'unavailable' in 
           [ states('sensor.ups_charger_energy'), 
             states('sensor.ups_input_energy') ] }}
# ---- UPS Energy to Battery ---- #

It has “state_class: total_increasing” how can it become 0 then?

And yes, that both independant Shellys have the same fault (Reported Energy dropping to 0) within 30 Seconds of each other is not very plausible…
Is there something going on with the Shelly Integration?

[tom_l]

There are other non numeric states than ‘unavailable’. Change your availability template to this:

- sensor:
    - name: "UPS Energy to Battery"
      unit_of_measurement: "kWh"
      device_class: energy
      state_class: total_increasing
      state: >
        {{ [ float(states('sensor.ups_charger_energy')), 
             float(states('sensor.ups_input_energy')) ]
             | sum }}
      availability: >
        {{ states('sensor.ups_charger_energy')|is_number and 
           states('sensor.ups_input_energy')|is_number }}
# ---- UPS Energy to Battery ---- #

[Zarox666]

Thanks for the hint, I changed the availabilty Check accrodingly

BUT
the problem was not, that the two input Sensors or the “UPS Energy to Battery” bacame anything other than a number… the input Values dropped to 0 and despite “state_class: total_increasing” UPS Energy to Battery dropped to 0 as well…

[tom_l]

The state class does not prevent the sensor from resetting. In fact it expects it to.

total_increasing Similar to total, with the restriction that the state represents a monotonically increasing positive total which periodically restarts counting from 0, e.g. a daily amount of consumed gas, weekly water consumption or lifetime energy consumption. Statistics of the accumulated growth of the sensor’s value since it was first added is updated every 5 minutes. A decreasing value is interpreted as the start of a new meter cycle or the replacement of the meter.

https://developers.home-assistant.io/docs/core/entity/sensor/#available-state-classes

[Zarox666]

I see…

but then the Answer is clear…

I will rebuild the “UPS Energy to Battery” Sensor that I do not sum the Energy reported by the Shellys but instead sum up the Power and then use a left Riemann Sum on it to get the kWh out of it. This works very well for all other Sensors.

[tom_l]

Why?

All you have to do is ensure the template goes to unavailable rather than 0 when it is not resolvable. Which the changes above will do.

It’s not the drop to 0 that introduces bogus data. It is the return to the total.

0 → total : add the whole total

Unavailable → to total : no change

[Zarox666]

It hurts so bad !!!
OBVIOUSLY!!! Thanks for clearing that one up!
Franticly-rewriting-all-availability-checks

Still does not explain, why the values on the Shellys themself dropped to 0.
And also, there was no return to the total.
The Sensor still show very small values… so it indeed reset to 0 and then counted forward from there.

caused by the two Input Sensors dropping to 0.

Compare with the Values from the History Graph in my earlier Post.

So neither was there a case of unavailbility, nor a return to total.
The Counter for Energy to Battery dropped to 0 (by some Error) and the Dashboard shows this as a Solar Production of 2,5kWh in the middle of the night ??

There is something not quite right here…

[tom_l]

@Zarox666

it indeed reset to 0 and then counted forward from there.

That’s ok. State class total_increasing can deal with that.

Your state template does not have defaults for your float conversion functions and the availability template might not have caught other non-numeric states so the template may have failed to load.

Using the float filter with a default is always recommended. e.g. (some_state)|float(0)

[Zarox666]

ah…

maybe that is why Summing up the Power and then building a Rieman Sum over it, seems to solve these issues…

But stilll… why does the Dashbord show a drop in Energy to Battery as an Increase in Solar production? Makes absolutely no sense to me.

[tom_l]

I don’t know. You have not shared the details of your solar production sensor.

[Zarox666]

The very first Screenshot in this post: Energy Dashboard showing bogus values - #4 by Zarox666

It did not drop, become unavailable, reset, jump or anything.
Its driven by a Riemann Sum:

[tom_l]

Why are you using the Riemann sum if you have a Shelly EM3?

The Shelly has energy sensors already that you can use.

[Zarox666]

Here is your answer

Shelly 3EM 3-phases Energy sensor

@teropop For what its worth and I hope it helps someone: I figured that my HA is giving me unrealistic Values for Energy Returned to the Grid out of my PV. After some thinking it became obvious whats going on. The Shelly measures Energy for each Phase seperately and for Consumption and Return seperately. My PV sits on Phase 1. If I produce more Power than what is consumed on Phase 1 the measured Power becomes negative and thus the Energy Returned Meter counts forward, despite the same Power…

[tom_l]

You can do those calculations with the energy sensors. No need to use the power and convert it to energy.

[Zarox666]

OK, enlighten me…
How do I get the correct values if the Shelly 3EM calculates Energy from-Grid and to-Grid seperately and independantly for all three phases?

But all this has nothing to do with the Question, why the Dashbord shows a reset of the Energy to Battery Sensor to 0 as increase in Solar Production.

So lets stay on topic please.
I am happy to discuss the need for the Calculations in the Thread for the Shelly 3EM.

[tom_l]

@Zarox666

But all this has nothing to do with the Question, why the Dashbord shows a reset of the Energy to Battery Sensor to 0 as increase in Solar Production.

It is because you do not have the correct sensors required for the energy dashboard.

What I have gathered so far is that you have a three phase system with solar connected to phase 1:

• Shelly EM phase 1: grid phase 1 to house and solar production.

• Shelly EM phase 2: grid phase 2 to house

• Shelly EM phase 3: grid phase 2 to house

These are not enough to provide the energy dashboard with the information it requires.

Phase 2 and 3 are fine, they are just loads. You can use the Shelly energy sensors.

Phase 1 is the issue.

You have no way of knowing how much solar energy you are producing. As you only know the net energy (load - solar). Without knowing how much your load is on this phase you can not create a solar production sensor.

P1 = load - solar. This will go negative when you are exporting and you can use templates to create sensors that track net export and import. You can use the export sensor (when the value is negative) as the “return to grid” sensor in the energy dashboard. You can not use a template to to send the positive value from this to the energy dashboard “solar production” sensor.

What you actually have when this value is positive is still load - solar not solar production which is what the energy dashboard requires for its calculations.

You need another sensor. One that measures the energy coming from your PV inverter. Does not matter if this goes to your load or to the grid or any mix of the two, it just measures how much is being produced by your solar panels. Is your solar inverter smart enough to connect to Home Assistant and provide this?

If so, set the energy dashboard up like this:

• You don’t need the Riemann sum sensor to split the positive and negative values on phase1.

1. • Create a utility meter with two tarrifs, export and import. Feed it a template sensor from phase 1 that always returns a positive value (use the |abs filter).
2. • Use an automation to switch the tariff depending on the phase 1 power being positive (import) or negative (export).

• Utility meter import tariff → Energy Dashboard Grid Consumption sensor.
• Shelly phase 2 energy → Energy Dashboard Grid Consumption sensor
• Shelly phase 3 energy → Energy Dashboard Grid Consumption sensor
• PV Inverter energy produced → Energy Dashboard Solar Production sensor
• Utility meter export tariff → Energy Dashboard Return to grid sensor.

[Zarox666]

The Values for the PV Production come from a Shelly 1PM.
The value of sensor.pv_anlage_power is used with a left rieman sum named sensor.pv_energy_produced which is listed in the Dashbord as value for the solar production.

It behaves analog to the 2 Rieman Sums for Engery from Grid and Energy to Grid.
The Value for sensor.pv_energy_produced has remained constant on that day since the PV stopped producing.

But all this has been documented on your request in the Screenshots in my previous posts.

So I am really not sure why you are going on about the usage of the Shelly 3EM as Powermeter while the only problem is, that the Dashboard shows an increase of 2.5kWh in a single hour, while the underlaying sensor remained stable, and another completely unrelated Sensor (Energy to Battery) dropped to 0 (for a yet unknown reason).

And as you can see from the inital post, I am not the only one affected by this.