Energy dashboard - untracked consumption showing up as battery consumed (negative)

[jakhei]

Hi there,

I feel somewhat lost when it comes to correctly displaying the power distribution in the energy dashboard.

I do have a Shelly 3EM set up, that combines all three phases (working fine).
I do also have a solar system, hooked up to only one phase (also fine so far)
I did also add 2 batteries (BP-2500v1), that are a bit tricky to integrate - but the values on the battery itself seem fine to me (DC batteries, so no charging from the grid).

When I have a look at my energy dashboard, I get the following picture(s):

Let’s break it down:

• Until ~6am, the battery is drained with roughly the amount my home consumes “idle”. The values are properly displayed, battery consumed is ~0.2 kWh (batteries are set to discharge at 200W in total), and a little extra is consumed from grid.
• In the morning, batteries stop discharging, and energy consumption goes up (positive values)
• Solar power starts kicking in, (first time visible at the 8:00 bar), batteries are being charged (negative values), and energy is consumed from grid and some solar
• until 9:00, more energy is being produced than produced, therefore power is returned to the grid (violet), some power goes into batteries (red), and some is still being consumed before solar power kicks in
• the story goes on “correctly” until the 12:00 bar. Solar power (not connected to batteries, but directly fed into the home grid) now exceeds power consumption (> return to the grid), the two batteries are being charged by separate modules (negative red). However, I do now have a consumed battery amount of ~ -0.2 kWh
  The same is true for the 13:00 bar. The batteries did not discharge (all sensors on the battery show a discharge of 0, also the entities in the energy dashboard used for battery discharge don’t change by even a fraction:
  
  
  
  
  

Where it does get interesting:
If I check further down the energy dashboard, I can see untracked consumption exactly when the dashboard displayed power consumption (discharge) from the battery:

Checking the total power sensor for, e.g. 12:00-13:00, we can see that it fluctuates between ~0W and -530W:

adding up the (tracked) return to the grid of -0.35kWh and the untracked power consumption of -0.22 kWh (displayed as battery consumed in on the electricity usage graph) results in ~ -0.57 kWh - meaning the total power consumption in that timespan would have had to be -570W on average. We can clearly see, that it was not staying at -570W throughout the hour, but was rather fluctuating around - so the 0.35kWh tracked consumption seem to be about right.

I’m happy to provide the config files, but since they are a couple of 100 lines of code for the various sensors, I’d appreciate any hints of how to debug further. Most interesting would be, how that consumed battery value is being calculated, to check what sensors are actually being used, in order to compare it to the total power consumption sensors. I’m pretty sure there is a bug involved, since the values, without the untracked consumption, would be correct.

[JohnFLoki]

In the energygraoh you have a negative battery discharge, that should not be happening. Also the second Battery discharge is missing.
From your cost section, it seems you set the entities in the wrong sections.
Battery Power Bypass should be at the Battery section as charging sensor and PV Power as Solar Production.

[jakhei]

The bypass is only “converting” the power, that is looped through from the battery when it’s full, so it doesn’t count as battery discharge (which would otherwise be the case).

Let me explain the setup, maybe it brings more clarity (or in worst case, adds more confusion )

Here are my energy dashboard settings:

At the moment only battery 1 & 3 are connected, battery 2 has issues with the firmware update and is therefore not in use.

Sample settings for battery 1:

PV Power BP-2500-1 is an integral sensor (left riemann sum), based on this sensor:

name: "DC Solar Power In - BP-2500-1"
      unit_of_measurement: "W"
      device_class: power
      state: >
        {% set inv_dc = states('sensor.b2500_abcdefg123456_w3') | float(0) %}
        {% set net_batt = states('sensor.battery_net_discharge_corrected_bp_2500_1') | float(0) %}
        {% set solar = inv_dc - net_batt %}
        {% if solar < 0 %}
          0
        {% else %}
          {{ solar }}
        {% endif %}

inv_dc is the total solar power coming into the battery (sensor.b2500_abcdefg123456_w3)

In order to not count solar power coming from the battery (when the battery is full) as power discharge, and to fix an issue with the battery (displaying output power incorrectly (ie. it displays ~500W output, even though the inverter limits at 400W)), there is another helper, that corrects that behavior: battery_net_discharge_corrected:

- name: "Battery Net Discharge Corrected - BP-2500-1"
      unit_of_measurement: "W"
      device_class: power
      state: >
        {% set p_out = states('sensor.b2500_abcdefg123456_g3') | float(0) %}
        {% set p_in = states('sensor.b2500_abcdefg123456_w3') | float(0) %}
        {% set inv_dc = states('sensor.hoymiles_hm_800_2t_powerdc') | float(0) %}
        {% if states('sensor.b2500_abcdefg123456_pe') == 100 %}
          {# If battery is full, net discharge is only the amount above what is “looped through.” #}
          {{ [p_out - p_in, 0] | max }}
        {% else %}
          {# Original logic if battery is not full #}
          {% set effective_output = [p_out, inv_dc] | min %}
          {{ [effective_output - p_in, 0] | max }}
        {% endif %}

The battery power out sensor (in the energy dashboard) is again a left riemann intergral sensor, based on Battery Net Discharge Corrected AC - BP-2500-1, that converts the battery_net_discharge_corrected sensor from DC to AC:

- name: "Battery Net Discharge Corrected AC - BP-2500-1"
      unit_of_measurement: "W"
      device_class: power
      state: >
        {% set dc_power = states('sensor.battery_net_discharge_corrected_bp_2500_1') | float(0) %}
        {% set efficiency = states('sensor.hoymiles_hm_800_2t_efficiency') | float(0) %}
        {{ (dc_power * efficiency / 100) | round(2) }}

The final piece is now a separate solar sensor, that counts the solar power being looped through if the battery is full as solar power (and not as battery discharged). Since there might be issues again, with the battery reporting power output too high, we select the minimum value here:

name: "Battery Bypass Solar - BP-2500-1"
      unit_of_measurement: "W"
      device_class: power
      state: >
        {% set battery_pct = states('sensor.b2500_abcdefg123456_pe') | float(0) %}
        {% set p_out       = states('sensor.b2500_abcdefg123456_g3') | float(0) %}
        {% set p_in        = states('sensor.b2500_abcdefg123456_w3') | float(0) %}

        {% if battery_pct == 100 %}
          {{ [p_out, p_in] | min }}
        {% else %}
          0
        {% endif %}

[JohnFLoki]

Yeah, I have difficulties understanding your setup :D, BUT I think you may be able to solve this issue by adding PV Power to Solar panels AND Battery Input Energy. That way the bilances stay the same and the values should end up correct. If I am thinking correctly, you dont even need the bypass sensors.
If it still doesnt work you can double check that the values show what you think they do. If yes, then maybe you can upload a schematic of your setup, the sensors it has and in which direction it shows the power flow etc.
With a name like b2500_abcdefg123456_w3 I cant really imagine what it does and if it is correct at this place.

[jakhei]

I’ve found two issues in my config that I do want to check now (waiting for a proper sunny day…)

• the battery_net_discharge_corrected sensor did use the total power output of the inverter ( sensor.hoymiles_hm_800_2t_powerdc instead of only the one channel, the battery is connected to)
• the battery bypass integral sensor for battery 3 was defined as trapezodial sensor instead of left riemann

The sensor values (abcdef…_xyz) are kind of explained in the variables:

• _g3 = power out of the battery
• _w3 = power into the battery (from the solar panel)
• _pe = battery level of the battery (in %)

I’ll try to provide a graphic that shows where each of the sensors is being used, but this might take a little time (since it’s quite complex already).

Maybe to summarize what I want to achieve:

I’d like to distinguish between battery consumed and solar power simply looped through the battery. Solar power looped through (e.g. battery is full) should be displayed as solar power in the energy dashboard, whereas battery consumed (in this example: 100W) should be displayed as battery power consumed.

The standard behavior in Home Assistant seems to be that all power coming from the battery out sensor is regarded as battery power, which could result in 4kWh per day coming from the battery, even though the capacity is only about 2kWh.

The battery itself can’t adjust the power output, it will always run on full power. It can only switch on/of battery discharge. Adjusting the power output is only possible be limiting the inverter power output.
So if the battery is full, it will loop through solar power if discharge is disabled. If discharge is enabled, it will discharge at full power (limited only by the inverter). I haven’t tried combining discharging while solar power is active, so far I’ve only used to battery to cover my energy consumption at night (therefore no solar power in).
I would expect, that if I start discharing the battery throughout the day (ie. solar power in is active), it will discharge the battery at full power, while also charging it from solar - therefore the difference between max power out (limited by the inverter) and solar power in would be the battery discharge power.

I think I did build in some logic errors, but I basically got lost and need help sorting my logic again

[jakhei]

What I really don’t understand:
When I remove the batteries from the energy dashboard altogether, I do get the consumed solar power displayed correctly:

As soon as I add at least one of my batteries, a fraction of the solar power is not being displayed any longer, when adding both batteries, the solar power consumed disappaears completely:

This is not true however… my energy consumption was still covered by solar power, a part of it was returned to the grid, and another part was stored in my batteries.

I have also compared the sensor values for that timeframe (13:00-14:00):

Sensor	13:00	14:00	Difference
Sum Power consumed from grid	7393,91 kWh	7393,91 kWh	0 kWh
Sum Power returned to grid	560,06 kWh	560,28 kWh	0,22 kWh
-	-	-	-
Battery 1 Power in	69816,87 Wh	69996,25 Wh	180,08 Wh
Battery 3 Power in	66814,81 Wh	66958,88 Wh	144,07 Wh
Battery 1 Power out	32575,08 Wh	32575,08 Wh	0 Wh
Battery 3 Power out	32886,07 Wh	32886,07 Wh	0 Wh
Battery 1 Power out (bypass)	3577,10 Wh	3577,10 Wh	0 Wh
Battery 3 Power out (bypass)	20453,48 Wh	20453,48 Wh	0 Wh
Power Hoymiles HMS-1600	399601,78 Wh	399932,10 Wh	330,32 Wh
Hoymiles HM-800 (Channel 2) YieldDay	238 Wh	354 Wh	116 Wh
		Sum solar power:	770,47 Wh

So the correct values for 13:00-14:00 should be:

• consumed solar: 0,226 kWh (770,47 Wh - 220 Wh - 180,08 Wh - 144,07 Wh)
• power returned to grid: -0,22 kWh
• power stored into battery 1: -0,18 kWh
• power stored into battery 3: -0,14 kWh
• consumed battery: 0 (there was no power drain from my batteries)

There appears to be a slight offset for the power returned to grid sensor (which is actually identical for both the energy dashboard, and the individual sensor values above), which still doesn’t explain the negative consumed battery value.

So in the end, the consumed energy from solar power simply disappears, while a new value for consumed battery power (negative) appears, which I can’t explain.

I still have no clue what could possible be wrong…