Energy dashboard - Bad cost calculation after HA restart

[one1tick]

Hi, earlier this month, I have posted that issue… Energy dashboard - Strange cost calculation

I know now that it start as soon as I restart HA, but I don’t know the reason why HA restart put the mess in the cost calculation.

before restart :

after restart :

Any idea what could be wrong in my setup ? Thx for any help.

[tom_l]

You r grid consumption sensor requires an availability template to prevent it reporting 0 KWh when unavailable.

[one1tick]

Thanks a lot for your quick reply!

Tried to have a look but I don’t understand how it works… Could you elaborate a bit ?
Below is the templates I use for for consumption and pv production (as well as on et off peak cost calulation):

 # Energy Dashboard
  - name: energy_tot_grid_consumption
    state: "{{ states('sensor.kp_energy_tot_grid_consumption') | float(0) / 1000 }}"
    unit_of_measurement: kWh
    state_class: measurement
    device_class: energy
    attributes:         
      last_reset: "1970-01-01T00:00:00+00:00"
  - name: energy_tot_solar_prod
    state: "{{ states('sensor.kp_energy_tot_solar_production') | float(0) / 1000 }}"
    unit_of_measurement: kWh
    state_class: measurement
    device_class: energy
    attributes:         
      last_reset: "1970-01-01T00:00:00+00:00"  
  - name: energy_tot_solar_consumption
    state: "{{ states('sensor.kp_energy_tot_solar_consumption') | float(0) / 1000 }}"
    unit_of_measurement: kWh
    state_class: measurement
    device_class: energy
    attributes:         
      last_reset: "1970-01-01T00:00:00+00:00"  
  - name: energy_tot_return_to_grid
    state: "{{ states('sensor.energy_tot_solar_prod') | float(0) - states('sensor.energy_tot_solar_consumption') | float(0) }}"
    unit_of_measurement: kWh
    state_class: measurement
    device_class: energy
    attributes:         
      last_reset: "1970-01-01T00:00:00+00:00"

  # Tarifs électricité
  - name: reh_tariff_price
    unit_of_measurement: CHF/kWh
    state: >
      {% if is_state('utility_meter.reh_daily', 'peak') %}
        {{ 0.2571 }}
      {% elif is_state('utility_meter.reh_daily', 'offpeak') %}
        {{ 0.1732 }}
      {% endif %}

Thanks again for you help.

[tom_l]

Here’s the first one as an example:

 # Energy Dashboard
  - name: energy_tot_grid_consumption
    state: "{{ states('sensor.kp_energy_tot_grid_consumption') | float(0) / 1000 }}"
    unit_of_measurement: kWh
    state_class: total_increasing
    device_class: energy
    availability: "{{ states('sensor.kp_energy_tot_grid_consumption') | float(0) / 1000 > 1400 }}"

For starters you were using the wrong state_class, and the last_reset attribute is no longer required.

That may have fixed it but just in case the availability template will also offer a bit of extra protection. I am assuming that sensor.kp_energy_tot_grid_consumption divided by 1000 is currently over 1400kWh from the erroneous result you got in your energy dashboard. if this is not the case, adjust the availability template 1400 number. The result of your template should be just greater than this.

[one1tick]

Excellent, I have modified my setup according to your example… hopefully this will solve my issuer, let’s see next hour !

2 more questions…

1. I don’t understand what the is role of availability ? You are right, the ‘sensor.kp_energy_tot_grid_consumption’ divided by 1000 is > 1400kwh (it is 4445 kWh) therefore condition will be always True so why don’t we put ‘availability: True’ ?

2. As I have many days with wong cost, is it possible to correct the past datas ?

[tom_l]

@one1tick

I don’t understand what the is role of availability ? You are right, the ‘sensor.kp_energy_tot_grid_consumption’ divided by 1000 is > 1400kwh (it is 4445 kWh) therefore condition will be always True so why don’t we put ‘availability: True’ ?

Because when your sensor goes into a fault condition it should report “unavailable” or “unknown”, the energy dashboard knows to ignore this.

However your template sensor uses a float filter that has a default of 0. So if it can’t convert the state of the watched sensor to a number it reports 0. So unavailable or unknown or none becomes 0.

This causes your energy to go from 4445 kWh to 0kWh when your watched sensor in the template goes off-line for whatever reason (including restarts). Then when the sensor comes on-line again it shoots back up to 4445 kWh. 0 to 4445 causes the energy dashboard to record that massive spike in energy use.

The availability template will return false when your template drops below 1400kWh, and this will change the template sensor state to unavailable protecting you from the massive spike in the energy dashboard when the sensor comes back on line as it will change from unavailable to 4445 kWh, and this will be ignored.

@one1tick

As I have many days with wong cost, is it possible to correct the past datas ?

Yes but it is not trivial:

How to fix statistics data (e.g. energy data)

Hey I have managed (I think) to solve the issue which was bothering me for a long time and I have decided to share the solution with you guys. I hope that someone finds it helpful. Problem I have OWL energy sensor which sends the data via RF433. The problem is that sometimes the received values are not correct. Pretty often I get “0” as the energy_total and once the value jumped significantly by 78M. [image] I don’t have the sensor data to investigate what has happened that day and what wer…

[one1tick]

Thanks again for explanations. I have modified according your suggestions and restart HA. It looks ok for consumption but I got a spike for ‘return to grid’.

My sensors after your suggestions:

# Energy Dashboard
  - name: energy_tot_grid_consumption
    state: "{{ states('sensor.kp_energy_tot_grid_consumption') | float(0) / 1000 }}"
    unit_of_measurement: kWh
    state_class: total_increasing
    device_class: energy
    availability: "{{ states('sensor.kp_energy_tot_grid_consumption') | float(0) / 1000 > 1400 }}"
  - name: energy_tot_solar_prod
    state: "{{ states('sensor.kp_energy_tot_solar_production') | float(0) / 1000 }}"
    unit_of_measurement: kWh
    state_class: total_increasing
    device_class: energy
    availability: "{{ states('sensor.kp_energy_tot_solar_production') | float(0) / 1000 > 1400 }}"
  - name: energy_tot_solar_consumption
    state: "{{ states('sensor.kp_energy_tot_solar_consumption') | float(0) / 1000 }}"
    unit_of_measurement: kWh
    state_class: total_increasing
    device_class: energy
    availability: "{{ states('sensor.kp_energy_tot_solar_consumption') | float(0) / 1000 > 1400 }}"
  - name: energy_tot_return_to_grid
    state: "{{ states('sensor.energy_tot_solar_prod') | float(0) - states('sensor.energy_tot_solar_consumption') | float(0) }}"
    unit_of_measurement: kWh
    state_class: total_increasing
    device_class: energy

  # Tarifs électricité
  - name: reh_tariff_price
    unit_of_measurement: CHF/kWh
    state: >
      {% if is_state('utility_meter.reh_daily', 'peak') %}
        {{ 0.2571 }}
      {% elif is_state('utility_meter.reh_daily', 'offpeak') %}
        {{ 0.1732 }}
      {% endif %}

[tom_l]

Looks like you forgot the availability template for that template sensor:

@one1tick

  - name: energy_tot_return_to_grid
    state: "{{ states('sensor.energy_tot_solar_prod') | float(0) - states('sensor.energy_tot_solar_consumption') | float(0) }}"
    unit_of_measurement: kWh
    state_class: total_increasing
    device_class: energy

So the others are working. Nice.

You should also change those 1400 numbers. Make them just below the current total.

[one1tick]

Dumb i am…

Should be like this right ? (I put 5000 because ‘sensor.energy_tot_solar_prod’ - ‘sensor.energy_tot_solar_consumption’ = 5800) :

- name: energy_tot_return_to_grid
    state: "{{ states('sensor.energy_tot_solar_prod') | float(0) - states('sensor.energy_tot_solar_consumption') | float(0) }}"
    unit_of_measurement: kWh
    state_class: total_increasing
    device_class: energy
    availability: "{{ states('sensor.energy_tot_solar_prod') | float(0) - states('sensor.energy_tot_solar_consumption') | float(0) > 5500 }}"

@tom_l

You should also change those 1400 numbers. Make them just below the current total.

Does it mean, I should increase these thresholds over time ?

Thanks a lot ofr you r much appreciated help.

[tom_l]

@one1tick

Should be like this right ?

A better choice would be:

availability: "{{ states('sensor.energy_tot_solar_prod') | float(0) > 1000 and states('sensor.energy_tot_solar_consumption') | float(0) > 1000 }}"

Change the 1000 values to just under the individual totals. This will catch the error if only one of those sensors goes off-line.

@one1tick

Does it mean, I should increase these thresholds over time ?

You shouldn’t have to.

I had one instance where my solar inverter total energy drooped down to zero rather than stepping directly down, but that was an issue with the SMA integration (which has been fixed).

[one1tick]

Thanks, I will close the case (switch to solved) tomorrow if everything is ok.

Last question, is there any way to split the consumption between Peak and Offpeak as I have the 2 sensors ?

Thanks for help.

[tom_l]

Sure, just add both sensors instead of your grid consumption sensor.

[one1tick]

Great thanks. I will check.

For the correction of the wrong past data, I have decided to setup a VM of HA on my Unraid server (to be able to test the repair without breaking my live setuo), so I :

1. Backup my live HA
2. Create the VM on Unraid based on this image ‘haos_ova-7.2.qcow2’
3. Restore the backup

But the HA VM has issue with history and the energy dashboard does not appear whilst it should be a clone on my live setup which is working… any idea of the problem ?
Thansk again.

The error messages

[tom_l]

The database does not backup well, more often than not it gets corrupted. There is work being done to fix this but it’s not there yet.

You could try copying the home-assistant_v2.db file from one Home Assistant to the other. Stop Home Assistant before copying, and stop the new Home Assistant before pasting.

Or if you are using the MariaDB addon, stop the addon, create a partial backup of MariaDB only. Restore this to the VM Home Assistant.

[one1tick]

@tom_l

The database does not backup well, more often than not it gets corrupted. There is work being done to fix this but it’s not there yet.

Thx, does it mean that daily backups is useless in case of crash ?

[tom_l]

Not at all. The changes to your configuration and addons are accurately preserved.

[one1tick]

Thanks again @tom_i , I have corrected all the past datas and it is working well. Moreover I have learned a bit of sql and the way statistics works i ha…

On the other hand backup/restore of mariadb is a nghtmare it barely never worked (corrupt database) but thats another story.

Appreciate the time you spend to help me…

[CChris (Christoph)]

Hi all,

Today, I’ve added static costs to my energy dashboard as well - but I don’t know - it seems not to be working as I would expect…

1. input is limited to two characters after the comma:
   I pay: 0.393 € / kWh → allowed: 0.39 € / kWh
   I receive: 0.0775 € / kWh → allowed: 0.07 € / kWh

further more:
After exporting something over 10 kWh to the grid, the Dashboard is showing 0.44 € earnings…
But 10 x 0.7 would be 0.70 € in total…

OK, this COULD be caused because I already had some production / export before I added the price information… can this be recalculated somehow?

[Katamave (Venca)]

I have got a strange calculation of total cost. Its adding grid import and export together.
This started today when i have added static costs for the export with 3 digits like 0,051 to my energy dashboard.

I could fix this issue with correcting value to 0,05 as i found out that 3 digits are not supported.
Now the computation is correct.