Utility_meter wrong calculation

Iam testing utlity_meter and found it has issues when HA is restarted.

I am using fibaro dimmers that is providing entity energy, this is a incrementing meter of consumed energy in kWh, it is persistant and is always counting up without possibility to soft reset.

when i am using energy entity with utility_meter everything works fine until you restart ha. Utility meter is saving last value of the meter in core.restore_state, when HA restarts the energy entity value is added to the saved state value resulting in calculation error. for each restart it add the value of the energy entity.

Something is clearly off here.

Is this a bug or intended behavior?

There is a github issue: https://github.com/home-assistant/core/issues/38710

You or the other poster didn’t answer the developers question. It could be intended based on your answer.

Ok!?

Anyway the issue is not related to utility_meter, but template sensor, seems there are som bad design here. Upon restart template sensor is reporting 0 as value until its calculations is done. In a perfect world this should have been unavailable and not 0. Some of this problems was tried fixed with availability_template but that dont cover this issue.

Fixes would be:

  1. Set unprocessed template sensors to unavailable until processed instead of 0

  2. Delay utility meter calculation until template sensor is done. This is not recommended.

  3. Expand functionality of availability_template.

It seems that the"0 as startup" issue is quite common problem.

If your sensor is 0 at startup then you don’t have it configured properly to be unavailable. It’s taking a code path and supplying an answer. Post it here and we can fix it.

Example: If your template is:

{{ states('sensor.xxx') | float + states('sensor.yyy') | float }}

this will result in 0 at startup because "unavailable" | float will return 0. If they are both zero you get 0 + 0, which of course equals zero. So your options are to add an availability_template:

{{ state('sensor.xxx') in ['unknown', 'unavailable'] or state('sensor.yyy') in ['unknown', 'unavailable'] }}

Or add it to the value_template itself:

{% if state('sensor.xxx') in ['unknown', 'unavailable'] or state('sensor.yyy') in ['unknown', 'unavailable'] %}
  unavailable
{% else %}
  {{ states('sensor.xxx') | float + states('sensor.yyy') | float }}
{% endif %}

Thank you!

- platform: template
  sensors:
    light_total_energy:
      friendly_name: Light Total Energy
      entity_id: sensor.time
      unit_of_measurement: kWh
      icon_template: mdi:counter
      value_template: >
        {% set ns = namespace(states=[]) %}
        {% for s in states.sensor %}
          {% if s.object_id.endswith('_dimmer_energy') %}
            {% set ns.states = ns.states + [ s.state | float ] %}
          {% endif %}
        {% endfor %}
        {{ ns.states | sum | round(2) }}

so this sensors gives 0 at startup unitl sensor.time trigger calculation.
I dont know how to use the availability_template to set this sensor to unavailable until calculated.

Starting in 0.115 this should now update when any sensor updates because it will be monitoring states.sensor. When you update to 0.115, you can safely remove entity_id: sensor.time.

Until then, try this:

- platform: template
  sensors:
    light_total_energy:
      friendly_name: Light Total Energy
      entity_id: sensor.time
      unit_of_measurement: kWh
      icon_template: mdi:counter
      value_template: >
        {% set ns = namespace(states=[], offline=[]) %}
        {% for s in states.sensor %}
          {% if s.object_id.endswith('_dimmer_energy') %}
            {% if s.state in ['unavailable', 'unknown' ] %}
              {% set ns.offline = ns.offline + [ s.entity_id ] %}
            {% else %}
              {% set ns.states = ns.states + [ s.state | float ] %}
            {% endif %}
          {% endif %}
        {% endfor %}
        {% if ns.offline | count > 0 %}
          unavailable
        {% else %}
          {{ ns.states | sum | round(2) }}
        {% endif %}

If that gives you problems, i.e. you still get goofy numbers, use the availability template

- platform: template
  sensors:
    light_total_energy:
      friendly_name: Light Total Energy
      entity_id: sensor.time
      unit_of_measurement: kWh
      icon_template: mdi:counter
      value_template: >
        {% set ns = namespace(states=[]) %}
        {% for s in states.sensor %}
          {% if s.object_id.endswith('_dimmer_energy') %}
            {% set ns.states = ns.states + [ s.state | float ] %}
          {% endif %}
        {% endfor %}
        {{ ns.states | sum | round(2) }}
      availability_template: >
        {% set ns = namespace(offline=0) %}
        {% for s in states.sensor %}
          {% if s.object_id.endswith('_dimmer_energy') and s.state in ['unavailable', 'unknown' ] %}
            {% set ns.offline = ns.offline + 1 %}
          {% endif %}
        {% endfor %}
        {{ ns.offline == 0 }}

Last one might work, that is a neat way of doing it. Let me try it.

Thank you!

Yes, this fixes the issue. Thanks for all the help :grin:

1 Like

which one did you use? the availability template?

I used the one with availability_template, i think that would be the most correct. i did not test the other one.

1 Like

Thanks so much for this discussion. I have the same problem and we were already desperate. I’ve tried several procedures and the ones you suggest, and the only one that was 100% stable when restarting the Home Assistant, restarting the router, turning the router off and on, is:

sensor:
  - platform: template
    sensors:
      energy_consumption_total:
        friendly_name: Energy Consumption Total
        icon_template: mdi:flash
        unit_of_measurement: kWh
        value_template: >
          {% set ns = namespace(states=[], offline=[]) %}
          {% for s in states.sensor %}
            {% if s.object_id.endswith('shelly_shem_3_40f52001972c_1_total_consumption') %}
              {% if s.state in ['unavailable', 'unknown' ] %}
                {% set ns.offline = ns.offline + [ s.entity_id ] %}
              {% else %}
                {% set ns.states = ns.states + [ s.state | float ] %}
              {% endif %}
            {% endif %}
          {% endfor %}
          {% if ns.offline | count > 0 %}
            unavailable
          {% else %}
            {{ ns.states | sum | round(2) }}
          {% endif %}

The second option, where the “availability_template:” is tested, did not work properly if I turned the router off and on (more delay in sensor unavailability)

You wrote somewhere here that this should be solved by Home Assistant in version 0.115 and this complicated solution will no longer be needed. I’m using 0.118.4 right now and I’m still solving problems. As? Who to contact? Is this a bug, or rather an unfortunate feature of templates in Home Assistant?

It was the need for entity_id: sensor.time that is not needed after 0.115.

I dont know why availability template is not working for you, but that way is also fine.

Oh, thanks for the explanation. And do you have any idea how to get this formula? When is it necessary to add the values of three sensors?

value_template: "{{ states('sensor.shelly_shem_3_40f52001972c_1_total_consumption')|float + states('sensor.shelly_shem_3_40f52001972c_2_total_consumption')|float + states('sensor.shelly_shem_3_40f52001972c_3_total_consumption')|float}}"

I have the same problem, somebody can help to adapt the solution for calculated sensor value, please?