Sensor(s) with only valid values for reliable energy dashboard and calculation

Hello,
I am trying quite a long time to create sensors for saving only valid values especially for energy entities. I have several energy / consumption entities (PV production, production from battery (LG Resu 16), battery charging, water from espcam / AI on the Edge, gas from esp32 reed and grid via tibber websocket…)

My problem: When devices are unavailable / unknown / way wrong values from ai on the edge watermeter etc. these “errors” cascade for even more invalide states (e.g. my calculated self consumption etc.)

So I want to create always available and valid sensors as a base for the energy dashboard and own calculations. And of course cycling utility meters.

I have tried several ways:
template-sensor like this:

- name: "Gas Valid"
  unique_id: "gas_valid"
  state: >
    {%- set current_value = states('sensor.gaszaehler_gaszaehler') | float -%}
    {%- set last_valid_value = states('sensor.gas_valid') | float -%}
    {%- if is_number(current_value) -%}
      {%- set new_value = max(current_value, last_valid_value) -%}
      {{ new_value if new_value <= last_valid_value + 0.05 else last_valid_value }}
    {%- else -%}
      {{ last_valid_value if is_state('sensor.gas_valid', 'unknown') else 0.0 }}
    {%- endif -%}
  unit_of_measurement: "m³"
  device_class: gas
  state_class: total_increasing
  icon: mdi:gas-cylinder

the problem with this approach: after reloading the yamls to get this active the “valid” sensor is unavailable as there is no former valid value. If I set a valid value manually it works until I reload my yamls to add further configuration / sensors (or just restart for updating HAOS…)

My “working” approach:
automation to set only valid states to input_number sensors like this:

platform: time_pattern
seconds: "30"

condition: template
value_template: >-
  {{ float(states('sensor.pv_produktion_shelly')) >
  float(states('input_number.produktion_aus_pv_nur_gueltige')) }}

service: input_number.set_value
data:
  value: "{{ float(states('sensor.pv_produktion_shelly')) }}"
target:
  entity_id: input_number.produktion_aus_pv_nur_gueltige

to get these values available for utility meters I have created template sensors getting the state of the input_numbers.

So this a very unconvenient way in my opinion.

Do you have any suggestions how to it better / more convenient / more efficient?

Thank you so much!
Andi

Can you share a history graph of the gas sensor and the pv sensor so we can see what the erroneous data looks like?

Then we should be able to create some template sensors to filter it for you.

Thanks for your quick reply. So this is the watermeter:


as a consequence the energy dashboard shows waterconsumption of 218m³ in every hour / day, cost etc, whenever the value is unavailable.
Btw. the watermeter sensor has a state class “total increasing” so I would have expected it to not accept those lower values…

I don’t have an example of the power consumption at the moment, but I think it would be similar. Especially with calculated values like self consumption (pv-prod + grid-import + battery-import - battery-charge - grid-export).
So the overall goal is to have sensors NOT setting below their previous value.
As of writing this I see a flaw in my logic: the calculated self consumption will vary by including the battery with import and export…

Nevertheless, I don’t want to have unavailable / unknown sensors :wink:

Thanks!

Nope. It wont register the decrease it will register the increase. Which is causing your issue.

Where is this data coming from?

Which integration?

watermeter = espcam (ai on the edge) → mqtt → home assistant.
Maybe the cause is following:
AI on the edge can be configured to check the pausibility of the new read raw value. If it is significant higher than the last (valid) value, there is no new value to be sent, so 0 or unknown or whatever is sent via mqtt which causes the sensor state in HA to be incorrect, as it is just the value of the mqtt telegram (I think).
That caused me to think about “helper” sensors that are “controlled” by HA.