Filtering **instant peaks** out of a sensor.... how can it be done?

[ludovicus]

I couldn’t find a way…

I want to measure the solar energy I consumed during the day.
This is the difference between the solar energy produced and the energy I sold to the provider.
I did it by creating a template (and to avoid peaks in one direction I have an if)

{% set solar_prod = (states('sensor.solar_energy_daily_yield') |float(0)/1000) %}
{% set energy_sold = states('sensor.to_netz_daily')|float(0) %}
{% set solar_consumed = (solar_prod - energy_sold) | float(0) %}

{% if solar_consumed <= 0 %}
  {{ 0 }}
{% else %}
  {{ solar_consumed }}
{% endif %}

But… I still get wrong results because there are sudden peaks… eg at 00:00.
→ I suppose this is because one of the sensors (the wrong! ) resets quicker than the other…and during the day, perhaps because the data is not there for a moment.

this is the pattern

Any ideas please?

[allard77 (Allard K)]

I would probably make an automation that runs every time one of the sensors is updated, and set the state to the from_state if the difference is too large to be physical. But if you do that the peak is still there of course, it’s just corrected more quickly. So you would then have to also delete the bad state so it isn’t recorded bythe recorder, or even remove from the records if it is recorded before the automation finishes. Not sure how to do that… I don’t know if it is possible to compare to the previous state directly in the template, that would be useful.

[diamant-x]

I used a filter entity based on a source entity that had a similar behavior for some similar use case. The pitty is then you can’t use the original one.

You could use the low-pass filter method. Resets to zero would take a couple updates longer to show but that’s what you have to give up in order to remove it.

Ideally you should contact the sensor manufacturer and ask them to fix it.

[ludovicus]

the thing is this isn’t a sensors problem (I think… )
The data comes from Tasmota, which reads the electricity meter (for the energy sold)…

I think the problem is that at 00:00 the sensors reset.
If the energy_sold_sensor resets and the solar_sensor doesn’t (even if this is a question of a few seconds), then the measurement shows this behaviour…

In the good old electronics world capacitors were used to “hold” a value and filter peaks…

[ludovicus]

to help others help me… perhaps this is useful.
(I looked in detail to the data on 2024-02-14:11:32

the “original” sensor for the solar_energy is so (i.e. data is not available for a time period…
to account for this I defined a template where if the data is not available it defaults to zero…

—> and this is what causes all the above because there are several calculations…

→ what I want is:
if has_value() = false then “use last known value”

but I dont know how to access the last value before the one currently being evaluated