New Filters: Increasing & Decreasing (including concrete examples)

Feature Requests
1

Use Case: sometimes you have sensors that are only allowed to increase in value, or decrease in value.

Example: a total kWh counter that takes the state of x individual kWh sensors and sums them together. Sometimes these sensors are unreliable: they can sometimes be unavailable, or report 0 values. As a result the total kWh counter can go down, which completely messes up the report generated by Utility Meter - Home Assistant.

Current Workaround: create an input_number to track the old value, and do a check in your sensor.template to see if the new value is less or more than the old value. The input number is updated using an automation. Requires a lot of confusing templating and an additional automation entry.

My request: add two filters that filter out values that are less than the previous value, or more than the previous value.

This would discard any new values that are less than the old value:

sensor:
  - platform: filter
    name: "filtered my total kwh"
    entity_id: sensor.my_total_kwh
    filters:
      - filter: increasing_only

And discarding new values that are more than the old value:

sensor:
  - platform: filter
    name: "filtered some sensor"
    entity_id: sensor.some_sensor_that_should_only_decrease
    filters:
      - filter: decreasing_only

How it would work in practice:

We would be able to reduce this:


input_number:
  energy_consumption_old_state:
    initial: 0
    min: 0
    max: 9999999999999999

sensor:
  - platform: template
    sensors:
      energy_consumption:
        friendly_name: "Energy Consumption"
        unit_of_measurement: "kWh"
        value_template: >
          {% set new_state = (states('sensor.fibarowp9_electric_consumed_kwh') | float + states('sensor.fibarowp10_electric_consumed_kwh') | float + states('sensor.fibarowp11_electric_consumed_kwh') | float + states('sensor.fibarowp12_electric_consumed_kwh') | float + states('sensor.fibarowp13_electric_consumed_kwh') | float + states('sensor.fibarowp14_electric_consumed_kwh') | float + states('sensor.fibarowp15_electric_consumed_kwh') | float + states('sensor.fibarowp16_electric_consumed_kwh') | float + states('sensor.fibarowp17_electric_consumed_kwh') | float) | round(3) %}
          {% if new_state >= states('input_number.energy_consumption_old_state') | float %}
            {{ new_state }}
          {% else %}
            {{ states('input_number.energy_consumption_old_state') | float }}
          {% endif %}

automation energy_consumption:
  - alias: "Check new state of sensor.energy_consumption and store it in input_number.energy_consumption_old_state if it has increased"
    trigger:
      - platform: state
        entity_id: sensor.energy_consumption
    condition:
      - condition: template
        value_template: "{{ (states('sensor.energy_consumption') | float > states('input_number.energy_consumption_old_state') | float) }}"
    action:
      - service: input_number.set_value
        data_template:
          entity_id: input_number.energy_consumption_old_state
          value: "{{ states('sensor.energy_consumption') | float }}"

To this:

sensor:
  - platform: template
    sensors:
      energy_consumption:
        friendly_name: "Energy Consumption"
        unit_of_measurement: "kWh"
        value_template: >
          {{ (states('sensor.fibarowp9_electric_consumed_kwh') | float + states('sensor.fibarowp10_electric_consumed_kwh') | float + states('sensor.fibarowp11_electric_consumed_kwh') | float + states('sensor.fibarowp12_electric_consumed_kwh') | float + states('sensor.fibarowp13_electric_consumed_kwh') | float + states('sensor.fibarowp14_electric_consumed_kwh') | float + states('sensor.fibarowp15_electric_consumed_kwh') | float + states('sensor.fibarowp16_electric_consumed_kwh') | float + states('sensor.fibarowp17_electric_consumed_kwh') | float) | round(3) }}
  - platform: filter
    name: "filtered energy consumption"
    entity_id: sensor.energy_consumption
    filters:
      - filter: increasing_only

Based on my experience here: Only allow a sensor value to increase - #10 by _Brian

2

I’m struggling with the same thing, to set up my energy sensors in a reliable way through filters before adding them as the input sensor for utility meters.
As you describe, sensors going offline will mess up the data.
For example if a utility meter has the option “net_consumption: false” it will ignore the decreasing value when a meter goes offline, but it will add the value when it comes back online.
For zwave entities that are never reset this value can be the total energy consumption for several years.
If you set “net_consumption: true” the total value will be correct though, but the graphs will be messed up, also if a entity is removed from the system this will also mess things up as the total will decrease.

I’ve now tried using the outlier function to restrict the raw data coming from my zwave entities before feeding the filtered value into utility meters for yearly/monthly/weekly/daily periods. But I’m not able to find the correct settings to avoid these things happening.

I’m adding a example scenario in case someone has any pointers

sensor:
# Electronics totals
  - platform: template
    sensors:
      total_electronics_energy:
        unit_of_measurement: kWh
        value_template: >-
          {{ states('sensor.server_energy')|float(default=0)
            + states('sensor.pc_energy')|float(default=0)
            + states('sensor.tv_energy')|float(default=0) }}

# Filter total electronics energy
  - platform: filter
    name: "total_electronics_energy_filtered"
    entity_id: sensor.total_electronics_energy
    filters:
# Only update consumption every minute
      - filter: time_throttle
        window_size: "00:01"
# Allow a maximum change of 5 kWh per 10 minutes (10 samples throttled to minute intervals)
      - filter: outlier
        window_size: 10
        radius: 5.0
        precision: 2

# Yearly electronics utility meter
utility_meter:
  electronics_energymeter_daily:
    source: sensor.total_electronics_energy_filtered
    cycle: daily
    net_consumption: false
3

Im in the exact same situation, really frustrating! Have you guys come up with any solution with the filter??

4

+1 for someone to implement the filter _Brian outlines. I would be a lot more efficient that the workaround I have used using an input number, automation and template sensor.

I hope this makes it onto the feature request list.

5

Correct me if I’m wrong, but I believe this can be addressed by the existing "outlier" filter. If window_size is set to 1, outliers will be replaced by the median of the N=1 last value(s), which is to say it will keep the last value. By doing so, it should preserve the strictly increasing/decreasing nature of the underlying signal.

I have a similar issue (outliers with my Qubino z-wave meter which trash my energy graphs every other week). I’m going to test this approach to see if I can fix it without introducing spurious decreasing values.

6

Nope, that wont work.

I think the solution is this.

1 Like