Need help figuring out low pass filters and periodic sensor updates

Configuration
1

I’m having an issue due to one sensor not updating when the state does not change and therefore the low-pass filter I’m applying never converges. Here’s an example (red: sensor, purple: low-pass)

16%20PM

What I’m trying to do is take the illumination filter and low pass filter it to get less fluctuation over a short amount of time. I’ve tried using a filter directly on the sensor output, as well as creating a template sensor that I update using automation. In either case, no luck.

Here’s the current implementation using the template sensor:

Sensor defintion:

sensor:
      - platform: illuminance
        name: DSW Illuminance
        entity_id: weather.dark_sky
      - platform: template
        sensors:
          outdoor_illuminance:
            friendly_name: Outdoor Illuminance
            unit_of_measurement: lux
            value_template: "{{ states('sensor.dsw_illuminance') }}"
      - platform: filter
        name: illuminance filter three
        entity_id: sensor.outdoor_illuminance
        filters:
          - filter: lowpass
            time_constant: 3
            precision: 2

Automation:

    - alias: update_illuminance
      trigger:         
        platform: time   
        minutes: '/5'
        seconds: 00
      action:
        service: homeassistant.update_entity
        entity_id: sensor.outdoor_illuminance
        service: homeassistant.update_entity
        entity_id: sensor.illuminance_filter_three

I also tried running the automation directly on the illumination sensor (rather than on it’s template version) without luck.

Any idea how I could force an update every 5 minutes regardless of sensor state change such that the filter works properly?

1 Like
2

By the way, I tried to trigger the automation manually and that didn’t help either. The states of the different sensor did not update…

3

Just posting to let you know that you are not crazy. This filter is driving me nuts because it never “zeros out” like it should. It just runs parallel to the value like your graph. Did you ever solve it?

4

I never got the filter working as I intended but I came up with a little workaround to achieve the same thing… I created a virtual sensor that takes the input periodically and then adds a random small number (in the range of 0 to 0.01 or something like that). Then I use that sensor to do the filtering.

If it’s not clear, I can try to dig up the code when I’m on my computer…

5

Yes, this is the problem of an ‘event-based’ data stream vs a ‘regular-interval’ data stream based on set-frequency polling.

With event-based data streams, filtering out noise is much more difficult where there are long periods where a senor does not change (and a data-generating event doesn’t trigger) - you can’t properly process the value until AFTER it changes (when you know BOTH the VALUE and the PERIOD that the measurement stayed at the value). This means that you get lags in registering changes that can be long but also very variable/inconsistent which can propagate unexpected behaviours.

As you note, you need to add some regular polling (at a period matching the length of lag that is acceptable for your application) if the lags are creating problems.
It also means that low-pass-filters need to take into account that the periods associated with each incoming (event-triggered) measured value, which can vary (and can’t assumed to be at a fixed frequency). You are effectively driving from the rear-view mirror, where that distance that objects appear way can vary wildly.

6

I had the same problem.
The solution was, to config the Sensor (via MQTT in my case) with “force_update: true”

see: How to force a template sensor to update even if the value hasn’t changed?

1 Like
7

Something which is very convenient is to use ha_sampler. Very efficient !