Filters like moving average, median and similar with deep sleep

ESPHome
1

I have various sensors that I want to mostly deep sleep due to power draw. The problem is that many of those sensors are jumpy, so I need filters like moving average and/or median. The problem is that when the device wakes from deep sleep, it seems the previous values are forgotten (which makes sense since deep sleep seems to work by resetting the device and booting fresh). Is there any way around that ? If not on the ESPHome side, would it be possible to apply filters like that on the HA side, and if so how ?

I just want my graphs to look a little less jumpy :frowning:

2

Statistics in HA

Use max age to keep the last data point. I forget whether that generates data points to fill the gap.

If not some charts will allow data generation to fill the gap. I think Apex does that.

Good luck

3

You can have the esp wake up, take several readings, then aggregate them into one and just send that to HA.

For example a median of 3 or 5 values.

I can provide example code if you need.

Medians are very effective at removing spikes.

4

Thanks, I know how to do that, but the sensor is just very jittery, and legitimately reports jumpy values, even with a median used like that. For me, the solution to a smooth and pratical graph was to do a median over measurements from a few minutes, and that worked perfectly, but it’ll drain a lot more battery that way.

5

From a first glance, that looks like sort of a meta entity; an entity that generates itself out of another entity ? At first glance that looks perfect for me, thank you! The Apex thing also looks exciting, will definitely try and tinker with that!

6

Hi Gaibhne,

Firstly the name “deep sleep” is a bit misleading - it’s actually a power off and on which is why all memory is lost (except for Global variables which are stored in NVRAM).

I have a similar situation with measuring LiPo battery voltage at my greenhouse. When the ESP32-S3 wakes up, the first battery voltage reading is always low. I don’t want to run the battery totally flat, so when the battery gets low (<3.4V) I send alerts, and if voltage < 3.2V I extend the deep_sleep time from 30 minutes to 3 hours.
Lately I have had spurious initial readings up to 0.4V lower than the voltage gauge is actually showing … resulting in my ESP32 going into long sleep without any notifications when the battery voltage is still a healthy 3.56V :angry:

I am still working on this. I need this to happen on the ESP32.

I have thought to simply ignore the first reading in each awake period … however I would still like to track (in HA) the actual INA3221 sensor readings. At the moment I am experimenting with a Copy Sensor Component to apply a median filter and take action based on that result.

7

FWIW, the copy sensor definitely is reducing the extent of the initial spikes …

  # copy of the battery voltage, allow smoothing
  - platform: copy
    source_id: batt_voltage
    name: "Copy of batt_voltage"
    # average readings out, so first reading won't shut down the ESP32
    filters:
      - median:
          window_size: 3
          send_every: 2
          send_first_at: 2
    accuracy_decimals: 2
    on_value:
      then:
        - script.execute: check_battery_voltage

image
image1382×499 31.9 KB

But I might just try with the skip_initial: 1 filter.

8

What is that script it is executing ? Can you post your whole config ?

9

I have some similar examples to the above here.

Personally I prefer to process/filter on the esp side. I send the processed values only but have toggles on the esp to send raw values for debugging if required.

I found a 5 or 10 point sample worked well even for jumpy values like ADC. Typically I could wake up, measure, send and sleep in about 10-15secs.

10

My whole greenhouse config is 785 lines long, most I think not relevant to your project. Even the check_battery_script is quite long (200 lines including lots of logger statements to aid debugging), but basically is

script:
  #
  # Checks for Li-Po battery under-voltage
  #
  - id: check_battery_voltage
    then:
      - logger.log: "called check_battery_voltage"

      #
      # check for low voltage
      #
      - if:
          condition: 
            lambda: return id(batt_voltage).state < 3.2;
          then:
            - logger.log: 
                format: "##### battery is <3.2v (%.2fv) LOW BATTERY - SHUTTING DOWN #####"
                args: [ 'id(batt_voltage).state' ]
            # raise an alarm / notification / email
            - homeassistant.action:
                action: notify.persistent_notification
                data:
                  title: Greenhouse Alert
                data_template:
                  message: "##### EXTREME LOW BATTERY {{ var }}v #####  GOING TO EXTRA LONG SLEEP #####"
                variables:
                  var:   'return id(batt_voltage).state;' 
            #
            # to have got here, should have already given warnings  ... so go into extended deep sleep
            # 
            - deep_sleep.enter:
                id: deep_sleep_1
                sleep_duration: $sleep_low_batt_duration ms

      - if:
          condition: 
            lambda: return id(batt_voltage).state < 3.3;
          then:
            - logger.log: 
                format: "### battery is <3.3v (%.2fv) #### Trying USB-in ###"
                args: [ 'id(batt_voltage).state' ]
            # raise an alarm / notification / email
            - homeassistant.action:
                action: notify.persistent_notification
                data:
                  title: Greenhouse Alert
                data_template:
                  message: "### LOW BATTERY {{ var }}v ####  Trying USB-in ###"
                variables:
                  var:  'return id(batt_voltage).state;' 
            - script.stop: check_battery_voltage      # end this script and return

      - if:
          condition: 
            lambda: return id(batt_voltage).state < 3.4;
          then:
            - logger.log: 
                format: "### battery is <3.4v (%.2fv) ### BATTERY LOW "
                args: [ 'id(batt_voltage).state' ]
            - homeassistant.action:
                action: notify.persistent_notification
                data:
                  title: Greenhouse Alert
                data_template:
                  message: "### LOW BATTERY warning {{ var }}v "
                variables:
                  var:  'return id(batt_voltage).state;'
            - script.stop: check_battery_voltage      # end this script and return

I do recommend taking a look at Mahko_Mahko’s ESPlanty project - I learnt a lot from it, and have even come around to programattically entering deep_sleep rather than using the official timing method (largely because my wi-fi at greenhouse can take longer to connect).