Battery powered: How to read sensors on boot, send them once, and immediately enter deep sleep?

Hello,

I have found many battery powered projects that use esphome, but almost none of them strive to be as power efficient as I’d like.

That’s why I’m hoping you, great members of the forum, can help me figure out … pretty much the title. For the sake of everyone and their batteries :heart:

Just to reiterate and make things clear: I want to achieve the following operation with the esp home:

  1. ESP Boot up - read the sensor values ONCE immediately or asap, as we might miss some important inputs. Filters should be applied such as averaging a value from ADC, the point is to do this asap, just once for every sensor, store it until we can send it out.

  2. After checking the sensors, do whatever else needs to be done normally (connect to wifi, run scripts by priority, …)

  3. As soon as wifi connection and MQTT connections are successfully established/ready, send out all the sensor values.

  4. When all values have been sent and successfully received by MQTT (it would be very welcome if there’d be some ack, not just ‘send UDP and Yolo it lol’), immediately enter deep sleep. No stalling here.

Maybe read the disable deep sleep for OTA MQTT message before that but generally, we ain’t wasting milliseconds in this project.

I think that’s the gist of it. Maybe it’s stupid easy and I’m just blind, but as I haven’t really found anything about this other than this blog post - Optimizing ESP8266/ESPHOME for battery power (and making an ice bath thermometer as well) – Necromancer's notes - which seems to have almost all of the answers, except I can’t, for example, manually call update for the binary sensor components, copy components etc.

I also haven’t found functions in esphome for the mentioned checks (are we connected to wifi? To the MQTT server? Were all sensor values already sent and successfully received by the server?)

What I know so far:

Appears that point 1. Can apparently be easily solved by assigning high setup priority values to the time sensitive sensors - in combination with setting ‘unrealistically high’ sensor update values (one hour, one day, etc…), to make them be read and updated only once,

At point 2. - we don’t generally need to do much ourselves here, just let the esphome handle things, maybe run early scripts if needed

The points 3. and 4. seem like the biggest hurdles to me, I haven’t found any way to check that all sensor values have been successfully sent and received by the server, therefore I cannot reliably assume that it’s safe to immediately enter deep sleep.

Once figured out, this would generally be useable for most kinds of ultra-low power sensors that output simple digital values.

Thanks to anyone that even just managed to read this all trough~!

P.s. if it turns out to be successful, I’m hoping to make a complete how-to.

2 Likes

I am not sure that esphome gives you that degree of control. For example, when a sensor is read the reading is transmitted to HA via the API or transmitted to the MQTT broker via MQTT. There doesn’t seem to be a concept of “store and forward”.

It does indeed appear that way, although from my testing, the debug messages indicate that it is possible to read sensors before setting up wifi, and they’ll get updated accordingly.

Now that I think about it, isn’t there a way to “pause” the sensor? I don’t want it to be unavailable, but it could be a way to ensure every sensor reads only once, by disabling “self” after first reading by on_value/on_state triggers…

You can take a look at these projects. They may get you closer.

1 Like

Maybe you could (re)import the last updated info for the sensor as a text sensor attribute and then do some checks to “wait until” that is recent?

Hello @Mahko_Mahko, and thank you for your replies, I took a look at the projects.

I was hoping to avoid exactly this kind of workaround of having like a “data received” sensor in home assistant.

I might just take a look and see if it would be too difficult to expose more functions, or ideally callbacks for this use case.

Imagine you could just define a “data received” callback to simply point to a function or even if just set the global variable to know that data from a particular sensor have just been successfully received by home assistant instead of … whatever this sensor send / receive duplication this is…

I hope there’s a better way, else I might’ve just coded my project all directly by hand from scratch like the rest of my projects. Which would be a real pity since esphome is otherwise amazing.

1 Like


Most of the time the esp will be awake is waiting for WIFI and MQTT to connect. Once connected it can send and go back to sleep in less than 10 milliseconds. From when esp led flashes to the logs showing connection to WIFI is about 6 seconds.

1 Like

Yes, I measured similar wifi connection times in my setup as well.

Just about in line with my expectations, no reason for it to be slow. Not sure how exactly you have measured it though.

If you have achieved what you’re describing, would you mind sharing more details or the relevant yaml? It would be very appreciated.

I’m keen to hear how you get on with this. Please post progress/ final solution. Thanks!

esphome:
  name: weathersmall
  platform: ESP8266
  board: esp01_1m

  on_boot:
    priority: 1000
    then:
      - lambda: |-
          pinMode(13, OUTPUT);
          digitalWrite(13, HIGH);

# Enable logging
logger: 
  level: INFO
  
ota:
  password: "******"

wifi:
  ssid: "*********"
  password: "******"
  manual_ip:
    static_ip: 192.168.1.50
    gateway: 192.168.1.1
    subnet: 255.255.255.0
    dns1: 192.168.1.1
    dns2: 8.8.8.8  
  output_power: 13dB
  fast_connect: true

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "smallweather Fallback Hotspot"
    password: "*"
    ap_timeout: 15min

captive_portal:
switch:
  - platform: restart
    name: "reset"


i2c:
    sda: 4
    scl: 5
    scan: False

sensor:

  - platform: bmp280
    address: 0x76
    update_interval: 60s 
    temperature:
      name: Temp
    pressure:
      name: Pres


 
  - platform: adc
    pin: A0
    name: cell
    id: sw_cell
    filters:
      - multiply: 4.55
    on_value:
      then:
      - if:
          condition:
            lambda: 'return id(sw_cell).state <3.5;'
          then:
          - lambda: |-
              id(deep_sleep_1).set_sleep_duration(3000000); 
          else:
          - lambda: |-
              id(deep_sleep_1).set_sleep_duration(300000);
    
    accuracy_decimals: 3
    update_interval: 60s

  - platform: wifi_signal
    name: " WiFi Signal"
    update_interval: 60s      
    
  - platform: mqtt_subscribe
    name: "Sleep time topic"
    id: swcustom_sleep_time
    unit_of_measurement: ms
    accuracy_decimals: 0
    topic: weathersmall/sleep_mode_time
    on_value:
      then:
       - lambda: |-
          id(deep_sleep_1).set_sleep_duration(id(swcustom_sleep_time).state);	    
    
mqtt:
  broker: 192.168.1.10
  username: *******
  password: ******
  birth_message:
    topic: weathersmall/birth
    payload: 'ON'
  will_message:
    topic: weathersmall/willdisable
    payload: disable
  discovery: true
  discovery_retain: true    
  on_message:
    - topic: weathersmall/ota_mode
      payload: 'ON'
      then:
        - logger.log: 'OTA Mode ON - Deep sleep DISABLED'
        - deep_sleep.prevent: deep_sleep_1

    - topic: weathersmall/sleepearly
      payload: 'ON'
      then:
        - deep_sleep.enter: deep_sleep_1              


deep_sleep:
  id: deep_sleep_1
  run_duration: 90s
  sleep_duration: 5min     						 

This is the ESPhome yaml. So once the esp connects to MQTT it sends info from sensors. In my case battery voltage, wifi signal, pressure and temp. Then an automation sends the esp to deep sleep early rather than having to wait 90s or 10s or whatever. This happens in less than 1 second. To reduce drain of battery I have set wifi power to only 13dB. Setting ip addresses and fast connect reduce connect time. I have set the sensors to be powered from a pin other than 3.3V so it’s not getting powered when in deep sleep.

alias: weather small sleep early
description: when gets battery state send mqtt to go to sleep
trigger:
  - platform: mqtt
    topic: weathersmall/sensor/cell/state
condition: []
action:
  - service: mqtt.publish
    data:
      qos: 0
      retain: false
      topic: weathersmall/sleepearly
      payload: "ON"
  - delay:
      hours: 0
      minutes: 0
      seconds: 30
      milliseconds: 0
  - service: mqtt.publish
    data:
      qos: 0
      retain: false
      topic: weathersmall/sleepearly
      payload: "OFF"
mode: single

This automation once it receives an MQTT from the esp it sends an MQTT message back to go to sleep early.

Still the big problem is I haven’t been able to reduce the time further from device beginning to wake and wifi connect which is about 6 seconds

Regarding the time measurement I checked the MQTTExplorer and time stamp suggest about 200ms for all mqtt messages to go back and forth then deep sleep. I don’t know where I got the idea that it was 10 milliseconds.

1 Like

Hello everyone, I’ll try to summarize here what I’ve learned:

First, context: I was building a mailbox notifier, that was just on a border of wifi availability. In addition to that, I made a power latch / cutoff circuit specifically tailored to the task, resulting in 0.0µA - that is, as far as my DMM can measure it. I tried to connect a li-poly battery to an onboard linear regulator, however, it seems that the voltage drop sets in rather quickly and is quite big. I’ll be moving over to lifepo4 cell, once they arrive, as I can bypass the regulator with their voltage range.

As far as the connectivity goes, I chose to go with MQTT. Regarding wifi, just a small note, in case you made the lowest allowed rate of the 802.11/b clients higher, you might experience connectivity issues - like I did.

About the core of the issue - I’m not happy with my solution as it isn’t much better than other solutions, already mentioned here, however I have settled with it for now.

Here’s the stripped-down yaml - hope I won’t remove anything essential:

esphome:
#...
globals:
  - id: mailbox_occupancy_verified
    type: bool
    restore_value: no
    initial_value: 'false'
  - id: batt_voltage_verified
    type: bool
    restore_value: no
    initial_value: 'false'
script:
  - id: invert_led
    then:
      - switch.toggle: led_builtin

  - id: blink
    then:
      - switch.toggle: led_builtin
      - delay: 100ms
      - switch.toggle: led_builtin
      - delay: 100ms

  - id: check_and_poweroff
    then:
      - repeat: 
          count: 10 #read the adc enough times to fill the filter window
          then:
            - component.update: batt_adc
            - delay: 10ms
      - component.update: wifi_signal_db
      - wait_until: 
          timeout: 10s
          condition: 
            - lambda: 
                return id(mailbox_occupancy_verified);
      - deep_sleep.prevent: deepsleep
      - wait_until: # if the mailbox doors are left open, it also serves as an OTA mode indicator - don't sleep
          condition:
            binary_sensor.is_off: access_doors
      - component.update: upT
      - delay: 100ms #hopefully uptime will make it in time, as the more stuff we do after now the less accurate the uptime metric will be.
      - deep_sleep.allow: deepsleep
      - deep_sleep.enter: deepsleep

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  fast_connect: True
  manual_ip: 
#   ...
  power_save_mode: HIGH
#  power_save_mode: LIGHT
#  power_save_mode: NONE
  on_connect:
    then:
      - script.execute: blink #nothing like a good old led debugging...
mqtt:
#  id: mqtt
  broker: 192.168.0.60
  username: !secret mqtt_uname_iot
  password: !secret mqtt_passwd_iot
  birth_message: 
  will_message: 
  on_connect: 
    then:
      - script.execute: blink
      - script.execute: check_and_poweroff

deep_sleep:
  run_duration: 30sec #should kick in in case wifi doesn't connect in 30 sec but for some reason it doesn't...
  id: deepsleep
  
sensor:
  - platform: wifi_signal # Reports the WiFi signal strength/RSSI in dB
    name: "WiFi Signal dB"
    id: wifi_signal_db
    update_interval: never
    entity_category: "diagnostic"

  - platform: copy # Reports the WiFi signal strength in %
    source_id: wifi_signal_db
    id: wifi_signal_percent
    name: "WiFi Signal Percent"
    filters:
      - lambda: return min(max(2 * (x + 100.0), 0.0), 100.0);
    unit_of_measurement: "Signal %"
    entity_category: "diagnostic"

  - platform: uptime
    name: "Uptime"
    id: upT
    update_interval: never
    unit_of_measurement: s
    accuracy_decimals: 3

  - platform: adc
    pin: 3
    name: "Battery ADC"
    id: batt_adc
    update_interval: never
    attenuation: 11db
    accuracy_decimals: 2
    filters: 
      - multiply: 2
      - offset: +0.05

  - platform: copy
    source_id: batt_adc
    id: batt_voltage_filtered
    icon: "mdi:battery"
    name:  Battery Voltage
    unit_of_measurement: V
    accuracy_decimals: 2
    filters:
      - max: 
          window_size: 10
          send_every: 10
          send_first_at: 10

  - platform: copy
    source_id: batt_voltage_filtered
    id: batt_level
    icon: "mdi:battery"
    name: Battery Percent
    unit_of_measurement: '%'
    accuracy_decimals: 0
    filters:
      # Map from voltage to Battery level
      - calibrate_linear:
          - 3.6 -> 0 
          - 4.1 -> 100
      #Overide values less than 0% and more than 100%
      - lambda: |
          if (x < 0) return 0; 
          else if (x > 100) return 100;
          else return ceil(x / 5) * 5;

  - platform: internal_temperature
    name: "Internal Temperature Sensor"

switch:
  - platform: shutdown
    name: "ESP deepsleep"

  - platform: gpio
    id: power_cutoff #pin to cut power to self, currently not in use since entering deep sleep has the equivalent effect, and that way esphome expects power loss.
    pin: 
      number: 12
      mode: 
        output: True
    name: "Power cutoff"
    setup_priority: 800
    restore_mode: ALWAYS_ON

  - platform: gpio
    pin:
      number: 15
      mode:
        output: True
    name: "onboard LED"
    id: led_builtin
    setup_priority: 800
    restore_mode: ALWAYS_OFF

binary_sensor:
  - platform: gpio
    pin:
      number: 7
      inverted: False
      mode: 
        input: True
        pulldown: True
    publish_initial_state: True
    id: access_doors
    name: "ACCESS DOORS"
    device_class: door
    setup_priority: 800

  - platform: gpio
    pin:
      number: 11
      inverted: False
      mode: 
        input: True
        pulldown: True
    name: "Occupancy"
    id: mailbox_occupancy
    device_class: occupancy
    setup_priority: 800
    publish_initial_state: True
    filters:
    - lambda: !lambda |-
        // Woke up. Modify my reported state depending on whether the mailbox doors are open.
        if (id(access_doors).state) {
          return 0; //The mailbox access doors are open. Therefore the mailbox will now be empty.
        } else {
          return 1; //Woke up but the mailbox doors aren't open. Therefore input door sensor must have been triggered. Something is now likely in the mailbox.
        }

text_sensor:
  - platform: mqtt_subscribe
    id: verify_mailbox
    name: "Occupancy verify"
    setup_priority: -100
    topic: mailbox-32-s2/binary_sensor/occupancy/state
    filters:
      - lambda: |-
          if (x == "OFF" && id(mailbox_occupancy).state == 0) {
            id(mailbox_occupancy_verified) = true;
            return (std::string) "VERIFIED"; }
          if (x == "ON" && id(mailbox_occupancy).state == 1) {
            id(mailbox_occupancy_verified) = true;
            return (std::string) "VERIFIED"; }

          id(mailbox_occupancy_verified) = false;
          return (std::string) "NOT_VERIFIED";

#  - platform: mqtt_subscribe
#    name: "Battery voltage verify"
#    id: verify_batt_level
#    setup_priority: -100
#    topic: 32-s2-wemoss2-mini/sensor/battery_voltage/state
#    filters:
#      - lambda: |-
#          std::string s = to_string(round(id(batt_voltage_filtered).state * 1000) / 1000);
#          s = s.substr(0, s.find(".")+3);
#          if (x == s) {
#          //if (strtof(x.c_str(), NULL) == id(batt_voltage_filtered).state) {
#          //if (x == to_string(round(id(batt_voltage_filtered).state * 100) / 100)) {
#            id(batt_voltage_verified) = true;
#            return (std::string) "VERIFIED"; }
#          //return to_string(strtof(x.c_str(), NULL));
#
#          id(batt_voltage_verified) = false;
#          //return (std::string) "NOT_VERIFIED";
#          //return to_string(id(batt_voltage_filtered).state);
#          return s;
#

An explanation is in order now:
By triggering either one of the physical switches on the mailbox doors, the ESP will be supplied power. ESP then asap set the power cutoff pin to high which will ensure the power will remain turned on.
All relevant input states should also be acquired right now due to a combination of setup_priority and publish_initial_state.

After the wifi connects, the MQTT server connection is right around the corner, and the relevant script is triggered. I believe the script itself is sufficiently clear, so I won’t go into any details unless requested.

The binary sensor is the most important to get its reading across. Due to the usage of MQTT this is achieved by simply subscribing to the same topic, where button state is published. This additionally gives the “VERIFIED” / “NOT_VERIFIED” status in home assistant as a bonus.

It doesn’t really matter if the correct reading is already in the MQTT since sending it again, would change nothing.

So far it works well, but has some serious limitations. For example. You can find remains of my attempts to do the same with the battery voltage. Here we have some issues with floating point numbers since the sensor sends out rounded values but .state returns “raw” value. Now this might be just a floats limitation, but it’s still a headache. Maybe I can copy the sensor…

The other limitation - besides obvious small but unnecessary traffic, the MQTT subscribe path has to be updated manually on hostname changes or the sensor changes, and I haven’t found any way to get it from some variables.

Overall it’s … disappointing, but perchance it’ll be more useful for someone else.

Well then, I think that’s about it~!

1 Like

Did you see that some people use RF door sensors for this kind of thing (with all the pros and cons that go with it).

Yes, thank you. I’m well aware. My mailbox is however pretty nonstandard, to the point no door / window sensor would work there. And since it already required custom uhhhh “mail insertion detector”, and I like building stuff, I’ve just done the needful.

That said, next time, I wouldn’t go with WiFi. However ideally I’d build something matter compatible that uses thread, however it’s a bit harder to build such devices than with just ESPs and the WiFi…

I don’t really see any other options than ZigBee and z-wave, both of which I don’t have any other devices of yet (so not receivers as well), while the other one that would be preferred is E X P E N S I V E .

Therefore all I can conceivably think of for the “next time” really is just a pair of 433mhz modules with some protocol or just straight up raw.

Or modifying one of the commercial door sensors :yum:

If there are some other options I’m missing here, please do tell~!

1 Like

Fair enough.

Not sure if you’ve seen but some of the door sensors you can hack a fair bit (for example run wires out to a more remote reed switch or maybe to your “mail insertion detector”. Probably you can do the same with an rf one?

Probably you only get one way communication with this approach though and could miss the signal etc.

I doubt you will touch Zigbee and Bluetooth battery powered level of functionality with WiFi for at least 2 years and some coin for WiFi mesh upgrades. I’ve had success using these Aqara/Xiaomi vibration sensors as mailbox “you’ve got mail” sensors with Zigbee2MQTT. The battery life has been good and with some ‘tuning’ the vibration action yields solid results when the mailbox door is opened and closed. While not designed for IPxx outdoor use, with proper placement they have survived this winter’s serious rains okay and at a USD 10 price point replacing every 24 months does not seem to be too big a cost. The biggest challenge is getting your Zigbee mesh network ‘out’ to the mailbox, I’ve found that a Zigbee router device in the form of a power plug mounted to a wall mains socket as close to the mailbox yields solid results. And you get the temperature at the mailbox about every hour…this gives you some comfort that the device is alive. Might be a bit ‘hill billy tech’ from the sounds of your mailbox, however adding a Zigbee door/window sensor ‘hack’ along with this vibration sensor and then do some ‘sensor fusion’ might be a path. Good hunting!