How to OTA flash sleeping esphome sensor

Hello all. Flashing new firmware to esphome in deep sleep OTA is quite complicated, because sensor is in deep sleep and does not have wifi active. You have to wait until it wakes up, but you never know when it will be because clock in esp chip is not accurate. The esphome uploader only waits few seconds and if it does not detect the chip awake, it fails.

What I am succesfully using is running esphome in loop and try to flash until it succeeds. I created this simple bash script that waits until esphome wakes up and then flashes sends firmware OTA. I leave it running for hours or over night.

I call it ota-esp.sh. The parameter is yaml file:

#!/bin/bash
esphome $1 compile
esphome $1 compile
while ! esphome $1 upload; do :; done
esphome $1 logs

I compile twice because I noticed that at first compilation there may be warnings that disappear when you compile for second time.

I am still struggling with OTA of sleeping sensor from within ESPhome add-on in HA OS Supervisor. I cannot create bash scripts there and neither it works from HA OS terminal. If anyone could help please. This would solve flashing sensors in deep sleep over internet, which would be kind of cool. I have port of Home Assistant open in my router, but now I can only flash if I am at same network as the sensor.

You could also use: deep-sleep-prevent-action

Not very experienced in the “deep sleep” of ESPs, but may I suggest another way, that at least in my head should work.

“Disconnect” the ESP via your router from your standard WiFi. In theory the captive portal should step in and open its own AccessPoint. There you can flash via OTA whatever you want. After rebooting it should use the new firmware.

EDIT: There is a solution right in the ESPHome examples:

For example, if you want to upload a binary via OTA with deep sleep mode it can be difficult to catch the ESP being active.

You can use this automation to automatically prevent deep sleep when a MQTT message on the topic livingroom/ota_mode is received. Then, to do the OTA update, just use a MQTT client to publish a retained MQTT message described below. When the node wakes up again it will no longer enter deep sleep mode and you can upload your OTA update.

Remember to turn “OTA mode” off again after the OTA update by sending a MQTT message with the payload OFF. To enter the the deep sleep again after the OTA update send a message on the topic livingroom/sleep_mode with payload ON. Deep sleep will start immediately. Don’t forget to delete the payload before the node wakes up again.

deep_sleep:
  # ...
  id: deep_sleep_1
mqtt:
  # ...
  on_message:
    - topic: livingroom/ota_mode
      payload: 'ON'
      then:
        - deep_sleep.prevent: deep_sleep_1
    - topic: livingroom/sleep_mode
      payload: 'ON'
      then:
        - deep_sleep.enter: deep_sleep_1

I had exactly this same issue with a garden sensor that would sleep most of the time and wake every minute to send it’s readings over MQTT.
What I did was to publish an MQTT message to the sensor with the retain flag set to true. When the sensor woke up it would connect to the broker and retrieve the retained message. If the message was true, then the ESP would not go back to sleep and OTA works normally. If the message was false, then the ESP would go back into a normal sleep cycle.

I am VERY new to ESPHome, so I don’t know if anything like this would work here.

So could there be script in Home Assistant that creates mqtt message for sensor to block it from going to sleep, then activates OTA and then sends another mqtt message to allow deep sleep? It seems to me a bit complicated because script in Home Assistant may need to wait many hours until sensor wakes up and then trogger OTA. It looks it is doing same thing as I do with one bash command.

I am combining your ideas but using the home assistant api directly instead of using mqtt.

  1. Create an input_boolean in Home Assistant. Call it my_device_disable_sleep. (This can also be done using the helper section in the UI.)
  2. Add it as a binary sensor to your esphome config:
    binary_sensor:
      - platform: homeassistant
        entity_id: input_boolean.my_device_disable_sleep
        id: disable_sleep
        publish_initial_state: true     # This is important!
        on_state:
          then:
            if:
              condition:
                lambda: return x;
              then:
                - logger.log: "Preventing deep sleep"
                - deep_sleep.prevent: deep_sleep_1
              else:
                - logger.log: "Allowing deep sleep"
                - deep_sleep.allow: deep_sleep_1
    
  3. Make the esp turn off the boolean after every OTA update:
    ota:
      # ...
      on_end:
        then:
          - homeassistant.service:
              service: input_boolean.turn_off
              data:
                entity_id:
                  input_boolean.my_device_disable_sleep
    

Now, in order to do an update, you need to first turn on the switch in Home Assistant and then run the OTA in a loop until succeeds, as shown by @janbenes:

while ! esphome $1 upload; do :; done

When the upload is finished, the esp will automatically turn off the disable switch. So after the reboot, it will use deep sleep again as normal.

3 Likes

My upload wrapper script looks like this. So you don’t need to manually turn on the switch in HA anymore:

#!/usr/bin/env bash
set -euo pipefail

HA_URL=http://homeassistant:8123
TOKEN=PUT_TOKEN_HERE

yaml="$1"
name="${2:-}"

if [[ -n "$name" ]]; then
  entity="input_boolean.${name}_disable_sleep"
  echo "Turning on $entity"
  curl -s -X POST -H "Authorization: Bearer $TOKEN" \
                          -H "Content-Type: application/json" \
                          -d "{\"entity_id\": \"$entity\"}" \
                          "$HA_URL/api/services/input_boolean/turn_on"
fi

while ! esphome upload "$yaml"; do sleep 1; done

Run it like this:

upload my_device.yaml my_device

Tried your solution, but it doesn’t work for me. This is my configuration:

esphome:
  name: dht22-bath

esp8266:
  board: nodemcuv2

# Enable logging
logger:

# Enable Home Assistant API
api:

ota:
  password: ""
  on_end:
    then:
      - homeassistant.service:
          service: input_boolean.turn_off
          data:
            entity_id: input_boolean.prevent_deep_sleep_esphome

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Dht22-Bath Fallback Hotspot"
    password: ""

captive_portal:

binary_sensor:
  - platform: homeassistant
    entity_id: input_boolean.prevent_deep_sleep_esphome
    id: disable_sleep
    publish_initial_state: true
    on_state:
      then:
        if:
          condition:
            lambda: return x;
          then:
            - logger.log: "Preventing deep sleep"
            - deep_sleep.prevent: deep_sleep_control
          else:
            - logger.log: "Allowing deep sleep"
            - deep_sleep.allow: deep_sleep_control

deep_sleep:
  id: deep_sleep_control
  sleep_duration: 3min

sensor:
  - platform: dht
    model: DHT22
    pin: 4
    temperature:
      name: "Bathroom Temperature"
    humidity:
      name: "Bathroom Humidity"

Unfortunately doesn’t send any readings. Maybe it goes sleep before it can connect to WiFi and send readings? Pin is connected correctly, setup is working with deepsleep (run_duration 15s) without the prevent deepsleep things. Also it takes ages when I set the flag until the device comes from deep sleep and preventing it.

It’s been a while since I did this. I have an ESP01 in my Jeep that I use for presence.
If the ESP connects, then the Jeep is present.
I use a binary helper that I turn on on the dashboard that is basically a semaphore to tell the ESP that I want it to not go to sleep. The test_ota script in the code below is where this is tested when the ESP boots.

So, my sequence is:

  1. Turn on OTA mode.
  2. Wait for the sleep period to end, then the ESP reboots and will not go to sleep.
  3. Perform my OTA upload.
  4. Turn off the OTA mode.
  5. Press “Restart” on the dashboard to reboot the ESP.

jeep

# This is the ESP device inside the Jeep to provide presence (status=connected).
substitutions:
  device_name: jeep
  
esphome:
  name: ${device_name}
  on_boot:
    priority: -100.0
    then:
      - delay: 1s
      - script.execute: test_ota

esp8266:
  board: esp01_1m

api:

ota:
  safe_mode: True
  
packages:
  wifi: !include common/wifi.yaml

logger:
  level: VERBOSE     # default is DEBUG


binary_sensor:
  - platform: status
    name: "Jeep Status"
  - platform: homeassistant
    id: otamode
    entity_id: input_boolean.jeep_ota_mode
    
    
#################################################
# Get the WiFi details
text_sensor:
  - platform: wifi_info
    ip_address:
      name: ${device_name} IP Address
    ssid:
      name: ${device_name} SSID
    mac_address:
      name: ${device_name} Mac Address
      
sensor:
  - platform: wifi_signal
    name: ${device_name} WiFi Signal Sensor"
    update_interval: 60s
    
    
#################################################    
# Script to test if the otamode switch is on or off
script:
  - id: test_ota
    mode: queued
    then:
      - logger.log: "Checking OTA Mode"
      - if:
          condition:
            binary_sensor.is_on: otamode
          then:
            - logger.log: 'OTA Mode ON'
            - deep_sleep.prevent: deep_sleep_handler
          else:
            - logger.log: 'OTA Mode OFF'
      - delay: 2s
      - script.execute: test_ota


    
#################################################
#Deep Sleep
deep_sleep:
  id: deep_sleep_handler
  run_duration: 5s
  sleep_duration: 120s
  

################################################
#Make a button to reboot the ESP device
button:
  - platform: restart
    name: ${device_name} Restart
6 Likes

Works like a charm. Tank you!

works great. Thanks.

In addition I also added deep_sleep.allow: deep_sleep_handler once toggled off in HA so no need to restart

HI
Somehow I can’t manage to integrate the OTO update code into my existing one
BR Markus

esphome:
  name: "tanne"
  platform: ESP8266
  board: d1_mini
  

# Enable logging

logger:

# Enable Home Assistant API
api:
  encryption:
    key: "BH3dK3eMeT7ACjzYw3L2tNA+U/dWgwa/MJ8IBeAzV6c="

ota:
  password: "soil"

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  fast_connect: true
 
  # Enable fallback hotspot (captive portal) in case wifi connection fails
  #ap:
   # ssid: "moisture-1 Fallback Hotspot"
    #password: !secret fallback_wifi_password

captive_portal:


# the ads1115 is i2c
i2c:
  sda: D2
  scl: D1
  scan: true

# Konfiguration der Batteriespannungssensoren
ads1115:
  - address: 0x48

sensor:

  - platform: ads1115
    multiplexer: 'A3_GND'
    gain: 6.144
    name: "ADS1115 Channel A3-GND"
    id: ads1115_1
    update_interval: 5s
    internal: true # Don't expose this sensor to HA, the template sensors will do that
    # Dry 2.734 - taken from in the air, probably should do it directly in dry soil
    # Wet 0.638 - taken from in a glass of water, probably should do it directly in saturated soil
    filters:
      - sliding_window_moving_average: # averages the last 10 results, probably overkill
          window_size: 3
          send_every: 3
      - lambda: |-
          if (x > 2.245) {        // if over 2.734 volts, we are dry
            return 0;
          } else if (x < 0.862) {       // if under 0.638 volts, we are fully saturated
            return 100;
          } else {
            return (2.245-x) / (2.245-0.862) * 100.0;   // use a linear fit for any values in between
          }
  - platform: template
    name: "Soil Saturation"
    id: zone1saturation
    unit_of_measurement: "%"
    icon: "mdi:water-percent"
    accuracy_decimals: 0
    lambda: |-
      return id(ads1115_1).state;

  - platform: ads1115
    multiplexer: 'A2_GND'
    gain: 6.144
    name: "ADS1115 Channel A2-GND"
    id: ads1115_2
    update_interval: 5s
    internal: true # Don't expose this sensor to HA, the template sensors will do that
    # Dry 2.734 - taken from in the air, probably should do it directly in dry soil
    # Wet 0.638 - taken from in a glass of water, probably should do it directly in saturated soil
    filters:
      - sliding_window_moving_average: # averages the last 10 results, probably overkill
          window_size: 3
          send_every: 3
      - lambda: |-
          if (x < 3.10) {         // if over 2.734 volts, we are dry
            return 0;
          } else if (x > 4.18) {      // if under 0.638 volts, we are fully saturated
            return 100;
          } else {
            return (3.10-x) / (3.10-4.18) * 100.0;  // use a linear fit for any values in between
          }
  - platform: template
    name: "Akku Voltage"
    id: zone2saturation
    unit_of_measurement: "%"
    # icon: "mdi:water-percent"
    icon: "mdi:battery-80"

    accuracy_decimals: 0
    lambda: |-
      return id(ads1115_2).state;  

… Why not?

i am a newbie in this topic, i tried it but only got error messages…

Markus

What are you trying to add? What code? Where’s your code? Error measages? Where are they? What is anyone supposed to do with

If you want help then help people help you.

Hi, thank you very much for your example, I am trying to implement it, but I have one problem:
I created the boolean helper in HA with the entity name “input_boolean.jeep_ota_mode”, and I can also see the value change from ON to OFF, when I check the HA variables.

My problem is, that the script, running the check for otamode is not reacting to my boolean from above. it will not recognize the boolean state on and always tells me OTA Mode OFF in the logs.

Is there any additional implementation of the boolean necessary than just adding it to the dashboard and entities?

my code:

esphome:
  name: esp32-c3-zockbock
  platformio_options:
   board_build.flash_mode: dio
  on_boot:
    priority: -100.0
    then:
      - delay: 1s
      - script.execute: test_ota

esp32:
  board: seeed_xiao_esp32c3
  variant: esp32c3
  framework:
    type: arduino
    platform_version: 5.4.0

# Enable logging
logger:
  hardware_uart: UART0
  level: VERBOSE


# Enable Home Assistant API
api:
  encryption:
    key: "EGBMB97pCl7k1wjavApwzw6oO8i5in8Cy6zre3F4tos="

ota:
  - platform: esphome
    password: "4d5cba2bf5b501ea29eff6624ea46b5a"

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Esp32-C3-ZockBock"
    password: "1a3yZx5BBtrO"

#################################################
# Erstelle einen binären Sensor, der mir den ESP Status zurückgibt im Dashboard
binary_sensor:
  - platform: status
    name: "ZockBock status"
  - platform: homeassistant
    name: "ZockBock_OTA_mode"
    id: otamode
    entity_id: input_boolean.zockbock_ota_mode

#################################################
# Definiere den Switch, der die LED auf GPIO6 steuert
switch:
  - name: LED
    platform: gpio
    pin: GPIO6

#################################################
# Definiere den Sensor, der die Batteirspannung misst
sensor:
  - platform: adc
    pin: GPIO2
    name: BatteryVoltage
    #raw : true
    unit_of_measurement: V
    update_interval: 60s
    attenuation: 11db
    filters:
      - multiply: 2 # wegen Voltage divider

#################################################    
# Script to test if the otamode switch is on or off
script:
  - id: test_ota
    mode: queued
    then:
      - logger.log: "Checking OTA flashing Mode"
      - logger.log: "OTA flashing Mode state:" {{ states('input_boolean.zockbock_ota_mode') }}
      - if:
          condition:
            binary_sensor.is_on: otamode
          then:
            - logger.log: 'OTA flashing Mode ON'
            - deep_sleep.prevent: deep_sleep_handler
          else:
            - logger.log: 'OTA flashing Mode OFF'
      - delay: 2s
      - script.execute: test_ota

# Das richtige Vorgehehen für das Flashen des ESP ist nun:
# Turn on OTA mode with dashboard switch
# Wait for the sleep period to end, then the ESP reboots and will not go to sleep.
# Perform my OTA flash upload.
# Turn off the OTA mode.
# Press “Restart” on the dashboard to reboot the ESP.


#################################################
#Deep Sleep
deep_sleep:
  id: deep_sleep_handler
  run_duration: 20s
  sleep_duration: 120s
  

################################################
#Make a button to reboot the ESP device
button:
  - platform: restart
    name: ${device_name} Restart

In your configuration YAML, you are looking at
entity_id: input_boolean.zockbock_ota_mode

Everything else looks right. Let me know if this is the problem and I’ll offer other suggestions.

Can you put a terminal on the UART to see what logger.log is saying?

Thank you for the quick answer.
input_boolean.zockbock_ota_mode is my actual entity, I just copied your name with “jeep” in my question :slight_smile:

But regarding your suggestion: do I need to manually add this entity id to the configuration yaml?

Here is my log:

[21:27:02][V][mdns:118]:   Services:
[21:27:02][V][mdns:120]:   - _esphomelib, _tcp, 6053
[21:27:02][V][mdns:122]:     TXT: version = 2024.7.0
[21:27:02][V][mdns:122]:     TXT: mac = 543204862320
[21:27:02][V][mdns:122]:     TXT: platform = ESP32
[21:27:02][V][mdns:122]:     TXT: board = seeed_xiao_esp32c3
[21:27:02][V][mdns:122]:     TXT: network = wifi
[21:27:02][V][mdns:122]:     TXT: api_encryption = Noise_NNpsk0_25519_ChaChaPoly_SHA256
[21:27:02][C][esphome.ota:073]: Over-The-Air updates:
[21:27:02][C][esphome.ota:074]:   Address: esp32-c3-zockbock.local:3232
[21:27:02][C][esphome.ota:075]:   Version: 2
[21:27:02][C][esphome.ota:078]:   Password configured
[21:27:02][C][safe_mode:018]: Safe Mode:
[21:27:02][C][safe_mode:020]:   Boot considered successful after 60 seconds
[21:27:02][C][safe_mode:021]:   Invoke after 10 boot attempts
[21:27:02][C][safe_mode:023]:   Remain in safe mode for 300 seconds
[21:27:02][C][api:139]: API Server:
[21:27:02][C][api:140]:   Address: esp32-c3-zockbock.local:6053
[21:27:02][C][api:142]:   Using noise encryption: YES
[21:27:02][C][homeassistant.binary_sensor:039]: Homeassistant Binary Sensor 'ZockBock_OTA_mode'
[21:27:02][C][homeassistant.binary_sensor:040]:   Entity ID: 'input_boolean.zockbock_ota_mode'
[21:27:02][C][deep_sleep:026]: Setting up Deep Sleep...
[21:27:02][C][deep_sleep:029]:   Sleep Duration: 120000 ms
[21:27:02][C][deep_sleep:032]:   Run Duration: 20000 ms
[21:27:02][D][script:100]: Script 'test_ota' queueing new instance (mode: queued)
[21:27:02][D][main:368]: Checking OTA flashing Mode
[21:27:02][D][main:414]: OTA flashing Mode OFF
[21:27:04][D][script:100]: Script 'test_ota' queueing new instance (mode: queued)
[21:27:04][D][main:368]: Checking OTA flashing Mode
[21:27:04][D][main:414]: OTA flashing Mode OFF
[21:27:06][D][script:100]: Script 'test_ota' queueing new instance (mode: queued)
[21:27:06][D][main:368]: Checking OTA flashing Mode
[21:27:06][D][main:414]: OTA flashing Mode OFF
[21:27:08][D][script:100]: Script 'test_ota' queueing new instance (mode: queued)
[21:27:08][D][main:368]: Checking OTA flashing Mode
[21:27:08][D][main:414]: OTA flashing Mode OFF
[21:27:10][D][script:100]: Script 'test_ota' queueing new instance (mode: queued)
[21:27:10][D][main:368]: Checking OTA flashing Mode
[21:27:10][D][main:414]: OTA flashing Mode OFF
[21:27:12][D][script:100]: Script 'test_ota' queueing new instance (mode: queued)
[21:27:12][D][main:368]: Checking OTA flashing Mode
[21:27:12][D][main:414]: OTA flashing Mode OFF
[21:27:14][D][script:100]: Script 'test_ota' queueing new instance (mode: queued)
[21:27:14][D][main:368]: Checking OTA flashing Mode
[21:27:14][D][main:414]: OTA flashing Mode OFF
[21:27:16][D][script:100]: Script 'test_ota' queueing new instance (mode: queued)
[21:27:16][D][main:368]: Checking OTA flashing Mode
[21:27:16][D][main:414]: OTA flashing Mode OFF
[21:27:17][I][deep_sleep:060]: Beginning Deep Sleep
[21:27:17][I][deep_sleep:062]: Sleeping for 120000000us

So here is something new… but I do not understand it, maybe this helps you:

I just did a restart of the ESP32 and suddenly, the following appeared in the log (I did not change the code before, except adding this state logger of the input boolean entity, which obviously didn’t print the value but just the dumb text {{ states(‘binary_sensor.otamode’) }} …

But what its visible in the log: suddenly the Homeassistant api said hello :smiley:
V][api.connection:1357]: Hello from client: ‘Home Assistant 2024.7.2’ | 192.168.2.128 | API Version 1.10
[22:00:15][D][api.connection:1375]: Home Assistant 2024.7.2 (192.168.2.128): Connected successfully

does this have something to do with the update timer of the Homeassistant api and this time I have just been lucky to have restarted the Esp32 to this exact moment?

Or (what I actually suspect): The binary sensor update usually changes during a deep sleep phase of the ESP32. I suspect, the “ON” value is not recognized by the ESP, when it wakes up from deep sleep ans still sees “OFF” from when it fell asleep.
I just tested a few toggles while the ESP has been on and this worked (recognized the value in the log)

22:00:11][D][main:378]: OTAMode is {{ states('binary_sensor.otamode') }}
[22:00:11][D][main:424]: OTA flashing Mode OFF
[22:00:13][D][script:100]: Script 'TestOTA' queueing new instance (mode: queued)
[22:00:13][D][main:375]: Checking OTA flashing Mode
[22:00:13][D][main:378]: OTAMode is {{ states('binary_sensor.otamode') }}
[22:00:13][D][main:424]: OTA flashing Mode OFF
[22:00:15][D][api:102]: Accepted 192.168.2.128
[22:00:15][V][api.connection:1357]: Hello from client: 'Home Assistant 2024.7.2' | 192.168.2.128 | API Version 1.10
[22:00:15][D][api.connection:1375]: Home Assistant 2024.7.2 (192.168.2.128): Connected successfully
[22:00:15][D][script:100]: Script 'TestOTA' queueing new instance (mode: queued)
[22:00:15][D][main:375]: Checking OTA flashing Mode
[22:00:15][D][main:378]: OTAMode is {{ states('binary_sensor.otamode') }}
[22:00:15][D][main:424]: OTA flashing Mode OFF
[22:00:15][D][homeassistant.binary_sensor:026]: 'input_boolean.zockbockotamode': Got state ON
[22:00:15][D][binary_sensor:034]: 'ZockBockOTAMode': Sending initial state ON
[22:00:17][D][script:100]: Script 'TestOTA' queueing new instance (mode: queued)
[22:00:17][D][main:375]: Checking OTA flashing Mode
[22:00:17][D][main:378]: OTAMode is {{ states('binary_sensor.otamode') }}
[22:00:17][D][main:383]: OTA flashing Mode ON
[22:00:19][D][script:100]: Script 'TestOTA' queueing new instance (mode: queued)
[22:00:19][D][main:375]: Checking OTA flashing Mode
[22:00:19][D][main:378]: OTAMode is {{ states('binary_sensor.otamode') }}
[22:00:19][D][main:383]: OTA flashing Mode ON
[22:00:21][D][script:100]: Script 'TestOTA' queueing new instance (mode: queued)
[22:00:21][D][main:375]: Checking OTA flashing Mode
[22:00:21][D][main:378]: OTAMode is {{ states('binary_sensor.otamode') }}
[22:00:21][D][main:383]: OTA flashing Mode ON
[22:00:23][D][script:100]: Script 'TestOTA' queueing new instance (mode: queued)
[22:00:23][D][main:375]: Checking OTA flashing Mode

I just tested my code again, and it is working. My hardware is an ESP8266. I have done very little with ESP32, so I have to wonder if the ESP32 sleep commands are different??