Error flashing EspHome after migration from Tasmota on ESP32 (Sonoff DUALR3)

Jethro97 · January 9, 2023, 4:57pm

Hello guys,

I’m experiencing a problem trying to flash images on DUALR3 Sonoff Smart Relays, based on ESP32 chip.

While flashing directly there’s no issues, but when trying to migrate:

If I select “Modern Format” Tasmota fail saying “Invalid file signature”
If I select “Legacy Format” Tasmota flash succed and ESPhome device works as expected, until I try to do an OTA upgrade.

The file is simple and works on other DUALR3

This is the error I got when trying to OTA upgrade:

Processing rele-scrivania (board: esp32dev; framework: arduino; platform: platformio/espressif32 @ 5.2.0)
--------------------------------------------------------------------------------
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch3
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
Dependency Graph
|-- WiFi @ 2.0.0
|-- ESPmDNS @ 2.0.0
|-- Update @ 2.0.0
|-- noise-c @ 0.1.4
|   |-- libsodium @ 1.10018.1
Compiling /data/rele-scrivania/.pioenvs/rele-scrivania/src/main.cpp.o
Linking /data/rele-scrivania/.pioenvs/rele-scrivania/firmware.elf
RAM:   [=         ]  12.2% (used 40120 bytes from 327680 bytes)
Flash: [=====     ]  47.9% (used 879205 bytes from 1835008 bytes)
Building /data/rele-scrivania/.pioenvs/rele-scrivania/firmware.bin
Creating esp32 image...
Successfully created esp32 image.
esp32_create_combined_bin(["/data/rele-scrivania/.pioenvs/rele-scrivania/firmware.bin"], ["/data/rele-scrivania/.pioenvs/rele-scrivania/firmware.elf"])
Wrote 0xe80f0 bytes to file /data/rele-scrivania/.pioenvs/rele-scrivania/firmware-factory.bin, ready to flash to offset 0x0
========================= [SUCCESS] Took 8.80 seconds =========================
INFO Successfully compiled program.
INFO Connecting to 10.10.12.82
INFO Uploading /data/rele-scrivania/.pioenvs/rele-scrivania/firmware.bin (884976 bytes)
Uploading: [=                                                           ] 1% 
ERROR Error sending data: [Errno 104] Connection reset by peer

I searched on www but found nothing

nickrout · January 9, 2023, 9:15pm

Keep trying, and possibly bring the device closer to your ap so the wifi strength is bettet.

Jethro97 · January 10, 2023, 7:38am

Thank you, already tried several times, device are under the AP, both with 50 dB or less, it shouldn’t be a problem.
Also tried power cycling some times with no results

EDIT: Tried with minimal firmware (only wifi, api, ota): same problem.

Jethro97 · January 12, 2023, 10:48pm

This is definitely a problem.

Flashed the same image via USB TTL adapter and is now working flawlessly.

Tried to upgrade a MINIR2 (ESP8266 based) and got Tasmota saying “Not enough space” even if program and flash size are correct. I think there is something with the latest version of Tasmota (12.3.1) as I had no problems migrating from the previous ones

Edit: *solved on the ESP8266 uploading a compressed .bin.gz source

Jethro97 · January 17, 2023, 6:17pm

I had to solder and flash with usb adapter ALL the Tasmota ESP32 devices I had, to migrate to ESPhome.

In the other hand, I only need to flash tasmota minimal and then ESPhome on ESP8266 devices.

daywalker03 · January 17, 2023, 7:48pm

This might sound strange, but if you can load the latest minimal tasmota image, you should be able to flash it over the air. That’s what I had to do with my Sonoff Basic R2 that already had tasmota on it.

Jethro97 · February 10, 2023, 12:18pm

That’s also what i did on MINIR2 and all other ESP8266 based devices, on ESP32 boards, but on DUALR3 it didn’t work.

MsG · April 16, 2023, 12:20pm

Same problem, pretty annoying with these boards. There seems to be no minimal for my board (ESP32-S2)

AndreKR · September 11, 2023, 11:56am

I have the exact same issue on two devices, a Shelly Plus i4 and a Sonoff TX Ultimate.

And if I retry the flash process, the second time it will look like the OTA update completed successfully, with upload progress bar and all, but the firmware will not change.

The way I put ESPHome on the devices is: I installed Tasmota on them via Shelly OTA update or Tasmota Web Flasher respectively, then I downloaded the ESPHome binary in “legacy” format and flashed it using Tasmota’s update form. Was that the correct procedure?

ferbulous · September 11, 2023, 6:25pm

There’s a different storage partition starting with tasmota v12 not compatible with esphome
If you have safe boot partition, then you need to serial flash it

AndreKR · September 11, 2023, 7:00pm

There was a message during the update “Switching to Safeboot” and after the update before the reboot there was a red text “SAFEBOOT” above the menu. So I think I have it, whatever it is.

Does that mean there is no way to go from Tasmota to ESPHome using OTA? Or do I simply need to do something different?

Does that explain why my OTA updates have no effect?

ferbulous · September 12, 2023, 1:04am

Yes, unfortunately
If you still need to use tasmota, you’ll need to flash tasmota v11, and you can still ota to v12 and back to esphome.

pbix · November 6, 2023, 2:39am

Same problem here.
“Modern format” produces the “Invalid file signature” error.
“Legacy format” works after a couple of tries but later updates via ESPHome produce the following:

INFO Successfully compiled program.
INFO Resolving IP address of esp-j1.local
INFO  -> 192.168.2.184
INFO Uploading /data/build/esp-j1/.pioenvs/esp-j1/firmware.bin (860976 bytes)
Uploading: [                                                            ] 0% 
ERROR Error sending data: [Errno 104] Connection reset by peer

Monitoring log output in a separate web page I can observe the following:

INFO Reading configuration /config/esphome/esp-j1.yaml...
INFO Starting log output from esp-j1.local using esphome API
INFO Successfully connected to esp-j1.local
[21:25:22][I][app:102]: ESPHome version 2023.10.6 compiled on Nov  5 2023, 21:04:36
[21:25:22][C][wifi:546]: WiFi:
[21:25:22][C][wifi:382]:   Local MAC: 84:F7:03:EC:B7:68
[21:25:22][C][wifi:383]:   SSID: [redacted]
[21:25:22][C][wifi:384]:   IP Address: 192.168.2.184
[21:25:22][C][wifi:386]:   BSSID: [redacted]
[21:25:22][C][wifi:387]:   Hostname: 'esp-j1'
[21:25:22][C][wifi:389]:   Signal strength: -7 dB ▂▄▆█
[21:25:22][C][wifi:393]:   Channel: 1
[21:25:22][C][wifi:394]:   Subnet: 255.255.255.0
[21:25:22][C][wifi:395]:   Gateway: 192.168.2.3
[21:25:22][C][wifi:396]:   DNS1: 192.168.2.3
[21:25:22][C][wifi:397]:   DNS2: 192.168.2.2
[21:25:22][C][logger:416]: Logger:
[21:25:22][C][logger:417]:   Level: DEBUG
[21:25:22][C][logger:418]:   Log Baud Rate: 115200
[21:25:22][C][logger:420]:   Hardware UART: UART0
[21:25:22][C][captive_portal:088]: Captive Portal:
[21:25:22][C][mdns:115]: mDNS:
[21:25:22][C][mdns:116]:   Hostname: esp-j1
[21:25:22][C][ota:097]: Over-The-Air Updates:
[21:25:31][I][ota:117]: Boot seems successful, resetting boot loop counter.
[21:25:32][D][esp32.preferences:114]: Saving 1 preferences to flash...
[21:25:32][D][esp32.preferences:143]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed
[21:27:09][D][ota:151]: Starting OTA Update from 192.168.2.217...
[21:27:17][D][ota:320]: OTA in progress: 0.1%
WARNING esp-j1.local: Connection error occurred: [Errno 104] Connection reset by peer
INFO Processing unexpected disconnect from ESPHome API for esp-j1.local
WARNING Disconnected from API
INFO Successfully connected to esp-j1.local
[21:32:27][I][ota:117]: Boot seems successful, resetting boot loop counter.
[21:32:27][D][esp32.preferences:114]: Saving 1 preferences to flash...
[21:32:27][D][esp32.preferences:143]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed

The following link indicates this should work and to use the “Modern format” but it does not work.
https://next.esphome.io/guides/migrate_sonoff_tasmota

I am unable to initialize the WEMO-S2 using web.esphome.io as well.

Seems “board: lolin_s2_mini” is broken in several places.

If someone knows what to try I can give it a go.

mbrevda · November 17, 2024, 4:44pm

Seems relevant:

github.com/esphome/esphome

Read/Write bootloader, partition table and any partition via OTA

esphome:dev ← angelnu:extend_ota

opened 02:41AM - 15 Oct 23 UTC

angelnu

+1207 -305

# What does this implement/fix? This adds support to update the bootloader an…d partition tables with OTA. It allows migrating from firmwares with different layouts (new Tasmota versions) or repair botloaders (such as the locked new Shellies). Updating the bootloader and partition tables only works with the esp-idf framework. For arduino [arduino-esp32](https://github.com/espressif/arduino-esp32/releases/tag/3.0.0-alpha3) is needed. This is currently in alpha so no platformio package is available. It is possible to move from esp-idf to arduino OTA so you can migrate Tasmota -> esphome(esp-idf) -> esphome(arduino). ## New CLI commands ```bash # update OTA the bootloader without rebooting python -m esphome -v upload test.yaml --bootloader --no-reboot # update OTA the partition table without rebooting (requires esp32-idf version > 5.1) python -m esphome -v upload test.yaml --partition-table --no-reboot # update OTA partition with type and subtype python -m esphome -v upload --partition-type=0x0 --partition-subtype=0x10 test.yaml # update OTA partition with label python -m esphome -v upload --partition-label=app0 test.yaml # download OTA the bootloader python -m esphome -v download test.yaml --bootloader --file=bootloader.bin # download OTA the partition table without rebooting (requires esp32-idf version > 5.1) python -m esphome -v download test.yaml --partition-table --file=partition_table.bin # download OTA partition with type and subtype python -m esphome -v download --partition-type=0x0 --partition-subtype=0x10 test.yaml --file=firmware.bin # download OTA partition with label python -m esphome -v download --partition-label=app0 test.yaml --file=firmware.bin # update OTA all esphome parts (equivalent to a factory reset) python -m esphome upload-factory-ota test.yaml ``` The `upload-factory-ota` command makes a backup of the partition table before flashing the new one. If any operation fails then the backup is automatically restored. This usually happens when the new firmware would override the running partition. ## Tasmota migration If you are migrating from Tasmota 1.12 or newer using the Safeboot partition you need the following steps. There are 2 options depending on the used esp-idf framework. ### Single command (requires esp-idf>5.1) This is how to do it without custom partition_tables. 1. Build an ESPHome (`test.yaml`) with a minimal configuration (with wifi and OTA) with esp-idf>5.1 ```yaml # Example for esp32-c3 (Shelly Plus 1PM Mini) esphome: name: test platformio_options: board_build.flash_mode: dio esp32: board: esp32-c3-devkitm-1 framework: type: esp-idf platform_version: 6.4.0 version: 5.1.1 variant: esp32c3 wifi: manual_ip: static_ip: 192.168.0.33 gateway: 192.168.2.1 subnet: 255.255.192.0 dns1: 192.168.2.1 networks: - ssid: my_wifi password: my_wifi_password ota: password: my_ota_password unprotected_writes: True # This is mandatory if you want to flash the partition table or bootloader! logger: hardware_uart: USB_SERIAL_JTAG web_server: button: - platform: restart name: Restart ``` 2. Use Tasmota firmware upgrade with the ESPHome legacy firmware binary built in the previous step. ESPHome should boot 3. Upgrade all ESPHome to factory: `python -m esphome -v upload-factory-ota test.yaml`. ESPHome should reboot and use ESPHome partition table and boot loader. Afterwards you might downgrade to the recommend esp-idf framework. ### Using multiple commands If you are using the recommend esp-idf framework you will need to use a custom partition table: 1. Build an ESPHome (`test.yaml`) with a minimal configuration (with wifi and OTA) and the following partition table (this uses the Tasmota OTA data partition and app1 address/size): ```CSV # Name, Type, SubType, Offset, Size, Flags otadata, data, ota, 0x00e000, 0x2000, # phy_init, data, phy, , 0x1000, app0, app, ota_1, 0x010000, 0x0d0000, app1, app, ota_0, 0x0e0000, 0x0d0000, nvs, data, nvs, 0x38C000, 0x6d000, ``` ```yaml # Example for esp32-c3 (Shelly Plus 1PM Mini) esphome: name: test platformio_options: board_build.partitions: "../../../custom_partitions.csv" board_build.flash_mode: dio esp32: board: esp32-c3-devkitm-1 framework: type: esp-idf variant: esp32c3 wifi: manual_ip: static_ip: 192.168.0.33 gateway: 192.168.2.1 subnet: 255.255.192.0 dns1: 192.168.2.1 networks: - ssid: my_wifi password: my_wifi_password ota: password: my_ota_password unprotected_writes: True # This is mandatory if you want to flash the partition table or bootloader! logger: hardware_uart: USB_SERIAL_JTAG web_server: button: - platform: restart name: Restart ``` 2. Use Tasmota firmware upgrade with the ESPHome legacy firmware binary built in the previous step. ESPHome should boot 3. Upgrade the partition table: `python -m esphome -v upload test.yaml --partition-table`. ESPHome should reboot and still run from address 0x0e0000 as in Tasmota 4. Upgrade the ESPHome (either using the dashboard or `python -m esphome -v upload test.yaml`). ESPHome should boot and it will be running out of the 0x010000 address as in regular ESPHome. At this point OTA works but in order to use all available flash we need the following steps. 5. Build ESPHome (`test.yaml`) without the custom partition table from step 1 6. Upgrade the partition table: `python -m esphome -v upload test.yaml --partition-table` ## Types of changes - [ ] Bugfix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Other **Related issue or feature (if applicable):** NA **Pull request in [esphome-docs](https://github.com/esphome/esphome-docs) with documentation (if applicable):** esphome/esphome-docs#<esphome-docs PR number goes here> ## Test Environment - [x] ESP32 - [x] ESP32 IDF - [ ] ESP8266 - [ ] RP2040 ## Example entry for `config.yaml`:  ```yaml # Example config.yaml ota: unprotected_writes: True # This is mandatory if you want to flash the partition table or bootloader! password: removedPassword ``` ## Checklist: - [x] The code change is tested and works locally. - [ ] Tests have been added to verify that the new code works (under `tests/` folder). If user exposed functionality or configuration variables are added/changed: - [ ] Documentation added/updated in [esphome-docs](https://github.com/esphome/esphome-docs).