Tried to OTA Update Mutliple ESPHome Devices - Seems to Have Broke Them

bryanbr · November 10, 2021, 5:02pm

A few more thoughts:

This one will sound stupid – doublecheck your host IP address on the pi. Supervisor | System – check IP address is correct. Just in case. Check Gateway there as well.
Maybe add a manual ip address to the esphome yaml build. Here is syntax:

wifi:
  ssid: $wifi_ssid
  password: $wifi_password
  #  manual_ip:
  #    static_ip: 192.168.178.230
  #    gateway: 192.168.178.1
  #    subnet: 255.255.255.0
  #    dns1: 8.8.8.8
  #    dns2: 4.4.4.4
  #----------------------------------------------------

Here is my log from esphome:

[17:48:10] INFO: Waiting for dashboard to come up...
[services.d] done.
[17:48:10] INFO: Starting ESPHome dashboard...
2021-11-10 17:48:11,198 INFO Starting dashboard web server on unix socket /var/run/esphome.sock and configuration dir /config/esphome...
[17:48:11] INFO: Starting NGINX...
2021-11-10 17:48:14,028 INFO 200 GET / (0.0.0.0) 5.40ms
2021-11-10 17:48:14,078 INFO 200 GET /static/fonts/material-icons/material-icons.css?hash=ead4c276 (0.0.0.0) 4.84ms
2021-11-10 17:48:14,088 INFO 200 GET /static/css/vendor/materialize/materialize.min.css?hash=ec1df3ba (0.0.0.0) 4.43ms
2021-11-10 17:48:14,098 INFO 200 GET /static/css/esphome-2.css?hash=95c08d5a (0.0.0.0) 2.64ms
2021-11-10 17:48:14,104 INFO 200 GET /static/js/vendor/jquery/jquery.min.js?hash=12108007 (0.0.0.0) 1.84ms
2021-11-10 17:48:14,105 INFO 200 GET /static/js/vendor/jquery-validate/jquery.validate.min.js?hash=aad785d4 (0.0.0.0) 2.28ms
2021-11-10 17:48:14,109 INFO 200 GET /static/js/vendor/materialize/materialize.min.js?hash=4be20daf (0.0.0.0) 1.52ms
2021-11-10 17:48:14,111 INFO 200 GET /static/js/vendor/jquery-ui/jquery-ui.min.js?hash=cab3392d (0.0.0.0) 2.67ms
2021-11-10 17:48:14,260 INFO 200 GET /static/js/esphome/index-e0440f6f.js (0.0.0.0) 1.56ms
2021-11-10 17:48:14,260 INFO 200 GET /static/js/vendor/ace/ace.js?hash=65322f49 (0.0.0.0) 4.42ms
2021-11-10 17:48:14,307 INFO 200 GET /static/fonts/material-icons/MaterialIcons-Regular.woff2 (0.0.0.0) 0.79ms
2021-11-10 17:48:14,362 INFO 200 GET /static/js/vendor/ace/theme-dreamweaver.js (0.0.0.0) 0.71ms
2021-11-10 17:48:14,364 INFO 200 GET /static/js/vendor/ace/mode-yaml.js (0.0.0.0) 0.83ms
2021-11-10 17:48:14,391 INFO 200 GET /devices (0.0.0.0) 1.83ms
2021-11-10 17:48:14,417 INFO 101 GET /ace (0.0.0.0) 0.98ms
2021-11-10 17:48:14,505 INFO Running command 'esphome --dashboard -q vscode --ace /config/esphome'
2021-11-10 17:48:19,398 INFO 304 GET /devices (0.0.0.0) 2.55ms
2021-11-10 17:48:24,398 INFO 304 GET /devices (0.0.0.0) 1.52ms
2021-11-10 17:48:29,403 INFO 304 GET /devices (0.0.0.0) 1.67ms
2021-11-10 17:48:34,414 INFO 304 GET /devices (0.0.0.0) 1.61ms
2021-11-10 17:48:39,422 INFO 304 GET /devices (0.0.0.0) 1.63ms
2021-11-10 17:48:44,434 INFO 304 GET /devices (0.0.0.0) 1.54ms
2021-11-10 17:48:49,441 INFO 304 GET /devices (0.0.0.0) 2.30ms
2021-11-10 17:48:54,449 INFO 304 GET /devices (0.0.0.0) 1.48ms
2021-11-10 17:48:59,450 INFO 304 GET /devices (0.0.0.0) 1.47ms
2021-11-10 17:49:04,594 INFO 304 GET /devices (0.0.0.0) 1.50ms
2021-11-10 17:49:09,471 INFO 304 GET /devices (0.0.0.0) 1.59ms
2021-11-10 17:49:14,475 INFO 304 GET /devices (0.0.0.0) 1.56ms
2021-11-10 17:49:19,479 INFO 304 GET /devices (0.0.0.0) 1.55ms
2021-11-10 17:49:24,492 INFO 304 GET /devices (0.0.0.0) 1.55ms
2021-11-10 17:49:29,505 INFO 304 GET /devices (0.0.0.0) 1.71ms
2021-11-10 17:49:34,524 INFO 304 GET /devices (0.0.0.0) 1.74ms
2021-11-10 17:49:39,520 INFO 304 GET /devices (0.0.0.0) 1.58ms
2021-11-10 18:01:14,576 INFO 304 GET / (0.0.0.0) 0.82ms
2021-11-10 18:01:14,597 INFO 200 GET /static/fonts/material-icons/material-icons.css?hash=ead4c276 (0.0.0.0) 0.93ms
2021-11-10 18:01:14,601 INFO 200 GET /static/css/vendor/materialize/materialize.min.css?hash=ec1df3ba (0.0.0.0) 1.06ms
2021-11-10 18:01:14,606 INFO 200 GET /static/js/vendor/jquery/jquery.min.js?hash=12108007 (0.0.0.0) 0.99ms
2021-11-10 18:01:14,612 INFO 200 GET /static/css/esphome-2.css?hash=95c08d5a (0.0.0.0) 0.95ms
2021-11-10 18:01:14,615 INFO 200 GET /static/js/vendor/materialize/materialize.min.js?hash=4be20daf (0.0.0.0) 2.15ms
2021-11-10 18:01:14,617 INFO 200 GET /static/js/vendor/jquery-ui/jquery-ui.min.js?hash=cab3392d (0.0.0.0) 3.18ms
2021-11-10 18:01:14,617 INFO 200 GET /static/js/vendor/jquery-validate/jquery.validate.min.js?hash=aad785d4 (0.0.0.0) 3.73ms
2021-11-10 18:01:14,704 INFO 200 GET /static/js/vendor/ace/ace.js?hash=65322f49 (0.0.0.0) 2.04ms
2021-11-10 18:01:14,706 INFO 200 GET /static/js/esphome/index-e0440f6f.js (0.0.0.0) 1.56ms
2021-11-10 18:01:14,743 INFO 200 GET /static/fonts/material-icons/MaterialIcons-Regular.woff2 (0.0.0.0) 2.04ms
2021-11-10 18:01:14,792 INFO 200 GET /static/js/vendor/ace/theme-dreamweaver.js (0.0.0.0) 1.01ms
2021-11-10 18:01:14,793 INFO 200 GET /static/js/vendor/ace/mode-yaml.js (0.0.0.0) 1.34ms
2021-11-10 18:01:16,013 INFO 304 GET /devices (0.0.0.0) 2.55ms
2021-11-10 18:01:16,016 INFO 101 GET /ace (0.0.0.0) 4.26ms
2021-11-10 18:01:16,018 INFO Running command 'esphome --dashboard -q vscode --ace /config/esphome'

snel6424 · November 10, 2021, 5:06pm

Host IP is correct.

I also have this in my yaml file, and everthing in the secrets file as well in my router lines up

  manual_ip:
    static_ip: !secret led_controller_ip
    gateway: !secret network_gateway
    subnet: !secret network_subnet

bryanbr · November 10, 2021, 5:06pm

Thinking outloud:

I thought I saw a reference to minimum password lengths (at github.com/esphome issues repository), but of course, I cannot find it any longer. I remember seeing passwords had to be 10 characters or more for wifi credentials which I use.

Perhaps your host HA instance + esphome is newer, yet you are using an older wifi-password.
Of course, that would mean no other devices could join…

Is this the only device not working?

bryanbr · November 10, 2021, 5:08pm

Nice job on secrets. Everything should line up with that. Compile would complain if it couldn’t find the secrets.yaml.

bryanbr · November 10, 2021, 5:09pm

Wait – I believe newer versions of esphome require ssid and password in wifi section.
My compiles complain without it.

wifi:
  ssid: !secret ssid
  password: !secret ssid_password
  manual_ip: 
    static_ip: 192.168.178.16
    gateway: 192.168.178.199
    subnet: 255.255.255.0

snel6424 · November 10, 2021, 5:11pm

Interesting. My password is 18 characters long though.

And no, I have 3 ESPHome devices, and all 3 of them are down after the OTA update. I have tried all of these steps on two different devices, the third device isnt super easy to get to.

esphome:
  name: led-controller
  platform: ESP8266
  board: esp8285

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  fast_connect: true
  output_power: 17db
  domain: ""
  power_save_mode: none
  manual_ip:
    static_ip: !secret led_controller_ip
    gateway: !secret network_gateway
    subnet: !secret network_subnet

  ap:
    ssid: "LED Controller Hotspot"
    password: !secret ap_password

logger:

api:

ota:
  password: !secret ap_password

bryanbr · November 10, 2021, 5:12pm

That looks solid. One additional thing I would add:

# Enable logging
logger:
  level: VERY_VERBOSE   #DEBUG is the default

Need to run my son to hockey – back online in few hours – will check back…

snel6424 · November 10, 2021, 5:17pm

oooh interesting, okay after adding that in and reflashing the device, this what the logs say

bryanbr · November 10, 2021, 7:16pm

Interesting. One source for searching is: Issues · esphome/issues (github.com)

problem uploading esp8266 · Issue #1665 · esphome/issues (github.com)

Your issue is certainly strange.

From the console again, could you docker exec ab123 default gateway IP just to confirm that esphome can see other IPs and working correctly?

snel6424 · November 10, 2021, 7:50pm

is that the correct command?

bryanbr · November 10, 2021, 8:32pm

Forgot the ping.

Docker exec containername ping your_defaultgateway_ip

This will confirm whether esphome container can see your gateway on your network segment. Proves that esphome networking is working as planned.

snel6424 · November 10, 2021, 8:35pm

that does indeed work

bryanbr · November 10, 2021, 8:44pm

Cool. That proves that esphome is communicating properly.

The compile/upload of the .bin file appears to be working.

If you reset the esp8266 device via reset button, can you monitor any serial logs?
Is the device pingable from another device on that vlan? Perhaps a PC/Mac/Pi?

bryanbr · November 10, 2021, 8:59pm

Here is a bit of a long-shot, but I found some pointers here:

SONOFF BASIC - Not stable above core 2.3.0 Will not connect to WiFi · Issue #2625 · esphome/issues (github.com)

I saw that you were runing esp8285 board and it may have some frameworks that are newer. Some people are mentioning that is may be helpful for Ubiquiti networks (not sure if you have that).

  board: esp8285
  framework:
    version: recommended

snel6424 · November 10, 2021, 9:01pm

So the device I am using is one of the MagicLED wifi controllers (flux integration in HA I believe), and so it does not have a reset button (that I know of?)

OOOOOHHHH If i change my laptop over to the same subnet as the device, I CAN PING it! So what the heck happened…why is my network all of a sudden not allowing cross subnet communication?

bryanbr · November 10, 2021, 9:06pm

Oh, I think I have one of those controllers – I haven’t used it in a while as my wife didn’t care for LEDs above a cabinet in our bedroom. Congrats on flashing that badboy! Respect!

So, I think you have narrowed down to cross-subnet communications. Perhaps easy check (which I doubt has changed), but perhaps checking secrets.yaml to see if anything has changed on your gateways and subnets. I’m guessing they haven’t changed. However at least you are narrowing it down.

bryanbr · November 10, 2021, 9:52pm

I’d be over my ski-tips guiding you on any vlan configurations, but I suspect that is what may be causing the cross-subnet communication issues.

I’ve read a little about 2.5mhz settings vs. 5mhz settings and IGMPv3 settings on Ubiquiti gear, etc, but I don’t have Ubiquiti gear in front of me to play with. My home has the consumer gear meaning no vlans (jealous).

Good luck on further troubleshooting. I’m sure you’ll get it. Let me know how things work out. I’m sure it is something small, yet so significant.

snel6424 · November 10, 2021, 10:01pm

Hmm… so playing with it a little further, it only seems to be the ESPHome devices that I cannot ping unless I am on the same subnet. If I put any other device on that vlan, I can ping it just fine.

snel6424 · November 10, 2021, 10:14pm

I have fixed it! It was due to an incorrect gateway in my ESPHome Secrets yaml. I had it as 192.168.10.1 (as that is what my router is), but it should be 192.168.20.1, as it is on vlan 20! Thank you so much for all of your help!

EDIT: ESPHome over VLAN became unreachable - #5 by Ronaldt helped me out as well!

bryanbr · November 11, 2021, 5:49am

Yeah! Fantastic. I knew you’d find root cause. So glad that you got it working.