ESPHome - initial setup - can't find device, can't manual download, basically can't do anything

Background: Recently I setup HA in a docker on a Synology NAS. Everything is working fine and I primarily I have been tinkering with custom sensors using raspberry pi pico Ws and MQTT (also in a docker on my NAS). I have seen a lot about ESPHome and the prospect of using that and being able to manage OTA seemed beneficial, so I set to testing it out, and basically, I can’t even get past the starting gate.

Issue(s): I installed ESPHome in a docker on my NAS and it seems to be running OK and I can access the dashboard. However when I go to add a device via the dashboard page on my Windows PC, I get the New device dialog stating I’m not using HTTPS and therefore I’m prevented from using connected device (rpipicow). Ok seems to make sense since the device is connected to a different computer than the docker is on, so I select the “OPEN ESPHOME WEB” follow the instructions, download the uf2 file, drag it to the rpipicow, then it finds the device and I set the wifi info and it shows up on my wifi network.

That’s as far as I can get with the device, I cannot see it and therefore adopt it into ESPHome using the dashboard. Yesterday, when I was trying various things it did show up once for adoption but ultimately it failed with some error that I didn’t document and I have never gotten to that point after uninstalling/reinstalling ESPHome docker, reflashing the picow etc. Interestingly I can add the device into HA using ESPHome, but at this point it’s not so useful since I haven’t been able to configure it as I want.

I’m not sure where to go from here to troubleshoot. In the end I just want a way to program the device/node and flash it. If there is some better way than using the ESPHome dashboard, I’m all for it. That being said one of the other issues is that if I create a device in the dashboard, select install and then “Manual download”, it hangs and the step “3 Download project preparing download…” the blue circle spins indefinitely.

Well I randomly decided to comment out the encryption and the related key in the test.yaml device I tried to create and what do you know it worked. I had previously added manual_ip info for the device, which appears to also be required. I guess I will have to dig some more to understand why it auto generates and api encryption key and why that causes issues.

Spoke too soon. That allowed me to get info when clicking on “log”. Still when I go to install it hangs as somewhere around 30%:

I’m not sure what the web setup of the picow does but it feels like there is some security key that is needed to access it for installing configs that’s needed but is not provided when it’s setup.

It hasn’t even compiled. Looks like the server can’t download the libraries needed to compile the code. Or rather it is still doing so. (31%, wait longer) .