Esp32e taking forever to load or modify code on via pi 3b+ is that normal?

[jumper-d-1981]

It is taking over 10 minutes to load code to the esp through the wireless upload. Only a minute or so for the actual upload, so I think it is just this pi3b+ being slow. Pretty much no addons other than a studio code viewer which is not on atm. The compiling and linking takes a really long time, would love to be able to speed this up. Is there a way to easily load code via the Arduino IDE and still be able to connect this to my HA? I have some really complex code for another esp project unrelated to HA and it only takes like 2 or 4 minutes.

Thanks.

[Mahko_Mahko]

Yes it’s the pi. Take a look at your memory and CPU with system monitor. Both the background level and then spikes when you compile.

I moved to an old laptop pretty much for this reason. Compile times dropped from 10minutes or
so to like 15sec.

You can actually compile and upload ESPHome fro a totally different machine that has better resources (say your pc).

System monitor

Optimizing old laptop for HA server use

I’m going to add my experiences to this post (ongoing). I want to speed up HA reboot and ESPHome compile times from my rpi4 4GB w microSD. I have an old laptop with HDD and 4GB RAM which was destined for the trash. Battery failed long ago. CPU is Intel i7-3610QM CPU @ 2.30GHz. Before investing in SSD and memory upgrades, I decided to pull out the HDD, throw it in an external enclosure I have, and flash the generic-x86-64 image to it to see if it would work. I followed the instructions, and …

[aceindy]

@Mahko_Mahko

You can actually compile and upload ESPHome fro a totally different machine

Even crazier, you can do that online on Esphome hone page
https://web.esphome.io/
It used to have an option. to download the bin, but it seems nowadays it requires web-serial

[Spiro]

It’s only been the last month or so since it started taking >10mins on the RPI3b +. Can only hope it will imorove.

[CaptTom (Tom)]

Yes, I’ve seen the slow-down over the past few versions, too.

I don’t expect it to improve. My own suspicion is that HA is close to dropping support for the RPi. Often in this forum I see people denigrating the PI, and recommending more robust hardware. Meanwhile, new and processor-intensive “features” are being added (see: voice.) It seems every new version thread has posts showing graphs of increased memory and/or CPU usage.

I hope I’m wrong. I hope it’s just that the hard-core, full-time HA enthusiasts who expend lots of time and money building out a bleeding edge system are over-represented here. Personally, I chose HA because it could run on the RPi, which was something I wanted to dabble in. But I do have to limit the HA budget and how much time I spend tweaking it. I hope part-time hobbyists like me aren’t left behind.

[Spiro]

I want to push my RPI3b+ as far as I can. This is the fun. The lower the power the better. If I could push this on a C64 I would. Every upgrade an orpertunity. Nowt worse than an upgrade going fine.

[UngluedChalice]

@Mahko_Mahko

You can actually compile and upload ESPHome fro a totally different machine that has better resources (say your pc).

I’m trying to figure out how to do this, but I only can find references that it is possible, or sometimes “do it with docker compose.” (I’ve never done anything with docker). You have any guesses at to what I should google to figure this out? Can’t really do anything with esphome now as the Pi3B doesn’t seem like it can handle it. Would like to use my macbook to compile the code and then upload it.

[Mahko_Mahko]

It seems to be more advanced users doing it who know what they’re doing and don’t need to much instructions (unlike me ;).

Searching around on the ESPHome Discord is probably a better bet, as there’s a lot of advanced users over there.

Join the Discord via ESPHome homepage or link below.

Here’s two threads I was involved in when I was interested in it. The second one is more relevant. I’m sure there are many more on Discord. I seem to have issues linking to Discord threads sometimes, so try searching for the title if links fail.

Join the Community

Targeting Hardware Upgrades to target reducing compile times

ESPHome in a VM (+HomeAssistant)

There’s also these sections of the docs:
Mac: Installing ESPHome Manually — ESPHome
Installation: Getting Started with the ESPHome Command Line — ESPHome

[UngluedChalice]

Got it! I thought there was a bunch of stuff that I wouldn’t know and it wouldn’t work, but I can’t believe I actually got it to work.

For those like me from the future who want some hints/instructions:

I googled “install python on mac.” This was super easy and typical “double click to install” kind of thing.

Then I opened up the terminal on my Mac and did

pip3 install esphome

It prompted me to upgrade python or something so I did that.

Then I tried to run esphome version, again all of this is on my Mac from a Terminal window and I got an error.

So then I did this (this is from the install page, I didn’t change anything)

echo 'export PATH=$PATH:$HOME/.local/bin' >> $HOME/.bashrc

I closed the Terminal window and reopened it and ran esphome version and it worked (showed me the version of esphome on there).

Then I used the cd command to change directory to my Desktop, as that is where the yaml file was. I downloaded this from the ESPHome add-on page in Home Assistant. I went to the ESP32 I was wanting to update and there’s an option to download the yaml.

Then, to compile (and a apparently upload, that was a surprise) I ran

esphome run esp32-poe-2.yaml where the esp32-poe-2.yaml is the name of the yaml file itself, so you’d have to change it to match your filename.

Then after about 1-2 minutes, it was done, and started showing the log of the esp32. So it not only compiled the code, but uploaded it to the esp32 as well.

My current problem/question is now this code is separate from the ESPHome add-on and what I see there in HA. So I have to make sure not to use that to update as it will again kill the pi and will push out the older code.

I’ve come to realize the addon is the thing that compiles the code, so I’ve basically just moved the add-on to my computer and I could get rid of it. Except, it is still helpful for when I’m setting up a new ESP32 because I think there are issues with USB and ESPHome on a Mac, but I’m not sure. And it is nice for pulling up the logs, as I don’t know how to do it (although I’m sure it’s just knowing the command) from the terminal.

Anyway, that’s how it worked for me on a 2019 MacBook Air with MacOS 13.0.1 (22A400). Hopefully this is helpful to someone else!

[CaptTom (Tom)]

Thanks for the detailed post! I did something similar on my Windows machine but it was just so much more convenient to do it all from the ESPHome UI on HA that I never really use it. I know I’m putting a strain on my poor old RPi when I try to compile though. I think the real solution would be to install an instance of HA on my laptop just to use to compile ESP stuff. Maybe one of these days…

[Mahko_Mahko]

Nice. Thanks for sharing back.

I believe you can still install “the dashboard” on your Mac. The thing you see in the add-on.

Someone suggested using GitHub as the means to keep config files in sync on both machines. I see quite a few advanced users doing that. Need to get secrets set up right etc.

I used to use an add-on and software called syncthing to do something similar a while ago. Think of a local Google Drive sync.

[CaptTom (Tom)]

This thread motivated me to try the Windows command-line ESPHome option again.

Wow, that was easy! And now I can compile and install to my ESP device much faster, and without taxing the HA machine or its SD card. The install instructions are here.

It’s still easy to create the yaml file in the HA ESPHome add-on. I then copy the file to my laptop where I have all my ESPHome yaml files, including samples, current and old versions.

From there I can run a few commands:

• esphome config filename.yaml to validate the YAML.
• esphome run filename.yaml --device [ip address] to compile and upload it to the device.

The run command can also be used to load a new ESP device that’s connected to the USB port on the laptop.

Of course I have ESPHome on HA open in a browser, so I can immediately see the results.

Thanks again for pushing me to revisit this!

[UngluedChalice]

Anyone else get the error:

ERROR Error while reading config: Invalid YAML syntax:
Error reading file secrets.yaml: [Errno 2] No such file or directory: ‘secrets.yaml’

I didn’t have this issue because I was using an ESP32 with PoE earlier, but now I am trying it with an ESP8266 over wifi and it doesn’t work. I assume it’s becuase the secrets.yaml is on the Pi and not my Macbook - anyone got a fix?

[2probro]

I was also tired of the slow compiling on my Raspberry Pi via the add-on in Home Assistant. I now use my computer for this (much faster).

How I solved this:

First, install your Home Assistant on your Rasberry Pi like usual.

Afterward, you need to make a folder on your desktop (I did this in windows) called “ESPHome”.
Open cmd and go to the folder with cd desktop/esphome
Then in cmd pip3 install wheel, pip3 install esphome and lastly you can check if it’s installed with
esphome version.

If you don’t get the esphome version, you probably didn’t install the packages in the correct directory or you might not have added the installation location to your PATH.

If you have successfully installed esphome you should now be able to use it in the cmd
First again go the folder in cmd with cd desktop/esphome
Then use esphome wizard <name you chose>.yaml (you can change the name to whatever you want)
There will now be a .yaml file in the esphome folder.
You can modify this to include sensors etc…

If you’re done with the code, you again go to the folder with cd desktop/esphome
Then use esphome run <name you chose>.yaml
Now it will compile the code with your much faster pc, instead of the Raspberry Pi.

After it’s compiled you’ll be asked to upload “via usb” or “over the air”.

You have now successfully installed new firmware without having to deal with the slow compiling time of the Raspberry Pi.

For more info:
https://esphome.io/guides/installing_esphome.html

Getting Started with the ESPHome Command Line