Surely everything isn't supposed to recompile every time!?

theronburger · October 22, 2023, 9:21pm

Hi there!

Im a little stumped here. Im playing around with configs, and ESPHome is recompiling EVERYTHING every time. It’s a 10 minute wait to change a single parameter. Surely this can’t be the default? Surely I forgot to check a cache checkbox somewhere. Please say it isn’t so! Any help would be lovely

Im running HA on a Raspi 3 B1.2 with a Class 10 SD card
Versions :
Home Assistant 2023.10.5
Supervisor 2023.10.0
Operating System 10.5
Frontend 20231005.0 - latest
ESPHome 2023.10.1

zoogara · October 22, 2023, 10:00pm

If you add or remove components (the ESPHome definition of component) then yes - it will do a full compile.

Anything that requires addition or removal of a library will force a full compile. But changing a sensor config, or adding a new sensor of component of the same type generally won’t.

10 minutes is long. My biggest device takes 5 at most on a Pi 4 with limited (2Gb) of memory.

stevemann · October 22, 2023, 10:13pm

Use a slow computer, get slow results.
Rpi3 is fine for running Home Assistant and many people do, but the ESPHome compiler wants lots of processor time. So, yes this is probably normal. I wouldn’t know if ten minutes on a Pi3 is normal because I haven’t run Home Assistant and the ESPHome plug in on a Pi3 for years.

I run Home Assistant on a used Intel NUC i3 that I bought on eBay for a little over $100. Compiling an ESPHome binary takes me about 60-seconds.

CaptTom · October 22, 2023, 10:59pm

Being basically impatient, I now compile on my laptop. It’s pretty easy to install and compile from there. Saves my poor RPi a lot of work, and me a lot of time.

kurtj · October 22, 2023, 11:03pm

I’ve been thinking about doing this as well. I have a nuc running HA, but right now I have 5 esphome devices, and once a month compiling them all gets my fans revving up already. And I’m looking to expand on the esp’s with the echo, and a few more plugs. So yeah might just be easier that way

stevemann · October 23, 2023, 3:13am

Lightweight. I have 71 ESPHome devices…

Just for fun, I did install ESPHome on my PC and a compile finished in:
“[SUCCESS] Took 3.68 seconds”

kurtj · October 23, 2023, 6:29am

Yeah I kept lights, temperature and motion sensors on zigbee (deconz) and it’s pretty low/no maintenance so I’m sticking to it.

Initially I only had a BT proxy and a DHT sensor on esphome, but lately it became difficult to find decent plugs with power monitoring that work on zigbee. Innr 120 is no longer produced, the 220 is without power monitoring. I’ve tried some aqara, but coupled with deconz there’s a bug that multiplies consumption 10x. So now I’m replacing the buggy plugs with esphome ones (athom.tech) and while at it, I bought their ir-blaster too. I’ll be replacing switches with ir-blasters in each room when replacement is needed, so nr of esphome will rise no doubt

The HA runs on a slightly older nuc i3, but literally next to it sits my ‘mini server’ for my dockers/dns/ect (asus pn51) so I’ll probably check to run esphome in docker on that host later on

theronburger · October 23, 2023, 1:12pm

Ah, wild. OK, Thats unexpected. Docker has a lot of caching functionality that would normally help with something like this, even on a low power machine. Sure, yes, the primary code needs to be but it’s literally downloading platformIO and ESPTools afresh each time.

theronburger · October 23, 2023, 1:22pm

Ah no, this doesn’t seem to be the intended behaviour.
Looking into the ESPHome repo the docker entry point tries to handle cacheing.

github.com

esphome/esphome/blob/e3fbf54a1aafad77ab3125855f21607ed35be74d/docker/docker_entrypoint.sh

#!/bin/bash

# If /cache is mounted, use that as PIO's coredir
# otherwise use path in /config (so that PIO packages aren't downloaded on each compile)

if [[ -d /cache ]]; then
    pio_cache_base=/cache/platformio
else
    pio_cache_base=/config/.esphome/platformio
fi

if [[ ! -d "${pio_cache_base}" ]]; then
    echo "Creating cache directory ${pio_cache_base}"
    echo "You can change this behavior by mounting a directory to the container's /cache directory."
    mkdir -p "${pio_cache_base}"
fi

# we can't set core_dir, because the settings file is stored in `core_dir/appstate.json`
# setting `core_dir` would therefore prevent pio from accessing
export PLATFORMIO_PLATFORMS_DIR="${pio_cache_base}/platforms"

This file has been truncated. show original

OK, need to dig a little

stevemann · October 23, 2023, 2:03pm

If you don’t change anything, then a recompile takes only a second. I will often do a “manual download” first to check for errors before doing a WiFi install. If I’ve changed nothing, the compile is like one or two seconds.

theronburger · October 23, 2023, 2:52pm

erm… poked it with a stick for a bit and suddenly caching works.
that is… less than satisfying

peter.vk · November 3, 2023, 7:20pm

Were you able to get ESPHOME to no re-compile everything for every update for identical devices? I have a bunch of Sonoff S31s that take a while to update every time. I’m running HA on a Dell 9020 so I would have assumed it was pretty quick.

purplepanda · June 20, 2024, 4:50am

I’m just guessing but it seems like it doesn’t know when it actually finished installing and so therefore never saves the cache?

I have to hit stop every time.

nickrout · June 22, 2024, 7:29am

It will start logging the new firmware to the screen. Be patient.

BeckyB · June 26, 2024, 10:53pm

i use 1st gen NUC with celeron takes 3 min per node i have 26 nodes. its like a 12 yo pc but is max 15 watts. im ok with slow and cheap and reliable. ill prolly use it till the caps blow out.