ESPhome freezes during compile

What? It’s called memory management

1GB simply isn’t enough… I can see on my Pi4 (4GB), that it needs more than 1GB to compile correctly or at all.

I’d recommend to use your PC/Laptop for compiling. Then you can decide how you want to go ahead, uploading via a Windows FlashTool (ESPHomeFlasher) or uploading the binary file to your Pi and then forward it to the device. There is no need to compile it on your Pi, besides the comfort. :slight_smile:

I didn’t know it was possible but super easy to install: pip install esphome.

I just went into one of my more powerful Linux machines to set up an ESPHome installation thinking this way I could compile on it instead of my pi hoping to avoid this common situation of freezing the whole pi and forcing me to power cycle it. What I found out is that I would have to recreate the whole structure of my YAML files from my home assistant so that my inherited secrets file would work… I’m not really interested in copying out my secrets file to another machine.

I wish I could just compile these on the pi without the risk of it hanging the whole pi.

You don’t need to. ESPhome has it’s own secrets.yaml file. In my secrets.yaml (esphome) there are only three or four entries listed, that would be the ssid, the wifi-password, mqtt-password and ota-password.

All the other entries shouldn’t be needed from the HA secrets.yaml. :slight_smile:
And you don’t have to rebuild any structure from HA, you just need to write one file, that you then could use for all of your future projects.

As I said, you will loose some comfort, but it’s your choice. Use esphome with a little less comfort or don’t use it at all, as the Pi will definitively not get more power over time, and esphome will not get smaller… :wink:

In the end it is one copy & paste to the secrets.yaml on esphome and you are good to go and can compile and upload very easy. :slight_smile:

Yeah. I’m doing this now but I’m not happy about it. Having two sets of config files is terrible especially two sets of secrets. I’ll have a hard time keeping everything sync’d up and if I ever need to do something remotely (through the ingress), I’ll have to remember subsequently to sync it up backwards – otherwise I’ll end up undoing some improvement/fix.

I guess I prefer having to do all of that versus risking esphome crashing the pi during compiles.

First time a compile locked my 'pi up today - found this thread and (after rebooting the 'pi) I tried shutting down my unifi plugin and the compile worked fine. So memory then. Not a problem - it’s not like I compile a lot so I’ll just need to remember to shut down any large plugins first. Happy to live with that rather than run things like unifi on a separate device.

FYI, my results have been VERY inconsistent. I’m not sure I’d call your results a correlation based on what I’ve experienced. I’ve just stopped using the pi to compile to avoid the lockups which sucks because if it’s ever fixed I’ll never know. :neutral_face:

Heh - true. Not exactly science based I’d admit, but for the heck of it I did try compiling again after starting the unifi plugin and my 'pi froze part way. A reboot, and - after shutting that plugin down first - it happily compiled again. Absolutely may be a coincidence, and perhaps if I was more patient it might even complete the compilation, but my initial knee-jerk reaction is to suspect an issue with resources whether that’s memory or disk so I’m going to try to remember to shut down a few things prior to doing a compile. Again, I only do this rarely so I can live with it. :wink:

Cool, I’ll be interested if you use it like this for dozens of builds without a freeze. Oh, I would so love to be able to use my HassIO to handle the management of ESPHome devices again… it’s really making me consider purchasing some higher-end hardware to run my home automation.

Maybe not dozens, but I did about 8 in a row on my RPi 3 B+ without an issue. I actually also shut down deconz to free up even more resources, but forgot I had the watchdog enabled on that so when I checked later I found that it had of course started back up again most likely before I even started to try to compile anything.

I was having the same problems as others mentioned here with my Pi3+. Wasn’t running unifi, but thought maybe some of the resources were taking up memory. turned off every non essential, and she compiled right up. Memory is the issue, so the ESPHOME guys should consider this, as I was about to consider it wasted funds and throw the parts on eBay before reading some of the other posts. Though the other posts were not a solution, it was enough for me to try something different.

You do not have to run it on a pi.

before I switched to a NUC I had HA running on a pi and compiled ESPhome in a VM on my desktop PC

Yeah, yesterday when I ran the update to 0.118.3, hassio was unavailable for about an hour as it “updated”, looks like it was running out of memory (the high point is ~95%):
image

My memory was sitting at ~87% usage just now and I just tried turning off the Unifi controller and the memory dropped to ~50%!!!

I tried turning off the ESPHome, Samba, and File Editor and they only accounted for ~3% memory usage total. The rest of the add-ons I’m using are non-negotiable. TBH, I don’t want to cut out functionality so I can build ESPHome stuff and running a separate build host is annoying, I’ve been doing it for weeks and keeping the two in sync is a pain in the butt.

I think this black Friday I’ll be graduating my hassio installation to a NUC…

How do you add the memory and the processor use to the dashboard?
I have daily 2-3 hours freezes that I can’t find the cause for, I managed to plot the CPU/GPU Temps and Freq but no RAM and CPU Load.

So I’m only into this now by about a year and loving the capabilities, but chasing how to make things work and keeping a day job are tough. Until this post, I hadn’t heard of NUC and no clue as to the steps to compile in VM.

I’m still learning

I use the systemmonitor integration:

- platform: systemmonitor
  resources:
    - type: disk_use_percent
      arg: /
    - type: memory_use_percent
    - type: processor_use
    - type: memory_free
    - type: load_1m
    - type: load_5m
    - type: load_15m
    - type: last_boot
1 Like

I finally broke down and did the upgrade… it wasn’t too expensive either. I upgraded my router a few days ago, from:


To:

So, the 4-port version was freed up. I decided to put an installation of Ubuntu on there and host a supervisor docker for HassIO. I’m amazed at how simple the transition was: the snapshot functionality is AMAZING. I did a full snapshot from the Pi, shut it down, renamed the new host to replace the old and launched the “Wipe & Restore” option with it on the new installation and voila! Everything worked (except for samba because the eth0 changed to enp2s0).

The results are fantastic! The green marks below the charts are where I made the transition:

This thing has 8GB of ram so the swap sits at zero, now. :stuck_out_tongue:

image
Looking at the cpu/memory/swap usage percentage chart, you can see that the Raspberry Pi 3b+ was memory limited previously, with the ram full, the CPU apparently was constantly waiting on swap IO so cpu percentage maxed out at about 50% utilization before. Now that I don’t have that problem, the CPU can spend much more time crunching important things and not stalling. Also, look at the CPU temperatures! I dropped 40° and gained performance. :smiley:

Best of all, I get to re-consolidate my ESPHome installations onto the HassIO device again! :tada:

I want to thank you for the memory and swap integrations, I could see that my RPi4 1Gb had only 100 MB of swap and it was all used, after changing to 2 Gb I got no more blocking for 4 days now!!!
So definitely 1 Gb is too small for HA Supervised.

Never had this issue until today as well. When you say “reboot” can you describe what exactly you mean? I’ve been restarting HA, I’ve restarted the core, restarted supervisor, and rebooted the core, restarted ESPhome, not to mentioned power cycled the Pi after it crashes - all with no luck.

I’ve also attempted @sermayoral’s suggestion with no luck either.

Edit: This might help some of you, so I figure I’ll share. I ended up stopping all the other addons that I had running and it seemed to resolve the issue. Obviously the Pi struggles a bit when things have to compete for memory. Not an ideal solution, but I finally got it working.