ESPHome setup .yaml file not in \config\esphome\

What I have is: Raspberry pi 3B+
PC with Windows 10
ESP32-WROOM-32 30pin module

  • Core2024.1.6
  • Supervisor2023.12.1
  • Operating System11.4
  • Frontend20240104.0
    ESPHome 2023.12.5
    My HASS dashboard is in Norwegian, the translation of terms may be a bit different from the correct English wording.

My question is, after I made all below, still the .yaml configuration file is solely local on my PC. There’s no .yaml on the Raspberry. Is that correct, that I am able to edit the ESPHome configuration only from this PC?
Anyway, this information may be helpful for others that struggle to seup the ESPHome. I found many other advices that did not help so this is yet another one as soon as I figure out this location of the .yaml

Problem and solution

The “missing” .yaml was just a result of conflicting methods of installation. Have look at the bottom to see the cause and solution.



History:

By trying I found that the Rpi 3B+ cannot compile (due to memory?) so I made it the Manual way but it failed.
So I went the Manual way :
https://esphome.io/guides/installing_esphome
Installed Python - in cmd window: ‘>pip3 install wheel and’ ‘> install esphome’
As you can see in

I struggled and failed to complete this.

This path made it better:

On the Raspberry I deleted all files of the \192.168.1.17\config\esphome+ directory. And in HASS I deleted Unit and Entities so there was only the ‘Configuration-’>Integration->ESPHome’ left to prepare for a fresh new device.
However there’s still an ESPHome dashboard here, empty.

Integration ESPHomme in HASS, this was made earlier:

As mentioned above I deleted all ESP module specifics after this.

Now, instead of the install above I did:

Run the wizard:

‘>esphome wizard brann.yaml’
Here I define ESPmodules name, function, name etc by wizard
Find the proper ESP device in the list, in my case Espressif ESP32. I have the ESP-WROOM-32 30pins with no identification of manufacturer.
And thereafter more specific, I used ESP32dev

Once complete, So where’s the .yaml file? I found it on my PC-disk as C:\Users'>my_name>\brann.yaml’
That’s where I open and edit it:
Added a switch to yaml like the example: (Note I mistyped the room to rook)
switch:
’ - platform: gpio’
’ name: “Living rook dehumidifier”’
’ pin: 5’

Compile by cmd:

‘>esphome run brann.yaml’
During this I get some queestions:
Select port (OTA Wireless or PORT)

Compilation printout stops, last lines are:
API Server:
Address: …
Using noise encryption: NO
5 minutes after this: 3 lines of Boot seems succesful, Saving preference etc
I checked the WiFi router, ESP is not in the device list
10 minutes later: More output in DOS: forcing reboot etc - Still not in routers list
4 minutes later: Same last 3 lines as above. Still no ESP in router device list, no .yaml in Raspberry’s \192.168.1.17\config\esphome directory
In HASS Integrations there’s a the ESPHome, click Configure and Password = And area (room)
Tada! Here it is, with 2 enitities ! The switch and firmware status. Based on this I should be able to add more to the .yaml and recompile the firmware. (But I thought it was sufficient to edit thee .yaml and refresh it from the HASS - Developer - Load Yaml, seems this is not the case)

(Note that my HASS panel on PC locked so only the main and Unit/Integration page showed data. I used my Galaxy pad to fulfill the integration.
At this stage, there’s a Raspberry file \192.168.1.17\config\esphome\secrets.yaml - but still no brann.yaml here.

And still there’s an ESPHome dashboard which was here from the start where the only option is to add a New device. This option is to add yet another (second) ESP, not to add the one I have added. And I’m not able to remove this dashboard.

What I learned is that when I setup the ESPHome Dashboard this made a .yaml file and I used the same name for the ESP device, Dashboard and .yaml file (Came from default). And when I edited this .yaml it did not result in any changes. So I came closer to my goal by this approach. I made some trouble to mayself by introducing a separate dashboard for the ESPdevice. I rather wait with this until I get control of this first part.
brann.yaml:

esphome:
  name: brannsentral

esp32:
  board: esp32dev
  framework:
    type: arduino

# Enable logging
logger:

# Enable Home Assistant API
api:
  password: "***"

ota:
  password: "***"

wifi:
  ssid: "***"
  password: "***"

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Brannsentral Fallback Hotspot"
    password: "EoZrlPFbjohy"

captive_portal:
    
switch:
  - platform: gpio
    name: "Living rook dehumidifier"
    pin: 5

I’m not sure if I understand your question entirely. But the ESP-home workflow is as follows:

  • On your HomeAssistant host, you set up the ESPHome integration, and you add a new yaml ‘configuration file’ for the device you want to connect (in your case: brann.yaml).
  • Then you need to install/update ESP-Home on the ESP device itself. Using the configuration from the yaml file, the ESPHome will build the correct firmware and flash it onto the device. If it already runs ESPHome and Wifi is configured, you can do so wirelessly, but doing that wired by connecting it through e.g. an USB cable is the more reliable option.
  • After that is successfully, the ESP Device should be up and running with the new ESPHome firmware. The yaml configuration file is (probably) not on the ESP device, as it is only used as input configuration for building the firmware. However, in the ESPHome integration in Home Assistant, the yaml file is still there and you can view or edit it again to make changes.
  • In order to push those changes to the actual ESPHome device, you will need to again compile a newer version of the firmware and upload it to the ESPHome device, using the ‘Install’ option in the ESPHome dashboard (again, either through Wifi or wired). After that, the device should be running the latest firmware and be up-to-date with the configuration file in ESPHome.

Thanks a lot for assisting me!
In previous attempts I tried that approach but it failed. I saw that the HASS Integration prepared a .yaml file. However that contradicts the way described by the Manual approach, where the .yaml file is defined and made by the Wizard. And this file remains on my local PC. So I deleted the .yaml on the Raspberry config directory as I expected the new one to be copied over.
I am not able to update via the ESPHome Install option as it fails to compile. Something like “Killed signal terminated program cc1plus”. Seems many other struggle by this, f. inst:

And the recommended fix here is to avoid the Install option and rather do the compile by Python on a PC.
So then it seems to me that I have to compile locally on the PC by the '>esphome ’ command line every time?

Hhm, that shouldn’t really be necessary. I’m running it on a Raspberry Pi 4, which is more powerful than the 3, but I don’t think that should be such a large issue. Sounds more like something else to me…

As this is your first/only ESPHome device, I would suggest to completely remove the ESPHome integration (Settings > Add-Ons > ESPHome > Remove). Then add it again, and make sure you update to the latest version (Current version: 2023.12.9) . Then, add a new device (using the green “+ New Device” button), answer the questions to make sure you use the right target device, and try to set it up with just the most basic configuration needed (the defaults will probably just do fine). Use the ‘validate’ option (before installing) to make sure the configuration does not contain errors, and then connect the device with a proper USB-cable and try the install option.

Once that works, you should be able to repeat the process: In the ESP-home integration, make changes to the yaml configuration, and install it again to the device.

Does that work? If not, can you describe what errors you get, post the configuration you have used, what the target device is (the one you are trying to flash ESPHome to), and how they are connected?

I did as you described, after have removed all I deleted the secrets.yaml in config/esphome config too. No traces of Esp as far as I could see. So I installed the standard way as you told, so far so good. Connected the ESP32 to USB and then added it by “Prepare for first use”. Then checked that the .yaml in \192.168.1.17\config\esphome was proper, just the basic and no entities here.
Clicked Install, and things went quick and OK, named the device by a new name - autroprime, and suddenly there were two instances her - both the one I have removed and the new… (By view of the left panel sidebar / ESPHome). The name of this one is corresponds with the MAC adress of the ESP32. And was used as the naming in a previous trial of installation.
Looking at the new ESPHome unit it has two entities, one of them was declared on the previous (deleted) ESP:


And here’s the two entities of the ESPHome Web 60a7dc:

The previous brann.yaml-file was listed above,
New aurtoprime.yaml file:

esphome:
  name: autroprime
  friendly_name: Autroprime

esp32:
  board: esp32dev
  framework:
    type: arduino

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: "4uXDJFCohmHkkglg/PGBFe/V4xwATwaMOl9tbNASYlc="

ota:
  password: "7a5abd95e29f651949c320eb91b3da18"

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Autroprime Fallback Hotspot"
    password: "nr9xBNSGCTw9"

captive_portal:
    

Downloaded diagnostics:

{
  "home_assistant": {
    "installation_type": "Home Assistant OS",
    "version": "2024.1.6",
    "dev": false,
    "hassio": true,
    "virtualenv": false,
    "python_version": "3.11.6",
    "docker": true,
    "arch": "aarch64",
    "timezone": "Europe/Oslo",
    "os_name": "Linux",
    "os_version": "6.1.63-haos-raspi",
    "supervisor": "2023.12.1",
    "host_os": "Home Assistant OS 11.4",
    "docker_version": "24.0.7",
    "chassis": "embedded",
    "run_as_root": true
  },
  "custom_components": {
    "tapo_control": {
      "version": "5.4.11",
      "requirements": [
        "pytapo==3.3.18"
      ]
    },
    "hacs": {
      "version": "1.33.0",
      "requirements": [
        "aiogithubapi>=22.10.1"
      ]
    }
  },
  "integration_manifest": {
    "domain": "esphome",
    "name": "ESPHome",
    "after_dependencies": [
      "zeroconf",
      "tag"
    ],
    "codeowners": [
      "@OttoWinter",
      "@jesserockz",
      "@kbx81",
      "@bdraco"
    ],
    "config_flow": true,
    "dependencies": [
      "assist_pipeline",
      "bluetooth"
    ],
    "dhcp": [
      {
        "registered_devices": true
      }
    ],
    "documentation": "https://www.home-assistant.io/integrations/esphome",
    "integration_type": "device",
    "iot_class": "local_push",
    "loggers": [
      "aioesphomeapi",
      "noiseprotocol",
      "bleak_esphome"
    ],
    "requirements": [
      "aioesphomeapi==21.0.1",
      "esphome-dashboard-api==1.2.3",
      "bleak-esphome==0.4.0"
    ],
    "zeroconf": [
      "_esphomelib._tcp.local."
    ],
    "is_built_in": true
  },
  "data": {
    "config": {
      "entry_id": "a0a6cb4eb6068c07a203c7e4e912579a",
      "version": 1,
      "minor_version": 1,
      "domain": "esphome",
      "title": "brannsentral",
      "data": {
        "host": "192.168.1.16",
        "port": 6053,
        "password": "",
        "noise_psk": "",
        "device_name": "esphome-web-60a7dc"
      },
      "options": {
        "allow_service_calls": true
      },
      "pref_disable_new_entities": false,
      "pref_disable_polling": false,
      "source": "zeroconf",
      "unique_id": "e0:5a:1b:60:a7:dc",
      "disabled_by": null
    },
    "storage_data": {
      "device_info": {
        "uses_password": false,
        "name": "esphome-web-60a7dc",
        "friendly_name": "ESPHome Web 60a7dc",
        "mac_address": "**REDACTED**",
        "compilation_time": "Jan 19 2024, 22:59:47",
        "model": "esp32dev",
        "manufacturer": "Espressif",
        "has_deep_sleep": false,
        "esphome_version": "2023.12.8",
        "project_name": "esphome.web",
        "project_version": "1.0",
        "webserver_port": 80,
        "voice_assistant_version": 0,
        "legacy_bluetooth_proxy_version": 0,
        "bluetooth_proxy_feature_flags": 0,
        "suggested_area": ""
      },
      "services": [],
      "api_version": {
        "major": 1,
        "minor": 9
      },
      "switch": [
        {
          "object_id": "living_rook_dehumidifier",
          "key": 4179117744,
          "name": "Living rook dehumidifier",
          "unique_id": "brannsentralswitchliving_rook_dehumidifier",
          "disabled_by_default": false,
          "icon": "",
          "entity_category": 0,
          "assumed_state": false,
          "device_class": ""
        }
      ]
    },
    "dashboard": "5c53de3b_esphome"
  }
}

Validating the autroprime:

INFO ESPHome 2023.12.9
INFO Reading configuration /config/esphome/autroprime.yaml...
esphome:
  name: autroprime
  friendly_name: Autroprime
  build_path: build/autroprime
  area: ''
  platformio_options: {}
  includes: []
  libraries: []
  name_add_mac_suffix: false
  min_version: 2023.12.9
  compile_process_limit: 1
esp32:
  board: esp32dev
  framework:
    version: 2.0.5
    source: ~3.20005.0
    platform_version: platformio/[email protected]
    type: arduino
  flash_size: 4MB
  variant: ESP32
logger:
  baud_rate: 115200
  tx_buffer_size: 512
  deassert_rts_dtr: false
  hardware_uart: UART0
  level: DEBUG
  logs: {}
api:
  encryption:
    key: 4uXDJFCohmHkkglg/PGBFe/V4xwATwaMOl9tbNASYlc=
  port: 6053
  password: ''
  reboot_timeout: 15min
ota:
  password: 7a5abd95e29f651949c320eb91b3da18
  safe_mode: true
  port: 3232
  reboot_timeout: 5min
  num_attempts: 10
wifi:
  ap:
    ssid: Autroprime Fallback Hotspot
    password: nr9xBNSGCTw9
    ap_timeout: 1min
  domain: .local
  reboot_timeout: 15min
  power_save_mode: LIGHT
  fast_connect: false
  passive_scan: false
  enable_on_boot: true
  networks:
  - ssid: 3Com
    password: xxx
    priority: 0.0
  use_address: autroprime.local
captive_portal: {}

INFO Configuration is valid!

The two steps of "Prepare for first use " and "Install went quickly, so the firmware of the ESP is (most probably) left as it was despite the start from scratch of ESPHome.

BTW as you say latest version is 2023.12.9, so it says her too:

I compiled the FW by Python today and it still made 2023.12.5
After this: I thought the compilation and upload from PC looked for the latest ESPHome, but I found that I have to update by:

pip3 install –U esphome
before I make the binary and upload.
Frequently Asked Questions — ESPHome

Glad to help! So if I understand correctly, you are now able to compile and upload/install the firmware correctly from ESP-home to the device, which previously did not work. Great to hear that that is working! You should then also be able to use the wireless installation method (of course only when the esp-device is powered on and connected to WiFi)!

Nope, I’m not quite there yet. I described that even if I prepared only one ESP32 I’ve got two instances of the ESP32 (see picture). And on the device I intend to use there are two entities that is a mix from both. The autroprime.yaml file does not contain any of these entities - “Living rook dehumidifier” and “ESPHome Web 60a7dc Firmware”.
But the other, ESPHome Web 60a7dc, was the one I deleted from HASS. As it pops up now I presume the physical ESP32 has kept it’s firmware and configuration unchanged.
Another reason I doubt it is really compiled as it went too fast, just some 10-15 seconds, hence I’m not convinced still that the compilation will work next time.
Tomorrow I will start all over by removing all I can of the ESPHome and pick a fresh ESP32 module and see if I manage then.

This still looks like a procedural error.

First correction, you are flashing from the ESPhome add-on. That’s important because there is a separate ESPhome integration. The add-on is for configuring/flashing devices and the integration is for adding them as devices in HA. Different things.

Two, I think you might be flashing this wrong but I want to confirm.

What machine was this USB connection made? The desktop/laptop/whatever you’re accessing HA from or directly to the Pi? I presume the former because of the device showing up as “Discovered” in the ESPhome add-on. That tells me you didn’t use the add-on to flash the device but used the web tool and the add-on is “discovering” it as a new device.

If what I’m thinking above is correct, then re-try the install from the ESPhome add-on, but plug the ESP32 board into a USB port on the Pi directly. Or, select the “manual” installation method, then select the modern format option to download a copy of the “.bin” file. Put that file on your laptop/desktop/whatever with the ESP32 connected via USB. Go to https://web.esphome.io/ and use that to install it. Click connect, select the port for the ESP, click on “Install” (NOT on “prepare for first use”), and select the “.bin” file.

Meanwhile, I tried to fix the dual instance of ESP32s.


The right one is what I intended to make, but the left is the “real” ESP32 that is connected. So I deleted “Autroprime”. And Adopted the other.
Which gave me the option to Update. And as I assumed this fails same way as it always did when I go this path. Here’s the printout during this:

INFO ESPHome 2023.12.9
INFO Reading configuration /config/esphome/esphome-web-60a7dc.yaml...
INFO Generating C++ source...
INFO Compiling app...
Processing esphome-web-60a7dc (board: esp32dev; framework: arduino; platform: platformio/[email protected])
--------------------------------------------------------------------------------
Library Manager: Installing esphome/AsyncTCP-esphome @ 2.0.1
INFO Installing esphome/AsyncTCP-esphome @ 2.0.1
Unpacking  [####################################]  100%
Library Manager: [email protected] has been installed!
INFO [email protected] has been installed!
Library Manager: Installing esphome/ESPAsyncWebServer-esphome @ 3.1.0
INFO Installing esphome/ESPAsyncWebServer-esphome @ 3.1.0
Unpacking  [####################################]  100%
Library Manager: [email protected] has been installed!
INFO [email protected] has been installed!
Library Manager: Resolving dependencies...
INFO Resolving dependencies...
Library Manager: Installing esphome/noise-c @ 0.1.4
INFO Installing esphome/noise-c @ 0.1.4
Unpacking  [####################################]  100%
Library Manager: [email protected] has been installed!
INFO [email protected] has been installed!
Library Manager: Resolving dependencies...
INFO Resolving dependencies...
Library Manager: Installing esphome/libsodium @ 1.10018.1
INFO Installing esphome/libsodium @ 1.10018.1
Unpacking  [####################################]  100%
Library Manager: [email protected] has been installed!
INFO [email protected] has been installed!
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Dependency Graph
|-- AsyncTCP-esphome @ 2.0.1
|-- WiFi @ 2.0.0
|-- FS @ 2.0.0
|-- Update @ 2.0.0
|-- ESPAsyncWebServer-esphome @ 3.1.0
|-- DNSServer @ 2.0.0
|-- ESPmDNS @ 2.0.0
|-- noise-c @ 0.1.4
Compiling .pioenvs/esphome-web-60a7dc/src/esphome/components/api/api_connection.cpp.o
Compiling .pioenvs/esphome-web-60a7dc/src/esphome/components/api/api_frame_helper.cpp.o
Compiling .pioenvs/esphome-web-60a7dc/src/esphome/components/api/api_pb2.cpp.o
xtensa-esp32-elf-g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
*** [.pioenvs/esphome-web-60a7dc/src/esphome/components/api/api_pb2.cpp.o] Error 1
========================= [FAILED] Took 102.85 seconds =========================

What machine was this USB connection made?
I use a Lenovo laptop, Windows 10. I connect the ESP32 to a USB and it is recognised as Port4.
Two, I think you might be flashing this wrong but I want to confirm.
Most probably, I am not much certain of the terminology. I struggle to interpret some basic knowledge. “Compile” - as shown above there are multiple ways of this - by Python, by the HASS Web, maybe Arduino IDE as I’ve also used. I also struggled a lot to understand where the Yaml file is, sure it’s obvious after a while but…
I tried to compile by the Update command as I show in my previous reply, and also by cmd/Python. That’s my confusion I try to describe here.
then re-try the install from the ESPhome add-on, but plug the ESP32 board into a USB port on the Pi directly. I believe this is described above or is this wrong?
Or, select the “manual” installation method I’ve done the Manual method, this is described in top of this page. But I used the Wizard and it automatically compiles (no errors) and uploads the firmware.
Click connect, select the port for the ESP, click on “Install” This should be similar to what I tried in the prevous reply, and this fails : “Killed signal terminated program cc1plus”
but plug the ESP32 board into a USB port on the Pi directly. Are you sure this is an option? I’ve not seen this method anywhere

This is definitely an option.

You’re conflating things which is causing some of the confusion.

ESPhome integration from settings - this is what Home Assistant uses to talk to the ESP device as a device.
ESPhome add-on - this is used to compile the ESP firmware in HA using the YAML configuration files without needing to install ESPhome on a separate device
ESPhome python library - this is what you installed using the “manual” method you linked to. This is for using ESPhome to compile and install firmware on an ESP device

The add-on and the python library you manually installed on your laptop are the same thing. The difference is where they are running from. This is why you are confused about the YAML location. When you created the YAML on your laptop and used that to install the firmware, that YAML file only exists on your laptop and will not be in the ESPhome folder in HA. You would either have to manually copy the file over or setup a new device (this really is just creating a new YAML file) in the add-on and copy the contents over from your laptop.

A quick Google search says this is caused by the Pi running out of memory during compilation. You could try disabling other add-ons while you compile to see if the error clears up. Or you could go back to using ESPhome on your laptop to compile/update the device, but you’d have to stop trying to use the add-on in HA if you go this route. If you want to make a change, then you would update the YAML on your laptop and run the update from there.

If I were you, I would remove the ESPhome add-on from HA. Delete any devices in the ESPhome integration. Re-flash the board from your laptop via USB and add it to HA using the integration (it should get auto-discovered like before). After compiling/flashing the firmware from your laptop, the device should connect to your wifi. After that, you can make changes to the YAML on the laptop and re-flash the device using the IP address for the WiFi connection. Just make sure you don’t change the OTA password if you configure one. If you do, you’ll have to flash the ESP device via USB again (well, there are ways around that but it requires some more in-depth understanding of how everything works).

1 Like

Thanks, I really appreciate this answer, this clarifies some of the frustrations I had. Obviously I’ve mixed methods in the belief they would comply with each other. I’ll try the method of flashing by Python separated from HASS as this seems to make a functional ESP32 easist for me. And then try to integrate it to ESPHome in HASS carefully. I’ll update here afterwards :slightly_smiling_face:

1 Like

Now I’ve removed all ESPHome in HASS, updated version of the ESPHome
(Use > pip3 install –U esphome ) and loaded the new firmware into it. As was described, the ESP popped up in HASS and I just enabled this. This first YAML had no entities, and none showed up. (Previously the Firrmware version popped up from nowhere).
So I added an input and output to the C:\Users\MyName\xxx.yaml file, recompiled and uploaded -
(> esphome run xxx.yaml ) - and - There they were!
Thanks a lot, now I presume I have a solid fundament for further improvements!
:smiley: :smiley: :smiley:

1 Like

Glad you were able to work through it.

1 Like