How to setup an IDE to test new espHome components

brazoayeye · March 12, 2024, 7:18am

Hello, I read

and I setup an enviroment to compile and test code faster.
I made an ubuntu image on Virtualbox, since seems that only linux is supported.
I installed docker, vsCode
In vsCode I installed PlatformIO extension

Downloaded GitHub - esphome/esphome: ESPHome is a system to control your ESP8266/ESP32 by simple yet powerful configuration files and control them remotely through Home Automation systems. ad described, open with PlatformIO, setup something…
I tried to compile having many errors (esp8266 and esp32-arduino was ok, esp32-idf and another wasn’t ok). I kept only esp32-arduino in platformio.ini

I don’t know after compilation which component is included.
Is there a yaml I can change to perform a compilation as if it is in Home assistant?

Thanks

koying · March 12, 2024, 8:09am

What are exactly trying to achieve?
You can test components with external_components in yaml, e.g.

external_components:
  # use ble_client_hid from this master branch in GitHub
  - source: github://koying/esphome-ble-remote@master
    components: [ ble_client_hid ]
    refresh: 5min

If you just want to compile faster, install esphome under docker in another machine.
Example docker-compose:

version: '2.1'
services:
    esphome:
        container_name: esphome
        volumes:
            - ./cache:/cache
            - ./config:/config
            - /etc/localtime:/etc/localtime:ro
        restart: unless-stopped
        image: esphome/esphome:stable
        #image: esphome/esphome:beta
        network_mode: host

brazoayeye · March 12, 2024, 10:15am

So at the end the easy solution is to go through gitHub and a real HA? I’ve seen I can also use a local repo that’s probably easier, but always slow since I have a rpi.

Isn’t possible to simulate something directly from the IDE / toolchain?

I don’t fully understand also the compilations phase in VSCode: since the C source code is made by a py script based on the inut yaml, what is compiling VS (with PlatformIO) since there are no yaml?

koying · March 12, 2024, 10:53am

You lost me.

This has nothing to do with HA

ESPhome is the IDE/toolchain.
What “IDE/toolchain” are you talking about? Behind the scene, ESPhome is using platformIO, but I don’t think this is what you’re talking about.

What has VSCode to do with all this. Are you using ESPhome in CLI mode?

Once again, what are you trying to achieve?

brazoayeye · March 12, 2024, 12:01pm

I don’t know exaclty what i’m using, but i can describe what i’m trying to do.
I have an HA and i want to make an Esp Home external component. I have the plugin in HA and I can build - load component from there, but it’s not suitable for developing purposes.

The main goal is to modify component code, to build a bin and to load it using ota. Runtime errors will be analyzed using logs because i don’t have jtag access and the device must act in an RS485 network i think it’s not easy to simulate.

I landed on External Components — ESPHome - where they link the Contributing — ESPHome page .

To keep all clean, i started in a virtualbox ubuntu system, I downloaded espHome in a folder and I Installed vsCode + PlatformIO after I read

Now you can open ESPHome in your IDE of choice (mine is CLion) with the PlatformIO addons (see PlatformIO docs for more info). Then develop the new feature with the guidelines below.

After opening PlatformIO plugin and project, it blamed me because i haven’t got docker installed. So I installed it using apt as described here .

Now it compiles (hitting the shortcut) and produces a bin (at least for esp32-arduino) after a long compilation time. I really don’t know what he compiled. He made many .o objects but (if I have understood it right) - these could be done only after a py pre-preprocessor have interpreted a yaml file regarding wanted components. I haven’t found the yaml file.

The next step I would try is to add a random git working external component, to test the produced firmware, and if it’s working i can proceed with a local component i can modify.

Thanks

koying · March 12, 2024, 12:38pm

I think you’re mucho confused
At least, I don’t understand what you’re trying to do…

Maybe start with

brazoayeye · March 12, 2024, 12:57pm

I don’t have problems using espHome in Home assistant, or to use already developed esp home components. You linked a general usage of espHome plugin, that’s not my main goal. I’ve already created 3 different devices with a combination of different components and they works well.

A 4th esp device uses a custom component, based on RS485, that’s not working well and i would like to modify the C code of the component and to test it.

The main topic is: how can I modify the component in an enviroment where I can use an IDE or something more practical than a plain text editor?
Probably, but i’ve not tested, I can copy in my HA (that’s in docker) the extension code and to import it directly using

  - source:
      type: local
      path: my_components

but it will be probably be a mess if there are compilation problems.

I suppose that there are different setups meant to develope a custom component, maybe something that allow to perform tests before going to a release environment.
Isn’t it?

I tried to setup that VM as described before, but i’m not understanding where is located the yaml that guide the component compilation (i.e. in homeassistant/esphome/mydevice.yaml for “release environment”)

koying · March 12, 2024, 1:03pm

Where is it?
See my very first post here on how to use a custom component from github.
Assumption here is that you have forked the custom component, made your modifications, and push those modifications back to your fork.

As you mentioned, the alternative is to use a local folder.

Not sure how you actually landed on “Contributing”, but that seem irrelevant to your request.

brazoayeye · March 12, 2024, 1:49pm

This is the component

Second row in External component is:

You can find some basic documentation on creating your own components at Contributing to ESPHome.

I thought the IDE setup was designed also to develop custom components, since they are the most of the code.

I haven’t forked it yet, because I tried another (maybe wrong) path.

How is it supposed to write code before pushing? I can’t believe all has been written without auto completion, syntax check, and so on (even if it’s possible, it’s a return to the stone age)

koying · March 12, 2024, 1:52pm

Likely using a “local” external component, possibly with ESPhome as a docker container as I mentioned above, obviously with vscode as IDE for the component itself.

Gaitolini · March 13, 2024, 8:04pm

Hello @brazoayeye, reading your topic I come across the same problems.
I already knew about platformio when I was still playing with Arduinos to run my sketch code, but now that I use HA + espHome I don’t want to know anything else.
However, over time we are evolving with the use of Home Assistant and its resources. We want to do some more complex things, to integrate our embedded electronics devices that we love (ESP822 and ESP3).

So I started to venture into creating my external component. At this point I identify with your doubts.

I can’t “Build” my projects, and no matter how many times I install the lib dependencies, it always breaks.
One thing is certain, espHome should have a more detailed step-by-step guide on how to install and run it via the platformio.
What I can tell you is that you don’t necessarily need to run it in the environment, if the code is perfect, just upload it to your git branch, declare the external component in YAML and with luck it will run or break, but at least this way you can debug. …

If anyone who knows how to configure and run the platformio environment for espHome can help, we would be grateful!

brazoayeye · March 14, 2024, 7:22am

At the end I found a decent compromise.

I program inside VSCode that can detect some errors
I installed pip install esphome and all worked well creating a starting yaml with esphome wizard myYaml.yaml and then esphome compile myYaml.yaml. I downloaded the component locally and included as described before, That runs the entire compilation phase including py script and so on.

Since i’m working from another network I also made a ssh tunnel to the HA server and then to the device with something like

sudo ssh -L 80:192.168.32.103:80 -L 3232:192.168.32.103:3232 -L 6053:192.168.32.103:6053 [email protected] -p myCustomPort

then added `127.0.0.1 myEspHomeDevice.local to /etc/hosts

This way I can compile in the ide, than compile better in a terminal, then updating the device using ota and testing it runtime with esphome run myYaml.yaml

There probably are better solutions, but i’m very bad on setting up enviroments because I made it few times in my life and, since it’s something set-and-forget, I always forgot it after I had set it.

TLCary · July 10, 2024, 6:15pm

Custom Components are being deprecated for External Components. If someone who has their head wrapped around how best to do this would make a Youtube walk-though on the process; that would be fantastic (especially if it included PlatformIO autocomplete and debugging).

Did find this:

github.com

DrCoolzic/ESPHomeCompDev/blob/774182f486dda07f29f2a521a116c1b9f20a92e5/Part1-introduction/part1-introduction-US.md

# ESPHome component design : Overview

## Introduction

Terminology:

- A **device** refers to a set of **components** grouped together by a user to perform a specific function.
- A **component** is a basic element of the ESPHome library. It can be a physical component, such as a temperature sensor, or a logical component, such as an I²C bus.

The main purpose of ESPHome is to enable **users** with no particular computer skills to automatically create the software for an electronic module. To do this, the user describes the module’s configuration in a YAML file. ESPHome then uses a library of components which it assembles according to the user’s needs, automatically generating the firmware required for the module to function correctly. The user must then assemble the various components on an electronic board and connect it to a computer to load and execute the code generated by ESPHome.

On the other end a **developer** will create new components to enrich the ESPHome library. However, this task is much more complex, as it involves describing the component’s behavior using the C++ and Python languages. A good knowledge of C++ and the basics of Python are required to complete this task.

Having been faced with the need to create new components that didn’t exist in the ESPHome library, I wondered about the steps and tools required to complete this task. Although there are many tutorials on creating **devices** in ESPHome, very few deals with creating **components**. I therefore proceeded by trial and error to finally establish an environment and a process to use. This experience has given me some valuable knowledge, which I’d now like to share.

It’s worth noting that there are certainly other (_and probably better_) methods for accomplishing this task.

## **ESPHome's high-level vision**

First, let me introduce you to how ESPHome works at a high level of abstraction, as I understand it.

This file has been truncated. show original