How to setup an IDE to test new espHome components

ESPHome
1

Hello, I read

and I setup an enviroment to compile and test code faster.
I made an ubuntu image on Virtualbox, since seems that only linux is supported.
I installed docker, vsCode
In vsCode I installed PlatformIO extension

Downloaded GitHub - esphome/esphome: ESPHome is a system to control your ESP8266/ESP32 by simple yet powerful configuration files and control them remotely through Home Automation systems. ad described, open with PlatformIO, setup something…
I tried to compile having many errors (esp8266 and esp32-arduino was ok, esp32-idf and another wasn’t ok). I kept only esp32-arduino in platformio.ini

I don’t know after compilation which component is included.
Is there a yaml I can change to perform a compilation as if it is in Home assistant?

Thanks

2

What are exactly trying to achieve?
You can test components with external_components in yaml, e.g.

external_components:
  # use ble_client_hid from this master branch in GitHub
  - source: github://koying/esphome-ble-remote@master
    components: [ ble_client_hid ]
    refresh: 5min

If you just want to compile faster, install esphome under docker in another machine.
Example docker-compose:

version: '2.1'
services:
    esphome:
        container_name: esphome
        volumes:
            - ./cache:/cache
            - ./config:/config
            - /etc/localtime:/etc/localtime:ro
        restart: unless-stopped
        image: esphome/esphome:stable
        #image: esphome/esphome:beta
        network_mode: host
3

So at the end the easy solution is to go through gitHub and a real HA? I’ve seen I can also use a local repo that’s probably easier, but always slow since I have a rpi.

Isn’t possible to simulate something directly from the IDE / toolchain?

I don’t fully understand also the compilations phase in VSCode: since the C source code is made by a py script based on the inut yaml, what is compiling VS (with PlatformIO) since there are no yaml?

4

You lost me.

This has nothing to do with HA

ESPhome is the IDE/toolchain.
What “IDE/toolchain” are you talking about? Behind the scene, ESPhome is using platformIO, but I don’t think this is what you’re talking about.

What has VSCode to do with all this. Are you using ESPhome in CLI mode?

Once again, what are you trying to achieve?

5

I don’t know exaclty what i’m using, but i can describe what i’m trying to do.
I have an HA and i want to make an Esp Home external component. I have the plugin in HA and I can build - load component from there, but it’s not suitable for developing purposes.

The main goal is to modify component code, to build a bin and to load it using ota. Runtime errors will be analyzed using logs because i don’t have jtag access and the device must act in an RS485 network i think it’s not easy to simulate.

I landed on External Components — ESPHome - where they link the Contributing — ESPHome page .

To keep all clean, i started in a virtualbox ubuntu system, I downloaded espHome in a folder and I Installed vsCode + PlatformIO after I read

Now you can open ESPHome in your IDE of choice (mine is CLion) with the PlatformIO addons (see PlatformIO docs for more info). Then develop the new feature with the guidelines below.

After opening PlatformIO plugin and project, it blamed me because i haven’t got docker installed. So I installed it using apt as described here .

Now it compiles (hitting the shortcut) and produces a bin (at least for esp32-arduino) after a long compilation time. I really don’t know what he compiled. He made many .o objects but (if I have understood it right) - these could be done only after a py pre-preprocessor have interpreted a yaml file regarding wanted components. I haven’t found the yaml file.

The next step I would try is to add a random git working external component, to test the produced firmware, and if it’s working i can proceed with a local component i can modify.

Thanks

1 Like
6

I think you’re mucho confused :wink:
At least, I don’t understand what you’re trying to do…

Maybe start with

7

I don’t have problems using espHome in Home assistant, or to use already developed esp home components. You linked a general usage of espHome plugin, that’s not my main goal. I’ve already created 3 different devices with a combination of different components and they works well.

A 4th esp device uses a custom component, based on RS485, that’s not working well and i would like to modify the C code of the component and to test it.

The main topic is: how can I modify the component in an enviroment where I can use an IDE or something more practical than a plain text editor?
Probably, but i’ve not tested, I can copy in my HA (that’s in docker) the extension code and to import it directly using

  - source:
      type: local
      path: my_components

but it will be probably be a mess if there are compilation problems.

I suppose that there are different setups meant to develope a custom component, maybe something that allow to perform tests before going to a release environment.
Isn’t it?

I tried to setup that VM as described before, but i’m not understanding where is located the yaml that guide the component compilation (i.e. in homeassistant/esphome/mydevice.yaml for “release environment”)

8

Where is it?
See my very first post here on how to use a custom component from github.
Assumption here is that you have forked the custom component, made your modifications, and push those modifications back to your fork.

As you mentioned, the alternative is to use a local folder.

Not sure how you actually landed on “Contributing”, but that seem irrelevant to your request.

9

This is the component

Second row in External component is:

You can find some basic documentation on creating your own components at Contributing to ESPHome.

I thought the IDE setup was designed also to develop custom components, since they are the most of the code.

I haven’t forked it yet, because I tried another (maybe wrong) path.

How is it supposed to write code before pushing? I can’t believe all has been written without auto completion, syntax check, and so on (even if it’s possible, it’s a return to the stone age)

10

Likely using a “local” external component, possibly with ESPhome as a docker container as I mentioned above, obviously with vscode as IDE for the component itself.

11

Hello @brazoayeye, reading your topic I come across the same problems.
I already knew about platformio when I was still playing with Arduinos to run my sketch code, but now that I use HA + espHome I don’t want to know anything else.
However, over time we are evolving with the use of Home Assistant and its resources. We want to do some more complex things, to integrate our embedded electronics devices that we love (ESP822 and ESP3).

So I started to venture into creating my external component. At this point I identify with your doubts.

I can’t “Build” my projects, and no matter how many times I install the lib dependencies, it always breaks.
One thing is certain, espHome should have a more detailed step-by-step guide on how to install and run it via the platformio.
What I can tell you is that you don’t necessarily need to run it in the environment, if the code is perfect, just upload it to your git branch, declare the external component in YAML and with luck it will run or break, but at least this way you can debug. …

image
image1916×1036 150 KB

image
image969×801 80.2 KB

If anyone who knows how to configure and run the platformio environment for espHome can help, we would be grateful!

12

At the end I found a decent compromise.

I program inside VSCode that can detect some errors
I installed pip install esphome and all worked well creating a starting yaml with esphome wizard myYaml.yaml and then esphome compile myYaml.yaml. I downloaded the component locally and included as described before, That runs the entire compilation phase including py script and so on.

Since i’m working from another network I also made a ssh tunnel to the HA server and then to the device with something like

sudo ssh -L 80:192.168.32.103:80 -L 3232:192.168.32.103:3232 -L 6053:192.168.32.103:6053 [email protected] -p myCustomPort

then added `127.0.0.1 myEspHomeDevice.local to /etc/hosts

This way I can compile in the ide, than compile better in a terminal, then updating the device using ota and testing it runtime with esphome run myYaml.yaml

There probably are better solutions, but i’m very bad on setting up enviroments because I made it few times in my life and, since it’s something set-and-forget, I always forgot it after I had set it.

13

Custom Components are being deprecated for External Components. If someone who has their head wrapped around how best to do this would make a Youtube walk-though on the process; that would be fantastic (especially if it included PlatformIO autocomplete and debugging).

Did find this:

14

Unfortunately, developer documentation for ESPHome is pretty much nonexistent. From what I can tell, there are different implementations of Custom Components. One style (older?) is basically a c++ file loaded into your project.

Another version of custom components (newer?) has your custom code located in config/custom_components/my_custom_component. And within that directory, you’ll have .cpp or .h (or both) files, and at least one python file __init__.py. This follows the standard structure for all ESPHome components (built-in and external). If you code this type of custom component properly, it’s easy to convert it to an external component, which can then be loaded from a git repo or locally anywhere on your build system. The ESPHome Contributing To guide shows the basics of that structure.

So, in building my external component, I first got it running from my custom_components directory (as mentioned above). Then I converted it to a bonafide external component. When that was working, I pushed it up to github, and referenced it accordingly from my ESPHome project.

The difficult part here, for me at least, was figuring out what to do in the python file. That’s where documentation is fully missing, and you pretty much have to reverse engineer the ESPHome built-in components to see what’s going on and how it works. And it’s not at all obvious - there are heavy layers of abstraction in using one language to code another language. I’m an experienced software developer, and I spent most of a work week before I had a functioning __init__.py file. I hope the devs put together some docs at some point.

As for build environment, I’m using the official Docker image esphome/esphome. I think they call this “ESPHome CLI”, and HomeAssistant is not part of this and is not required to make use of this. Once you configure and boot up the esphome container, you don’t need to use the CLI to run ESPhome or PlatformIO commands. Instead, use the web GUI to configure, build, and upload your ESPHome projects. For coding your C++ components, use whatever text editor (or fancy IDE) that you want.

EDIT: Regarding testing, I have no idea how to test external components, other than building and running on actual hardware. I’ve been experimenting with platformio unit tests, but I can’t get them to work natively on the desktop. If you want to test components that use ESPHome libraries, it looks like you have to build your tests as actual firmware and upload them to your devices.

2 Likes
15

Now I’m pretty new to EspHome, but I’ve been developing software professionally for 30 years. Adding on to this thread, in order to develop espHome devices in a more “traditional” development process, I feel like some things are missing. It may be they exist (and I would REALLY appreciate a how-to!) or they need to be created (and I would be happy to develop some of these components, but I’d need some architectural discussions first).

  • The ability to “compile in the IDE”. That is, transform YAML into C++ and then build it locally. For me, this is VSCode & PlatformIO.
  • The ability to run the compiled code in a simulated runtime environment, not actual hardware.
  • The ability to have a library of unit tests that the IDE can execute against the compiled runtime.
16

That’s possible using the host platform. No idea how you would configure VSCode to compile and run it though.

1 Like
17

I wasn’t familiar with the host platform, so thanks for the tip.

I’ve been using the VSCode terminal, and either running

esphome compile .\wallController.yml

which only compiles the code, or

esphome run .\wallController.yml

which compiles it and uploads it to the target. I guess I just need to change the target from the ESP device I’ve developing against to the host. I’ll post again with the solution if end up figuring it out.

1 Like