How Can I Understand ESPHome Native API

From the doc:

The ESPHome native API is used to communicate with clients directly, with a highly-optimized network protocol. Currently, only the ESPHome tool and Home Assistant use this native API.

Who are the clients? Are ESPHome tool and Home Assistant the clients? Then these clients must have the encryption key in advance to communicate with servers (devices with ESPHome firmware). As per the doc:

encryption (Optional): Enable transport encryption of the API layer.

  • key (Required, string): The pre-shared key for the encryption. This is a 32-byte base64 encoded string.

But I don’t remember I have set up those keys in HA. I also don’t know which ESPHome Tool the doc talks about.

Normally, a client will call the API of a server to change it’s state, but from the doc:

When using the native API with Home Assistant, you can create Home Assistant service calls straight from ESPHome Automations.

I just can’t understand if HA is the server. If not how can an ESPHome device call a service on it?

The devices you program with ESPhome code.

It is.

If HA is the server, then why the doc says:

  • port (Optional, int): The port to run the API Server on. Defaults to 6053.

I think HA is the client. It keeps scanning the whole subnet it connects to on port 6053 to detect any ESPHome device.

But I still don’t know why this API allow HA service call. Does HA send it’s credentials when it detects ESPHome devices so that ESP devices can call HA service?

And when the encryption key is shared. Seems lots of puzzles here.

HA is a server running the ESPhome add-on, on port 6053. This referring to the add-on which allows you to write the ESPhome code and send it to the devices.

The API which ESPhome devices and HA use to communicate with each other is a seperate thing. ie: You don’t need to have the ESPhome add-on (as I mentioned above) running on the HA server for this part to work. eg: some people just run the ESPhome coding application on a laptop to program their devices and not on the HA machine.

This is all dependent on what you write to your ESP device. eg: If you use the 'Sprinkler’ component in your ESPhome device, it will allow a bunch of services to be called in HA relative to that ESPhome component.

You put this in your ESPhome code as per the docs.

All solved by reading the ESPhome site, but really, you don’t need to stress about the intricacies of how it works.

This is probably the source of some of the confusion. ESPHome’s bundling as an HA add-on is not how it was originally intended — it can be run on a completely separate server. I run it in a separate Docker container on my NAS, for example.

EDIT: to be clear, the ESPHome container is not “the server” in terms of API communication: that is HA.

image

The Native API is the comms link between the ESP devices and your HA instance, nothing to do with the add-on (in the sense that you don’t need the add-on running on your HA instance for the API to work).

If I understand it right. Do you mean the ESPHome container is the server and it listens on port 6053? I use terminal add-on to run docker command:


But it seems the esphome container hasn’t expose port 6053 for public access.

Do you mean both HA and ESPHome devices are clients and ESPHome container is the server listen on port 6053? Since ESPHome container is installed as an add-on to HA, so HA already has the key. If I use ESPHome dashboard to install firmware on devices, they will all share the same encryption key.

Edit:
I just checked all the ESPHome configuration files, and each has a different encryption key. Seems ESPHome container allows multiple encryption keys.

As does the HAOS add-on. You can either use one key for all devices, or a fresh one for each.

As per my post above, the Native API is separate from the application where you complie your ESPhome code and write files to the devices.

I guess I kind of understand the whole process.

HA and ESPHome devices are clients of the ESPHome server (the add-on, the dashboard, the container). And ESPHome server is also a client of HA. So when an ESPHome device calls a HA service, it in fact talks to the ESPHome server through the native API with the encryption key, and the ESPHome server calls the HA API.

But as I mentioned above, why I can’t see the ESPHome server expose 6053 through docker ps command? I haven’t configured the IP of the server, how can my device know it?

From the integration configuration page, it still seems it is my ESPHome device (the node) which runs the native API, not the add-on.

No. The add-on has nothing to do with the communication between your ESP devices and HA. That is purely the Native API.

The ESPhome add-on (or Docker container if installed that way) is purely for writing and compiling ESPhome code and producing a binary file to go on the device.

Oh, I got it!

So every ESPHome device is the server themselves. HA communicates with ESPHome devices through ESPHome integration. Then the integration is actually the client. I guess the way the ESPHome integration finds new devices is to scan the whole subnet.

As for the encryption key, how can the ESPHome integration know which key it should use to communicate with a device, as the doc says, this key should be pre-shared.

Yeah, that’s what I’ve been trying to say to UchihaYuki. What we use to create the ESPhome files is totally separate to the Native API that ESPhome devices and HA use to communicate with each other.

HA is the server, the ESP devices are clients.

You enter it in your code when creating the device, then paste that in when you add a device via the HA Integrations page.

I also think the ESPHome integration somehow transmits the credentials to ESPHome devices through the ESPHome native API, or how can an ESPHome device make a service call on HA?

It’s a 2-way communication.

If HA is the server, then why do I need to enter the IP and port of an ESPHome device, every time I add one? So it is the device that runs the server on port 6053.

I do write the key in the configuration file and flash the firmware into my device. But I never paste this code when I add a device, the device is added automatically somehow.

HA is talking to the device on that port, using the API. It wouldn’t matter which one was the server or client, the port still gets used for comms.

My bad, I thought it was like HTTP protocol. I just remembered that it uses a custom TCP protocol, then I guess it just works like WebSocket.

Do you really get prompted for the encryption key? I have already added 3 devices, no prompt by far.