Can't figure out MQTT and it's discovery feature

PapaSierra82 · January 16, 2025, 8:39am

Hi,

I’m using HA for number of years and I’m pretty competent with it, but I’m completely lost when it comes to MQTT. I read so much about it online but I feel like I’m only getting more confused. Long story short, my biggest struggle it to use MQTT with Zigbee devices.

My setup is HA running on ondroin-n2 with USB zigbee dongle.

Core 2025.1.2
Supervisor 2024.12.3
Operating System 14.1
Frontend 20250109.0

I have ZHA configured with multiple devices on it, all working fine. I’m running EMQX broker with currently 7 clients showing up - all of them added by configuring MQTT server IP and login details directly on the devices (all of them wifi). I also have MQTT integration added and configured with EMQX details - all appears to be wroking fine. Under MQTT integration options the auto discovery option is enable with the default homeassistant prefix.

Despite all of the above seemingly configured properly, none of the zigbee devices are showing in the EMQX broker. Ialso tried connecting MQTT Explorer and it connects to the host correctly, but then only shows the host’s IP address with nothing else, no device tree.

What can I be doing wrong?

All I found in the logs was this inside the EMQX log:

EMQX 5.8.2 is running now!

2025-01-16T07:27:54.192903+00:00 [warning] tag: AUTHZ, clientid: mqtt-explorer-ba681002, msg: authorization_permission_denied, peername: 192.168.1.45:61222, username: mqttuser, topic: #, source: file, action: SUBSCRIBE(Q0)

2025-01-16T07:28:24.473968+00:00 [warning] msg: log_events_throttled_during_last_period, dropped: #{authorization_permission_denied => 1}, period: 1 minutes, 0 seconds

2025-01-16T07:29:25.163735+00:00 [warning] tag: AUTHZ, clientid: mqtt-explorer-ba681002, msg: authorization_permission_denied, peername: 192.168.1.45:61376, username: mqttuser, topic: #, source: file, action: SUBSCRIBE(Q0)

2025-01-16T07:30:24.475967+00:00 [warning] msg: log_events_throttled_during_last_period, dropped: #{authorization_permission_denied => 1}, period: 1 minutes, 0 seconds

2025-01-16T07:37:24.609109+00:00 [warning] msg: alarm_is_activated, message: <<"85.32% cpu usage">>, name: high_cpu_usage

2025-01-16T07:39:22.168833+00:00 [warning] tag: AUTHZ, clientid: mqtt-explorer-ba681002, msg: authorization_permission_denied, peername: 192.168.1.45:62280, username: mqttuser, topic: #, source: file, action: SUBSCRIBE(Q0)

2025-01-16T07:39:24.489001+00:00 [warning] msg: log_events_throttled_during_last_period, dropped: #{authorization_permission_denied => 1}, period: 1 minutes, 0 seconds

2025-01-16T07:39:24.632009+00:00 [warning] msg: alarm_is_deactivated, name: high_cpu_usage

peterxian · January 16, 2025, 4:15pm

MQTT is a tcp/ip protocol for message passing — think of it like WhatsApp for devices. Home Assistant can subscribe to messages and see what devices are saying, then create entities to use their data.

Zigbee is not a tcp/ip protocol, it has its own network stack for defining home automation data exchange. The ZHA coordinator software implements this network stack and creates all the entities directly in HA for Zigbee devices. MQTT is neither needed nor compatible with this approach.

There does exist coordinator software that instead converts Zigbee device data to MQTT messages for transmission over TCP/IP networks, for people that either prefer IP or need to backhaul over a larger network. But it cannot be run on the same network as ZHA; you must pick one coordinator or the other.

If you have Zigbee devices working fine with with ZHA, and WiFi devices working fine with MQTT, then you are using HA as it is meant to be used. Can you explain more precisely what you are trying to accomplish that isn’t working?

PapaSierra82 · January 16, 2025, 6:02pm

Thank you so much, that gave me a lot more clarity on the subject now

The reason I started leaning towards MQTT is that I now have some 30-40 WiFi and maybe 15 zigbee (but increasing quickly) devices and I thought switching to MQTT will result in a lighter network traffic and reduce latency. I also thought that maybe when using MQTT, in some cases it will be easier to make the devices talk to each other directly, rather than through HA automations.

An example would be Aqara Zigbee TRV taking temperature data directly from a third party zigbee wall sensor, or when I have two wall switches controlling the same light where one switch has the actual relay and the other is just a zigbee button, it will reduce a delay in response when using this switch. Or am I completely wrong here?

peterxian · January 16, 2025, 7:31pm

First off, even for devices that do support MQTT, communications are never “direct” between devices, they only communicate with the broker (server), never with each other. Traffic is generally sparse and lightweight though, so many dozens (hundreds?) of devices can easily share WiFi 6 spectrum if needed.

Some MQTT-over-WiFi devices, including those running Tasmota and ESPhome, have web REST API methods for direct control of each other to avoid the server middleman.

Similarly, Zigbee has a feature called bindings that provides limited direct (serverless) data exchange but support is not universal and capabilities are small. You’d have to consult the product manuals to see what, if anything, your devices can do with Zigbee bindings. This feature doesn’t really reduce network traffic as much as it provides some resiliency so triggers can work even when the coordinator is offline.

To provide an example from my Z-wave devices (which support “direct associations” similar to bindings) — a temperature sensor has an endpoint where high or low set temp can be configured to directly trigger a switch on another device, but the other device does not “monitor” the temperature or even know why it is being triggered.

PapaSierra82 · January 22, 2025, 10:41pm

Apologies for my late reply.

Ok, that all makes sense now… almost. One thing I don’t understand is that on my Aqara TRVs it says they support external temp sensor. When I connected them to ZHA, they even report, under the Diagnostics sections, a status of that external sensor (on or off).

I googled it and if I had an aqara hub and aqara app on my phone, I would be able to “add” an external zigbee temperature sensor to that TRV and I imagine it would then say “External sensor ON” in the diagnostics section. How would that work?

I’m basically trying to achieve just that, add ext. temp. sensors to my TRVs, but without a use of a dedicated hub. I know I can use Better Thermostat to control heating using the external sensor, but that would only control the “Call for Heat” function based on that sensor value. It wouldn’t control the actual valve to prevent temperature overshoot in the room. Am I right?