ZHA vs Zigbee2Mqtt - which is the most stable?

Configuration Zigbee
42

Are you doing all of your automations in NR outside of HA?

Part of my quest to tame the beast that is home automation is to bring all automations into one platform. There is nothing more frustrating than to have a light turn on/off through an automation but not be quite sure which automation did it.

(ie. Lutron has a countdown timer for lights to auto turn off, and my Hue motion sensor has an HA automation to turn lights on temporarily if it is dark when someone approaches, and smartthings puts everything to “bed” at midnight, so who turned out the lights? lol)

I am currently mostly using automations within HA, but like the way NR looks for visualizing flows, but have been wary of dipping my toes into it as having three ways to control stuff within HA isn’t any better than three separate apps.

43

hello
yes, except literally a few I’ve created at the very beginning or borrowed from somewhere (like sending notifications about system restart) all my automations are in NodeRed.

I agree with you that scattering automations across different places and/or technologies is far from optimal, and in extreme situations might lead to race conditions.

44

Sorry but you are very wrong here and I like to ask you to please stop recommending network-attached Zigbee Coordinator adapters to everyone, (especially when the topic is specifically about stability).

There can definitely be some disadvantages to using a network-attached Zigbee Coordinator adapter, and that is why I persoanlly do not recommend them to most users and instead think those should only be used as the exception in edge cases when it is not at all practically possible to just use a very long USB extension cable instead, epecially when you can simply convert any Ethernet-cable into a extremly long USB extension cable, see my Zigbee Coordinatoir adapter recommendations here → Zigbee buyer's guide - #2 by Hedda.

If a network-attached Zigbee Coordinator adapter is used and the users LAN is a little unstable or drop any packages for whatever reason then they will have problems and troubleshooting Zigbee Coordinator issues due to unstable LAN connection can be hard and will give the users an unnessesary bad experience which is not the fault of the Zigbee Gateway software and can put a strain on ZHA and Zigbee2MQTT developers, which is why there are now warnings against using network-attached Zigbee Coordinator adapters like that onder WiFi or VPN in both the ZHA and Zigbee2MQTT documentation:

Most users of Zigbee find out sooner or later that Zigbee can be very finicky/fussy about its needs and requirements even without adding the extra complexity that tunneling/piping serial over LAN adds. See:

To quote myself: “in a few edge cases it might be worth looking at a network-attached Zigbee Coordinator adapter (also referred to as Zigbee LAN adapters) like the Zigbee-to-Ethernet/USB Serial Coordinator solutions from TubeZB, and similar products from ZigStar, SMLIGHT, or cod.m as another alternative way to workaround the problem of relocating a Zigbee Coordinator radio adapter to a remote location for better reception, but please understand and respect that I personally think that using a such network-attached Zigbee Coordinator adapter should a solution should only ever be the exception when there is no other solutions and not the go-to rule, as please note that all those are proxy-servers uses Serial-over-IP tunneling which adds a little more complexity than simply using a very long USB extension cable (which I personally recommend using instead). I therefore personally believe that in most use bases it will be both easier and better to just use a very long shielded USB extension cable and a tip there is that you can easily achieve up to around 30 meters by using inexpensive “USB Ethernet RJ45 Extender Adapter” converters which easily and practically convert any single CAT5e/CAT6 shielded Ethernet cable with RJ-45 connectors into a very long passive USB extension cable (as it does not use LAN or IP but only converts an RJ45 Ethernet cable to a dumb USB extension cable). With such a solution you can use any Zigbee Coordinator USB dongle and easily replace the USB dongle later too (which should make for a less expensive solution in the long run). See example → https://www.amazon.com/Male-RJ45-Female-Extension-Adapter/dp/B083W3D65G Note! An additional caution warning about using Zigbee Coordinator network adapters over Wi-Fi/WAN/VPN. Be aware that using a Zigbee Coordinator via a Serial-Proxy-Server (also known as Serial-to-IP bridge or Ser2Net remote adapter) over a Wi-Fi, WAN, or VPN connection is not recommended. Serial protocols used by the Zigbee Coordinator do not have enough robustness, resilience, or fault tolerance to handle packet loss and latency delays that can occur over unstable connections. A Zigbee Coordinator requires a stable local connection to its serial port interface with no drops in communication between it and the Zigbee gateway application running on the host computer.”

Before buying such adapters users of those adapters should at least be aware those products are just Serial-over-LAN applicances with proxy server software that only uses Serial-over-IP to pipe/tunnel the serial communication port over a TCP/IP connection unchanged. Meaning that the serial data need to arive in-time and still serialized (in synchronous order with a start bit and and end bit), while Ethernet/LAN tecnology is asynchronous so does not have a start bit or is coded in a way that facilitates clock recovery which can cause tranfer delays and data packets being dropped because they did not arrive in time. There is nothing the makers of network-attached Zigbee Coordinator adapters can do to ensure that it can deal with stability issues of that connection as the serial protocol that is piped/tunneled is proprietary from the Zigbee stack firmware manufacturer (e.i. Silicon Labs and Texas Instruments), so it is not like makers of network-attached Zigbee Coordinator adapters can add buffers or other technology that could help ensure that the connection remains stable.

So from a technical and architecture point-of-view it is a bad idea to use a network-attached Zigbee Coordinator adapter as not only does it add extra complexity and dependencies that your LAN is stable but more importantly the developers who wrote the Zigbee stack applications interface serial communication protocol that is running in the Zigbee Coordinator firmware have designed the serial communication protocol as a synchronous data transfer protocol and made specifically for a direct-connection and thus it expects to have a 100% stable connection, as such the Zigbee stack firmware developers at Silicon Labs and Texas Instruments, etc. have not added much software fault tolerance, robustness, and resilience into the Zigbee stack applications interface serial communication protocol to deal with latency delays or packet loss that are more likely to occur if you use a serial stream proxy server over TCP/IP tunnel to pass-though that communication over a LAN.

Now most experienced Home Assistant users probably have such a network-attached Zigbee Coordinator adapter connected via a near 100% stable wired Ethernet LAN and will therefore not see problems often, but there will be problems for those users who do not have a a near near 100% stable wired Ethernet LANm and even worse for those who connect it via a Wi-Fi network or via VPN over WAN/internet connection, (because Wi-Fi network or a VPN are features built-in to some network-attached Zigbee Coordinator adapter firmware and the makers of is marketing as available without any warnings in the product or its documentation)

Sure the Zigbee gateway connection might recover if there is a total disconnect but the result will be worse when it only drops some packages intermittently and that will make troubleshooting much harder for the end-users and developers of Zigbee Gateway host appllications such as Zigbee2MQTT and the ZHA integration. Thus the end-users must be warned and recoomened to try to connect a network-attached Zigbee Coordinator over WiFi or to a remote WAN/VPN. That applies to a DIY solution using Ser2Net or any other Serial Proxy/Forwarding Tunnel is not supported for normal operation. This it should never be recommended to use a Zigbee Coordinator via a Serial-Proxy-Server / Serial-over-IP solution over a WiFi, WAN, or VPN connection.

45

@neildsbm No! Just no! That is wrong in everyway possible, and not supported in any bay in the Zigbee specification, Zigbee gateways, or by Zigbee Coordinator makers. The serial interface protocol on those Zigbee stack firmware not a bus, the serial connection is exclusive. Such a bad idea it borders on trolling trolling, especially when you bring this up in a thread that is specifically about stabilty.

You always need to use a separate dedicated Zigbee Coordinator for each Zigbee Gateway, period!

And as a wrote in the post above, using a network-attached Zigbee Coordinator adapter is a dirty workaround regardless that even when only using a single Zigbee Gateway should only be used as the exception in edge cases when it is not at all practically possible to just use a very long USB extension cable instead, epecially when you can simply convert any Ethernet-cable into a extremly long USB extension cable, see my Zigbee Coordinatoir adapter recommendations here → Zigbee buyer’s guide - #2 by Hedda.

46

I have to disagree with this. LAN-based coordinators that are strictly Ethernet based are usually more powerful in terms of mesh processing and coordination and are not susceptible to USB3 interference like a USB coordinator can be. Yes, using a RJ-45 adapter can place the USB coordinator far away from interference sources and is a viable solution.

Yes, however, almost all LAN connections are very stable and the TCP/IP protocol itself (even used over Ser2Net) has redundancy built-in to handle any kind of packet loss. Even then, it’s usually a case of a bad Ethernet cable or possibly a port that will cause problems. USB-based coordinators suffer from the same issue though; A bad extension cable or port will also cause issues for the user.

Both of the links you posted talk about WiFi, not LAN based (Ethernet) connections. In that case, yes, WiFi based LAN coordinators are very bad and I never have recommended those. In fact, most users that say they are using WiFi based LAN coordinators are told to stop using that and go with a directly connected Ethernet coordinator.

This is completely incorrect. While Ethernet does not have a start bit, it does use start-of-frame and either scrambling or Manchester encoding so that a clock wire isn’t required. The transmitter encodes clock data and the receiver decodes the clock data. The ONLY time this would be even remotely accurate is if the user is on something like token-ring or some other ancient technology. Any modern router produced in the past 20 years will use either encoding method with start-of-frame/end-of-frame packets to ensure synchronous delivery. I would suggest learning the OSI layered network model before making claims like this as it is wrong.

Again, no one ever suggests using a LAN based coordinator over WiFi or VPN. We actively discourage that in all of our support mediums. It’s the same for using HA itself over a WiFi connection. We never, ever recommend WiFi for either LAN-based coordinators or HA itself. Nowhere have I ever seen someone (including myself) recommend WiFi/VPN based coordinators.

If you could provide some statistics showing how a LAN-based coordinator behaves when packets are dropped (which, again, TCP/IP handles internally in the stack through re-transmission), I’d be happy to change my opinion here. However, after many years of being in the networking space, I can tell you first hand that what you are describing rarely happens except in the worst of circumstances. Even on very cheap consumer networking gear, the TCP/IP protocol is still operates the exact same way.

1 Like
47

“No one”? At least “XZG Firmware” (which is used by several network-attached Zigbee Coordinator adapter manufacturers including ZigStar) and “SMLIGHT” Zigbee network-adapters both have WiFi configuration as well as a VPN client built-in available via easy-to-use settings in the UI. They are marketing them as " Key features" without any warnings about why it is a bad idea to use those feature, instead just making it easy for users to believe it is a good idea to use them WiFi and/or VPN. I have not checked but think there are other similar firmware that at least have WiFi configurations built directly into the UI too simply because the ESP32 chip they use has Wi-Fi capability(?).

I mean, if a feature is there then why should a user not assume that it should work without any problems.

IMHO best would be if the XZG Firmware developers would just remove those any Wi-Fi and VPN features completely from their firmware as using them can give users a bad experience and put an extra strain on the developers of Zigbee Gateway project whose users buy those.

Much better to not make it easy to create a bad setup.

After many years of me helping many ZHA and Zigbee2MQTT end-users here in this community forum or other support channels for those two projects I can tell you from my own first-hand experience that while it is true that most users do tend to use stable wired Ethernet LAN connection and therefore those often seem to have no problems there are still many end-users here reporting issues and asking for help here where it turns out that they connecting a network-attached Zigbee Coordinator adapter over Wi-Fi or VPN (which is why that warning against using Wi-Fi or VPN had to be added specifically).

These are are just a few examples:

And as I do not follow all other projects that uses such adapters what I have seen is only a fraction of the recorded cases, which I guess should mean that there is a multitude of number of unrecorded cases that are using a network-attached Zigbee Coordinator adapter over Wi-Fi or VPN, and might be having problems because that today or in the future, which in turn can create an unfair reputation via word-of-month, etc. the ZHA and Zigbee2MQTT projects being unstable when the root cause is really the users network or setup not being stable enough for a network-attached Zigbee Coordinator adapter.

1 Like
48

“No one” meaning us as helpers/experts here on the forums, Discord, Reddit and FB. Yes, the companies market those features, but in terms of what WE can control and suggest, myself and many others always discourage WiFi/VPN usage. Granted, we cannot control what the user does with their device, but generally speaking, they either end up here or on one of the other support mediums when they have issues. Case in point, I had a user on Discord just about a week(?) ago asking about a WiFi connected Sonoff coordinator and major instability on their mesh. Lots of discussion back and forth and finally got them to understand what the problem was with their mesh and it was the WiFi coordinator that was causing the majority of their issues. Once explained why it was bad, the user decided to get another coordinator and connect it via LAN. This happens more often than not.

Yup. Same. I almost always defer to you for Zigbee related matters when it comes to mesh communication and interference issues. With that said, pretty much the first question I ask is what coordinator and how it’s connected. When I see WiFi/VPN, I launch into a tirade of “STOP THAT” comments. Usually after a few minutes of explaining, I get the point across. :wink:

I get what you’re saying and I do agree about the WiFi/VPN connected coordinators 100%. But, lumping in all LAN-based coordinators isn’t correct or accurate. At the end of the day, most users that have issues with their mesh ends up here or on the other support channels and we usually have to reiterate what our experiences are with Zigbee equipment “as experts”.

4 Likes
49

Just to chime in here:

We fork XZG (previously UZG) for our cod.m ZigBee Coordinator (CZC) and are currently working on a new release. We will put a disclaimer on the page of the Wi-Fi and WireGuard settings in the webinterface to discourage users to use it and try to explain why this is not a good idea.

Already updated our documentation for the coordinator: Overview - cod.m Knowledge Base

Our coordinator - as well as every variant with a appropriate chip out there - will support working as a Thread border router, so the Wi-Fi option makes sense. At least for that use case. Correct?

1 Like
50

My esp border routers use the Ethernet hat, but since it is optional, most use them with WiFi. So for a border router, WiFi does make sense.

2 Likes
51

@pmayer Off-topic but yes, if you are instead specifically only using it as a Thread Border Router (OpenThread Border Router) then it makes more sense to allow Wi-Fi and/or VPN, but then with the disclaimer that the user should really have more than one Thread Border Router for each location/site for redundacy. As one of the main benefits with the Thread standard/protocol is that you can have multiple Thread Border Routers per Thread network (and Matter fabric for Thread based Matter standard devices), while with Zigbee you can only ever have a single Zigbee Coordinator. It might however cause confusion if the user thinks that they can add multiple Thread Border Routers in different sites connected to he same Thread network, because as I understand it the Thread network self-regulate itself but can only has one master at a time so could lead to problems if the user put Thread Border Routers from the same Thread network on different sites. The goal in a Thread Border Router deployment would be to add many Thread Border Routers to each site so get no single-point-failure.

1 Like
52

OMG, Thank you man! I’m literally fighting with this for days now. Why there is no any note for this on the config sources…

53

Not sure if that limitation is covered in the Zigbee2MQTT installation and configuration or getting-started documentation but it is at least explained in their FAQ, see → General limitations that apply to all Zigbee implementations

I think that the ZHA inetgration documenation does make that fact perfectly clear in both its “Introduction” section at the top as well as in “Limitations” under the troubleshooting section, see → Zigbee Home Automation - Home Assistant

As that is a limitation in the official Zigbee specification it applies to all Zigbee gateway implementations, so I guess it is assumed to be self-evident and as such presumed is so obvious that there is no need for proof or further explanation.

However since both the ZHA integration and the Zigbee2MQTT projects (as well as their dependencies) have community-maintained documentation were it is meant that anyone can contribute to make it better please feel free to improve them, and that means you too!

See here for ZHA:

and here for Zigbee2MQTT (Z2M):

"Introduction

This ZHA integration is a hardware-independent Zigbee gateway implementation that can replace most proprietary Zigbee gateways/bridges/hubs/controllers. Zigbee is a low-bandwidth communication protocol that relies on using small low-power digital radios to connect compatible devices to local Zigbee wireless private area networks. ZHA will create a single Zigbee network to which you can then pair/join most Zigbee-based devices that are made for home automation and lighting.

Before installing the ZHA integration in Home Assistant, you need to connect a Zigbee Coordinator radio adapter that will connect to your Zigbee network. Those normally come in the form of a USB dongle that plugs directly into the same computer that is running your Home Assistant installation. The ZHA integration is compatible with many different “Zigbee Coordinator” adapters from various manufacturers. Be sure to note the recommendations in the respective sections below before buying a Zigbee Coordinator. A Zigbee network always needs to have one Zigbee Coordinator (it can never have more than one), and Zigbee devices can never be connected to more than a single Zigbee network, however, a Zigbee network can have multiple “Zigbee Router” devices and “Zigbee End Device” products.

Once ZHA has been set up with a Zigbee Coordinator it will automatically create a Zigbee network and you will be able to join/pair any Zigbee Router devices and Zigbee End Devices. With only a few limitations, most devices will join/pair directly regardless of brand and manufacturer.

…"

"Limitations

Note that ZHA only supports connecting a single dedicated Zigbee Coordinator radio adapter or module with a single Zigbee network and that the Zigbee Coordinator cannot already be connected or used by any other application. Any devices that are or have previously been connected to another Zigbee implementation will also need to first be reset to their factory default settings before they can be paired/joined to ZHA, please see each device manufacturer’s documentation.

Any Zigbee device can only be connected to a single Zigbee Coordinator (only one Zigbee gateway). This is a limitation in the current (as well as previous) Zigbee protocol specifications, governed by the CSA (Connectivity Standards Alliance). As such, it is a limit that applies to all Zigbee implementations, not just the ZHA implementation.

…"