Not many people know that... A random collection of Zigbee trivia

Community Guides
, , ,
1

Don’t forget that the first post in a community guide is a wiki - feel free to add to it or correct it.

This guide was written with reference to:

Core 2023.12.3
Supervisor 2023.11.6
Operating System 11.2
Frontend 20231208.2

Some subjects come up quite a lot...

That map

That map...
That map...1134×867 105 KB

The ZHA “network visualisation” was originally a dashboard card. It was absorbed into ZHA and the card is now deprecated, but the repository is still on Github.

The lines on the map show every route between nodes that Zigbee has discovered - not just the ones it is using. They are colour coded according to their LQI:

  • Green >192
  • Yellow 129-192
  • Red 80-128
  • Grey < 80

The map is a snapshot. It will be different each time you generate it.

There are two numbers on each link because messages pass in both directions. A value is supplied by the router at each end of the hop.

It is normal to see nodes on the map that don’t have a link to anything else. These are usually end devices and they’re probably sleeping to conserve battery. They will wake immediately if one of their sensors is triggered, and in any event they should wake periodically to check in with their parent router. A router with no connections is unavailable, which may indicate a problem.

LQI Values

We talk about link quality rather loosely. Strictly speaking LQI is a measure of error rates. An LQI of 255 means that no messages have been lost between two nodes, and in theory this is possible even if signal strength (RSSI) is not great.

Each router in a network maintains a table of information about its neighbours so that it can direct messages efficiently. It tracks the LQI of incoming links and the number we see is the average for recent messages.

Can I control the route my messages take?

No.

If you watch the videos in this post you’ll see that even the standard’s developers talk about Zigbee as a black box that “just works” - and it does, providing your network has enough routers and not too much interference.

When a message passes from the coordinator to a device at the other end of the house each router along the way will make a decision about how to forward it and the outcome will be different each time, depending on traffic and the number of other routers in range. If you don’t have enough routers messages will be forced along the same path and that’s when you get problems.

When you pair an end device with the network you can do it through a nearby router rather than the coordinator - sometimes this is easier. Devices from some manufacturers tend to “stick” with the original router they were paired through, but they shouldn’t - if it becomes unavailable they will fail. If a parent router fails, its children should connect elsewhere and inform the rest of the network of the fact.

Interference

Zigbee only has a weak signal which is easily swamped by wi-fi if you or your neighbours are using the same channel. Before you embark on major surgery, however, bear in mind that Zigbee channel numbers and wi-fi channel numbers are not the same.

Channels
Channels885×431 68.7 KB

Ironically, Channel 11 Zigbee and Channel 11 wi-fi are a good combination, at opposite ends of the 2.4 GHz spectrum.

There’s a post about this here.

What's the difference between ZHA and Z2M?

Zigbee Home Automation is Home Assistant’s built in Zigbee integration. If you buy a SkyConnect coordinator, it will be installed by default; with a different coordinator installation and device discovery is very simple through the GUI. If a Zigbee device is standards compliant it will work with ZHA out of the box. In practice this is less common than you might think and many devices need custom device handlers, or quirks. More details here.

Zigbee2MQTT is an open source gateway application for controlling Zigbee networks - in other words, it performs the same function as (say) a Philips Hue bridge but it is not restricted to any particular manufacurer’s devices. It works well with HA, but also with other smart home systems. If you wish you can run it on a machine separate from your HA server - which means your lights will still work while HA is being restarted. As the name suggests, it needs a MQTT broker, which makes it more complicated to set up and maintain than ZHA, however Z2M supports more non-compliant devices. More details here.

These are the two main Zigbee integrations, and with either of them you will be able to mix devices from different manufacturers. Which one you use is a matter of personal preference (some people use both, but they need separate coordinators and a device can’t be on both networks at the same time). Both are equally stable. Whichever one you choose, the Zigbee protocol is the same.

Where do I find ZHA quirks?

There’s a list here, sorted by manufacturer:

It’s a good idea to look up your device on the issues page before you download anything - some have several variants.

Can I use a Zigbee integration in HA and run a hub by [insert manufacturer's name here] at the same time?

Yes, if you must. But devices can’t be on both networks.

How can I find out which devices work well?

Search the forum. Ask.

There is a useful database here.

If you’re using Z2M, there’s a catalogue of supported devices here. With pictures! If you’re using ZHA, compliant devices should all work, so you need to look for warnings about buggy ones that don’t. The GitHub issues list is a good place to start.

I've just restarted HA and lots of my devices are unavailable

This is normal for end devices, which may have been sleeping, and it can sometimes take a long time for all of them to report in.

Bear in mind that “reporting in” takes place in stages. A device may poll its parent every few seconds to see whether there are any buffered messages, but it may only update link values every five minutes and battery status every couple of hours. All this is part of the device designer’s strategy to conserve battery and it will vary from device to device. If a sensor is triggered the device should update immediately.

I have lots of devices but Zigbee is still unstable

OK. But what kind of devices? Routers do the heavy lifting in a Zigbee network; end devices are just along for the ride. Best practice is to build a solid network of mains-powered routers first (bearing in mind that not all mains-powered devices are routers) and add the fun stuff afterwards.

Other Zigbee guides

14 Likes
2

This is a great write-up, thanks for doing this. I was recently looking for much of this information as I added a router to my ZHA to try and increase the mesh for some of my Christmas lighting outside. The map I originally found confusing but you clarified it well. To date my Texas Instruments, Sengled, TRADFRI, LEDVANCE, and new USB router are working well.

3

That is not true.

4

No, you’re right - it doesn’t tolerate non-compliant devices out of the box, so to speak. Guide edited. I meant in the sense that each device supported by Z2M has its own device handler, so a large number of variants work with Z2M and new devices tend to get support more quickly.

I’m not a fan myself - its seems suspiciously like a huge collection of hacks (waits for the sky to fall :scream:).

5

Well, only if you just have one Zigbee device and one controller :laughing:

Your post is excellent- it should be pinned.

6

Thank you for this. Lots of good info and insights in how the zigbee network works.

7

Yeah the real Zigbee2MQTT get full support for new devices quicker is that it has more contributing developers working activly on device handlers, which is probably due to the Zigbee2MQTT project having been around longer so it have had a head start at growing its community and the fact that it also works with other open-source home automation platforms meant that it has had the benifit of getting a broader audiende from the start of developers that are likley to contribute.

The same could be done with ZHA Device Handlers (quirks) for the ZHA integration as the community grows and more developers activly contribute to the development of new ZHA Device Handlers (quirks).

The userbase of both Home Assistant and its built-in ZHA integration have also grown a lot in the last few years, and more users usually means more developers, so it look for the future.

8

It’s just occurred to me that a list of ZHA quirks is always going to be relatively short because compliant devices don’t need them. We’re not comparing like with like - for Z2M you need a list of supported devices; for ZHA you need a list of non-compliant devices.

9

I’m confused by statements like 'because compliant devices don’t need them", can you expand on this?

As I understand the Zigbee specifications, example cited below, and the reality of the fast changing world of sensors/controls and home automation devices. Saying that the ZHA system does not require a new ‘handler’ for a new device, however Zigbee2MQTT does, does a real disservice to new users of the Home Assistant system. Whether it is a new functions, an example would be the new radar based occupancy sensors, or new manufactures that are following (and paying CSA) correctly to produce ‘clone’ devices under new manufacturing ID’s, it is rather frustrating for a new HA user to purchase ‘compliant’ Zigbee devices and setup a ZHA Zigbee system only to find the devices don’t work. And the guru advice on these forums is ‘just write a quirk in Python’… not too helpful and inclusive IMHO. And to stay on the soapbox, to what appears to be the large amount of ‘oh just go look at the code over at the Zigbee2MQTT github and copy it, oh and don’t bother giving any proper credit/attribution’, really sux!

Zigbee2MQTT is the leading place for open Zigbee development and support because there is a large community of helpful users and developers who are passionate about making Zigbee products work in an open way. And there is some very good leadership in this focused community. ZHA is in a sense similar, but does not and IMHO will probably not have the traction and size that Zigbee2MQTT has obtained.

2.3.3 Manufacturer Specific Extensions
Manufacturers are free to extend the standard in the following ways:
10

Posting a reply here to @dproffer post in Zigbee Connectivity in my house sucks 😢 - #24 by dproffer (which perhaps went a little too much off on a tangent) as it so not to hijack the topic thread by @mombro even though the subject questions and problems posted there are also indirectly related so should be in his interest to read as well. Thought this discussion might fit better in this thread with the title “a random collection of Zigbee trivia”.

Firstly, I am not saying that Zigbee is an overall bad protocol, it does work very well for its intended purpose if you set it up correctl, and I have been promoting Zigbee devices to IoT beginners and those on a smaller budget for years because it does have a few benefits over other home automation wireless IoT protocols used for products; with those primarily that practically all Zigbee products made for home automation uses the same 2.4 GHz radio frequency spectrum throughout the world (making it possible to sell the same products worldwide), and the cheap price of Zigbee devices due to cost of the chips being inexpensive and include license fees (with even the certification fees and royalty rights to use the official trademarked Zigbee logo on products is not expensive).

Secondly, a little background is that Zigbee technology/protocol is relatively old compared to most other wireless IoT protocol standards so some of its limitations are due to that, (Zigbee was conceived in 1998, standardized in 2003, and Zigbee Pro which we use for home automation devices was finalized in 2007, but while the specification has it has been revised with extensions and improvements since then it is still backwards compatible so carries legacy burdens).

Thirdly, all wireless IoT protocols have common limitations, problems, and challenges that device product manufacturers and end-users need to accept or work around. They also all have some common benefits, like being wireless (which all in all usually makes for a simpler installation).

So if we limit the scope to only comparing Zigbee, Z-Wave, Thread, and Bluetooth as those are currently or will soon dominate the market then the fact is that Zigbee technology, protocol and gateway + device implementations do have several limitations, I like to highlight these important key points for example, which some other protocols share too but those have different implementations with ways to work around.

Disclaimer: I do not think that it is fair at all to compare Zigbee with Wi-Fi for many reasons, and I do not see those two as competing protocols as they are intended for very different purposes. WiFi is not low-power nor low-bandwidth/low-datarate like Zigbee, and in the case of home automation products those serving a single purpose while WiFi is used for general wireless networking), however, that is a whole separate discussion, but let us just say that Zigbee has many limitations/problem that WiFi does not, and the current WiFi specification can almost not be used for battery-operated products (at least not as well and not without large and expensive batteries). 802.15.4 (Zigbee/Thread) and Z-Wave radios also have very tight constraints on power and bandwidth compared to Wi-Fi. 802.15.4 (Zigbee/Thread) and Z-Wave were purposely designed with long battery life requirements and from the start meant to utilize mesh network.technology function to extend range and coverage.

Now to two major downsides to Zigbee and the major benefit of competing protocols for one of those:

  • A Zigbee network can only have a single Zigbee Coordinator which not only means it is a SPOF (Single-Point-Of-Failure) but that is also the only point of ingress to the Zigbee gateway application layer which also means that all Zigbee communication messages must be routed to the same Zigbee Coordinator.

    • Thread protocol allows you you have active-active Thread Border Routers so it does not have a SPOF, and you can have several Thread Border Routers on the same Thread network, allowing you to have multiple points of ingress.They do this by featuring a technique called TREL (Thread Radio Encapsulation Link) which can tunnel Thread traffic over other IP-based networks (i.e.LAN and WiFi) and load load-balanced the traffic between all your Thread Border Routers in your home. So you can spread Thread Border Routers around your house in a single Thread network for a rock-solid setup
    • Z-Wave protocol does support multiple Z-Wave controllers on a single Z-Wave network, meaning that you can at least have two (or more) Z-Wave controllers on a single Z-Wave network to avoid SPOF, (though the Z-Wave JS implementation does not yet support adding a second Z-Wave controller on the same Z-Wave network). However you can only have one primary controller per network, but any second or third controller, etc. are “active-standby” mode so ready to take over if the primary controller should fail.
    • Bluetooth Low Energy (BLE) devices do not normally support or use the available Bluetooth Mesh technology and there is no unified network standard for it between products from different manufacturers, but at least users can partially work around the SPOF and by using multiple ESPHome Bluetooth Proxy → https://esphome.io/components/bluetooth_proxy.html

  • Radio propagation of Zigbee low-energy signals is poor because Zigbee’s higher 2.4 GHz frequency radio waves have a shorter wavelength which translates into less penetration depth and passed through building materials or other obstructions that absorb medium frequency spectrum better than lower frequency spectrum, (e.i. compared to Z-Wave which uses Sub-GHz low frequency ranges that are better at passing through building materials and other obstructions). (This is by the way also why 2.4 GHz Wi-Fi have a longer range than 5GHz Wi-Fi signals).

  • This radio propagation problem is compounded to produce more packet losses when the signals are low-energy and low-bandwidth as the messages are very short for Zigbee communication, so if a message does not go through then it needs to be re-send which can cause issues with battery-life of other Zigbee devices on the network so is only done a few times, hence causing messages to newer reach its intended target, however this problem is meant to be solved with mesh networking but that does not work if the users does not have enough Zigbee Router devices.

    • 2.4 GHz radio frequency range is also prone to more EMF/EMI/RMI interference than Sub-GHz radio frequencies because there are a lot of other existing standards that intentionally use the 2.4 GHz radio frequency range precisely because it can be used globally. For example; Wi-Fi, Bluetooth, Thread, ISM band devices, security cameras, video senders, mobile phones, cordless phones, baby monitors, amateur radio, utility meters, etc… And there are also many appliances and other electronics that are electronically noisy and unintentionally emit EMF/EMI/RMI interference in the 2.4 GHz radio frequency range, like for example; USB 3.x/4.x devices, Thunderbolt, microwave ovens, computers, power-supplies, etc.
    • This is why the CSA (formerly Zigbee Alliance) and chip manufacturers began developing Sub-GHz Dual-Band IEEE 802.15.4 radio chips for Zigbee and Thread. Manufacturers of 802.15.4 radio chips have measured 2-3 times better range performance of Sub-GHz Zigbee in indoor and suburban environments compared to 2.4GHz on average. The reason that Sub-GHz Zigbee products have not been viable before is that they have previously not been able to make chips for global (but now they have managed to make Sub-GHz chips that can digitally tune radio frequencies from 779MHz to 928MHz a single product can be shipped worldwide and not have to make chips and products specific for different RF regions. The downside to Sub-GHz Zigbee is that it is limited to 20-40 kbit/s datarate compared to 250 kbit/s datarate at 2.4GHz before packet overhead, (actual data throughput will be even less, however, that is a non-issue for most IoT devices that are not meant to transfer audio or video). For reference see → Alliance’s Zigbee Sub-GHz – The Power of Reliable Zigbee Mesh - CSA-IOT and https://www.silabs.com/documents/public/white-papers/Key-Priorities-for-Sub-GHz-Wireless-Deployments.pdf
    • Sub-GHz radios can also offer longer battery life as lower frequencies take less energy to transmit and less packed-loss means the message to not need to be re-sent.
    • These are facts in the physics of radio waves so should not be disputed unless just like to argue.

Hopefully, these limitations is something that will be addressed in future Zigbee revisions, (we can at least hope that they sooner or later will add support for multiple Zigbee Coordinators + that Zigbee sub-GHz Dual-Band radio chips will become readily available at lower costs and start to become used in popular home automation products). But even if so then it might just be to little and to late now that both the Matter standard and the Thread protocol specifications are finally evolving faster and faster as more companies are seeing it and its future versions/revisions as a larger merging market for profit growth.

Anyway, I do believe that manufacturers of home automation products will still continue to make Zigbee 3.0 variants of their popular products for many more years to come but eventually, (because you can use the same hardware to make Thread variant and a Zigbee variant of the same product, as it takes only a little effort to make a different firmware build and separate certifications), but when the Matter specification offers feature parity and has surpassed Zigbee then I think that manufacturers will slowly abandon the manufacturing of Zigbee in favour only making Thread based products. My guess is that it will only take 3-years or so before some companies who previously made Zigbee products will not bother making a Zigbee variant when they make a Thead variant of a new product, (as it is more economical to only maintain a single firmware variant even if the effort two make two variants is low).

PS: Not going to be answering any replies to this for at least a few weeks as plan to stay away from distractions during the holidays, but replies should not be needs as these are not secrets but facts that anyone can find easily if just do some basic research on those most popular wireless IoT protocols.

5 Likes
11

While you intentional left out wifi in your comparison I can very much confirm that a former zigbee network of around 30 devices which wasn’t reliable became solid like a rock when replaced with 30 wifi esphome devices.

Also very appreciated for giving such an extensive answer even when it is expected that the questioner might not even understand a third of it. Very valuable post and information you got in stock!