By the way, I read that EMF/EMI/RMI is the reason why the Home Assistant Yellow only has USB 2.0 ports and is sold with a non-WiFi variant of Raspberry Pi CM4 (without Wi-Fi), so at least they did that to mitigate having an onboard Zigbee/Thread radio module, …but I bet that from what they learned since the Home Assistant Yellow shipped if they would redesign that board today then they would probably optimize the whole board design further for better Zigbee module placement as well as add some basic EMF shielding to the module, (i.e. add metal covers to components for EMF shielding), and maybe even add an external antenna.
FYI, it is also not recommended to use Multi-PAN/Multi-protocol firmware (i.e. RCP firmware with both Zigbee and Thread enabled in a single radio SoC at the same time. Instead, it is recommended to use one dedicated radio chip/module/adapter for Zigbee and a separate radio chip/module/adapter for Thread. At least if you plan on adding more than a few devices of each.
@korayberk I am replying here below in this existing hread instead of in your new thread as you asked not to be lectured there and this might feel like a lecture → Is Home Assistant Yellow really unusable?
Choose to ignore this if you want but I am telling you that this will explain your problem and the solution.
Firstly, I believe that you have misunderstood what “zigbee network utilization” values mean here. That “zigbee network utilization” has nothing to do with the amount of Zigbee devices on your Zigbee network, so it has nothing to do with how many more Zigbee devices you can add. You see that “zigbee network utilization” is just an the the result of a frequency scan that is meant to be used indication of how much radio noise is on those frequency bands which is meant to help you select the the Zigbee channel with the least radio noise. Most of that radio noise will be EMI (electromagnetic interference, also called RMI / radio-frequency interference) from different EMF (electromagnetic fields) due to all kinds of electric charges, meaning everything with electricity, including all types of cables and electric appliances.
You can use that “zigbee network utilization” for troubleshooting to do tests as you move your Zigbee Coordinator away from sources of radio interference, usually starting by moving it away from any devices with Wi-Fi router/access, but also all other electronics. FYI, other than WiFi, know that USB 3.x/4.0 devices (like example USB devices such as harddrives) are infamous for causing serious interference with Zigbee communication. It is therefore highly recommended that you read and understand then follow all these tips → Zigbee networks: how to guide for avoiding interference and optimize for getting better range + coverage
It is not unusable, you just need to make sure you are not using the built-in Zigbee radio as a dedicated Zigbee Coordinator and not in the experimental RCP Multi-PAN/Multi-protocol mode (please see the detailed explanation of why that is further down), and you also need to understand how to work around the inherent limitations and network design of the Zigbee technology.
The radio module built into the Home Assistant Yellow is based on Silicon Labs EFR32MG21 SoC and that has been proven to run stable when handling around 200 Zigbee 3.0 devices if running as a dedicated Zigbee Coordinator in NCP mode for a single network and as long as you also are following Zigbee best practices. (But if you have enabled the experimental RCP Multi-PAN/Multi-protocol mode then that goes out the window, again please see the detailed explanation of why that is further down).
That however depends on you also adding enough Zigbee Router devices (i.e. mains-powered devices that work as good Zigbee signal repeaters / Zigbee range extenders), as well as do your best to keep sources of EMF/EMI/RMI away from all Zigbee radios. That is, you have to understand that Zigbee heavily depends on its Zigbee network mesh technology which depends on having many stable Zigbee Router devices on the same Zigbee network. Again, I highly recommend that you read and follow all the tips here → Zigbee networks: how to guide for avoiding interference and optimize for getting better range + coverage
Note that all the above is only true if you run it using the default Zigbee-only mode (with the standard NCP firmware) and you have not enabled RCP Multi-PAN/Multi-protocol mode to also use it for a Thread network. Please see the detailed explanation below of why that is and why you should not use RCP Multi-PAN/Multi-protocol mode if you want to use more than a few Zigbee devices.
Just to clarify my recommendation about running in NCP mode with a single private network instead of running in RCP Multi-PAN/Multi-protocol mode with two active private networks on a single radio (one radio SoC/MCU module);
If you have enabled RCP Multi-PAN/Multi-protocol mode support on the built-in Silicon Labs EFR32MG21 radio module in the Home Assistant Yellow (and thus switched that single radio to be used in the experimental RCP / Radio Co-Processor mode) then connected both Zigbee devices and Thread (Matter over Thread) devices to that single radio at the same time you can not expect it to concurrently be able to handle as many total devices as it would be able to handle if it was instead being used as dedicated Zigbee Coordinator in single network mode.
Because of the way Multi-PAN/Multi-protocol technology works, running in two networks with two protocols simultaneously will significantly limit the total amount of devices that a single radio can handle, so just because that single EFR32MG21 radio SoC can handle around 200 Zigbee 3.0 devices when running as a dedicated Zigbee Coordinator in NCP mode does not mean that same single radio can handle 100 Zigbee devices and 100 Thread devices concurrently. Also understand that the way Zigbee works the resource constraints of that single radio is a bottleneck for the whole Zigbee network.
The fact is that the overhead and complexity of running two different networks in RCP Multi-PAN/Multi-protocol mode will realistically restrict the total amount of devices that the single radio can handle. Hence the RCP Multi-PAN/Multi-protocol mode is still being labeled as experimental, and the general recommendation is to instead use a separate dedicated radio adapter for Zigbee once you start adding many Zigbee devices.
So do not get me wrong, the built-in radio in Home Assistant Yellow is great to get started with both Zigbee and Thread, but once you added 50+ devices then I would strongly recommend that you separate those two networks so that they run on their own dedicated radio adapter/module.
Finally, to keep things fair you should start by comparing apples and apples. To compare, the Philips Hue Bridge officially only supports connecting up to a maximum of 50 Zigbee devices, and that upper maximum is similar to most other commercial Zigbee gateway/hub/bridge/controller appliances, so compared to having multiple Zigbee gateways/hubs/bridges/controllers it will be much better to migrate all your Zigbee devices to a single network, read my related comment here → Zigbee buyer's guide - #3 by Hedda
apologies for my poor choice of words due to sleep deprivation and frustration. my bad, I am sorry. in proper etiquette should i delete my other, frustrated thread (in which i also apoligized)?
i have read your reply -full of information- only twice, just to reply asap. I will re-read it several more times.
First off, I had never turned on this setting, to keep things simple.
I had removed the BLE dongle (at the end of a cable) (hope to hook it back one day)
I have tried to seperate as much as i can from the router, i will look for more distance.
my 2.4 ghz wifi is on channel 6. unfortunately, my isp currently doesnt let me change this. if this is the key, i will fight with ISP, or get my own wifirouter.
seeing my energy scan, 25 has a lot of breating space. do you think moving to channel 25 might help?
and finally, please, you said “make sure you are not using the builtin zigbee radio as a dedicated zigbee coordinator”… how do i test or configure this?
i understand that Yellow will have its limitations. I hope i will be able to improve the situation. (out of the 120 devices, 25 are thermometers. i have around 15 lamps and 15 power plugs (both are zigbee routers, right?) They are scattered all around the house. so the zigbee mesh should be fine.
apologies once more @Hedda
Looking forward to more of your feedback.
Yes changing Zigbee channel will probably make things much better for you. Though be aware that when changing Zigbee channel you might need to power-cyckle a few devices and sometimes even have to re-pair a few battery powered devices if they do not reconnect automatically.
Just make sure that you first already physically moved Home Assistant Yellow away from all other electronics (especially any WiFi and USB 3.x/4.x devices) and then re-scan to find the least noisy channel.
Product type does not determinate if a device is a Zigbee Router or not. While no battery powered fevices are Zigbee Router devices it is only a general rule that most mains-powered devices are Zigbee Router devices, there are still many mains-powered devices that are not Zigbee Router devices. You have to check their device type information inside the Zigbee gateway application (i.e. device information inside ZHA).
Important to keep in mind is to never power off Zigbee Router devices as they always need to be available or devices connected to them will loose their connection, and even if that is just temporarly it will make for an unstable Zigbee network. A common mistake there is to connect Zigbee lightbulbs/lights that acts as Zigbee Router devices to dumb switches that can be switches on an off. So not connect such Zigbee Router lightbulbs to dumn switches. If you can to connect Zigbee lightbulbs to dumb switches then you need to buy Zigbee lightbulbs that are not Zigbee Router devices, like example those from Sengled.
For a large home I personally recommend buying a few (suggest three or more) dedicated Zigbee Router devices, meaning products that was desigbed to be nothing other than Zigbee Routers.
So for a quick no-effort fix on this topic I normally just recommend that people new to Zigbee who have big houses and large areas or buildings with dense building materials make it easy for themselves and just buy a bunch of “IKEA Trådfri Signal Repeater” devices from the start to get them a good mesh network backbone as a baseline. This is because while not as strong as these Zigbee USB dongles with an external antenna, the “IKEA Trådfri Signal Repeater” comes with good firmware by default and are very inexpensive, so you can make up for them not having the highest performance by simply buying more of them, (and they are still more powerful than almost all other commercial Zigbee products that are not designed to be a dedicated Zigbee Router device).
However, if you want to get the best setup then the best AND most cost-effective Zigbee Router that money can buy today is just to convert a few of ITead’s “Sonoff Zigbee 3.0 USB Dongle Plus V2” (model “ZBDongle-E” based on Silicon Labs EFR32MG21) into dedicated Zigbee signal repeater / Zigbee range extenders by flashing them with Zigbee Router firmware and powering them with a simply USB-chargers. While maybe not the prettiest solution to look at, if you make sure they are permanently powered then joining/pairing three or more to your Zigbee network and spreading them around in your home will create an extremely stable backbone in your Zigbee network.
That must have been a typo, I again only meant that you should not enable multiprotocol. That is, use one dedicated radio adapter/module for Zigbee and a seperate radio adapter/module for Thread, (concurrant multiprotocol on a single radio is bad, mmmkey).
The reason why Home Assistant SkyConnect’s firmware developer is putting RCP Multi-PAN effort on backburner is further explained here → The State of Matter
There is a third, experimental, firmware option that supports multiprotocol, which allows the Silicon Labs chip in these products to connect to both Zigbee and Thread networks with one radio. We announced our intent to release a firmware supporting multiprotocol when we launched Home Assistant Yellow and Home Assistant SkyConnect, and this firmware has been available since December 2022. It integrates the Silicon Labs SDK, which adds this support for multiprotocol. During the further development and testing of the multiprotocol firmware, we have concluded that while Silicon Labs’ multiprotocol works, it comes with technical limitations. These limitations mean users will not have the best experience compared to using dedicated Zigbee and Thread radios. That is why we do not recommend using this firmware, and it will remain an experimental feature of Home Assistant Yellow and Home Assistant SkyConnect. If you currently have the multiprotocol firmware installed but don’t actively use it to connect to Thread devices, we recommend that you disable multiprotocol.
PS: Off-topic byt FYI; when using the ZHA integration it is by the way super easy to migrate to a new Zigbee Coordinator radio adapter if you would later want to use the build-in radio adapter/module for just Thread.
I need the weekend to try everything you mention. Many thanks.
Right off the bat, I can say: i do have router lamps that are connected to dumb switches that some people insist on using to turn off!
I didnt know that with such “users”, I should pick special kind (not router) of lamps for stability. When all I had was a hue hub, even if people closed lamps, it would be fine. Lagging lamps began after i started using rasberry Pi, but I thought that was due to anemic cpu (rPi3), ram(1G), i/o to sd card. I was naive to assume Yellow would solve all my woes.
Lets check if i got this right: even if i have a nice zigbee mesh with repeaters and routers, 2-3 router lamps turned off (from dumb switches) can screw it all up? I am not psychologically ready to disable the dumb switches
I have two parting questions:
1- is the zigbee channel utilization warning the main indicator of the poor performance i have? or is it one of the many parameters?
2- i also have a bunch of EmberStatus.DELIVERY_FAILED: 102 errors. are they related to my current woes, or this is another issue entirely?
Yeah that could very well be one of the root causes of your problems other than EMF/EMI/RMI interference. There is no other way around it. You need to disable all dumb switches connected to Zigbee Router lightbulbs or replace those lightbulbs with models that are not Zigbee Router devices.
Maybe the Philips Hue Bridge handles recovery in that scenario better but the fact is that using dumb switches to Zigbee Router lightbulbs is dumb, it goes against the architectural design of a Zigbee network mesh and breaks the technology.
That is only an indicator of radio noise (i.e interference) which is just one of many parameters and one that is usually among the easist to fix when you understand that interference is a serious problem for Zigbee as all you need is to relocate the Zigbee Coordinator away from all sources of interference and maybe switch either WiFi channel or Zigbee channel so that they do not conflict. Again read and follow these tips → Zigbee networks: how to guide for avoiding interference and optimize for getting better range + coverage
Those errors mean that Zigbee communication messages failed to be delivered to one or more devices, and that can be due to either interference jamming the signals or you switching of Zigbee lightbulbs acting as Zigbee Router devices with dumb switches.
The root causes of these problems can not be fixed with software configuration so I think it is now time for you to understand that you need to change your mindset and grasp that you need to take some actions, whether you like the idea of them or not.
To address this particular point, I’d say yes, it is very likely to help. I had a dodgy Zigbee network for a year, finally bit the bullet and did some channel scans, identified 25 as a good candidate (and it often is since it is “above” all the wifi channels), and switched. It is a pain because you have to repair everything, but it worked great.
I’m now on a POE powered coordinator (a TubeZB - earlier version of this), channel 25, about 60 devices from multiple vendors, and the network is rock solid. I haven’t had to repair or fiddle with a device in over a year (truly).
On the routers-connected-to-dumb-switches topic, I use Sengled bulbs which are nice because they intentionally DON’T try to act as routers since lights being on regular switches is so common. However, I do have a number of power plugs that get moved around the house, or end up being disconnected for a while, and I haven’t found that this causes any issues with my network. Maybe if I was flicking them on and off constantly it would confuse devices, but the mesh seems to work correctly and routes around my plugs when I remove them or plug them in somewhere else in the house. At any given time this is probably only 1-2 out of my 15 router capable devices.
If you do decide to change channels, I’d export a list of all your devices to use as a checklist, get a few other things ready like replacement batteries that you can save time by replacing at the same time, and then plan on pairing the first 15-20 devices and a good collection of routers to monitor the network for a day or two. If things are feeling good, keep pairing, but if you are still having trouble, you can try a different channel without much lost time.
FYI, while not updated the ZHA docs, it is no longer such a hassle when changing the Zigbee Channel.
With later versions of Home Assistant’s ZHA integration, there should no longer be a need to re-pair ALL your devices as you had to before. As mentioned above; be aware that when changing Zigbee channel you might need to power-cycle a few devices and sometimes even have to re-pair a few battery-powered devices if they do not reconnect automatically.
That is, if you simply click the change channel inside the ZHA UI now then the Zigbee Coordinator will send out a broadcast message to all connected Zigbee devices on the network informing them that the Zigbee network will change the channel and thus they will need to change too, and if devices receive that message then they should automatically switch channel if they have properly written firmware.
The chance is that all your sleepy end devices (i.e. battery-powered devices) are not awake when that broadcast is sent and will thus not receive the message to change the channel, and depending on how their firmware is written you might just need to power-cycle them for them to start looking for other channels or you might need to factory reset and re-pair them.
However at least most if not all mains-powered devices do now normally change the channel without the end user having to power-cycle or re-pair them as before. Just give them an hour or two and most mains-powered devices should reconnect automatically to the new channel.
You obviously need to make sure all devices have before and are connected or else they will not get that channel change announcement broadcast. Again, do also make sure that you first already physically moved Home Assistant Yellow away from all other electronics (especially any WiFi and USB 3.x/4.x devices), as otherwise the is a chance that the channel change announcement broadcast will be jammed so more devices do not receive it.
Wow, that is great news. I’m a Z2M user so I haven’t followed ZHA as much, but that would be a very welcome feature for people who may have to try several channel combinations in order to land on one that works.
during my research, obviously a few details fell through the cracks.
you have opened my eyes to bulb/routers/off being the problem.
(in my defence) kudos to hue hub experience for allowing me to stay completely oblivious to this zigbee design outcome (you see how my perception has changed?)
i physically moved my yellow a half a meter away from anything else, to an experimental spot. i see improvements in energy measurements, especially channel 25 is nicely used. so i no longer think of moving my zigbee channel, if you dont urge, in the current conditions.
wifi is on channel 6, thats supposed to be acceptable. Currently I cannot go to channel 1. But I read shi!storm begins north to channel 6. so i should also be fine here, right?
but with my opened eyes, when all router bulbs are on, the experience is muuuuuuuuuuuuch more smooth (exactly, theeeeres my problem). There are still a lot of errors in my logs, thats my next priority.
before i make a strategic change (deleting dumb switches is suddenly an option, moving yellow to another spot is another) i would like to understand one thing about “physical location of things”
As far as i read, i should preferably pair stuff at their ultimate location. There are a number of devices (I lost account) that have been moved after pairing. How bad is that? and if thats bad, then when i move my yellow, thats also bad??? for people who move things around, isnt there a “rebuild the network” option.
I bet you would see even better results if you could move your Yellow further away from everything else.
My advice would again be to buy and add at least three dedicated Zigbee Router devices. The products I suggested for this are not expensive yet adding them should give you a much better setup as more devices should connect through those instead of your other non-dedicated Zigbee Router devices.
As I wrote above:
For a large home I personally recommend buying a few (suggest three or more) dedicated Zigbee Router devices, meaning products that was desigbed to be nothing other than Zigbee Routers.
So for a quick no-effort fix on this topic I normally just recommend that people new to Zigbee who have big houses and large areas or buildings with dense building materials make it easy for themselves and just buy a bunch of “IKEA Trådfri Signal Repeater” devices from the start to get them a good mesh network backbone as a baseline. This is because while not as strong as these Zigbee USB dongles with an external antenna, the “IKEA Trådfri Signal Repeater” comes with good firmware by default and are very inexpensive, so you can make up for them not having the highest performance by simply buying more of them, (and they are still more powerful than almost all other commercial Zigbee products that are not designed to be a dedicated Zigbee Router device). → https://www.google.com/search?q=IKEA+Tradfri+Signal+Repeater
However, if you want to get the best setup then the best AND most cost-effective Zigbee Router that money can buy today is just to convert a few of ITead’s “Sonoff Zigbee 3.0 USB Dongle Plus V2” (model “ZBDongle-E” based on Silicon Labs EFR32MG21 into dedicated Zigbee signal repeater / Zigbee range extenders by flashing them with Zigbee Router firmware and powering them with a simply USB-chargers. While maybe not the prettiest solution to look at, if you make sure they are permanently powered then joining/pairing three or more to your Zigbee network and spreading them around in your home will create an extremely stable backbone in your Zigbee network. → https://community.home-assistant.io/t/itead-s-sonoff-zigbee-3-0-usb-dongle-plus-v2-model-zbdongle-e-based-on-silicon-labs-efr32mg21-20dbm-radio-soc-mcu/442695
There is no “rebuild the network” feature with Zigbee (like there is with Z-Wave), the actual meshing of your Zigbee network mesh is done automatically by every one of the Zigbee devices themselves and there is nothing you can do to affect it via software settings or configuration, so the only thing you can do to affect it is to add more Zigbee Router devices and physically relocate devices for more optimal placement. That is why my go-to saying is; when in doubt, just add more Zigbee Router devices
It is true that if possible then the general best practice recommendation to pair devices in their final location, and it can be a good idea to repair a device if you relocate it as it will then connect directly to the Zigbee Router with the best connection for it. Also, all battery-powered devices can be very slow to connect to better Zigbee Routers and thus it might still be a good idea to simply re-pair it if you know that you will be permanently moving it from one area to a other area.
But all in all, if you have already done as I recommend above and have added several “know good” (or better yet “known great”) Zigbee Router devices in your network that are in their permanent t locations and always have power then it should rarely matter as those will in practice make up a stable backbone-network/core-network of your Zigbee network mesh. There are of course always edge cases.
You see, Zigbee devices with properly written firmware should by design constantly reevaluate their possible connections and move to the best Zigbee Router for it about once every 24 hours or so, or in a scenario when the Zigbee Router it is connected to is no longer available then it should move quicker.
The problem is Zigbee devices with poorly or bad written firmware which will not automatically move to a better Zigbee Router and instead always stick to the first Zigbee Router it is connected to no matter if that is still available or not. Aqara and Xiaomi devices are for example infamous for needing to be manually repaired/rejoined if a Zigbee Router device is no longer available or if you add a new Zigbee Router later than you want devices to connect via instead. Devices with bad firmware are rare than Aqara and Xiaomi.
So as long as a device has a properly written firmware then you can in theory pair/join it at any location and move it later then wait around 24 hours for it to move connection, but the issue with that is then that move can affect other devices if that device was a Zigbee Router device, as then you can get a cascade effect that could, in theory, take several weeks for the whole Zigbee network mesh to rebalance itself and settle down.
thank you.
ikea repeaters have been ordered.
i am eyeballing the sonoff alternative in case things dont pick up.
thank you for clarifying the design principles.
I can always throw away incorrect implementations.
even when all my routers bulbs and power switches are on, sometimes some devices dont react on time. when the ikea repaters are added, i will reevaluate my whole situation. I already explained to the family that they need to part ways with “dumb switches”. I also need to nurture that. I will possible mechanically bypass them. (They used to make an excellent backup for when the zigbee mesh is screwing around)
Have you considered changing the dumb switches for smart switches? This would allow the removal of many smart bulbs, and would resolve the issue of the power being removed from the bulbs. Smart switches are often a much better solution, as they allow for manual control via the in-wall smart switch, with no negative impact to the Zigbee mesh network. This usually vastly improves the end-user experience and reduces the frustration level. Also, smart switches can control multiple dumb bulbs in a lighting fixture. Thus, the total number of nodes in a Zigbee mesh can be greatly reduced. Most in-wall smart zigbee switches/dimmers will perform better than smart bulbs as Zigbee repeater devices. Many older Zigbee bulbs, like GE Link, Cree Connected, gen 1 Hue, and Osram, are known to perform very poorly as Zigbee repeaters in large Zigbee networks. Newer Zigbee 3.0 bulbs seem to fare better, due to updated Zigbee radios and firmware. As previously mentioned, Sengled bulbs are designed as non-repeaters to help mitigate the issue of them being powered off manually via a dumb switch.
Hope you are able to get things resolved and running smoothly!
Note, do not yet update the firmware on the IKEA Trådtfri Signal Repeater as many are having problems with the very latest IKEA firmware on many different IKEA devices.
This the solution I also prefer for any lights connected to wall-switches. That is, use smart switch modules with dumb lightbulbs. There are are cord-switches and cord-dimmers that tou can replace dumb cord switches with and thus use with dumb lights.
I have personally relaced all our wall switches with smart switches as well as replaced dumb cord switches where we still have them, though in the cases where we have smart lightbulbs I have removed the cord switch completly.
I do not have any switches at all where we have smart lightbulbs as they do not make a good combination because smart switches also break the curcuit.
Pro tip is to buy quality dumb dimmable lightbulbs, i prefer Philips warm glow series of dumb lightbulbs.
I live in the European Union and here there is not one standard format for wall switches so instead it is we more common that we convert our dumb switches into smart switches via smart dimmer modules and smart switch modules.
In our house have just as many Z-Wave devices as Zigbee devices and personally I have installed Qubino, Fibaro and Aeotec Z-Wave dimmer modules that convert our dumb switches into smart switches.
Replacing all switches in the whole house also have the benifit of creating awesome backbone-network/core-network since they are installed in every room at a relativly regular distance from each other.
Because if you connect them in serie then the smart switch will kill the power to the smart lightbulbs, and thus having both in a serie is just as bad as having a dumb switch to a smart lightbulb that is a Zigbee Router device.
