Reliability issues and pairing issues (even after trying multiple coordinators and following all advice I could find)

Configuration Zigbee
1

Hello! I am seeing some significant issues with setting up Zigbee w/ ZHA at my house, and have no idea what to do next.

Baseline info
  • I am using ZHA
  • HomeAssistant is running in Docker on a PC with few other services, and seems to run fine on its own
  • Zigbee coordinators are passed into the container under devices in docker-compose with - /dev/serial/by-id/device-id-here:/dev/serial/by-id/device-id-here
  • I’ve tried two coordinators:
  • I am using a USB extension cable
    • I’m fairly sure I’m using a USB 2.0 port. This computer is an old-ish compact gaming PC, and the ports do not show the “super speed” logo and are not blue.
  • I’m trying to connect two kinds of devices in the same room, with direct line-of-sight
    • Sengled Zigbee Bulbs (link omitted due to limit)
    • Lutron Aurora (link omitted due to limit)
  • I have read ZigBee and Wifi Coexistence (link omitted due to limit)

    • I have moved my own 2.4 GHz networks to WiFi channel 11

    • I live in a suburb with many neighboring wifi networks I cannot change. Channels 1, 6, and 11 all have at least one network with at most -70 dBm strength.

      Screenshot of networks

      screenshot_networks
      screenshot_networks1080×2220 189 KB

      The networks in channel 3 and 9 show up inconsistently. They just happen to be in this screenshot, but most of the time I don’t see them.

      The highest channels on 11 are mine, but 11 still has the highest channels that aren’t mine, too!

    • I have kept the coordinators on the default channel 15 as it seems to be the least noisy spot

  • I typically group bulbs in a zigbee group to lower network traffic and see the same unreliability, including group commands not affecting every bulb.
  • For bulbs in the same room – less than 2 meters from the coordinator – I’m getting LQI ranging from 120 to 178. I do not have an RSSI reading.
  • I have tried removing the ZHA integration, deleting zigbee.db, and restarting from scratch multiple times

Problems:

  • With Conbee II:
    • Extreme unreliability: If I changed lights that the coordinator has a direct line-of-sight to 5 times or so it would suddenly drop changes for at least a minute, if not more. Experimenting with different light colors or different temperatures was unworkable, much less using a dimmer.
    • The Aurora switches were oddly more reliable than the lights, but still had significant problems.
  • With CC2652P2:
    • Unreliability: I can more consistently change lights that I have a line-of-sight to, but I will still occasionally see drop-outs, especially if I make changes too often. Drop-outs were typically shorter than with the ConBee, but still significant. I’m doubtful about dimmer usability.
    • The Aurora switches do not work at all when they pair, if they even pair. (That is, they do not emit zha_event when pressed.) Recently, both of my switches will show as connected in Home Assistant but be stuck in pairing mode. I have reset the switches multiple times and tried connecting them on a freshly-reset ZHA integration and DB.
      • This occurs even when the switches are less than a foot from the coordinator.

My questions:

  • Is this a software problem or a hardware problem?
  • Is Zigbee even workable with so many 2.4 GHz networks around me?
    • Do I need more powerful equipment? What do I need to make this work? I already have a CC2652P2 coordinator with an external antenna.
    • Is channel 15 the best channel in this instance?
      • Channel 20 would get interference from my own wifi.
      • Channel 26 I’ve read is not always supported, and would have some interference from my own wifi.
      • All other channels would have strong interference with neighbors’ wifi.
  • Would Zigbee2mqtt fix this somehow?
    • Nearly all of these – the unreliability at least – seem like hardware or radio issues so I haven’t tried zigbee2mqtt yet.
    • I’ve already spent hours on this so I’m hesitant to spend hours more on another dead-end.
  • Could this be caused by the computer running Home Assistant?
    • Is there an easy way to test these zigbee coordinators on other hardware / computers that I have on hand?
  • Am I “resetting and starting from scratch” properly? I didn’t find much information about resetting ZHA, and if I don’t delete zigbee.db it tries to re-add existing devices instead of making a new network.
  • Has anybody been in a similar situation with severe unreliability and many neighboring wifi networks and found a fix?

And, as a final note: trying to set up this zigbee network has been very frustrating! I expected zigbee to work well because I’d heard good things. It probably would work well if circumstances were right, but I’m finding the debug-ability in these wrong circumstances to be really difficult! Common solutions to problems don’t seem to work in my case, and it’s not clear why.

Please help me! Even if it’s just to let me know that zigbee is unworkable in this circumstance!

2

Welcome to community, very sorry that your zigbee issues are the entry point.
I wish I could pull a rabbit out of the hat and find your issue. There is something that is a blaring issue. Hoping someone can find it quickly.

You possible paths are all good thoughts to a solution, but as you know you probably can only do one at time.

I hate to offer that you throw more money at it, but here I go… buy yourself a Hue hub version 2, hopefully you can find one on the cheap or in bundle with some useful bulbs or other devices. My though here, is just get a stand alone proprietary but ‘solid’ zigbee network up and running with the Hue hub and an Lutron Aurora switch. If that works solid with the Aurora located at various places, then you have a base to work from to debug your DIY components. If that too does not work, then you have a bad RF issue, time for dixie cups and strings (sorry bad joke). If you can buy the Hue hub from somewhere like Amazon or Best Buy with a easy return policy, you can do 15 to 30 days of debugging and return. That said, I have a Hue hub as one of zigbee network for two reasons: it just works and it is only reliably way to do firmware upgrades of Hue and Friends of Hue devices, sux but that is the reality.

I don’t have any kind of RF signal viewing equipment, but I have a 40 node wifi with 4 access points, 4 zigbee networks with 20 nodes … all with no issues, even when the microwave is running…

Good hunting, keep the faith. One step at a time and a good bottle of wine!

1 Like
3

You probably have tried, however have you installed a 2m USB extender cable and moved the coordinator far away from any computer/USB3/Wifi equipment?

Some people report that a USB2 is working better for them, even using a USB2 hub. Not sure, I use USB3 my self with no problems.

2 Likes
4

This is a good way to narrow down if it’s a hardware issue! I found a hue hub online for pretty cheap. I didn’t realize they could be had for so little! I also didn’t look into them because I didn’t want to be tied to hue proprietary stuff, to be fair. It sounds good to have as an option for firmware upgrades, at the least.

This gives me hope that it’s a hardware issue with the computer I’m using, and not an RF issue.

I’ve tried some extension cables that I’m fairly sure are USB2, and moved the coordinator at least 3 meters away from other equipment across the room as a test. For the Conbee at least, this produced little-to-no difference. I didn’t try using an extension cable with the CC2652P2 as it came with a moderately long USB-to-micro-USB cable, and I felt it was unlikely that proximity to other devices was the main reason for its problems.

Thank you both for the help! I now feel like this is a hardware issue with the computer so my next steps are the following:

  • Return the Conbee II – the CC2652P2 seems more reliable, higher power, and better-supported by open source software.
  • Buy a Hue Hub 2 as “known good” standalone zigbee equipment. I’ll keep this for future troubleshooting and for upgrading firmware.
  • Try the CC2652P2 with zigbee2mqtt over ethernet. If this is some sort of hardware issue with the computer I’m using, having the CC2652P2 function on its own should also eliminate the hardware issue.

If the CC2652P2 with zigbee2mqtt works on its own, I’ll use that. Otherwise, I’ll use the Hue Hub for now. And if that doesn’t work then something’s really wrong, and I don’t think I can find a workable solution in that case.

5

Good that you have a plan to move forward. I think you will find a solid config. Have many of us wasted a lot of time getting to a solid solution, yes… Do use the search in this forum and reddit equiv as there is a lot of ‘experience’ that is good to not repeat…

On the topic of the Lutron Aurora, I recently purchased one and I have been experimenting with it. It is an interesting product, however it does have it’s share of quirks. So keep that in mind as you get your zigbee network stable and useful. This is one of the examples of searching the history of other folks experiences so you do not have to reinvent the wheel. It seems the Aurora seem to be bit of new/outlier, there do not seem to be a lot of rotary dimmers in the zigbee world and in general it seems to have some quirks with different coordinators, for example various firmware revs of the conbee either work or don’t work with it. I had similar experiences with the Aurora and my Sonoff/Tasmota zigbee coordinator, and as a result I have the Aurora connected to my Home Assistant via my Hue bridge. However, while the on/off functions work solid in HA this way, I am not able to use the dimming features of the Aurora via this route. However, that is ‘better’ than what I had with it connected to the Sonoff zigbee coordinator, where the on/off presses only came through about 10% of the time. Both the on/off and dimming functions come to the Hue hub 100% of the time, it is just that the current release of HA, 2021.12.10 does NOT decode the dimming events of the Aurora. Gotta love technology, huh? Any way good luck and good hunting!

1 Like
6

I got the Hue Hub and it works better, but still has issues.

Lights generally work well. Changing brightness / color constantly (to simulate using a dimmer) can leave some lights slightly delayed, but no major drop-outs.

The Aurora, however, still has problems. The zigbee network can’t seem to handle both Aurora messages and light messages at the same time. Using the dial only works for 2 seconds, after which light changes are delayed or never received even if the app shows a change in brightness level. I can generally change the brightness with the app instantly after that, at least, which is an improvement.

Finding the lights (4) and Aurora (1) can take over a minute on set-up and sometimes I have to try searching for them multiple times.

The hub is about 2 meters from the nearest wifi router. I tried zigbee channels 11 and 15 which don’t overlap my wifi network.

Does any of this scream “connection issues”, or does all of this seem normal? Is there anything else I should do to eliminate problems that might be non-obvious? Are there any tricky or non-obvious wifi settings, physical placement issues, interference-generating household items, etc. that I should be aware of?

With the Aurora connected to the Hue hub, do you see reliability / connection issues with it, or does it work consistently? It sounds like you hit some issues originally, but that it connects and works consistently with the Hue hub, which isn’t matching my experience. I’m seeing significant delays and dropped changes with the Aurora as I mentioned above, even on the Hue hub.

Some of this feels like there are still connection / reliability issues.

If this does seem like connection issues, are there any relatively unexpensive ways to analyze the 2.4 GHz spectrum in my area and locate any sources of interference? Checking that out directly seems like an effective way to eliminate RF connection issues, but the tools I found after a quick search were all $500 or so.

7

Sorry to hear even with a Hue hub your experience is still what sound to me below reasonable. I still have the Aurora connected to my Hue hub and I am only using it’s on/off commands sent from the Home Assistant Hue integration to control a wifi based ceiling light. It is working fine. Not sure if I wrote about this, however the Hue integration in HA does not send the rotate/dimming events thru to Home Assistant. I have made a request of the development team to add, but I have no idea whether it is possible or the time frame if so. In the Hue app, my Aurora firmware shows as version 3.8. Not sure if there are different versions, but you might check yours. I found no issues with on/off and dimming function of a Hue bulb from the Aurora within the Hue local function, I only tested it for under and hour however. I have seen no issues with the on/off event coming through to Home Assistant. It is possible to ‘miss press’ on the Aurora and not generate an on/off event. If you are looking at the switch, you can see this because the LED in the Aurora does not flash white. But, often, you are not facing the switch when you press it. This does not happen often, but it is a little annoying when it does. As I say, I think just a physical design issue with the Aurora.

To you question about checking your 2.4 GHz spectrum to see how busy it is. I am far from being an RF engineer, however as I understand it the price of spectrum analyzers that can truly look at the frequencies around 2.4 GHz cost at least 2 to 3 thousand USD. While USD 500 is not inexpensive, I am not sure you would get useful results at that price. Not sure what the device is that you are considering.

One thing you can do for low or no cost is run on of the wifi channel viewing apps on a laptop. This might give you sense for at least how many wifi stations are visible in your location. Below is a screen capture of my location for both 2.4 and 5 GHz. This program is called ‘WiFi Explorer’ on MacOS. Both my zigbee2mqtt and zha instances are on zigbee channel 11, my Hue is on 15 and I have three other zigbee networks, Ikea, Aqara and Energy Meter that are on other channels.

wifi-explorer
wifi-explorer1920×411 64.5 KB

wifi-ble-zigbee-2-4-Ghz
wifi-ble-zigbee-2-4-Ghz753×359 25.1 KB

1 Like
8

Suggest you begin by follow some of these general recommendation before troubleshooting any further:

https://github.com/home-assistant/home-assistant.io/pull/18864 → https://github.com/home-assistant/home-assistant.io/pull/18864/commits/b21c49589d898d60a1a235afa7b9c148d013cfee

Recommend that you start by upgrading to the very latest firmware on your Zigbee Coordinator adapter.

https://flemmingss.com/how-to-update-conbee-conbee-ii-firmware-in-windows-10/

And of course, do add a long USB extension cable to get the Zigbee Coordinator a bit further away.

Also read the additional generic tips here:

https://www.home-assistant.io/integrations/zha#best-practices-to-avoid-pairingconnection-difficulties

Another tip is to shield your computer and USB devices or peripherals connected to computers as well as other appliances or cables/wires located close to your Zigbee Coordinator. As already established if you read those other tips, Zigbee radio can be very susceptible to electromagnetic interference (EMI, also called radio-frequency interference or RFI), and one very common source that generates electromagnetic fields is unshielded computers and unshielded peripherals so make sure that you use for example enclosures for external hard drives and single-board-computers that are made of metal instead of plastic. Especially known to have a huge interference impact on wireless devices using radio frequency like Zigbee radios are USB 3.0 devices/peripherals (for more details on that read white paper report on the subject by Intel at https://www.usb.org/sites/default/files/327216.pdf).

1 Like
9

General recommendation is not change Zigbee channel since not all Zigbee devices support all Zigbee channels. The advice is to instead only change Wi-Fi channels on your WiFi router/access-points.

https://www.home-assistant.io/integrations/zha#defining-zigbee-channel-to-use

This is a good reference for channel selection Zigbee and Wifi Coexistance.

Note that the recommendation is, however, not to change the Zigbee channel from default as not all Zigbee devices support all channels. If you have issues with overlapping frequencies, then it will generally be a better idea to change Wi-Fi channels on your Wi-Fi-router or Wi-Fi Access Points.

10

FWIW, I have two Ledvance zigbee bulbs and they haven’t been particularly reliable. I have a solid mesh based on the Conbee II and a bunch of lights/switches as routers. The Ledvance bulbs don’t seem to behave well when they are either in a Home assistant Group or in a Zigbee group: one will sometimes fail to turn on or off, or they will come on in two different colors.

Have you tried your setup with other Zigbee devices than the Ledvance and Lutron? I have 27 Zigbee devices in my network so far (including 3 Aurora dimmers), and the Ledvance bulbs are the only ones that have misbehaved. It’d be a shame if all your hassles were just because of those bulbs!

Edit: sorry, you might disregard this post…I mistakenly thought my two misbehaving bulbs were Sengled, but they are Ledvance. Although it still wouldn’t be a bad idea to test your setup with some other devices just to be sure.

1 Like
12

The issue sounds like interference to me- you mentioned a lot of other 2.4 ghz wifi networks around, and problems with multiple coordinators and the Hue hub. I think the suggestions above about changing your wifi Chanel and other topics are some good ideas. I use zigbee2mqtt myself but don’t think switching to that would really solve your problems honestly.

I would look at a wifi analyzer program you can download on a laptop, or even app on a phone like wifi analyzer. While it won’t show everything on the 2.4ghz sample, it will show how crowded everything is wifi wise at least.

Screenshot_20220205-114427_Google Play Store
Screenshot_20220205-114427_Google Play Store1080×2400 174 KB

If there are lots of other networks though, changing your own wifi network’s channel might not help much since you can’t change other people’s. If interference is overwhelming, zigbee might just not work for you and should be avoided. The 2.4 ghz spectrum is definitely crowded. I have a lot of zwave devices and the alternative band they use really helps with minimizing interference.

Maybe just adding some additional mains powered repeaters could help. I recently bought 3 ikea zigbee outlets- they were pretty affordable and really helped me with devices dropping out.

1 Like
13

I’d suggest giving Zigbee2MQTT a go. I made the move over from ZHA and have found things significantly more reliable for my 15 device zigbee network. Also:

  • better device support, exposing more options for some of my devices
  • better options for debugging
  • install is separate from HA meaning restarting core doesn’t restart z2m
  • to me, it seems the mesh is more robust and constantly optimising. The ZHA map for me seemed to get stuck in its ‘way’ and following advice to prompt rebuilding mesh or repairing devices achieved nothing.

My experience getting Zigbee set up was also frustrating, and I also followed the same guidance as you. After moving over to z2m everything seemed to work better with only the occasion dropped message, but I then noticed the Conbee was on zigbee channel 11 (my wifi was on channel 1). Wifi now moved to 6 and everything is grand.

I don’t have the same devices as you, and realise there isn’t really any reason why z2m should perform better, but just thought I would share my experience.

1 Like
14

+1 though personally recommend IKEA Tradfri Signal Repeater (IKEA Trådfri Signal Repeater E1746).

IKEA Tradfri Signal Repeater is known to be a much better Zigbee Router than their control outlet plug.

2 Likes
15

Thank you all for the help! Your personal experiences and anecdotes gave me the will to keep trying here and gave me some real-world examples to compare my own with.

I got everything to work!

But before we get to that, here’s what happened since the last time I posted:

  • I converted my usb-only Tube coordinator to usb+ethernet
  • The microusb port disconnected from the Tube coordinator
  • I tried to repair it but I’m a complete noob at soldering so I failed
  • I bought a Tube PoE coordinator and PoE injector to replace the non-PoE one that broke.
  • I tried Zigbee2MQTT

So what worked?

  • Standalone PoE coordinator
    I assume this worked well because it:
    • Eliminates any potential USB3 2.4 GHz interference
    • Eliminates any potential hardware issues with connection to the PC
    • Allows me to move the coordinator to the most optimal location
    • Probably beefier than a little usb stick, plus it has its own external antenna!
  • Zigbee2MQTT
    I have no idea if this actually affected anything!
    I didn’t try the Tube PoE coordinator with ZHA, so I don’t know whether ZHA vs Zigbee2MQTT is an important factor here.
  • Zigbee channel not overlapping with my own WiFi
  • Zigbee channel in between neighbors’ WiFi
    For me, this is Zigbee channel 15.
    Notably, this setup worked great even before changing from channel 11 to channel 15. Channel 11 was not overlapping my WiFi, however, but it is directly overlapping neighbors’ WiFi.
    It’s worth noting: I tried changing my WiFi channel before and past setups didn’t work well even with my WiFi channel changed. Changing my WiFi channel was not the sole fix here.

How does it perform?

  • Good enough!
  • New Zigbee devices connect instantly – I reset all 12 zigbee bulbs and they nearly all instantly paired.
  • I can easily change lights using Home Assistant without problems.
  • The Auroras work okay.
    I can definitely toggle lights with them, but changing brightness is iffy and occasionally causes some short dropouts.
    One of my Auroras struggled to connect properly and I had to reset it a few times, but it eventually worked fine.
  • If a connection issue occurs, it’s usually resolved within a few seconds instead of taking over a minute.

A note for anyone in a similar circumstance in the future: don’t be scared to try Zigbee2MQTT! It’s surprisingly easy, it has a web interface, and it supports home assistant auto-discovery! I got the impression that it would be difficult to use but it was really straight-forward!

Thank you for this advice! It gave me more reason to try Zigbee2MQTT than just a hunch. Z2M is working great so far!

Thank you both for the advice and recommendations! Now that Zigbee is working acceptably, I may look into buying routers/repeaters. Sadly, the Sengled bulbs don’t double as routers/repeaters on their own.

As it is though, connection to all my devices seems pretty solid! This is a small house though, so that’s not surprising, really.

Again, thank you all for your help! I hope what I’ve posted here is useful for anyone in similar circumstances in the future!

2 Likes
16

If it helped appreciate a thumbs up on https://github.com/home-assistant/home-assistant.io/pull/18864