ZHA extremely slow and unreliable with ConBeeII stick

Configuration Zigbee
, ,
#2

Oh, and btw, the logs:

Logger: homeassistant.components.zha.core.channels.base
Source: components/zha/core/channels/base.py:428
Integration: Zigbee Home Automation (documentation, issues)
First occurred: 08:55:20 (6 occurrences)
Last logged: 08:56:01

[0x7D54:1:0x0008]: async_initialize: all attempts have failed: [DeliveryError('[0x7d54:1:0x0008]: Message send failure'), DeliveryError('[0x7d54:1:0x0008]: Message send failure'), DeliveryError('[0x7d54:1:0x0008]: Message send failure'), DeliveryError('[0x7d54:1:0x0008]: Message send failure')]
[0x84D9:1:0x0300]: async_initialize: all attempts have failed: [TimeoutError(), TimeoutError(), TimeoutError(), TimeoutError()]
[0x84D9:1:0x0008]: async_initialize: all attempts have failed: [TimeoutError(), TimeoutError(), TimeoutError(), TimeoutError()]
[0x84D9:1:0x0006]: async_initialize: all attempts have failed: [TimeoutError(), TimeoutError(), TimeoutError(), TimeoutError()]
[0x7D54:1:0x0006]: async_initialize: all attempts have failed: [DeliveryError('[0x7d54:1:0x0006]: Message send failure'), DeliveryError('[0x7d54:1:0x0006]: Message send failure'), DeliveryError('[0x7d54:1:0x0006]: Message send failure'), DeliveryError('[0x7d54:1:0x0006]: Message send failure')]

and the long one:
hatebin

EDIT: I see that the hatebin link had cut off the last 85% of the logs. I’ve made a new one only with the level >= warning. hatebin. The crossed out one has a lot of entries with level = DEBUG, but all the WARNING ones were in the part that was too long, I guess.

#3

Hey mate, of the 60+ devices do you have a number of powered ones as well to act as repeaters/routers? I was having the same issue recently with ZHA/Conbee II, and a remote switch that was a fair distance from the Conbee II. I purchased a Philips Hue globe yesterday to act as a repeater, and now the remote switch is turning lights on and off instantly…

#4

That would include lightbulbs, right? If so, I have a few dozen.

And the exact same physical setup worked for years with deCONZ, which is why I thought it so weird. I literally did not move a single device or the ConBeeII stick between the deCONZ and ZHA approaches. I just passed the stick to the VM, it’s even physically connected to the same server that ran deCONZ in docker…

#5

A few dozen lightbulbs would create a pretty strong zigbee network I would have thought. What does the network map look like under ZHA?

#6

I still have some devices (only battery powered, as far as I remember) that I didn’t bother to add yet, in case I’m going to have to go back to deCONZ and re-pair them all back again. But so far:

Screenshot 2022-02-08 at 11.34.18
Screenshot 2022-02-08 at 11.34.181628×1550 214 KB

However, it’s mostly the lights that are a problem. And there’s no obvious correlation with ones far not working, while those close do. But it is mostly certain lights, that don’t work well (or at all, in some cases).

And I do get all those warnings in the logs that I’m not really sure what mean.

EDIT: I see that the hatebin link had cut off the last 85% of the logs. I’ve made a new one only with the level >= warning. hatebin

#7

Note that most Zigbee coordinators and especially ConBee/Raspbee are known for being very susceptible to signal interference, so other than updating the ConBee firmware, be sure to take actions to avoid radio frequency interference and electromagnetic interference by using suggested long USB extension cable to a USB 2.0 port (or use it via a USB 2.0 hub) and add electromagnetic shielding to computer, peripherals, and appliances, as well consider getting better shielded cables/wires. Read:

https://github.com/home-assistant/home-assistant.io/pull/18864

and

https://www.home-assistant.io/integrations/zha#best-practices-to-avoid-pairingconnection-difficulties

Just some examples of others with ConBee that expensive similar symptom due to signal interference:

#8

Setting Zigbee channel is done via YAML configuration but is is recommended to change WiFi instead:

https://www.home-assistant.io/integrations/zha#defining-zigbee-channel-to-use

Note that setting Zigbee channel is only done when initially forming the network so require re-pairing.

Since ZHA formed the network so do not assume that you are using any other Zigbee channel than 15.

#9

Thanks.

I have gone through pretty much all the “best practices” for my old deCONZ network, which worked pretty well for a few years. Updating the ConBeeII firmware has improved things, however. Somewhat.

So unless there is some underlying difference between deCONZ and ZHA that can explain, it would seem to me that the issue is most likely that ZHA is using a ZigBee channel that overlaps with WiFi (because any two APs with non-overlapping 2.4 GHz channels would somewhat overlap with ZigBee channel 15, right? At leas the sideband lobes), and deCONZ was not?

Luckily I live in Europe, so we also have WiFi channels 12 and 13 available; I’ve set the AP close to the ConBeeII to channel 13, and the other one to channel 8. I’ll test a bit and see if it helps.

I would still have preferred just staying with ZigBee channel 25, and have no overlap with any WiFi, but just the sound of

after just having done that gives more than a few grey hairs.

Is it a technical limitation in ZHA that you cannot change the channel without having to re-pair everything?

#10

One idea. Not sure it is correct. I have a Conbee2/ZHA network with 60+ devices, and it is solid and fast.

You mention you have some devices that you have not included in the network again. As the coordinator is the same, and might be using the same network key, they will actually connect again. They will then be part of the zigbee network, however not known HA/ZHA, they might create problem? Im not sure.

One other idea, that I experienced early in my HA setup. The DECONZ supervisor should not be installed. Only ZHA. It created strange behavior when I had both installed.

#11

Thanks for the advice, I’ll try re-pairing the last few devices (hope I can remember all!).

Nothing deCONZ has ever been included in this current Home Assistant install. This is hassOS in VM that I’m in the process of migrating to from my old setup, which was just HA Core running in Docker (now stopped). That’s the install where deCONZ was used.

#12

I don’t use ConBee any longer but curious to know if ZHA can use “source routing” via zigpy-deconz?

https://github.com/zigpy/zigpy-deconz/issues/180

ZHA does support “source routing” for Silicon Labs and Texas Instruments Zigbee Coordinators but those are using different radio libraries for zigpy, and not sure if zigpy-deconz radio library support it?

Without having “source routing” enabled it will be much more limited with amount of Zigbee 3.0 devices that you can add to a Zigbee Coordinator as its Zigbee Routers is not off-loading size if routing tables.

https://github.com/dresden-elektronik/deconz-rest-plugin/wiki/Source-Routing

Off-topic but FYI, ZHA/zigpy developers are working on a tool that will allow you to backup Zigbee network from ConBee or RaspBee and restore to other brand Zigbee Coordinator with more resourses:

https://community.home-assistant.io/t/zha-libraries-will-soon-support-conbee-raspbee-backup-and-restore-that-can-be-used-for-zigbee-network-migrations/374782

I am not saying that ConBee or RaspBee are bad as Zigbee Coordinators but it is a fact that they use a MCU chip that has less RAM and flash storage and will therefore be much more limited in the amount of Zigbee 3.0 devices compared to Silicon Labs EFR32MG21 and Texas Instruments CC2652 chips.

I am not actually sure, it might be a limitation in zigpy or in Zigbee specifications. Suggest you ask here:

https://github.com/zigpy/zigpy/discussions

For reference; ZHA integration in Home Assistant depends on zigpy library and radio libraries for zigpy:

https://github.com/zigpy

Cool, however note and remember “Zigbee channels” and "Wi-Fi channels are not directly translatable:

https://support.metageek.com/hc/en-us/articles/203845040-ZigBee-and-WiFi-Coexistence

image
image726×597 42.2 KB

image
image714×498 17.2 KB

#13

Thanks for all the info. A bit to go through, but I think that as soon as there’s an easy way to backup/restore from a ConBeeII stick, maybe it is time for an upgrade.

Any recommendations? If using a Ethernet connected version (recently noticed this was an option, e.g. Tube’s PoE version), I could move it far away from any APs. Are there any real disadvantages of using those instead of simple USB connections?

Exactly why I really liked my old (deCONZ days) setup using WiFi channels 1, and 7 (no channel 11 in use), and ZigBee Channel 25. As soon as I also used an extension USB, I hardly had any issues for 2+ years.

Of course, as soon as I tried WiFi channels 8 and 13, some of my ESP devices (e.g. openEVSE) started having issues, and I found that ESP devices don’t necessarily play nice with channels > 11 (though I haven’t done very thorough testing of this myself). So maybe that’s not the way to go…

#14

Texas Instruments CC2652P and Silicon Labs EFR32MG21 or EFR32MG24 are the most powerful and have the most RAM so can control more Zigbee 3.0 devices which security schemes used a lot of RAM.

I personally recommend CC2652P as the first choice today because both Home Assistant’s native ZHA integration and Zigbee2MQTT have great support for Texas Instruments CC2652 adapters (with latest Z-Stack Zigbee Coordinator NCP firmware), as the only reason for not recommending Silicon Labs EFR32MG2x (Silabs EFR32MG21 or later) as the first choice today is that while those work great in Home Assistant’s native ZHA integration they currently only have experimental support in Zigbee2MQTT (due to Zigbee2MQTT developers are few and they primarily use Texas Instruments CC2652 based adapters themselves), but that might of course change in the future.

The ZHA (and zigpy) developers have proven that Silicon Labs EFR32MG2x based adapter hardware and firmware are certainly at least as good as Texas Instruments CC2652 adapters as long as there are developers that actively use those themselves and fix any issues that arise via bug-reports, etc…

Also note that there are disadvantages of using a network-attached Zigbee coordinator instead of just a long USB extension cable because the serial protocol they need is not robust so need a stable connection, which is the reason why using a Zigbee coordinator over Wi-Fi is especially warned against, see ex:

https://www.home-assistant.io/integrations/zha#warning-about-wi-fi-based-zigbee-to-serial-bridgesgateways

and

https://www.zigbee2mqtt.io/advanced/remote-adapter/connect_to_a_remote_adapter.html

Btw, testers testing backup mentioned pull request confirmed that patches to backup ConBee to the new Open ZigBee Coordinator Backup Format required to restore backups to different stacks/solutions:

https://github.com/zigpy/zigpy-cli/pull/2

2022: Good hardware for HA?
#15

Great, the CC2652 is also what’s in more of the ones I’ve been looking at.

I’d never use WiFi for this, but is the “serial protocol robustness” also an issue for a reliable hardwired connection? I couldn’t see any claims of this? If using e.g. the PoE CC2652P2 coordinator I mentioned, I could then place it away from the server room (which does probably have one of the highest level of RF interference in the house).

Alternately, I’d probably just add another “Remote Home Assistant” on an RPi with an USB ZigBee controller and pass ZigBee entities. But that would still ultimately be an ethernet connected ZigBee controller, so it seems to me like the RPi and additional HA instance to manage should be redundant?

Good timing for me :slight_smile:

#16

That would be a much more stable solution because not tunnel Zigbee serial protocols over a network. Such a solution cannot be compared with serial connection to a “network-attached” Zigbee Coordinator. If you link a second Home Assistant instance with a “direct-attached” Zigbee Coordinator USB adapter then that would convert the serial protocol to MQTT or other protocol meant for network communication.

There is more discussion about it here → https://github.com/home-assistant/architecture/issues/246

There should not be any serious issues in practice if have a wired Ethernet “network-attached” Zigbee Coordinator connection over a local LAN. Many people in this community use wired Ethernet connected network-attached Zigbee Coordinator adapters like Tube’s Zigbee Gateway and ZigStar LAN Gateway without any problems at all. You can read many reports from their users in the HA community here:

https://community.home-assistant.io/t/zigstar-zigbee-coordinators-and-routers/338586

https://community.home-assistant.io/t/tubes-zb-coordinators-and-routers-was-zigbee-router-on-steroids/280896/

Also, note that based on the fact that there are fewer reports about it in the communities, Texas Instruments ZNP serial protocol is generally believed to have slightly more robust than the Silicon Labs EZSP (EmberZNet Serial Protocol) in the way that Silicon Labs based “WiFi-attached” Zigbee Coordinator connection is infamously known to be notoriously unstable when connected over a Wi-Fi network that is not extremely stable. Though to be fair there are today a lot more Silicon Labs WiFi-attached Zigbee Coordinator products on the market that has been hacked for use with ZHA and Zigbee2MQTT than there are Texas Instruments based WiFi-attached Zigbee Coordinator products).

I think that in practice the only common scenario that causes issues for even Texas Instruments based wired Ethernet “network-attached” Zigbee Coordinators is when the LAN/switch does down for whatever reason (like a locally blown fuse) and your ZHA/Z2M instance is still up, then it would be like pulling out the adapter during operation. That would be to the effect same as in a scenario where you would pull out a Zigbee Coordinator USB adapter without stopping ZHA/Z2M. In that case and the wired Ethernet “network-attached” Zigbee Coordinator comes back then you might need to restart your ZHA/Z2M instance afterwards if the serial protocol the Zigbee stack uses is not robust enough to recover.

To workaround that issue you could as a suggestion make sure that both your computer running ZHA/Z2M and the wired Ethernet “network-attached” Zigbee Coordinator are both powered by the same PoE (Power over Ethernet) switch as that way if one dies due to power loss then other also dies too.

#17

I’m following your trail as I have pretty much the same story. Conbee II started loosing it after the end of december update. I have +60 devices, using 3 ikea wifi range extenders, in a mix of ikea, xiomi and others. I sometimes wake up and 30% of the devices became offline during the night. Look at the image. The repeaters are sitting there, doing nothing (they were reactivated some 5 hours ago).

Problem is I’m not “that” techical. So eager to read the outcome of your discussions.

image
image1087×696 99.4 KB

#18

Well, my initial thoughts is that it’s an RF interference issue for me.

I’m basing this on the fact that

(1) It happened when I changed from deCONZ to ZHA, and didn’t change anything else, but this apparently results in an automatic change to ZigBee channel 15 (if you set up the integration from UI at least, there’s a workaround if you use yaml, and specify channel when setting up),

(2) my old setup with ZigBee channel 25 (well away from any WiFi interference from me or any neighbors) worked quite well, and

(3) I found a chance to disable the WiFi AP near the ConBeeII stick for a few days while I was home alone (for testing), and I saw a significant improvement in the ZigBee network; almost every device responds near-instantaneously, just like with the deCONZ setup. Only problem is that I still can’t pair a few sensors and a single light “in place”, but have to move them close to the coordinator for it to work (which is not recommended).

So not sure if this is any help to you. If it happened after a Hass update, then it’s probably not the same issue…

1 Like
#19

Hi, thanks for the answer - I’m about to try something but I’m not sure of the consequences… My setup is still fullly with Deconz. You’re saying I could try to change to channel 25 here? Ands I get a warning… is it “everything might collapse” or some devices might need re-pairing? Thanks

image
image612×557 29.2 KB

#20

I have changed my Conbee II channel to 25 too with 60+ devices without any issues.
The only thing that could happen (happened to a friend of mine) is that your endpoints (like door/window sensors) need to send a state to the coordinator. Simply open/close the window/door and it binds again.

#21

Honestly, I did that so long ago that I can’t quite remember what consequences there were. But everything has worked in that configuration for several years (with 60+ devices, mixed Philips Hue, IKEA, Xioami (both old and new “Aqara”), Danfoss).

Given that I don’t quite remember the consequences, I’m assuming it wasn’t too bad…