I can conform both
1 Like
Noticed as well zigbee-herdsman have added initial support for “Backup” as merged, so believe that “Restore” should be the biggest feature still missing for Silicon Labs EZSP support in Zigbee2MQTT(?):
https://github.com/Koenkk/zigbee-herdsman/issues/319
TODO:
- Improving the stability work of the network
- Groups
- Touchlink
- Backup / Restore
- OTA update
- Tests
- ZGP (Zigbee Green Power)
I then guess that extending “Backup” and “Restore” interoperability in the zigbee-herdsman library will then also be another related big feature for migration compatibility in order to allow backup and restore between different manufacturers’ Zigbee Coordinator stacks in Zigbee2MQTT(?). That is, being able to backup a Texas Instruments-based Zigbee Coordinator with Z-Stack/ZNP firmware in Zigbee2MQTT and then restore from the backup files to a Silicon Labs based Zigbee Coordinator with EmberZNet/EZSP firmware in Zigbee2MQTT, or vice versa migrations, in order to update to a newer Zigbee Coordinator adapter inside the Zigbee2MQTT application.
The use case for such cross-manufacturer backup and compatibility feature in zigbee-herdsman is primarily wanted by Zigbee2MQTT users who currently have an old TI CC253x (CC2530/CC2531) based Zigbee Coordinator adapter and want to migrate to a newer EFR32MGxx (EFR32MG13/EFR32MG21) based Zigbee Coordinator adapter. Same for those with old Silabs EM358x /ETRX357US based Zigbee Coordinator adapter wanting to migrate to a newer TI CC2652/CC1352 based Zigbee Coordinator adapter.
So just to update to a newer Zigbee Coordinator adapter inside Zigbee2MQTT or IoBroker without having to re-pair or configure any devices.
I believe that both of those migration paths should be possible to achieve someday within Zigbee2MQTT? It relies on the “Open ZigBee Coordinator Backup Format” specs. for hardware-independent backups → GitHub - zigpy/open-coordinator-backup: Open Zigbee coordinator backup format
1 Like
Could you give me a step-by-step instruction how to do it?
I’m finding that the 9888010100045 dongle is better suited for larger Zigbee networks than the newer ZBDongle-E. Here’s my story;
The older dongle has an issue where once or twice a month it seems to disconnect and require a restart of HA to restore ZHA. It seems (seemed) the ZBDongle-E had become established over the past year so I decided to take the chance and order it. The migration to the new dongle was seamless using the migration tool in the ZHA integration.
But post migration is where the issues started to show… There were “Out of Memory” startup errors thrown by ZHA in the logs when the integration attempted to set CONFIG_APS_UNICAST_MESSAGE_COUNT to 64. I dropped it down to 32, but eventually was able to get it to 48 after flashing one of the community firmware images. The same errors were also true when trying to set CONFIG_ROUTE_TABLE_SIZE to 32. It turns out that 26 was as high as I could get that to go without throwing an error.
The show stopper was that ZHA started off good after a restart then the system eventually stopped responding altogether. The logs were full of messages of maximum concurrency reached like these…
DEBUG (MainThread) [zigpy .application] Max concurrency (48) reached, delaying request (3144 enqueued)
Thousands of messages queued up? I also noticed commands sent to devices 20-30 minutes prior would eventually work.
It’s worth noting that I kept the both Zigbee sticks in the same location, connected by a 12 foot shielded USB cable to a USB2 port with a ferrite choke on each end. WiFi interference is ruled out a scan shows no utilization on Channel 1 (Mesh uses Zigbee ch 12).
Tonight I migrated back to the 9888010100045 dongle and the concurrency messages still intermittent but the number of mesage enqueued was usually between 1-4.
DEBUG (MainThread) [zigpy .application] Max concurrency (64) reached, delaying request (1 enqueued)
Responsiveness is still a quite a bit laggy compared to the near instant response that I saw before the migration to the ZBDongle-E, but migrating back to the 9888010100045 stick mostly solved backlogged message queue.
I think part of that is driven by the fact the older dongle can do 64 concurrent messages where the E can only do 48, but that still doesn’t explain why the backlog of message went into the thousands with the E, but never goes higher than 3-4 on the older stick.
As a disclaimer, I have 214 Zigbee devices with about 70ish Zigbee outlets reporting power usage, so the network is busy.
Curious if anyone or any of the Zigbee devs might have additional thoughts or insights.
1 Like
Goal: To convert my Sonoff ZBDongle-E to router firmware to make it work as an Zigbee extender on my home assistant as a coordinator
I followed the instructions available on GitHub - Elelabs/elelabs-zigbee-ezsp-utility: Elelabs Zigbee EZSP Utility to perform firmware update on a range of Elelabs EZSP products as well as other generic EZSP adapters.
Can anybody help me with this?
I am able to successfully connect to the dongle, but getting an error:
$ python Elelabs_EzspFwUtility.py flash -f ncp-uart-sw_EZNet6.10.3_V1.0.1.gbl -p com7
2022/12/21 09:22:38 Elelabs_EzspFwUtility: Generic Zigbee EZSP adapter detected:
2022/12/21 09:22:38 Elelabs_EzspFwUtility: Firmware: 6.10.3-41
2022/12/21 09:22:38 Elelabs_EzspFwUtility: EZSP v8
2022/12/21 09:22:38 Elelabs_EzspFwUtility: Launch in bootloader mode
2022/12/21 09:22:48 Elelabs_EzspFwUtility: EZSP adapter in bootloader mode detected:
2022/12/21 09:22:48 Elelabs_EzspFwUtility: Gecko Bootloader v1.12.00
2022/12/21 09:22:49 Elelabs_EzspFwUtility: Successfully restarted into X-MODEM mode! Starting upload of the new firmware… DO NOT INTERRUPT(!)
send error: expected ACK; got b’\x18’ for block 1
send error: expected ACK; got b’\x18’ for block 1
send error: expected ACK; got b’\x18’ for block 1
send error: expected ACK; got b’\r’ for block 1
send error: expected ACK; got b’\n’ for block 1
send error: expected ACK; got b’S’ for block 1
send error: expected ACK; got b’e’ for block 1
send error: expected ACK; got b’r’ for block 1
send error: expected ACK; got b’i’ for block 1
send error: expected ACK; got b’a’ for block 1
send error: expected ACK; got b’l’ for block 1
send error: expected ACK; got b’ ’ for block 1
send error: expected ACK; got b’u’ for block 1
send error: expected ACK; got b’p’ for block 1
send error: expected ACK; got b’l’ for block 1
send error: expected ACK; got b’o’ for block 1
send error: expected ACK; got b’a’ for block 1
send error: NAK received 17 times, aborting.
2022/12/21 09:22:56 Elelabs_EzspFwUtility: Firmware upload failed. Please try a correct firmware image or restart in normal mode.
I believe that the old 9888010100045 (previous barebone Silabs EFR32MG21 based Zigbee 3.0 USB adapter from ITead uses the exact same SoC chip as the new ITead’s “Sonoff Zigbee 3.0 USB Dongle Plus V2” (model “ZBDongle-E”) so the symptoms you describe will likely be due to the firmware version and build configuration rather than the hardware, and you should, in theory, be able to flash a firmware image meant for the old 9888010100045 dongle onto the new ZBDongle-E adapter and vice versa, (but do not hold me to that). Best to ask and confirm firmware compatibility with ITead/Sonoff:
ZBDongle FW + issue reporting → https://github.com/itead/Sonoff_Zigbee_Dongle_Firmware/issues
Unofficial 9888010100045 FW + discussion → https://github.com/xsp1989/zigbeeFirmware/issues/
Other realted unofficial firmware discussions → https://github.com/grobasoz/zigbee-firmware/issues/
https://github.com/itead/Sonoff_Zigbee_Dongle_Firmware/issues/15
1 Like
Firstly, the file name you posted does not look like it is the Zigbee Router firmware image variant, see:
- https://github.com/itead/Sonoff_Zigbee_Dongle_Firmware/tree/master/Dongle-E/Router
- Z3RouterUSBDonlge_EZNet6.10.3_V1.0.0.gbl
Tip is then to use alternative flashing tools for flashing firmware Silicon Labs EFR32 based adapters:
- Elelabs Firmware Update Utility (multi platform Python based command line tool)
- puddly Universal Silicon Labs Flasher (multi platform Python based command line tool)
- agners silabs-flasher - Silicon Labs Firmware flashing utility (multi platform Python based command line tool)
- walthowd husbzb-firmware script (community maintained multi platform bash script)
- Manual Xmodem sending commands over a terminal console (note that any terminal application with “Xmodem(N)” send can be used)
- Silicon Labs Simplicity Studio included “Flash Programmer” (instructions) (can’t find your device? read below!)
- Additional programming options for Silicon Labs MCU based devices
1 Like
In theory. I’m gonna let this project go until after Christmas. The 988 is back in service and the mesh has settled down such that I hardly even see any concurrency messages in the logs now. All of the versions are variants of 6.10.3 with the 988 running the official iTead release. I tried iTeads official 6.10.3 firmware that shipped on the stick plus grobasozs’ 6.10.3 build.
I think the transmit power (-20dB) could actually be overpowering and causing interference to the network. I do not believe it’s possible to adjust the power in the config.
I have another theory that the high gain antenna on the E might be picking up interference through the antenna itself. The 988 dongle just has a negative gain trace antenna that should be considerably more “deaf” than the E. I may try another swap after Christmas is over.
I think that tx_power should be configurable for bellows config under zigpy_config in YAML? Ask here:
https://github.com/zigpy/bellows/
FYI, the 9888010100045 board hardware has no electromagnetic shielding (plus it is said by many in the community to have a poorly designed circuit board antenna) and it has become infamously known for being extremely sensitive to electromagnetic interference (a.k.a. EMF/EMI/RMI), especially on the receiving side. So far that many people want to lengths to try to add at least some basic DIY electromagnetic shielding to the board by wrapping everything on the 9888010100045 board (with the exception of the circuit board antenna) in first a layer of Kapton-tape (which is not conductive) and then on top of that a layer of conductive copper-tape or other conductive metallic-tape which is grounded to the USB plug to prevent EMF/EMI/RMI to affect the sensitive components on the board directly.
Regardless of which Zigbee Coordinator USB adapter you use be sure to read and follow the best practice tips in regard to avoiding sources of EMF/EMI/RMI by getting the Zigbee Coordinator USB adapter as far as possible away from all electronic devices/appliances/cables/wires/antennas, etc. by for starters using it via a longer shielded USB extension cable (tip is that thicker cables ussually are better shielded) connected to a USB 2.0 port or USB 2.0 hub (and absolutely not to a USB 3.0 port as USB 3.0 ports and USB 3.0 devices are infamously known for being a serious source of EMF/EMI/RMI that will very strongly affect all low-energy digital radios that uses 2.4GHz frequency band like Zigbee, Thread, and Bluetooth. See:
I took a quick look at the source code and didn’t spot anything to configure that but I will dive into that more after the holidays.
Mine is wrapped in kaptan tape covered with a layer of aluminum foil tape. The trace antenna is not covered, of course. Those type of antennas operate with negative gain meaning instead of amplifying power it’s attenuated. I don’t see anything in the trace antenna design that would cause it to pick up any more interference. I think that device is victim of poor shielding over its own components and thats where the noise is getting in. The Dongle-E with its external high gain antenna is more susceptible to receiving interference thats the 988 would be. It’s called front-end overload, and it’s essentially out-of-band RF (i.e. EMI) that raises the entire noise floor such that the receiver cannot discriminate the noise from the actual signal. I don’t see anything in terms of band-pass filtering on the 988, so that could be why this is happening, but looking at photos of the DongleE PCB I have a similar opinion on it as well.
As mentioned in an earlier post, I kept the both Zigbee sticks in the same location, connected by the 12 foot shielded USB cable to a USB2 port. I also installed a ferrite choke on each end to reduce any radiated interference picked up by the USB cable.
That said, 6 days of using the Dongle-E I could not stabilize the network after the transition. About 32 hours after going back to the 988 the mesh is stable and concurrency errors are gone.
I think the concurrency errors are a major clue. The E simply couldn’t process messages as fast as the 988… But I have no idea why. Thats the weird part of all this.
1 Like
Oh men, i’m experiencing so much issues with this dongle-e. I had a hell of a time moving everything from deConz to Z2M, because when there was a FW update oid for a device, other endpoint devices wouldn’t join (took me a while to draw this conclusion).
I have a few devices (Aqara multi sensor or door sensor) which cannot join on the same location as they did join with deConz (with same router locations). Some devices couldn’t join after being removed from deConz (mostly powerplugs). Had to rejoin them to deconz to try again. Even after rebooting Z2M and everything, they just wouldn’t join, after joining to deConz, removing and trying again to join to Z2M it started to join.
So after a very painful transition period I thought it was getting more stable, but now after a week or two, today most of the devices were offline. Rebooting Z2M solved most offline messages but my Aqara opple devices left the network without notification. It did this twice today. And nothing has changed in my meter cupboard.
I also often get red error messages saying it is trying to reconfigure aqara light sensors. This fails, but the devices seem to be working.
Example:
Failed to configure ‘Kantoor_light_lux’, attempt 1 (Error: Bind 0x04cf8cdf3c787550/1 msIlluminanceMeasurement from ‘0xe0798dfffe718659/1’ failed ({“address”:8795,“clusterId”:32801,“sequence”:35} after 10000ms) at Timeout._onTimeout (/app/node_modules/zigbee-herdsman/src/utils/waitress.ts:64:35) at listOnTimeout (node:internal/timers:559:17) at processTimers (node:internal/timers:502:7))
All in all i’m not really happy with this device.
I’m using a decent USB extension cable, connected to USB2.0. Z2M is 1.28.4 (which is the latest I’m assuming) and the FW i’m using is 6.10.3.0 build 297 on my dongle-e, also latest official supported. All connected to a HP T620 with Hassio, all latest version’s too (2022.12.8)
I honestly don’t know what I need to do to get this more reliable, should I switch to the dongle-p or even just push my conbee2 sick into z2m (and migrate everything over again). Or should I try a experimental built and if yes; which one is best?
For z2m, yes.
Or try ZHA, but even then in my limited testing, the 2652 still seems to perform better. I’m sure there are situations one may be better than the other. Maybe my device mix just prefers the TI chip.
If want to use the ZBDongle-E with Zigbee2MQTT then you should switch the Z2M dev branch (also known as as ”edge” version) and report issues to zigbee-herdsman instead of here as it still experiment in it in so know to be a bit unstable there.
I switched back from edge to stable because of the 1.28.4 update, it held some EZSP updates and hoped it would resolve my issues.
I just tried switching back to the EDGE, but it didn’t start. It tries starting but in the end nothing happens. I’ll look in to this next year deleting and reinstalling the edge solved it. So now back on EDGE.
With experimental FW I meant for the dongle, but as far as I can tell it is still recommended to stay on the official FW. Weird thing though, in the latter link of your post it says it supports touchlink, but as far as i can tell, it only supports this in experimental FW builds.
Thanks for you reply.
That is correct. Itead/Sonoff has confirmed that they missed enabling Touchlink in their currently available official firmware. So they are aware and next official should have it enabled, but for now you must use unofficial firmware if you absolutly need to have Touchlink today. Thus the hardware do support Touchlink but you need a firmware build with it enabled.
Hi All, I am completely new to HA and just started setting up my system a few days ago, but today I ran into an issue with this dongle. I apologize in advance, if this exact issue has been covered somewhere else in the forum, I tried my best to go through the seemingly relevant posts, but did not find the answer to my question. (found only this thread where the guy ended up with a clean install of HA)
I am running Home Assistant with the OS installation on a Linux VM (Oracle Virtualbox). When I plugged in the dongle initially, it discovered it automatically and I managed to configure the device flawlessly via ZHA. Today, however, I noticed that devices went offline and surely enough when I looked into devices & integrations I saw that the ZHA integration threw some error (PFA screenshot)
To go around this I thought I would remove and re-add the integration, but looks like the device is not getting auto-discovered. If I am trying to locate the hardware itself, looks like HA no longer sees it.
VM and even the host (W11 desktop) have been both restarted, dongle has been physically unplugged and plugged in again. Windows seems to discover the device, it also appears in VB, but strangely not in HA.
Sounds like a USB pass-through issue with the virtual machine in Oracle Virtualbox virtualization hypervisor and not an issue with Home Assistant operatingsystem.
Hopefully that will help you narrow down the problem but not much more we can help with that here as probably not issue specifically related to Home Assistant, as Home Assistant at least need to be able to see the dongle hardware as a serial port.
Better find a other forum that instead deal with configuring USB passthrough with Oracle Virtualbox.
Indeed that was the issue, thanks for the suggestion, I did look up some additional material on VM’s and USB passthrough, now this is sorted out!
Would you like to share how / what exactly you did to solve the USB passthrough issue?
I am browsing the forum, and looks like @rperkins ran into Oracal VirtualBox USB passthrough issue the other day… also with Virtualbox
… figured maybe you could help each other out. 
It’s been a couple weeks since I last posted about this issue. In the time since I have done extensive testing with the Dongle-E as have come to the conclusion that I do not recommend this adapter for more than an a handful of services due to a flaw with the UART. Here’s a recap…
-
I decided to replace the 988 dongle with the Dongle-E, in part because the 988 is no longer recommended for use with ZHA for stability reasons. I used the ZHA migration tool to flawlessly migrate to the Dongle-E, and flashed the recommended 6.1.3 firmware prior to migration. After migrating to the Dongle-E ZHA would gradually slow down and then stop responding altogether. I gave up after 24 hours.
-
The migration tool was used again to migrate back to the 988 dongle, and again the transition was flawless. Within minutes of completing the migration ZHA was once again stable. I left it this way through Christmas.
-
Following the holidays, I once again used the migration tool to switch over to the Dongle-E. Before attempting this, I flashed one of the community firmware builds. This time I replaced the 12 foot USB extension and used a different USB port on the server, just in case either of these could be a factor. Still no change. Within an hour or two ZHA was barely responsive (10-20 second lag). To rule out interference I used a USB over CAT6 extender to move the Dongle-E about 30 feet from the servers. Still no change. HA logs were full of concurrency errors but nothing else.
-
The migration too was used once again to migrate back to the 988, and again within a few minutes the Zigbee network stabilized.
-
Thinking ZHA might be at fault, I stood up an instance of Zigbee2Mqtt on a spare Raspberry PI 4 using the Dongle-E on the 12 foot USB extension. With this option, I placed the PI in my Attic, 3 floors above the HA server and began to migrate some devices from ZHA to Z2M on the Pi for testing. Serious issues were present right away with Z2M crashing during the interview process. At the time I dismissed that as a facet of the experimental EZSP support. It wasn’t until about 40 devices were moved over to Z2M when I started noticing Z2M crashing every few hours. The symptoms of this were quite obvious with Zigbee outlets momentarily cycling off and on at random times.
-
The next step in the process was to build a similar test bed using the Dongle-P with a second instance of Z2M on the Raspberry Pi using a different Zigbee channel. Another 50 devices of a similar mix of device type swere moved from ZHA to the Z2M instance using the Dongle-P, and another 10 to the Z2M instance using the Dongle-E. (By this point I was sick of re-pairing devices). The instance with the Dongle-P was rock solid, while the Z2M instance with the Dongle-E frequently crashed.
-
Herdsman logging was enabled and a console session was left running to capture any crash data. That did not take long. The logs were revealing.
zigbee2mqtt_ch12 | 2023-01-05T03:36:22.222Z zigbee-herdsman:adapter:ezsp:ezsp Close ezsp
zigbee2mqtt_ch12 | 2023-01-05T03:36:22.222Z zigbee-herdsman:adapter:ezsp:driv Close driver
zigbee2mqtt_ch12 | 2023-01-05T03:36:22.223Z zigbee-herdsman:adapter:ezsp:erro Reset error Error: Error while closing serialport 'Error: Port is not open'
zigbee2mqtt_ch12 | at SerialPort.<anonymous> (/app/node_modules/zigbee-herdsman/src/adapter/ezsp/driver/uart.ts:314:40)
zigbee2mqtt_ch12 | at SerialPort._error (/app/node_modules/@serialport/stream/dist/index.js:76:22)
zigbee2mqtt_ch12 | at /app/node_modules/@serialport/stream/dist/index.js:83:37
zigbee2mqtt_ch12 | at processTicksAndRejections (node:internal/process/task_queues:77:11)
zigbee2mqtt_ch12 | 2023-01-05T03:36:22.224Z zigbee-herdsman:adapter:ezsp:driv Pause 10sec before try 1
zigbee2mqtt_ch12 | 2023-01-05T03:36:22.225Z zigbee-herdsman:adapter:ezsp:ezsp Close ezsp
zigbee2mqtt_ch12 | 2023-01-05T03:36:22.225Z zigbee-herdsman:adapter:ezsp:driv Close driver
zigbee2mqtt_ch12 | 2023-01-05T03:36:22.226Z zigbee-herdsman:adapter:ezsp:erro Reset error Error: Error while closing serialport 'Error: Port is not open'
zigbee2mqtt_ch12 | at SerialPort.<anonymous> (/app/node_modules/zigbee-herdsman/src/adapter/ezsp/driver/uart.ts:314:40)
zigbee2mqtt_ch12 | at SerialPort._error (/app/node_modules/@serialport/stream/dist/index.js:76:22)
zigbee2mqtt_ch12 | at /app/node_modules/@serialport/stream/dist/index.js:83:37
zigbee2mqtt_ch12 | at processTicksAndRejections (node:internal/process/task_queues:77:11)
Following the burst of errors in the log Z2M would most of the time, automatically restart and reconnect to the dongle without any issue. On a couple occasions I had to restart the docker container to get Z2M to come back up. All the while, the Z2M instance with the Dongle-P was rock solid.
In conclusion I believe there is a flaw in the UART chip on the Dongle-E. I believe this was also the cause of the Dongle-E’s inability to reliably process the message traffic when used with ZHA, Those Concurrency Errors were essentially queued commands that Zigpy could not offload to the radio, so they were queued for future delivery. It’s quite possible that the dongle is just plain defective, but the stability of the device seemed to be inversely proportional to the number of devices on the network.
A second Dongle-P will be arriving today. It will replace the Dongle-E used on the Z2M instance. If that bring stability I’ll stay with the bifurcated Zigbee networks and move the remaining devices off of ZHA. It’s been a lot of work, but I am impressed with the stability and responsiveness of Z2M with the Dongle-P.
Did I mention I’m tired of pairing devices?
2 Likes