Migrating zigbee devices from sonoff dongle E

sapstar · October 4, 2023, 7:01am

This is not going well. I flashed the new firmware using the silicon labs web flasher. But the addon is stuck in an endless loop with the below error.

[07:58:43:020727] *** ASSERT *** : FATAL in function 'capabilities_checks' in file /usr/src/cpc-daemon/server_core/server_core.c at line #714 : UART flow control configuration mismatch between CPCd (enabled) and Secondary (disabled)
[07:58:44:021432] Info : Daemon exiting with status EXIT_FAILURE
Logger buffer size = 28672, highwater mark = 2471 : 8.62%. Lost logs : 0
[07:58:44] INFO: CPC ended with exit code 1 (signal 0)...
[07:58:45] INFO: Starting cpcd...
WARNING in function 'main' in file /usr/src/cpc-daemon/main.c at line #186 : Running CPCd as 'root' is not recommended. Proceed at your own risk.
FATAL in function 'capabilities_checks' in file /usr/src/cpc-daemon/server_core/server_core.c at line #714 : UART flow control configuration mismatch between CPCd (enabled) and Secondary (disabled)
[07:58:45:072315] Info : [CPCd v4.3.1.0] [Library API v3] [RCP Protocol v4]
[07:58:45:072360] Info : Git commit: 133b29678b3d0bc7578e098d2f46b4d5bcd2ebb4 / branch: 
[07:58:45:072361] Info : Sources hash: ff8300587e7e4ab1def7a89a272c0baef32f9eb3bff9b0ba06b94e655d652367
[07:58:45:072363] WARNING : In function 'main' in file /usr/src/cpc-daemon/main.c at line #186 : Running CPCd as 'root' is not recommended. Proceed at your own risk.

I also tried reflashing and didn’t make any difference. I looked up online to see one person who had this issue had firmware updater disabled which caused the problem. So I made sure it’s enabled. But no change.

francisp · October 4, 2023, 7:20am

Try to use this add-on

Worked for me (but not using a Dongle E but a zb-gw04-1v2)

sapstar · October 4, 2023, 7:55am

Thanks for your help @francisp. I was able to get this working after disabling the Hardware flow control option which was selected by default in the multi protocol addon.

Below are the final steps for anyone going through this process.

Intall the multi protocol addon but don’t configure and start it.
Start migrate radio on ZHA.
When it tells you to disconnect old radio, then disconnect and flash it with multi pan firmware.
Leave this screen as it is and open a new home assistant window.
Connect the upgraded radio.
Now configure multi protocol addon. Make sure you disable Hardware flow control and it is fully started.
Go back to your ZHA migration screen and continue migration and select the new silicon labs device that is detected.

If everything goes well your device should be migrated.

sapstar · October 4, 2023, 8:03am

I see my ZHA is running on channel 25. Where do I change channel for Matter? When I tried changing it in the Thread configuration it tells me I have to do it in hardware settings menu. Not sure where this is.

francisp · October 4, 2023, 8:09am

The hardware settings menu is only available for the Skyconnect dongle and the HA Yellow

See this issue for work-arounds :

github.com/home-assistant/addons

Multiprotocol consistently mismatches channels

opened 06:50PM - 29 Jun 23 UTC

Jonathanese

add-on: silabs-multiprotocol

### Describe the issue you are experiencing Addon: Silicon Labs Multiprotocol …Version: 2.2.0 Device: Sonoff Zigbee Dongle-E Firmware: RCP Multi-Protocol Every time I set up the SiLabs multiprotocol plugin and integrations, I immediately get an error saying the channels are mismatched. It sets the Zigbee channel to 25, and the Thread channel to 15. I try to change the Thread channel by running Form on the OTBR page, setting it to 25. This tends to get rid of the error, but I can't pair anything via Matter. It finds the device, connects, generates the credentials, sends the credentials, then checks the connection. And it always fails at that last step, which I assume is a Thread issue because of a new network which perhaps the rest of the system hasn't detected. I try to change the Zigbee channel, but rather than letting me change it, it says "This is configured by the Multiprotocol addon and must be changed in the hardware page". But there's no hardware page to set this. So it basically configures the channels wrong and prevents me from fixing it. ### What type of installation are you running? Home Assistant OS ### Which operating system are you running on? Home Assistant Operating System ### Which add-on are you reporting an issue with? Silicon Labs Multiprotocol ### What is the version of the add-on? 2.2.0 ### Steps to reproduce the issue 1. Flash Dongle-E using the online flashing tool. Use the RCP Multiprotocol firmware. 2. Install the Multiprotocol addon. 460kbaud. Software flow control. OTBR enabled. Ports enabled. 3. Point ZHA to the multiprotocol virtual device 4. Thread and OTBR are auto-detected 5. Reboot Home Assistant OS to re-run detection 6. <Mismatched Channel" error usually occurs here as a settings notification> 7. In ZHA configuration, try to change channel. Prompt tells you you can't do so. References a page in "Hardware" that doesn't exist. **Try matter** 8. Install Matter addon. Autodetected/Auto-Added to integrations. 9. On phone, try to add Nanoleaf A12 bulb. 10. Sequence: Finding Device -> Connecting to device -> Generating Matter Credentials -> Testing connection -> Connection failed **Try Change Channel** 11. Open OTBR page 12. Go to "Form", and configure network for channel 25 with the name home-assistant. Defaults elsewhere 13. Click "Form" 14. After a bit, or perhaps a refresh, in the Thread integration -> Configure page, a second home-assistant network shows up with a border router. This time saying channel 25 instead of channel 15. Set this as the preferred network. 15. Retry the matter settings. Same failure condition. ... ### System Health information Raspberry PI 4 Sonoff Zigbee Dongle-E ### Anything in the Supervisor logs that might be useful for us? _No response_ ### Anything in the add-on logs that might be useful for us? _No response_ ### Additional information _No response_

francisp · October 4, 2023, 8:20am

Does

settings → devices & services → thread → configure → 3 dots → Change channel

not work ?

sapstar · October 4, 2023, 9:01am

I did the same as your above screenshot, but I got a popup which said I have to change the channel in hardware settings.

But anyways, my happiness is shortlived. The Zigbee network went down and there are a lot of errors in the addon repeating while zha is trying to connect to the coordinator.

[09:57:27:999848] Info : Connecting to Secondary...
[09:57:30:000035] Info : Failed to connect, secondary seems unresponsive
[09:57:30:000045] Info : Connecting to Secondary...
[09:57:32:000446] Info : Failed to connect, secondary seems unresponsive
[09:57:32:000457] Info : Connecting to Secondary...
[09:57:34:000782] Info : Failed to connect, secondary seems unresponsive
[09:57:34:000792] Info : Connecting to Secondary...
[09:57:36:001146] Info : Failed to connect, secondary seems unresponsive
[09:57:36:001156] Info : Connecting to Secondary...
[09:57:38:001264] Info : Failed to connect, secondary seems unresponsive
[09:57:38:001271] Info : Connecting to Secondary...
[09:57:40:001344] Info : Failed to connect, secondary seems unresponsive

I can also see timeouts in the homeassistant logs.

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 209, in async_initialize
    await self.application_controller.startup(auto_form=True)
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 201, in startup
    await self.shutdown()
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 406, in shutdown
    await self._dblistener.shutdown()
  File "/usr/local/lib/python3.11/site-packages/zigpy/appdb.py", line 187, in shutdown
    await self._set_isolation_level(None)
  File "/usr/local/lib/python3.11/site-packages/zigpy/appdb.py", line 205, in _set_isolation_level
    await self._db._execute(lambda: setattr(self._db, "isolation_level", level))
  File "/usr/local/lib/python3.11/site-packages/aiosqlite/core.py", line 126, in _execute
    raise ValueError("Connection closed")
ValueError: Connection closed
2023-10-04 09:56:46.035 ERROR (MainThread) [zigpy.application] Couldn't start application
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 197, in startup
    await self.connect()
  File "/usr/local/lib/python3.11/site-packages/bellows/zigbee/application.py", line 141, in connect
    await ezsp.startup_reset()
  File "/usr/local/lib/python3.11/site-packages/bellows/ezsp/__init__.py", line 159, in startup_reset
    await self.reset()
  File "/usr/local/lib/python3.11/site-packages/bellows/ezsp/__init__.py", line 185, in reset
    await self._gw.reset()
TimeoutError
2023-10-04 09:56:46.037 WARNING (MainThread) [homeassistant.components.zha.core.gateway] Couldn't start EZSP = Silicon Labs EmberZNet protocol: Elelabs, HUSBZB-1, Telegesis coordinator (attempt 3 of 3)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 197, in startup
    await self.connect()
  File "/usr/local/lib/python3.11/site-packages/bellows/zigbee/application.py", line 141, in connect
    await ezsp.startup_reset()
  File "/usr/local/lib/python3.11/site-packages/bellows/ezsp/__init__.py", line 159, in startup_reset
    await self.reset()
  File "/usr/local/lib/python3.11/site-packages/bellows/ezsp/__init__.py", line 185, in reset
    await self._gw.reset()
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 209, in async_initialize
    await self.application_controller.startup(auto_form=True)
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 201, in startup
    await self.shutdown()
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 406, in shutdown
    await self._dblistener.shutdown()
  File "/usr/local/lib/python3.11/site-packages/zigpy/appdb.py", line 187, in shutdown
    await self._set_isolation_level(None)
  File "/usr/local/lib/python3.11/site-packages/zigpy/appdb.py", line 205, in _set_isolation_level
    await self._db._execute(lambda: setattr(self._db, "isolation_level", level))
  File "/usr/local/lib/python3.11/site-packages/aiosqlite/core.py", line 126, in _execute
    raise ValueError("Connection closed")
ValueError: Connection closed
2023-10-04 09:57:10.276 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry SONOFF Zigbee 3.0 USB Dongle Plus V2 for zha
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 197, in startup
    await self.connect()
  File "/usr/local/lib/python3.11/site-packages/bellows/zigbee/application.py", line 141, in connect
    await ezsp.startup_reset()
  File "/usr/local/lib/python3.11/site-packages/bellows/ezsp/__init__.py", line 159, in startup_reset
    await self.reset()
  File "/usr/local/lib/python3.11/site-packages/bellows/ezsp/__init__.py", line 185, in reset
    await self._gw.reset()
TimeoutError

Now I am trying to find a way to go back to the old firmware.

francisp · October 4, 2023, 9:13am

If you really want to use thread, a nRF52840 dongle costs about €12, is a thread certified component, and works well with HA.

https://openthread.io/vendors/nordic-semiconductor

sapstar · October 4, 2023, 10:16am

Migrated back to the old firmware and was able to restrore the previous backup.

Looks like it is difficult to source a nRF52840 dongle in the UK. I may get another dongle E and flash it with a matter firmware instead of a multi pan.

PS: Found this one which is nRF52840.

darkxst · October 5, 2023, 3:13am

This indicates that multiprotocol can’t connect to the dongle, usually caused by wrong settings or bad USB connection. However very occasionally the dongle might hang and require unplug/replug (this mostly seems to happen in VM’s )

sapstar · October 5, 2023, 5:38am

Thanks very much for your response. Yes I am running this in a VM, but not sure if this could be the USB connection. I have re flashed the firmware twice by disconnecting and reconnecting the usb and saw the same errors. At this time I also started my cpu spiking to 50% which is usually around 3%.

My settings are below which from what I have read online should be right for this dongle.

selected the right usb adapter.
baud rate 460800
disabled hardware flow control
disabled auto update
Rest are defaults

When I switched back to zigbee firmware, it immediately started working.

I will give it one more try.

sapstar · October 5, 2023, 7:05am

Tried one more time today. I have not seen the failed to connect error this time, but the behaviour is the same. After restart zigbee wouldn’t connect to the new coordinator.

I did see a message to change my zigbee and thread channels to match. Below is the help link provided.
https://skyconnect.home-assistant.io/about-multiprotocol/#resolve-channel-conflict

I couldn’t change the channel in both zha and thread settings. Below is the message I get for thread, but I get a similar message when tried to change the channel in zha.

So switched back to zigbee firmware agian. I don’t think I will try this experimental feature on my only zigbee adapter. As I can order a ZB-GW04 for about £6, I will buy one to experiment.

darkxst · October 5, 2023, 7:17am

Did you reconfigure ZHA to connect to silabs multiprotocol? via socket://core-silabs-multiprotocol:9999

There is no easy solution to the channel mismatch right now, but you can change the Thread channel to match Zigbee via the api

github.com/home-assistant/addons

Multiprotocol consistently mismatches channels

opened 06:50PM - 29 Jun 23 UTC

Jonathanese

add-on: silabs-multiprotocol

### Describe the issue you are experiencing Addon: Silicon Labs Multiprotocol …Version: 2.2.0 Device: Sonoff Zigbee Dongle-E Firmware: RCP Multi-Protocol Every time I set up the SiLabs multiprotocol plugin and integrations, I immediately get an error saying the channels are mismatched. It sets the Zigbee channel to 25, and the Thread channel to 15. I try to change the Thread channel by running Form on the OTBR page, setting it to 25. This tends to get rid of the error, but I can't pair anything via Matter. It finds the device, connects, generates the credentials, sends the credentials, then checks the connection. And it always fails at that last step, which I assume is a Thread issue because of a new network which perhaps the rest of the system hasn't detected. I try to change the Zigbee channel, but rather than letting me change it, it says "This is configured by the Multiprotocol addon and must be changed in the hardware page". But there's no hardware page to set this. So it basically configures the channels wrong and prevents me from fixing it. ### What type of installation are you running? Home Assistant OS ### Which operating system are you running on? Home Assistant Operating System ### Which add-on are you reporting an issue with? Silicon Labs Multiprotocol ### What is the version of the add-on? 2.2.0 ### Steps to reproduce the issue 1. Flash Dongle-E using the online flashing tool. Use the RCP Multiprotocol firmware. 2. Install the Multiprotocol addon. 460kbaud. Software flow control. OTBR enabled. Ports enabled. 3. Point ZHA to the multiprotocol virtual device 4. Thread and OTBR are auto-detected 5. Reboot Home Assistant OS to re-run detection 6. <Mismatched Channel" error usually occurs here as a settings notification> 7. In ZHA configuration, try to change channel. Prompt tells you you can't do so. References a page in "Hardware" that doesn't exist. **Try matter** 8. Install Matter addon. Autodetected/Auto-Added to integrations. 9. On phone, try to add Nanoleaf A12 bulb. 10. Sequence: Finding Device -> Connecting to device -> Generating Matter Credentials -> Testing connection -> Connection failed **Try Change Channel** 11. Open OTBR page 12. Go to "Form", and configure network for channel 25 with the name home-assistant. Defaults elsewhere 13. Click "Form" 14. After a bit, or perhaps a refresh, in the Thread integration -> Configure page, a second home-assistant network shows up with a border router. This time saying channel 25 instead of channel 15. Set this as the preferred network. 15. Retry the matter settings. Same failure condition. ... ### System Health information Raspberry PI 4 Sonoff Zigbee Dongle-E ### Anything in the Supervisor logs that might be useful for us? _No response_ ### Anything in the add-on logs that might be useful for us? _No response_ ### Additional information _No response_

Other work arounds are also mentioned in that thread.

sapstar · October 5, 2023, 8:31am

Thank you very much for the info. Yes my settings are correct. I picked the right adapter socket://core-silabs-multiprotocol:9999. As soon as the migration is done it did work and was able to control my zigbee devices. Only breaks after a restart of home assistant.

I will try again once I get my new adapter.

sapstar · October 12, 2023, 8:30pm

@darkxst I got the ZB-GW04 1.2 today and flashed the Openthread RCP using the web flasher. Flashing was successful, but after this homeassistant wasn’t able to find it as a border router.

I tried to use the web flasher and the connection was failing this time with the below logs.

2023-10-12 18:26:17 emscripten universal_silabs_flasher.flasher[42] INFO Probing ApplicationType.EZSP at 115200 baud
2023-10-12 18:26:17 emscripten zigpy.serial[42] DEBUG Opening a serial connection to '/dev/webserial' (115200 baudrate)
2023-10-12 18:26:17 emscripten bellows.ezsp[42] DEBUG Resetting EZSP
2023-10-12 18:26:17 emscripten bellows.uart[42] DEBUG Resetting ASH
2023-10-12 18:26:17 emscripten bellows.uart[42] DEBUG Sending: b'1ac038bc7e'
2023-10-12 18:26:22 emscripten bellows.uart[42] DEBUG Connection lost: None
2023-10-12 18:26:22 emscripten bellows.uart[42] DEBUG Closed serial connection
2023-10-12 18:26:22 emscripten universal_silabs_flasher.flasher[42] INFO Probing ApplicationType.SPINEL at 460800 baud
2023-10-12 18:26:22 emscripten zigpy.serial[42] DEBUG Opening a serial connection to '/dev/webserial' (460800 baudrate)
2023-10-12 18:26:22 emscripten universal_silabs_flasher.common[42] DEBUG Connection made: <webserial_transport.WebSerialTransport object at 0x18e29c8>
2023-10-12 18:26:22 emscripten universal_silabs_flasher.spinel[42] DEBUG Sending frame SpinelFrame(header=SpinelHeader(transaction_id=3, network_link_id=0, flag=2), command_id=<CommandID.PROP_VALUE_GET: 2>, data=b'\x02')
2023-10-12 18:26:22 emscripten universal_silabs_flasher.common[42] DEBUG Sending data b'~\x83\x02\x02\xe65~'
2023-10-12 18:26:23 emscripten universal_silabs_flasher.spinel[42] DEBUG Failed to send SpinelFrame(header=SpinelHeader(network_link_id=0, flag=2), command_id=<CommandID.PROP_VALUE_GET: 2>, data=b'\x02'), trying again in 0.10s (attempt 1 of 4)
2023-10-12 18:26:23 emscripten universal_silabs_flasher.spinel[42] DEBUG Sending frame SpinelFrame(header=SpinelHeader(transaction_id=3, network_link_id=0, flag=2), command_id=<CommandID.PROP_VALUE_GET: 2>, data=b'\x02')
2023-10-12 18:26:23 emscripten universal_silabs_flasher.common[42] DEBUG Sending data b'~\x83\x02\x02\xe65~'
2023-10-12 18:26:24 emscripten universal_silabs_flasher.spinel[42] DEBUG Failed to send SpinelFrame(header=SpinelHeader(network_link_id=0, flag=2), command_id=<CommandID.PROP_VALUE_GET: 2>, data=b'\x02'), trying again in 0.10s (attempt 2 of 4)
2023-10-12 18:26:24 emscripten universal_silabs_flasher.spinel[42] DEBUG Sending frame SpinelFrame(header=SpinelHeader(transaction_id=3, network_link_id=0, flag=2), command_id=<CommandID.PROP_VALUE_GET: 2>, data=b'\x02')
2023-10-12 18:26:24 emscripten universal_silabs_flasher.common[42] DEBUG Sending data b'~\x83\x02\x02\xe65~'
2023-10-12 18:26:26 emscripten universal_silabs_flasher.spinel[42] DEBUG Failed to send SpinelFrame(header=SpinelHeader(network_link_id=0, flag=2), command_id=<CommandID.PROP_VALUE_GET: 2>, data=b'\x02'), trying again in 0.10s (attempt 3 of 4)
2023-10-12 18:26:26 emscripten universal_silabs_flasher.spinel[42] DEBUG Sending frame SpinelFrame(header=SpinelHeader(transaction_id=3, network_link_id=0, flag=2), command_id=<CommandID.PROP_VALUE_GET: 2>, data=b'\x02')
2023-10-12 18:26:26 emscripten universal_silabs_flasher.common[42] DEBUG Sending data b'~\x83\x02\x02\xe65~'
2023-10-12 18:26:27 emscripten universal_silabs_flasher.spinel[42] DEBUG Failed to send SpinelFrame(header=SpinelHeader(network_link_id=0, flag=2), command_id=<CommandID.PROP_VALUE_GET: 2>, data=b'\x02'), trying again in 0.10s (attempt 4 of 4)

I then tried the universal-silabs-flasher python script and that was not able to probe as well. I was finally able to flash a zigbee firmware using SecureCRT. After this, the web flasher detects it again and able to flash the Openthread firmware. This was again bricked. Flashed Zigbee firmware again with SecureCRT and the tried the Multi PAN firmware. This seems to work fine. After flashing multi pan, The web flasher can still connect to the device.

Now I have multi pan firmware running, but will only using this stick for Thread.

Wondering if there could be a bug with the Openthread flashing with the web flasher…

darkxst · October 13, 2023, 12:06am

It is likely to do with baudrates
Web flasher for ZB-GW04 uses 230400 baudrate for Openthread and Multipan firmwares. This device simply cant handle any faster baudrates even with the hardware flow control in v1.2.
Universal-silabs-flasher by default only probes Spinel (Openthread) for 460800 though, this was also the default in webflasher. I just pushed a commit to fix probing of web flasher in this case. If you are using universal-silabs-flasher directly you need to specify the baudrate for spinel manually. Also when you setup OTBR, make sure you select 230400 baudrate.

sapstar · October 13, 2023, 12:26pm

Just tried again with the web flasher. Still the same issue. I get an installation successful after flashing openthread. After this, the device is basically bricked. Can only be reflashed using secureCRT.

Once flashed the zigbee firmware with secureCRT again the web flasher can detect it.

darkxst · October 13, 2023, 1:36pm

I doubt its bricked, but if baudrates dont match, it wont probe.

Btw are you using this link for web flasher (I havent updated the older sl-test links)

After flashing the Openthread firmware try this command
universal-silabs-flasher --device /dev/ttyUSB0 --spinel-baudrate 230400 probe

sapstar · October 16, 2023, 7:33pm

@darkxst u are right. I was using the sl-test webpage. I tried from the above provided link and it works this time. After flashing the openthread firmware, I can still connect to the device.Thanks very much for your help.

I have now connected the flashed usb to Home Assistant and was expecting to see a open thread border router, but don’t see any in home assistant. Is this the expected outcome? When I had the Multi protocol firmware and the silicon labs addon, I was seeing a border router.

sapstar · October 16, 2023, 8:20pm

All sorted now. I had to manually install the Openthread border router add on .