ZHA lost complete network and fails to initialize

Configuration
, ,
1

After a restart my ZHA integration failed to initialize. I tried all options in the reconfituration menu and reconfiguring the coordinator ended in “Success” but was still no success.
After one week of trying, it all of a sudden managed to initialize and my network was accessible again (although had lost references in HA and needed to update all scenes, scripts etc.). I was happy but not for long, because after the next restart of HA, the problems have returned.

Anyone that experience the same?

Here is a short part of the log of the exception, that is thrown.
Before this, an extremely huge number of retransmissions have occured with a CRC-error in return.

Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/bellows/ezsp/protocol.py", line 129, in command
    return await future
           ^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/zha/__init__.py", line 151, in async_setup_entry
    await zha_gateway.async_initialize()
  File "/usr/local/lib/python3.13/site-packages/zha/application/gateway.py", line 275, in async_initialize
    await self._async_initialize()
  File "/usr/local/lib/python3.13/site-packages/zha/application/gateway.py", line 258, in _async_initialize
    await self.application_controller.startup(auto_form=True)
  File "/usr/local/lib/python3.13/site-packages/zigpy/application.py", line 221, in startup
    await self.initialize(auto_form=auto_form)
  File "/usr/local/lib/python3.13/site-packages/zigpy/application.py", line 178, in initialize
    await self.start_network()
  File "/usr/local/lib/python3.13/site-packages/bellows/zigbee/application.py", line 203, in start_network
    await self.load_network_info(load_devices=False)
  File "/usr/local/lib/python3.13/site-packages/bellows/zigbee/application.py", line 299, in load_network_info
    can_burn_userdata_custom_eui64 = await ezsp.can_burn_userdata_custom_eui64()
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ezsp/__init__.py", line 421, in can_burn_userdata_custom_eui64
    return await self._get_mfg_custom_eui_64() is None
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ezsp/__init__.py", line 405, in _get_mfg_custom_eui_64
    data = await self.get_mfg_token(t.EzspMfgTokenId.MFG_CUSTOM_EUI_64)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ezsp/__init__.py", line 390, in get_mfg_token
    (value,) = await self.getMfgToken(tokenId=token)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ezsp/__init__.py", line 208, in _command
    return await command(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ezsp/protocol.py", line 128, in command
    async with asyncio_timeout(EZSP_CMD_TIMEOUT):
               ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/asyncio/timeouts.py", line 116, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2

A larger part of the logfile:

2024-12-06 17:23:38.147 DEBUG (bellows.thread_0) [bellows.ash] Sending frame AckFrame(res=0, ncp_ready=0, ack_num=1) + FLAG
2024-12-06 17:23:38.147 DEBUG (bellows.thread_0) [bellows.ash] Sending data  8160597e
2024-12-06 17:23:38.149 DEBUG (MainThread) [bellows.ezsp.protocol] Received command getConfigurationValue: {'status': <EzspStatus.SUCCESS: 0>, 'value': 5}
2024-12-06 17:23:38.150 DEBUG (MainThread) [bellows.ezsp.protocol] Sending command  getKey: () {'keyType': <EmberKeyType.CURRENT_NETWORK_KEY: 3>}
2024-12-06 17:23:38.151 DEBUG (bellows.thread_0) [bellows.ash] Sending frame DataFrame(frm_num=0, re_tx=False, ack_num=1, ezsp_frame=b'?\x00\x01j\x00\x03') + FLAG
2024-12-06 17:23:38.151 DEBUG (bellows.thread_0) [bellows.ash] Sending data  017d5d21a93e2a16bcea7e
2024-12-06 17:23:38.163 DEBUG (bellows.thread_0) [bellows.ash] Received data 7d317d5da1a93e2a15b1599786756c45e9ebd22ece36dc549dd4db0f62a9c67d5e3fa7ebcdde6f8fffc7dbd5d26999947e
2024-12-06 17:23:38.164 DEBUG (bellows.thread_0) [bellows.ash] Received frame DataFrame(frm_num=1, re_tx=0, ack_num=1, ezsp_frame=b'?\x80\x01j\x00\x00\x03\x00\x03\xccP\xc6\x10{\xa2N`\xe9\x9d1\x9a\xfa_&\xc9\x01 :\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
2024-12-06 17:23:38.164 DEBUG (bellows.thread_0) [bellows.ash] Sending frame AckFrame(res=0, ncp_ready=0, ack_num=2) + FLAG
2024-12-06 17:23:38.164 DEBUG (bellows.thread_0) [bellows.ash] Sending data  82503a7e
2024-12-06 17:23:38.169 DEBUG (MainThread) [bellows.ezsp.protocol] Received command getKey: {'status': <EmberStatus.SUCCESS: 0>, 'keyStruct': EmberKeyStruct(bitmask=<EmberKeyStructBitmask.KEY_HAS_SEQUENCE_NUMBER|KEY_HAS_OUTGOING_FRAME_COUNTER: 3>, type=<EmberKeyType.CURRENT_NETWORK_KEY: 3>, key=cc:50:c6:10:7b:a2:4e:60:e9:9d:31:9a:fa:5f:26:c9, outgoingFrameCounter=3809281, incomingFrameCounter=0, sequenceNumber=0, partnerEUI64=00:00:00:00:00:00:00:00)}
2024-12-06 17:23:38.172 DEBUG (MainThread) [bellows.ezsp.protocol] Sending command  getKey: () {'keyType': <EmberKeyType.TRUST_CENTER_LINK_KEY: 1>}
2024-12-06 17:23:38.175 DEBUG (bellows.thread_0) [bellows.ash] Sending frame DataFrame(frm_num=1, re_tx=False, ack_num=2, ezsp_frame=b'@\x00\x01j\x00\x01') + FLAG
2024-12-06 17:23:38.176 DEBUG (bellows.thread_0) [bellows.ash] Sending data  120221a93e2a14f3ce7e
2024-12-06 17:23:38.188 DEBUG (bellows.thread_0) [bellows.ash] Received data 2202a1a93e2a15a85995da3880891bc46827f14f27687464c50f63a9fc7d5e3fa7ebcddefa4e053924648e052e2a7e
2024-12-06 17:23:38.189 DEBUG (bellows.thread_0) [bellows.ash] Received frame DataFrame(frm_num=2, re_tx=0, ack_num=2, ezsp_frame=b'@\x80\x01j\x00\x00\x1a\x00\x01\x90\x1d*\xdc\x89\x8d\xf4i\xd6\xe4\xca\xa6\x13\xef8\xc9\x00 \x00\x00\x00\x00\x00\x00\x00\x95\xc1\xfa\xfe\xff\xb1\\l')
2024-12-06 17:23:38.189 DEBUG (bellows.thread_0) [bellows.ash] Sending frame AckFrame(res=0, ncp_ready=0, ack_num=3) + FLAG
2024-12-06 17:23:38.189 DEBUG (bellows.thread_0) [bellows.ash] Sending data  83401b7e
2024-12-06 17:23:38.191 DEBUG (MainThread) [bellows.ezsp.protocol] Received command getKey: {'status': <EmberStatus.SUCCESS: 0>, 'keyStruct': EmberKeyStruct(bitmask=<EmberKeyStructBitmask.KEY_HAS_OUTGOING_FRAME_COUNTER|KEY_HAS_PARTNER_EUI64|KEY_IS_AUTHORIZED: 26>, type=<EmberKeyType.TRUST_CENTER_LINK_KEY: 1>, key=90:1d:2a:dc:89:8d:f4:69:d6:e4:ca:a6:13:ef:38:c9, outgoingFrameCounter=8192, incomingFrameCounter=0, sequenceNumber=0, partnerEUI64=6c:5c:b1:ff:fe:fa:c1:95)}
2024-12-06 17:23:38.193 DEBUG (MainThread) [bellows.ezsp.protocol] Sending command  getCurrentSecurityState: () {}
2024-12-06 17:23:38.197 DEBUG (bellows.thread_0) [bellows.ash] Sending frame DataFrame(frm_num=2, re_tx=False, ack_num=3, ezsp_frame=b'A\x00\x01i\x00') + FLAG
2024-12-06 17:23:38.197 DEBUG (bellows.thread_0) [bellows.ash] Sending data  230321a93d2a8f3c7e
2024-12-06 17:23:38.205 DEBUG (bellows.thread_0) [bellows.ash] Received data 3303a1a93d2a154659018bdf54aa2315f005567e
2024-12-06 17:23:38.205 DEBUG (bellows.thread_0) [bellows.ash] Received frame DataFrame(frm_num=3, re_tx=0, ack_num=3, ezsp_frame=b'A\x80\x01i\x00\x00\xf4\x00\x95\xc1\xfa\xfe\xff\xb1\\l')
2024-12-06 17:23:38.206 DEBUG (bellows.thread_0) [bellows.ash] Sending frame AckFrame(res=0, ncp_ready=0, ack_num=4) + FLAG
2024-12-06 17:23:38.206 DEBUG (bellows.thread_0) [bellows.ash] Sending data  8430fc7e
2024-12-06 17:23:38.207 DEBUG (MainThread) [bellows.ezsp.protocol] Received command getCurrentSecurityState: {'status': <EmberStatus.SUCCESS: 0>, 'state': EmberCurrentSecurityState(bitmask=<EmberCurrentSecurityBitmask.GLOBAL_LINK_KEY|HAVE_TRUST_CENTER_LINK_KEY|TRUST_CENTER_USES_HASHED_LINK_KEY|96: 244>, trustCenterLongAddress=6c:5c:b1:ff:fe:fa:c1:95)}
2024-12-06 17:23:38.209 DEBUG (MainThread) [bellows.ezsp.protocol] Sending command  getMfgToken: () {'tokenId': <EzspMfgTokenId.MFG_CUSTOM_EUI_64: 12>}
2024-12-06 17:23:38.210 DEBUG (bellows.thread_0) [bellows.ash] Sending frame DataFrame(frm_num=3, re_tx=False, ack_num=4, ezsp_frame=b'B\x00\x01\x0b\x00\x0c') + FLAG
2024-12-06 17:23:38.211 DEBUG (bellows.thread_0) [bellows.ash] Sending data  340021a95f2a19caab7e
2024-12-06 17:23:38.218 DEBUG (bellows.thread_0) [bellows.ash] Received data 44a1a95f2a1d4da66bb5da55aa6dabb07e
2024-12-06 17:23:38.218 DEBUG (bellows.thread_0) [bellows.ash] Failed to parse frame bytearray(b'D\xa1\xa9_*\x1dM\xa6k\xb5\xdaU\xaam\xab\xb0')
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 473, in data_received
    frame = parse_frame(data)
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 353, in parse_frame
    return frame.from_bytes(data)
           ~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 188, in from_bytes
    control, data = cls._unwrap(data)
                    ~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 159, in _unwrap
    raise ParsingError(
    ...<2 lines>...
    )
bellows.ash.ParsingError: Invalid CRC bytes in frame bytearray(b'D\xa1\xa9_*\x1dM\xa6k\xb5\xdaU\xaam\xab\xb0'): expected 2bb8, got abb0
2024-12-06 17:23:38.222 DEBUG (bellows.thread_0) [bellows.ash] Sending frame CANCEL + NakFrame(res=0, ncp_ready=0, ack_num=4) + FLAG
2024-12-06 17:23:38.223 DEBUG (bellows.thread_0) [bellows.ash] Sending data  1aa4149e7e
2024-12-06 17:23:38.228 DEBUG (bellows.thread_0) [bellows.ash] Received data 4ca1a95f2a1d4da66bb5da55aa6d8cf57e
2024-12-06 17:23:38.228 DEBUG (bellows.thread_0) [bellows.ash] Failed to parse frame bytearray(b'L\xa1\xa9_*\x1dM\xa6k\xb5\xdaU\xaam\x8c\xf5')
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 473, in data_received
    frame = parse_frame(data)
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 353, in parse_frame
    return frame.from_bytes(data)
           ~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 188, in from_bytes
    control, data = cls._unwrap(data)
                    ~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 159, in _unwrap
    raise ParsingError(
    ...<2 lines>...
    )
bellows.ash.ParsingError: Invalid CRC bytes in frame bytearray(b'L\xa1\xa9_*\x1dM\xa6k\xb5\xdaU\xaam\x8c\xf5'): expected c0d3, got 8cf5
2024-12-06 17:23:38.233 DEBUG (bellows.thread_0) [bellows.ash] Sending frame CANCEL + NakFrame(res=0, ncp_ready=0, ack_num=4) + FLAG
2024-12-06 17:23:38.233 DEBUG (bellows.thread_0) [bellows.ash] Sending data  1aa4149e7e
2024-12-06 17:23:38.238 DEBUG (bellows.thread_0) [bellows.ash] Received data 4ca1a95f2a1d4da66bb5da55aa6d8cf57e
2024-12-06 17:23:38.239 DEBUG (bellows.thread_0) [bellows.ash] Failed to parse frame bytearray(b'L\xa1\xa9_*\x1dM\xa6k\xb5\xdaU\xaam\x8c\xf5')
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 473, in data_received
    frame = parse_frame(data)
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 353, in parse_frame
    return frame.from_bytes(data)
           ~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 188, in from_bytes
    control, data = cls._unwrap(data)
                    ~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 159, in _unwrap
    raise ParsingError(
    ...<2 lines>...
    )
bellows.ash.ParsingError: Invalid CRC bytes in frame bytearray(b'L\xa1\xa9_*\x1dM\xa6k\xb5\xdaU\xaam\x8c\xf5'): expected c0d3, got 8cf5
2024-12-06 17:23:38.242 DEBUG (bellows.thread_0) [bellows.ash] Sending frame CANCEL + NakFrame(res=0, ncp_ready=0, ack_num=4) + FLAG
2024-12-06 17:23:38.242 DEBUG (bellows.thread_0) [bellows.ash] Sending data  1aa4149e7e
2024-12-06 17:23:38.248 DEBUG (bellows.thread_0) [bellows.ash] Received data 4ca1a95f2a1d4da66bb5da55aa6d8cf57e
2024-12-06 17:23:38.248 DEBUG (bellows.thread_0) [bellows.ash] Failed to parse frame bytearray(b'L\xa1\xa9_*\x1dM\xa6k\xb5\xdaU\xaam\x8c\xf5')
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 473, in data_received
    frame = parse_frame(data)
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 353, in parse_frame
    return frame.from_bytes(data)
           ~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 188, in from_bytes
    control, data = cls._unwrap(data)
                    ~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 159, in _unwrap
    raise ParsingError(
    ...<2 lines>...
    )
bellows.ash.ParsingError: Invalid CRC bytes in frame bytearray(b'L\xa1\xa9_*\x1dM\xa6k\xb5\xdaU\xaam\x8c\xf5'): expected c0d3, got 8cf5
2024-12-06 17:23:38.252 DEBUG (bellows.thread_0) [bellows.ash] Sending frame CANCEL + NakFrame(res=0, ncp_ready=0, ack_num=4) + FLAG
2024-12-06 17:23:38.252 DEBUG (bellows.thread_0) [bellows.ash] Sending data  1aa4149e7e
2024-12-06 17:23:38.258 DEBUG (bellows.thread_0) [bellows.ash] Received data 4ca1a95f2a1d4da66bb5da55aa6d8cf57e
2024-12-06 17:23:38.258 DEBUG (bellows.thread_0) [bellows.ash] Failed to parse frame bytearray(b'L\xa1\xa9_*\x1dM\xa6k\xb5\xdaU\xaam\x8c\xf5')
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 473, in data_received
    frame = parse_frame(data)
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 353, in parse_frame
    return frame.from_bytes(data)
           ~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 188, in from_bytes
    control, data = cls._unwrap(data)
                    ~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ash.py", line 159, in _unwrap
    raise ParsingError(
    ...<2 lines>...
    )
bellows.ash.ParsingError: Invalid CRC bytes in frame bytearray(b'L\xa1\xa9_*\x1dM\xa6k\xb5\xdaU\xaam\x8c\xf5'): expected c0d3, got 8cf5
2024-12-06 17:23:38.262 DEBUG (bellows.thread_0) [bellows.ash] Sending frame CANCEL + NakFrame(res=0, ncp_ready=0, ack_num=4) + FLAG
2024-12-06 17:23:38.262 DEBUG (bellows.thread_0) [bellows.ash] Sending data  1aa4149e7e