ZHA/EZSP gateway (lidl) can't initialize after update to HA 24.12.0 (XncpCommandId.undefined_0x03e8: 1000)

Hi,

After update to HA 24.12.0 my Lidl silvercrest zigbee gateway (hacked to use ZHA/EZSP) can’t initialize because of <XncpCommandId.undefined_0x03e8: 1000>. Complete traceback:

2024-12-05 09:28:48.260 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/aiohttp/web_protocol.py", line 480, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/aiohttp/web_app.py", line 569, in _handle
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 92, in security_filter_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 210, in forwarded_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 26, in request_context_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 86, in ban_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 242, in auth_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 32, in headers_middleware
    response = await handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/http.py", line 73, in handle
    result = await handler(request, **request.match_info)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 81, in with_admin
    return await func(self, request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 285, in post
    return await super().post(request, flow_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 74, in wrapper
    return await method(view, request, data, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 122, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 367, in async_configure
    result = await self._async_configure(flow_id, user_input)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 414, in _async_configure
    result = await self._async_handle_step(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        flow, cur_step["step_id"], user_input
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 517, in _async_handle_step
    result: _FlowResultT = await getattr(flow, method)(user_input)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/config_flow.py", line 286, in async_step_manual_port_config
    return await self.async_step_verify_radio()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/config_flow.py", line 331, in async_step_verify_radio
    return await self.async_step_choose_formation_strategy()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/config_flow.py", line 347, in async_step_choose_formation_strategy
    await self._radio_mgr.async_load_network_settings()
  File "/usr/src/homeassistant/homeassistant/components/zha/radio_manager.py", line 255, in async_load_network_settings
    await app.load_network_info()
  File "/usr/local/lib/python3.13/site-packages/bellows/zigbee/application.py", line 303, in load_network_info
    flow_control = await self._ezsp.xncp_get_flow_control_type()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ezsp/__init__.py", line 705, in xncp_get_flow_control_type
    rsp = await self.send_xncp_frame(xncp.GetFlowControlTypeReq())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/bellows/ezsp/__init__.py", line 663, in send_xncp_frame
    rsp_frame = xncp.XncpCommand.from_bytes(data)
  File "/usr/local/lib/python3.13/site-packages/bellows/ezsp/xncp.py", line 70, in from_bytes
    payload, rest = COMMANDS[command_id].deserialize(data)
                    ~~~~~~~~^^^^^^^^^^^^
KeyError: <XncpCommandId.undefined_0x03e8: 1000>

Gateway was modified by this tutorial: Hacking the Silvercrest (Lidl) Smart Home Gateway - PaulBanks.Org

2 Likes

I have exactly the same error with the same Lidl gateway device. (also hacked using Paul’s tutorial)

1 Like

Running

ha os update --version 13.2
ha core update --version=2024.11.3

to revert back did fix the issue (FCVO “fix”, obviously)

Exactly the same problem with the same gateway Lidl and the same Hack :disappointed_relieved:

1 Like

Same problem, same gateway, same hack… I start to see a pattern here… :smiling_face_with_tear:

Same scenario. Worse: W.A.F. is SINKING! Wife Acceptance Factor is DOWN!

2 Likes

I have also the same problem, with the same gateway and same hack :thinking:

Looks like it might have been fixed upstream Fix XNCP parsing for hacked LIDL gateway by puddly · Pull Request #657 · zigpy/bellows · GitHub

1 Like

I have 2 home assistant with 2 LIDL gateways. one was working, the other no, i was thinking that i used the same tutorial, but no, i used another one.
So i compared both and there are some diferences, the mainis and updated EZZP version so … upgrading this solves the problem, no new config needed, only upgrade.

Solution:

Upgrade the EZSP Version to 6.7.8.0 with this tutorial procedure, only necesari to upgrade the firmware not all tutorial.

4 Likes

Thank you! ZHA works well after upgrading gateway to EZSP 6.7.8.0 with HA 2024.12.0. :+1:

Btw firmware_upgrade.sh script did not work for me. It stucked everytime when “/tmp/sx” started. I had to ssh into gateway and manually run every command separatedly, then it was flashed succesfully.

2 Likes

Works fine after upgrade of the EZSP to version 6.7.8.0. Thanks a lot.

Thank you so much!
Updating the FW fixed the issue.

I just got this when I moved my gateway to my IOT VLAN and thought it must be the firewall rule config I messed up.

Worked also for me, but I had to tune the ssh commands to add :

-oHostKeyAlgorithms=+ssh-rsa -oPubkeyAcceptedAlgorithms=+ssh-rsa

Great thanks for your help :smile:

1 Like

I had to add -vv to the sx command line from the firmware update script.
And I also was doing it manually in the SSH session… (pasted lines from the “script”)

same problem here, fixed with the upgrade of the EZSP to version 6.7.8.0. thanks guys

Same problem here but upgrade fails with no response after this:

/tmp/sx /tmp/firmware.gbl < /dev/ttyS1 > /dev/ttyS1

I ran it from the script and directly on the bridge. Same results. :frowning:

Eventually I get this:
Xmodem sectors/kbytes sent: 22/ 2kRetry 0: NAK on sector
Xmodem sectors/kbytes sent: 24/ 3kRetry 0: NAK on sector
Xmodem sectors/kbytes sent: 51/ 6kRetry 0: NAK on sector
Xmodem sectors/kbytes sent: 53/ 6kRetry 0: NAK on sector
Xmodem sectors/kbytes sent: 55/ 6kRetry 0: NAK on sector
Xmodem sectors/kbytes sent: 101/12kRetry 0: NAK on sector
Xmodem sectors/kbytes sent: 103/12kRetry 0: NAK on sector
Xmodem sectors/kbytes sent: 118/14kRetry 0: NAK on sector
Xmodem sectors/kbytes sent: 120/15kRetry 0: NAK on sector
Xmodem sectors/kbytes sent: 122/15kRetry 0: NAK on sector
Xmodem sectors/kbytes sent: 124/15kRetry 0: NAK on sector
Xmodem sectors/kbytes sent: 126/15kRetry 0: NAK on sector
Xmodem sectors/kbytes sent: 134/16kRetry 0: NAK on sector
Xmodem sectors/kbytes sent: 136/17kRetry 0: NAK on sector
Retry 0: Cancelled

but still fails

EDIT2:

It took like 150 tries but finally
Xmodem sectors/kbytes sent: 1254/156kRetry 0: NAK on sector
Bytes Sent: 183680 BPS:2750

Transfer complete

And working now, thanks!

Yep, had the same issue after upgrading HA to 24.12
Solution: upgraded the LIDL Gateway to `6.7.8.0’ .

I had to try several times, but the script worked in the end! And fixed the problem.