Trouble setting up my zigbee coordinator (first time)

[Porteffroi (Pierre Lorent)]

Hi there,

I have trouble integrating my first zigbee device and for once have not found a clear explanation on the internet on where I am making a mistake, or how to diagnose what is wrong.

For context, here is my setup :

• Debian OS running on 64 bit machine
• Docker running various containers including HA, Esphome and portainer
• Configured and running EspHome device (emporia vue 2) and watermeter (home wizard)
• A Sonoff zigbee dongle from https://www.amazon.com.be/dp/B0B6P22YJC?psc=1&ref=ppx_yo2ov_dt_b_product_details
• An aqara zigbee door sensor https://www.amazon.com.be/dp/B07D37VDM3?psc=1&ref=ppx_yo2ov_dt_b_product_details

I went through the following to add the zigbee device :

1. Plug in the USB key, device appeared as /dev/ttyACM0 in debian
2. Bind /dev/ttyACM0 to the HA container and restart it. It is visible from within the container
3. HA detected it as a new device to integrate, I confirmed and it made some background init.
4. I opened the zigbee home automation device and tried to pair my first node
   → After multiple resets of the device, I cannot get it to pair.
5. I found out the “Manage Zigbee Device” menu and tried to read app version
   → Impossible to read any of the fields from the coordinator node.

What I already tried :

• Pairing putting the device next to the coordinator.

• Deleting the integration and re-adding it

• Resetting the dongle THEN rebooting the container (to make sur it is well binded after reset)
  

  

• Playing with debug and trying to get a log to figure out what is wrong (I can not understand most of it)

Summary

2023-05-28 20:04:00.204 DEBUG (MainThread) [bellows.ezsp.protocol] Send command readCounters: ()
2023-05-28 20:04:00.206 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’042d21a9a52a242b7e’
2023-05-28 20:04:00.223 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’412da1a9a52a15b251944a25aa5592499c4e26abecce628bfec66389fc7e3fa7ebcdde6f8fffc7dbd5d2698c4623a9ec763ba5ea758241984c2613b1e070381c0e07bbe5ca658a459a4d9e4f9ff7c3d9d46a35a251904824156e7e’
2023-05-28 20:04:00.223 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’8520dd7e’
2023-05-28 20:04:00.224 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received readCounters: [[0, 8, 0, 0, 0, 0, 1, 1, 5, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]
2023-05-28 20:04:00.224 DEBUG (MainThread) [bellows.ezsp.protocol] Send command getValue: (<EzspValueId.VALUE_FREE_BUFFERS: 3>,)
2023-05-28 20:04:00.225 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’153221a9fe2a16c3b37e’
2023-05-28 20:04:00.229 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’5232a1a9fe2a15b3aed71e7e’
2023-05-28 20:04:00.229 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’8610be7e’
2023-05-28 20:04:00.230 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received getValue: [<EzspStatus.SUCCESS: 0>, b’\xf7’]
2023-05-28 20:04:00.230 DEBUG (MainThread) [bellows.zigbee.application] Free buffers status EzspStatus.SUCCESS, value: 247
2023-05-28 20:04:00.230 DEBUG (MainThread) [bellows.zigbee.application] ezsp_counters: [MAC_RX_BROADCAST = 0, MAC_TX_BROADCAST = 8, MAC_RX_UNICAST = 0, MAC_TX_UNICAST_SUCCESS = 0, MAC_TX_UNICAST_RETRY = 0, MAC_TX_UNICAST_FAILED = 0, APS_DATA_RX_BROADCAST = 1, APS_DATA_TX_BROADCAST = 1, APS_DATA_RX_UNICAST = 5, APS_DATA_TX_UNICAST_SUCCESS = 3, APS_DATA_TX_UNICAST_RETRY = 0, APS_DATA_TX_UNICAST_FAILED = 0, ROUTE_DISCOVERY_INITIATED = 0, NEIGHBOR_ADDED = 0, NEIGHBOR_REMOVED = 0, NEIGHBOR_STALE = 0, JOIN_INDICATION = 0, CHILD_REMOVED = 0, ASH_OVERFLOW_ERROR = 0, ASH_FRAMING_ERROR = 0, ASH_OVERRUN_ERROR = 0, NWK_FRAME_COUNTER_FAILURE = 0, APS_FRAME_COUNTER_FAILURE = 0, UTILITY = 0, APS_LINK_KEY_NOT_AUTHORIZED = 0, NWK_DECRYPTION_FAILURE = 0, APS_DECRYPTION_FAILURE = 0, ALLOCATE_PACKET_BUFFER_FAILURE = 0, RELAYED_UNICAST = 0, PHY_TO_MAC_QUEUE_LIMIT_REACHED = 0, PACKET_VALIDATE_LIBRARY_DROPPED_COUNT = 0, TYPE_NWK_RETRY_OVERFLOW = 0, PHY_CCA_FAIL_COUNT = 0, BROADCAST_TABLE_FULL = 0, PTA_LO_PRI_REQUESTED = 0, PTA_HI_PRI_REQUESTED = 0, PTA_LO_PRI_DENIED = 0, PTA_HI_PRI_DENIED = 0, PTA_LO_PRI_TX_ABORTED = 0, PTA_HI_PRI_TX_ABORTED = 0, ADDRESS_CONFLICT_SENT = 0, EZSP_FREE_BUFFERS = 247]
2023-05-28 20:04:06.330 DEBUG (MainThread) [homeassistant.components.zha.websocket_api] Requested attributes for: cluster_id: 0, cluster_type: ‘in’, endpoint_id: 1, response: [{‘id’: 0, ‘name’: ‘zcl_version’}, {‘id’: 1, ‘name’: ‘app_version’}, {‘id’: 2, ‘name’: ‘stack_version’}, {‘id’: 3, ‘name’: ‘hw_version’}, {‘id’: 4, ‘name’: ‘manufacturer’}, {‘id’: 5, ‘name’: ‘model’}, {‘id’: 6, ‘name’: ‘date_code’}, {‘id’: 7, ‘name’: ‘power_source’}, {‘id’: 8, ‘name’: ‘generic_device_class’}, {‘id’: 9, ‘name’: ‘generic_device_type’}, {‘id’: 10, ‘name’: ‘product_code’}, {‘id’: 11, ‘name’: ‘product_url’}, {‘id’: 12, ‘name’: ‘manufacturer_version_details’}, {‘id’: 13, ‘name’: ‘serial_number’}, {‘id’: 14, ‘name’: ‘product_label’}, {‘id’: 16, ‘name’: ‘location_desc’}, {‘id’: 17, ‘name’: ‘physical_env’}, {‘id’: 18, ‘name’: ‘device_enabled’}, {‘id’: 19, ‘name’: ‘alarm_mask’}, {‘id’: 20, ‘name’: ‘disable_local_config’}, {‘id’: 16384, ‘name’: ‘sw_build_id’}, {‘id’: 65533, ‘name’: ‘cluster_revision’}, {‘id’: 65534, ‘name’: ‘attr_reporting_status’}]
2023-05-28 20:04:09.144 DEBUG (MainThread) [zigpy.zcl] [0x0000:1:0x0000] Sending request header: ZCLHeader(frame_control=FrameControl(frame_type=<FrameType.GLOBAL_COMMAND: 0>, is_manufacturer_specific=False, direction=<Direction.Server_to_Client: 0>, disable_default_response=0, reserved=0, *is_cluster=False, *is_general=True), tsn=9, command_id=<GeneralCommand.Read_Attributes: 0>, *direction=<Direction.Server_to_Client: 0>)
2023-05-28 20:04:09.145 DEBUG (MainThread) [zigpy.zcl] [0x0000:1:0x0000] Sending request: Read_Attributes(attribute_ids=[1])
2023-05-28 20:04:09.146 DEBUG (MainThread) [bellows.zigbee.application] Sending packet ZigbeePacket(src=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), src_ep=1, dst=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), dst_ep=1, source_route=None, extended_timeout=False, tsn=9, profile_id=260, cluster_id=0, data=Serialized[b’\x00\t\x00\x01\x00’], tx_options=<TransmitOptions.NONE: 0>, radius=0, non_member_radius=0, lqi=None, rssi=None)
2023-05-28 20:04:09.147 DEBUG (MainThread) [bellows.ezsp.protocol] Send command sendUnicast: (<EmberOutgoingMessageType.OUTGOING_DIRECT: 0>, 0x0000, EmberApsFrame(profileId=260, clusterId=0, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=0, sequence=9), 10, b’\x00\t\x00\x01\x00’)
2023-05-28 20:04:09.149 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’263321a9602a15b259904b25aa5493499d4e27a2e7cb6782fdc763b41b7e’
2023-05-28 20:04:09.157 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’6333a1a9602a156eba577e’
2023-05-28 20:04:09.158 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’87009f7e’
2023-05-28 20:04:09.159 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received sendUnicast: [<EmberStatus.SUCCESS: 0>, 220]
2023-05-28 20:04:09.164 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’7333b5a9112a15b658944a24ab5593499c92d8abedce9874f8c66a89fd7edfe17e’
2023-05-28 20:04:09.165 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’8070787e’
2023-05-28 20:04:09.166 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received incomingMessageHandler: [<EmberIncomingMessageType.INCOMING_UNICAST: 0>, EmberApsFrame(profileId=260, clusterId=0, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=0, sequence=220), 255, 0, 0x0000, 255, 255, b’\x00\t\x00\x01\x00’]
2023-05-28 20:04:09.167 DEBUG (MainThread) [bellows.zigbee.application] Received incomingMessageHandler frame with [<EmberIncomingMessageType.INCOMING_UNICAST: 0>, EmberApsFrame(profileId=260, clusterId=0, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=0, sequence=220), 255, 0, 0x0000, 255, 255, b’\x00\t\x00\x01\x00’]
2023-05-28 20:04:09.167 DEBUG (MainThread) [zigpy.application] Received a packet: ZigbeePacket(src=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), src_ep=1, dst=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), dst_ep=1, source_route=None, extended_timeout=False, tsn=220, profile_id=260, cluster_id=0, data=Serialized[b’\x00\t\x00\x01\x00’], tx_options=<TransmitOptions.NONE: 0>, radius=0, non_member_radius=0, lqi=255, rssi=0)
2023-05-28 20:04:09.168 DEBUG (MainThread) [zigpy.zcl] [0x0000:1:0x0000] Received ZCL frame: b’\x00\t\x00\x01\x00’
2023-05-28 20:04:09.168 DEBUG (MainThread) [zigpy.zcl] [0x0000:1:0x0000] Decoded ZCL frame header: ZCLHeader(frame_control=FrameControl(frame_type=<FrameType.GLOBAL_COMMAND: 0>, is_manufacturer_specific=0, direction=<Direction.Server_to_Client: 0>, disable_default_response=0, reserved=0, *is_cluster=False, *is_general=True), tsn=9, command_id=0, *direction=<Direction.Server_to_Client: 0>)
2023-05-28 20:04:09.169 DEBUG (MainThread) [zigpy.zcl] [0x0000:1:0x0000] Decoded ZCL frame: Basic:Read_Attributes(attribute_ids=[1])
2023-05-28 20:04:09.169 DEBUG (MainThread) [zigpy.zcl] [0x0000:1:0x0000] Received command 0x00 (TSN 9): Read_Attributes(attribute_ids=[1])
2023-05-28 20:04:09.171 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’0333b1a96b2a15b259904b25aa5493499d4e2777e7ce67e13d7e’
2023-05-28 20:04:09.171 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’8160597e’
2023-05-28 20:04:09.171 DEBUG (Thread-8) [aiosqlite] executing functools.partial(<built-in method execute of sqlite3.Connection object at 0x7fe9ed324a40>, ‘UPDATE devices_v11\n SET last_seen=:ts\n WHERE ieee=:ieee AND :ts - last_seen > :min_last_seen_delta’, {‘ts’: 1685304249.168006, ‘ieee’: e0:79:8d:ff:fe:a8:89:ab, ‘min_last_seen_delta’: 30.0})
2023-05-28 20:04:09.172 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received messageSentHandler: [<EmberOutgoingMessageType.OUTGOING_DIRECT: 0>, 0, EmberApsFrame(profileId=260, clusterId=0, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=0, sequence=220), 10, <EmberStatus.SUCCESS: 0>, b’’]
2023-05-28 20:04:09.173 DEBUG (MainThread) [bellows.zigbee.application] Received messageSentHandler frame with [<EmberOutgoingMessageType.OUTGOING_DIRECT: 0>, 0, EmberApsFrame(profileId=260, clusterId=0, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=0, sequence=220), 10, <EmberStatus.SUCCESS: 0>, b’’]
2023-05-28 20:04:09.174 DEBUG (Thread-8) [aiosqlite] operation functools.partial(<built-in method execute of sqlite3.Connection object at 0x7fe9ed324a40>, ‘UPDATE devices_v11\n SET last_seen=:ts\n WHERE ieee=:ieee AND :ts - last_seen > :min_last_seen_delta’, {‘ts’: 1685304249.168006, ‘ieee’: e0:79:8d:ff:fe:a8:89:ab, ‘min_last_seen_delta’: 30.0}) completed
2023-05-28 20:04:09.175 DEBUG (Thread-8) [aiosqlite] executing functools.partial(<built-in method commit of sqlite3.Connection object at 0x7fe9ed324a40>)
2023-05-28 20:04:09.176 DEBUG (Thread-8) [aiosqlite] operation functools.partial(<built-in method commit of sqlite3.Connection object at 0x7fe9ed324a40>) completed
2023-05-28 20:04:10.233 DEBUG (MainThread) [bellows.ezsp.protocol] Send command readCounters: ()
2023-05-28 20:04:10.234 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’313021a9a52a6d867e’
2023-05-28 20:04:10.252 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’1430a1a9a52a15b250944a25aa5592499c4e26abecce618bf9c66389fc7e3fa7ebcdde6f8fffc7dbd5d2698c4623a9ec763ba5ea758241984c2613b1e070381c0e07bbe5ca658a459a4d9e4f9ff7c3d9d46a35a2519048242ecf7e’
2023-05-28 20:04:10.252 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’82503a7e’
2023-05-28 20:04:10.254 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received readCounters: [[0, 9, 0, 0, 0, 0, 1, 1, 6, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]
2023-05-28 20:04:10.255 DEBUG (MainThread) [bellows.ezsp.protocol] Send command getValue: (<EzspValueId.VALUE_FREE_BUFFERS: 3>,)
2023-05-28 20:04:10.256 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’423121a9fe2a16fe807e’
2023-05-28 20:04:10.262 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’2531a1a9fe2a15b3ae051b7e’
2023-05-28 20:04:10.262 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’83401b7e’
2023-05-28 20:04:10.264 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received getValue: [<EzspStatus.SUCCESS: 0>, b’\xf7’]
2023-05-28 20:04:10.264 DEBUG (MainThread) [bellows.zigbee.application] Free buffers status EzspStatus.SUCCESS, value: 247
2023-05-28 20:04:10.264 DEBUG (MainThread) [bellows.zigbee.application] ezsp_counters: [MAC_RX_BROADCAST = 0, MAC_TX_BROADCAST = 9, MAC_RX_UNICAST = 0, MAC_TX_UNICAST_SUCCESS = 0, MAC_TX_UNICAST_RETRY = 0, MAC_TX_UNICAST_FAILED = 0, APS_DATA_RX_BROADCAST = 1, APS_DATA_TX_BROADCAST = 1, APS_DATA_RX_UNICAST = 6, APS_DATA_TX_UNICAST_SUCCESS = 4, APS_DATA_TX_UNICAST_RETRY = 0, APS_DATA_TX_UNICAST_FAILED = 0, ROUTE_DISCOVERY_INITIATED = 0, NEIGHBOR_ADDED = 0, NEIGHBOR_REMOVED = 0, NEIGHBOR_STALE = 0, JOIN_INDICATION = 0, CHILD_REMOVED = 0, ASH_OVERFLOW_ERROR = 0, ASH_FRAMING_ERROR = 0, ASH_OVERRUN_ERROR = 0, NWK_FRAME_COUNTER_FAILURE = 0, APS_FRAME_COUNTER_FAILURE = 0, UTILITY = 0, APS_LINK_KEY_NOT_AUTHORIZED = 0, NWK_DECRYPTION_FAILURE = 0, APS_DECRYPTION_FAILURE = 0, ALLOCATE_PACKET_BUFFER_FAILURE = 0, RELAYED_UNICAST = 0, PHY_TO_MAC_QUEUE_LIMIT_REACHED = 0, PACKET_VALIDATE_LIBRARY_DROPPED_COUNT = 0, TYPE_NWK_RETRY_OVERFLOW = 0, PHY_CCA_FAIL_COUNT = 0, BROADCAST_TABLE_FULL = 0, PTA_LO_PRI_REQUESTED = 0, PTA_HI_PRI_REQUESTED = 0, PTA_LO_PRI_DENIED = 0, PTA_HI_PRI_DENIED = 0, PTA_LO_PRI_TX_ABORTED = 0, PTA_HI_PRI_TX_ABORTED = 0, ADDRESS_CONFLICT_SENT = 0, EZSP_FREE_BUFFERS = 247]
2023-05-28 20:04:14.175 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [140642881771872] Error handling message: Timeout (timeout) Porteffroi from 192.168.2.14 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36)

All attempt lead to the same outcome. Am i missing a step somewhere ?

Is it expected that the coordinator does not respond to read request or the symptom that the problem is between the dongle and HA, not between the dongle and the ZigBee device ?

I have seen multiple posts talking about zigbee2mqtt. Am i right to assume that this is redundant with what HA integration already does if I do not want to expose data through mqtt ?

Any help to try answering these will be appreciated.

[Hedda]

Sounds like it is probably related to Docker somehow, but do also take actions to avoid interference.

Try installing Docker Compose and adding the serial port device path via docker-compose.yml file:

https://www.home-assistant.io/integrations/zha#cant-connect-to-usb-device-and-using-docker

Also connect it via a long USB extension cable to a USB 2.0 port (not USB 3.0). Follow this guide:

Zigbee networks: how to guide for avoiding interference and optimize for getting better range + coverage

Who is this guide for? I have tried to make this guide to be a comprehensive collection of the most basic yet essential tips with a set of actions and best practice steps that should be applicable to all types of Zigbee setups, regardless of what type of Zigbee adapter/gateway/hub/bridge/devices that you are using, however, it has been written with a point-of-view in mind of Zigbee solutions that use a Zigbee Coordinator adapter locally via USB dongle/stick or remotely using a serial server ove…

[Porteffroi (Pierre Lorent)]

Thanks Hedda, I went through your advices and links.

Regarding the container config, I made some changes to match the docker-compose example

• The usb device is already mapped as /dev/usbACM0. What I don’t know is if being ACM0 instead of USB0 makes a difference for HA. For what I read it does not matter much but if your device appears differently maybe a driver is different.
  

  

• The volume configuration with different as localtime was not synced, I added it for sake of completeness.
  

  

• The network was set to bridge rather than host. I would prefer keeping it bridged but for the test I set it to host.
  

  

For the interference risk, I will order a long cable and place the coordinator much further from the wifi hotspot and host laptop. Wip.

Something I would really like to know for debugging is if someone with a similar coordinator is able to query it through read requests. Mine does not work for this and it should not depend on interferences as I query the coordinator.

[dbs (David)]

Pierre,
Two thoughts:

1. I cannot read from my coordinator endpoint either.
2. Try a different ZigBee device so you’re sure it isn’t the device that’s the problem.
   Sorry I can’t be of more help.
   -David

[ddaniel (Daniel Dekovic)]

I believe I had similar issue as yours. I could add zigbee device but than it will became unavailable and I couldn’t pair it again. I had to remove it from z2m and mqtt pair it again and the loop was on.
The problem was I think in user permissions for usb adapter if I recall things correctly.
Check permissions of /dev/usbACM0. Are you as a user member of that group?

[Porteffroi (Pierre Lorent)]

Great news, it is now working !

Thanks David for letting me know you have the same issue with the read command. That means I was biased in my attempt to diagnose towards an unrelated behavior. Helps a lot.

Hedda, thanks for your detailed article. Proximity with the wifi access point might definitively have been a source of trouble (< 1m apart). That makes a lot of sense but I was not aware that frequencies were so similar. I have not yet an extender to put the coordinator further from the laptop it is connected to but will plan on upgrading that once I move it to its final spot.

The note about some sensor being imperfect also helped. In most videos I watched it was successful at first attempt but I needed to insist much more, even being further from my wifi access point.

Some devices, like example those from Xiaomi/Aqara, are also known to not be 100% compliant with the standard Zigbee specifications for joining/pairing and may therefore require many paring attempts over 10-20 minutes or even more.

Now I have my sensor updating super fast on events.