Trouble setting up my zigbee coordinator (first time)

Hi there,

I have trouble integrating my first zigbee device and for once have not found a clear explanation on the internet on where I am making a mistake, or how to diagnose what is wrong.

For context, here is my setup :

I went through the following to add the zigbee device :

  1. Plug in the USB key, device appeared as /dev/ttyACM0 in debian
  2. Bind /dev/ttyACM0 to the HA container and restart it. It is visible from within the container
  3. HA detected it as a new device to integrate, I confirmed and it made some background init.
  4. I opened the zigbee home automation device and tried to pair my first node
    → After multiple resets of the device, I cannot get it to pair.
  5. I found out the “Manage Zigbee Device” menu and tried to read app version
    → Impossible to read any of the fields from the coordinator node.

What I already tried :

  • Pairing putting the device next to the coordinator.

  • Deleting the integration and re-adding it

  • Resetting the dongle THEN rebooting the container (to make sur it is well binded after reset)

  • Playing with debug and trying to get a log to figure out what is wrong (I can not understand most of it)

Summary

2023-05-28 20:04:00.204 DEBUG (MainThread) [bellows.ezsp.protocol] Send command readCounters: ()
2023-05-28 20:04:00.206 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’042d21a9a52a242b7e’
2023-05-28 20:04:00.223 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’412da1a9a52a15b251944a25aa5592499c4e26abecce628bfec66389fc7e3fa7ebcdde6f8fffc7dbd5d2698c4623a9ec763ba5ea758241984c2613b1e070381c0e07bbe5ca658a459a4d9e4f9ff7c3d9d46a35a251904824156e7e’
2023-05-28 20:04:00.223 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’8520dd7e’
2023-05-28 20:04:00.224 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received readCounters: [[0, 8, 0, 0, 0, 0, 1, 1, 5, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]
2023-05-28 20:04:00.224 DEBUG (MainThread) [bellows.ezsp.protocol] Send command getValue: (<EzspValueId.VALUE_FREE_BUFFERS: 3>,)
2023-05-28 20:04:00.225 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’153221a9fe2a16c3b37e’
2023-05-28 20:04:00.229 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’5232a1a9fe2a15b3aed71e7e’
2023-05-28 20:04:00.229 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’8610be7e’
2023-05-28 20:04:00.230 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received getValue: [<EzspStatus.SUCCESS: 0>, b’\xf7’]
2023-05-28 20:04:00.230 DEBUG (MainThread) [bellows.zigbee.application] Free buffers status EzspStatus.SUCCESS, value: 247
2023-05-28 20:04:00.230 DEBUG (MainThread) [bellows.zigbee.application] ezsp_counters: [MAC_RX_BROADCAST = 0, MAC_TX_BROADCAST = 8, MAC_RX_UNICAST = 0, MAC_TX_UNICAST_SUCCESS = 0, MAC_TX_UNICAST_RETRY = 0, MAC_TX_UNICAST_FAILED = 0, APS_DATA_RX_BROADCAST = 1, APS_DATA_TX_BROADCAST = 1, APS_DATA_RX_UNICAST = 5, APS_DATA_TX_UNICAST_SUCCESS = 3, APS_DATA_TX_UNICAST_RETRY = 0, APS_DATA_TX_UNICAST_FAILED = 0, ROUTE_DISCOVERY_INITIATED = 0, NEIGHBOR_ADDED = 0, NEIGHBOR_REMOVED = 0, NEIGHBOR_STALE = 0, JOIN_INDICATION = 0, CHILD_REMOVED = 0, ASH_OVERFLOW_ERROR = 0, ASH_FRAMING_ERROR = 0, ASH_OVERRUN_ERROR = 0, NWK_FRAME_COUNTER_FAILURE = 0, APS_FRAME_COUNTER_FAILURE = 0, UTILITY = 0, APS_LINK_KEY_NOT_AUTHORIZED = 0, NWK_DECRYPTION_FAILURE = 0, APS_DECRYPTION_FAILURE = 0, ALLOCATE_PACKET_BUFFER_FAILURE = 0, RELAYED_UNICAST = 0, PHY_TO_MAC_QUEUE_LIMIT_REACHED = 0, PACKET_VALIDATE_LIBRARY_DROPPED_COUNT = 0, TYPE_NWK_RETRY_OVERFLOW = 0, PHY_CCA_FAIL_COUNT = 0, BROADCAST_TABLE_FULL = 0, PTA_LO_PRI_REQUESTED = 0, PTA_HI_PRI_REQUESTED = 0, PTA_LO_PRI_DENIED = 0, PTA_HI_PRI_DENIED = 0, PTA_LO_PRI_TX_ABORTED = 0, PTA_HI_PRI_TX_ABORTED = 0, ADDRESS_CONFLICT_SENT = 0, EZSP_FREE_BUFFERS = 247]
2023-05-28 20:04:06.330 DEBUG (MainThread) [homeassistant.components.zha.websocket_api] Requested attributes for: cluster_id: 0, cluster_type: ‘in’, endpoint_id: 1, response: [{‘id’: 0, ‘name’: ‘zcl_version’}, {‘id’: 1, ‘name’: ‘app_version’}, {‘id’: 2, ‘name’: ‘stack_version’}, {‘id’: 3, ‘name’: ‘hw_version’}, {‘id’: 4, ‘name’: ‘manufacturer’}, {‘id’: 5, ‘name’: ‘model’}, {‘id’: 6, ‘name’: ‘date_code’}, {‘id’: 7, ‘name’: ‘power_source’}, {‘id’: 8, ‘name’: ‘generic_device_class’}, {‘id’: 9, ‘name’: ‘generic_device_type’}, {‘id’: 10, ‘name’: ‘product_code’}, {‘id’: 11, ‘name’: ‘product_url’}, {‘id’: 12, ‘name’: ‘manufacturer_version_details’}, {‘id’: 13, ‘name’: ‘serial_number’}, {‘id’: 14, ‘name’: ‘product_label’}, {‘id’: 16, ‘name’: ‘location_desc’}, {‘id’: 17, ‘name’: ‘physical_env’}, {‘id’: 18, ‘name’: ‘device_enabled’}, {‘id’: 19, ‘name’: ‘alarm_mask’}, {‘id’: 20, ‘name’: ‘disable_local_config’}, {‘id’: 16384, ‘name’: ‘sw_build_id’}, {‘id’: 65533, ‘name’: ‘cluster_revision’}, {‘id’: 65534, ‘name’: ‘attr_reporting_status’}]
2023-05-28 20:04:09.144 DEBUG (MainThread) [zigpy.zcl] [0x0000:1:0x0000] Sending request header: ZCLHeader(frame_control=FrameControl(frame_type=<FrameType.GLOBAL_COMMAND: 0>, is_manufacturer_specific=False, direction=<Direction.Server_to_Client: 0>, disable_default_response=0, reserved=0, *is_cluster=False, *is_general=True), tsn=9, command_id=<GeneralCommand.Read_Attributes: 0>, *direction=<Direction.Server_to_Client: 0>)
2023-05-28 20:04:09.145 DEBUG (MainThread) [zigpy.zcl] [0x0000:1:0x0000] Sending request: Read_Attributes(attribute_ids=[1])
2023-05-28 20:04:09.146 DEBUG (MainThread) [bellows.zigbee.application] Sending packet ZigbeePacket(src=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), src_ep=1, dst=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), dst_ep=1, source_route=None, extended_timeout=False, tsn=9, profile_id=260, cluster_id=0, data=Serialized[b’\x00\t\x00\x01\x00’], tx_options=<TransmitOptions.NONE: 0>, radius=0, non_member_radius=0, lqi=None, rssi=None)
2023-05-28 20:04:09.147 DEBUG (MainThread) [bellows.ezsp.protocol] Send command sendUnicast: (<EmberOutgoingMessageType.OUTGOING_DIRECT: 0>, 0x0000, EmberApsFrame(profileId=260, clusterId=0, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=0, sequence=9), 10, b’\x00\t\x00\x01\x00’)
2023-05-28 20:04:09.149 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’263321a9602a15b259904b25aa5493499d4e27a2e7cb6782fdc763b41b7e’
2023-05-28 20:04:09.157 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’6333a1a9602a156eba577e’
2023-05-28 20:04:09.158 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’87009f7e’
2023-05-28 20:04:09.159 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received sendUnicast: [<EmberStatus.SUCCESS: 0>, 220]
2023-05-28 20:04:09.164 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’7333b5a9112a15b658944a24ab5593499c92d8abedce9874f8c66a89fd7edfe17e’
2023-05-28 20:04:09.165 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’8070787e’
2023-05-28 20:04:09.166 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received incomingMessageHandler: [<EmberIncomingMessageType.INCOMING_UNICAST: 0>, EmberApsFrame(profileId=260, clusterId=0, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=0, sequence=220), 255, 0, 0x0000, 255, 255, b’\x00\t\x00\x01\x00’]
2023-05-28 20:04:09.167 DEBUG (MainThread) [bellows.zigbee.application] Received incomingMessageHandler frame with [<EmberIncomingMessageType.INCOMING_UNICAST: 0>, EmberApsFrame(profileId=260, clusterId=0, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=0, sequence=220), 255, 0, 0x0000, 255, 255, b’\x00\t\x00\x01\x00’]
2023-05-28 20:04:09.167 DEBUG (MainThread) [zigpy.application] Received a packet: ZigbeePacket(src=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), src_ep=1, dst=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), dst_ep=1, source_route=None, extended_timeout=False, tsn=220, profile_id=260, cluster_id=0, data=Serialized[b’\x00\t\x00\x01\x00’], tx_options=<TransmitOptions.NONE: 0>, radius=0, non_member_radius=0, lqi=255, rssi=0)
2023-05-28 20:04:09.168 DEBUG (MainThread) [zigpy.zcl] [0x0000:1:0x0000] Received ZCL frame: b’\x00\t\x00\x01\x00’
2023-05-28 20:04:09.168 DEBUG (MainThread) [zigpy.zcl] [0x0000:1:0x0000] Decoded ZCL frame header: ZCLHeader(frame_control=FrameControl(frame_type=<FrameType.GLOBAL_COMMAND: 0>, is_manufacturer_specific=0, direction=<Direction.Server_to_Client: 0>, disable_default_response=0, reserved=0, *is_cluster=False, *is_general=True), tsn=9, command_id=0, *direction=<Direction.Server_to_Client: 0>)
2023-05-28 20:04:09.169 DEBUG (MainThread) [zigpy.zcl] [0x0000:1:0x0000] Decoded ZCL frame: Basic:Read_Attributes(attribute_ids=[1])
2023-05-28 20:04:09.169 DEBUG (MainThread) [zigpy.zcl] [0x0000:1:0x0000] Received command 0x00 (TSN 9): Read_Attributes(attribute_ids=[1])
2023-05-28 20:04:09.171 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’0333b1a96b2a15b259904b25aa5493499d4e2777e7ce67e13d7e’
2023-05-28 20:04:09.171 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’8160597e’
2023-05-28 20:04:09.171 DEBUG (Thread-8) [aiosqlite] executing functools.partial(<built-in method execute of sqlite3.Connection object at 0x7fe9ed324a40>, ‘UPDATE devices_v11\n SET last_seen=:ts\n WHERE ieee=:ieee AND :ts - last_seen > :min_last_seen_delta’, {‘ts’: 1685304249.168006, ‘ieee’: e0:79:8d:ff:fe:a8:89:ab, ‘min_last_seen_delta’: 30.0})
2023-05-28 20:04:09.172 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received messageSentHandler: [<EmberOutgoingMessageType.OUTGOING_DIRECT: 0>, 0, EmberApsFrame(profileId=260, clusterId=0, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=0, sequence=220), 10, <EmberStatus.SUCCESS: 0>, b’’]
2023-05-28 20:04:09.173 DEBUG (MainThread) [bellows.zigbee.application] Received messageSentHandler frame with [<EmberOutgoingMessageType.OUTGOING_DIRECT: 0>, 0, EmberApsFrame(profileId=260, clusterId=0, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=0, sequence=220), 10, <EmberStatus.SUCCESS: 0>, b’’]
2023-05-28 20:04:09.174 DEBUG (Thread-8) [aiosqlite] operation functools.partial(<built-in method execute of sqlite3.Connection object at 0x7fe9ed324a40>, ‘UPDATE devices_v11\n SET last_seen=:ts\n WHERE ieee=:ieee AND :ts - last_seen > :min_last_seen_delta’, {‘ts’: 1685304249.168006, ‘ieee’: e0:79:8d:ff:fe:a8:89:ab, ‘min_last_seen_delta’: 30.0}) completed
2023-05-28 20:04:09.175 DEBUG (Thread-8) [aiosqlite] executing functools.partial(<built-in method commit of sqlite3.Connection object at 0x7fe9ed324a40>)
2023-05-28 20:04:09.176 DEBUG (Thread-8) [aiosqlite] operation functools.partial(<built-in method commit of sqlite3.Connection object at 0x7fe9ed324a40>) completed
2023-05-28 20:04:10.233 DEBUG (MainThread) [bellows.ezsp.protocol] Send command readCounters: ()
2023-05-28 20:04:10.234 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’313021a9a52a6d867e’
2023-05-28 20:04:10.252 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’1430a1a9a52a15b250944a25aa5592499c4e26abecce618bf9c66389fc7e3fa7ebcdde6f8fffc7dbd5d2698c4623a9ec763ba5ea758241984c2613b1e070381c0e07bbe5ca658a459a4d9e4f9ff7c3d9d46a35a2519048242ecf7e’
2023-05-28 20:04:10.252 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’82503a7e’
2023-05-28 20:04:10.254 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received readCounters: [[0, 9, 0, 0, 0, 0, 1, 1, 6, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]
2023-05-28 20:04:10.255 DEBUG (MainThread) [bellows.ezsp.protocol] Send command getValue: (<EzspValueId.VALUE_FREE_BUFFERS: 3>,)
2023-05-28 20:04:10.256 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’423121a9fe2a16fe807e’
2023-05-28 20:04:10.262 DEBUG (bellows.thread_0) [bellows.uart] Data frame: b’2531a1a9fe2a15b3ae051b7e’
2023-05-28 20:04:10.262 DEBUG (bellows.thread_0) [bellows.uart] Sending: b’83401b7e’
2023-05-28 20:04:10.264 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received getValue: [<EzspStatus.SUCCESS: 0>, b’\xf7’]
2023-05-28 20:04:10.264 DEBUG (MainThread) [bellows.zigbee.application] Free buffers status EzspStatus.SUCCESS, value: 247
2023-05-28 20:04:10.264 DEBUG (MainThread) [bellows.zigbee.application] ezsp_counters: [MAC_RX_BROADCAST = 0, MAC_TX_BROADCAST = 9, MAC_RX_UNICAST = 0, MAC_TX_UNICAST_SUCCESS = 0, MAC_TX_UNICAST_RETRY = 0, MAC_TX_UNICAST_FAILED = 0, APS_DATA_RX_BROADCAST = 1, APS_DATA_TX_BROADCAST = 1, APS_DATA_RX_UNICAST = 6, APS_DATA_TX_UNICAST_SUCCESS = 4, APS_DATA_TX_UNICAST_RETRY = 0, APS_DATA_TX_UNICAST_FAILED = 0, ROUTE_DISCOVERY_INITIATED = 0, NEIGHBOR_ADDED = 0, NEIGHBOR_REMOVED = 0, NEIGHBOR_STALE = 0, JOIN_INDICATION = 0, CHILD_REMOVED = 0, ASH_OVERFLOW_ERROR = 0, ASH_FRAMING_ERROR = 0, ASH_OVERRUN_ERROR = 0, NWK_FRAME_COUNTER_FAILURE = 0, APS_FRAME_COUNTER_FAILURE = 0, UTILITY = 0, APS_LINK_KEY_NOT_AUTHORIZED = 0, NWK_DECRYPTION_FAILURE = 0, APS_DECRYPTION_FAILURE = 0, ALLOCATE_PACKET_BUFFER_FAILURE = 0, RELAYED_UNICAST = 0, PHY_TO_MAC_QUEUE_LIMIT_REACHED = 0, PACKET_VALIDATE_LIBRARY_DROPPED_COUNT = 0, TYPE_NWK_RETRY_OVERFLOW = 0, PHY_CCA_FAIL_COUNT = 0, BROADCAST_TABLE_FULL = 0, PTA_LO_PRI_REQUESTED = 0, PTA_HI_PRI_REQUESTED = 0, PTA_LO_PRI_DENIED = 0, PTA_HI_PRI_DENIED = 0, PTA_LO_PRI_TX_ABORTED = 0, PTA_HI_PRI_TX_ABORTED = 0, ADDRESS_CONFLICT_SENT = 0, EZSP_FREE_BUFFERS = 247]
2023-05-28 20:04:14.175 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [140642881771872] Error handling message: Timeout (timeout) Porteffroi from 192.168.2.14 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36)

All attempt lead to the same outcome. Am i missing a step somewhere ?

Is it expected that the coordinator does not respond to read request or the symptom that the problem is between the dongle and HA, not between the dongle and the ZigBee device ?

I have seen multiple posts talking about zigbee2mqtt. Am i right to assume that this is redundant with what HA integration already does if I do not want to expose data through mqtt ?

Any help to try answering these will be appreciated.

Sounds like it is probably related to Docker somehow, but do also take actions to avoid interference.

Try installing Docker Compose and adding the serial port device path via docker-compose.yml file:

https://www.home-assistant.io/integrations/zha#cant-connect-to-usb-device-and-using-docker

Also connect it via a long USB extension cable to a USB 2.0 port (not USB 3.0). Follow this guide:

Thanks Hedda, I went through your advices and links.

Regarding the container config, I made some changes to match the docker-compose example

  • The usb device is already mapped as /dev/usbACM0. What I don’t know is if being ACM0 instead of USB0 makes a difference for HA. For what I read it does not matter much but if your device appears differently maybe a driver is different.

  • The volume configuration with different as localtime was not synced, I added it for sake of completeness.

  • The network was set to bridge rather than host. I would prefer keeping it bridged but for the test I set it to host.

For the interference risk, I will order a long cable and place the coordinator much further from the wifi hotspot and host laptop. Wip.

Something I would really like to know for debugging is if someone with a similar coordinator is able to query it through read requests. Mine does not work for this and it should not depend on interferences as I query the coordinator.

Pierre,
Two thoughts:

  1. I cannot read from my coordinator endpoint either.
  2. Try a different ZigBee device so you’re sure it isn’t the device that’s the problem.
    Sorry I can’t be of more help.
    -David
1 Like

I believe I had similar issue as yours. I could add zigbee device but than it will became unavailable and I couldn’t pair it again. I had to remove it from z2m and mqtt pair it again and the loop was on.
The problem was I think in user permissions for usb adapter if I recall things correctly.
Check permissions of /dev/usbACM0. Are you as a user member of that group?

Great news, it is now working !

Thanks David for letting me know you have the same issue with the read command. That means I was biased in my attempt to diagnose towards an unrelated behavior. Helps a lot.

Hedda, thanks for your detailed article. Proximity with the wifi access point might definitively have been a source of trouble (< 1m apart). That makes a lot of sense but I was not aware that frequencies were so similar. I have not yet an extender to put the coordinator further from the laptop it is connected to but will plan on upgrading that once I move it to its final spot.

The note about some sensor being imperfect also helped. In most videos I watched it was successful at first attempt but I needed to insist much more, even being further from my wifi access point.

Some devices, like example those from Xiaomi/Aqara, are also known to not be 100% compliant with the standard Zigbee specifications for joining/pairing and may therefore require many paring attempts over 10-20 minutes or even more.

Now I have my sensor updating super fast on events. :slight_smile: