[SOLVED] ZHA + Nortek + Sengled Non-Performant Network

jlento · February 13, 2022, 3:42pm

I am having many issues with my ZHA/Zigbee network using a Nortek stick and all Sengled devices. The issues range from lag to turn on/off, color swapping without being set, and even some light groups coming on without an automation (0430 this morning in the bedroom).

I would like to have all lights be Zigbee in my house and controlled via ZWave switches so they are never actually powered off and are controlled through adaptive lighting such that they are the right color and brightness throughout the day.

Currently I have 83 ZHA devices, a Nortek stick (coordinator), 10x Sengled E1C-NB7’s (routers), 23x Sengled E21-N1EA bulbs, and 49x E21-N1E floods. This is all via a Raspberry Pi. At one point I thought it was the SD card so I migrated to a brand new A2 rated card.

I started with a handful of lights in Light Groups but realized that flooded the network so I converted to ZHA groups which allows lights to be in sync for turn off/on. I updated my Nortek stick to the latest recommended firmware, and re-paired all of the coordinators afterward. I then shut down my Pi and forced all devices to re-compute routes.

Currently my network map looks like:

As a benchmark I am trying to toggle all lights on and then off and looking for near real time responsiveness. What I observe is a handful of groups turn on, some toggle on but then retoggle off within seconds to minutes, re-toggling causes failures such as EZSP not running. I have seen issues with Adaptive lighting where states are poorly synced and it causes a turned off light to turn back on and that is being fixed in v2 however what I really need is my lights to be responsive purely under ZHA.

I am sort of stuck with where to go from this point and have turned on pertinent logging. Here are some of the errors I see:

2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Checking quirks for None None (00:0d:6f:00:17:36:c5:0a)
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'zhaquirks.xbee.xbee_io.XBeeSensor'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Fail because endpoint list mismatch: {232, 230} {1}
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'zhaquirks.xbee.xbee3_io.XBee3Sensor'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Fail because endpoint list mismatch: {232, 230} {1}
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'zhaquirks.smartthings.tag_v4.SmartThingsTagV4'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Fail because device_type mismatch on at least one endpoint
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'zhaquirks.smartthings.multi.SmartthingsMultiPurposeSensor'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Fail because device_type mismatch on at least one endpoint
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'zhaquirks.netvox.z308e3ed.Z308E3ED'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Fail because device_type mismatch on at least one endpoint
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'zhaquirks.gledopto.soposhgu10.SoposhGU10'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Fail because endpoint list mismatch: {11, 13} {1}
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'bellows.zigbee.application.EZSPCoordinator'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Found custom device replacement for 00:0d:6f:00:17:36:c5:0a: <class 'bellows.zigbee.application.EZSPCoordinator'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Checking quirks for sengled E12-N1E (b0:ce:18:14:03:53:16:bb)
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'zhaquirks.xbee.xbee_io.XBeeSensor'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Fail because endpoint list mismatch: {232, 230} {1}
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'zhaquirks.xbee.xbee3_io.XBee3Sensor'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Fail because endpoint list mismatch: {232, 230} {1}
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'zhaquirks.smartthings.tag_v4.SmartThingsTagV4'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Fail because device_type mismatch on at least one endpoint
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'zhaquirks.smartthings.multi.SmartthingsMultiPurposeSensor'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Fail because device_type mismatch on at least one endpoint
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'zhaquirks.netvox.z308e3ed.Z308E3ED'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Fail because device_type mismatch on at least one endpoint
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'zhaquirks.gledopto.soposhgu10.SoposhGU10'>
2022-02-13 08:35:06 DEBUG (MainThread) [zigpy.quirks.registry] Fail because endpoint list mismatch: {11, 13} {1}

2022-02-13 08:37:30 DEBUG (MainThread) [zigpy.device] [0x65e1] Delivery error for seq # 0x6f, on endpoint id 1 cluster 0x0702: message send failure
2022-02-13 08:37:30 DEBUG (MainThread) [homeassistant.components.zha.core.channels.base] [0x65E1:1:0x0702]: failed to get attributes '['instantaneous_demand', 'current_summ_delivered', 'status']' on 'smartenergy_metering' cluster: [0x65e1:1:0x0702]: Message send failure
2022-02-13 08:37:30 WARNING (MainThread) [homeassistant.components.zha.core.channels.base] [0x65E1:1:0x0702]: async_initialize: all attempts have failed: [DeliveryError('[0x65e1:1:0x0702]: Message send failure'), DeliveryError('[0x65e1:1:0x0702]: Message send failure'), DeliveryError('[0x65e1:1:0x0702]: Message send failure'), DeliveryError('[0x65e1:1:0x0702]: Message send failure')]

Failed to call service light.turn_on

2022-02-13 10:38:21 DEBUG (MainThread) [bellows.ezsp.protocol] Send command sendMulticast: (EmberApsFrame(profileId=260, clusterId=6, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=13, sequence=63), 0, 3, 64, b'\x01?\x01')
2022-02-13 10:38:21 DEBUG (MainThread) [bellows.ezsp.protocol] Send command sendMulticast: (EmberApsFrame(profileId=260, clusterId=6, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=9, sequence=65), 0, 3, 66, b'\x01A\x01')
2022-02-13 10:38:21 DEBUG (MainThread) [bellows.ezsp.protocol] Send command sendMulticast: (EmberApsFrame(profileId=260, clusterId=6, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=14, sequence=67), 0, 3, 68, b'\x01C\x01')
2022-02-13 10:38:21 DEBUG (MainThread) [bellows.ezsp.protocol] Send command sendMulticast: (EmberApsFrame(profileId=260, clusterId=6, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=15, sequence=69), 0, 3, 70, b'\x01E\x01')
2022-02-13 10:38:23 DEBUG (MainThread) [homeassistant.components.zha.core.channels.base] [0xA365:1:0x0300]: failed to get attributes '['color_mode', 'color_temperature', 'current_x', 'current_y', 'color_loop_active']' on 'light_color' cluster:
2022-02-13 10:38:23 DEBUG (MainThread) [bellows.ezsp.protocol] Send command setExtendedTimeout: (b0:ce:18:14:03:52:e5:be, True)
2022-02-13 10:38:23 DEBUG (MainThread) [bellows.ezsp.protocol] Send command sendMulticast: (EmberApsFrame(profileId=260, clusterId=6, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=16, sequence=71), 0, 3, 72, b'\x01G\x01')
2022-02-13 10:38:23 DEBUG (MainThread) [bellows.ezsp.protocol] Send command readCounters: ()
2022-02-13 10:38:23 DEBUG (MainThread) [bellows.ezsp.protocol] Send command sendMulticast: (EmberApsFrame(profileId=260, clusterId=6, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_ENABLE_ROUTE_DISCOVERY: 256>, groupId=17, sequence=73), 0, 3, 74, b'\x01I\x01')

Its almost like the broadcast floods the network and takes forever to process then the controller times out before zigbee devices respond. Very hard to explain. I have not yet updated the firmware on the Sengled devices because I haven’t yet figured out how to do it without tearing down the network and pairing it all to a Sengled hub. Any thoughts on how to further troubleshoot would be very welcomed.

THanks!

finity · February 13, 2022, 5:29pm

I don’t know if explains all of your issues or not but you need to know that (most if not all?) Sengled bulbs don’t act as routers/repeaters so they act as if they were battery powered endpoint devices.

if you are counting on them to improve your mesh maybe that’s why things are failing.

NathanCu · February 13, 2022, 7:26pm

Also how are these ‘groups’ setup? Are they HA groups, HA light groups, Zigbee groups exposed through ZHA?

jlento · February 13, 2022, 7:37pm

Definitely surprised by that but have been adding the sengled repeaters as necessary.

jlento · February 13, 2022, 7:37pm

They are ZHA groups now

jlento · February 14, 2022, 12:31pm

As of this morning two sengled routers appear offline and are causing network issues. Not sure why they just drop.

jlento · February 14, 2022, 12:33pm


Logger: homeassistant.core
Source: components/zha/light.py:242 
First occurred: 7:27:16 AM (4 occurrences) 
Last logged: 7:27:16 AM

Error executing service: <ServiceCall light.turn_on (c:adapt_lgt_4356_interval_338): entity_id=['light.entryway_lights_zha_group_0x0011'], params=transition=45.0, brightness=255, color_temp=438>
Error executing service: <ServiceCall light.turn_on (c:adapt_lgt_0186_interval_338): entity_id=['light.kitchen_bar_zha_group_0x000a'], params=transition=45.0, brightness=255, color_temp=438>
Error executing service: <ServiceCall light.turn_on (c:adapt_lgt_fe55_interval_338): entity_id=['light.coffee_bar_zha_group_0x0008'], params=transition=45.0, brightness=255, color_temp=438>
Error executing service: <ServiceCall light.turn_on (c:adapt_lgt_a2de_interval_338): entity_id=['light.breakfast_room_zha_group_0x0010'], params=transition=45.0, brightness=255, color_temp=438>
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 492, in wait_for
    fut.result()
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/core.py", line 1648, in catch_exceptions
    await coro_or_task
  File "/usr/src/homeassistant/homeassistant/core.py", line 1667, in _execute_service
    await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)(
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 204, in handle_service
    await self.hass.helpers.service.entity_service_call(
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 668, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 930, in async_request_call
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 705, in _handle_entity_call
    await result
  File "/usr/src/homeassistant/homeassistant/components/light/__init__.py", line 504, in async_handle_light_on_service
    await light.async_turn_on(**filter_turn_on_params(light, params))
  File "/usr/src/homeassistant/homeassistant/components/zha/light.py", line 591, in async_turn_on
    await super().async_turn_on(**kwargs)
  File "/usr/src/homeassistant/homeassistant/components/zha/light.py", line 242, in async_turn_on
    result = await self._level_channel.move_to_level_with_on_off(
  File "/usr/local/lib/python3.9/site-packages/zigpy/group.py", line 44, in request
    res = await self.application.mrequest(
  File "/usr/local/lib/python3.9/site-packages/bellows/zigbee/application.py", line 490, in mrequest
    res = await self._ezsp.sendMulticast(
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 494, in wait_for
    raise exceptions.TimeoutError() from exc
asyncio.exceptions.TimeoutError

jlento · March 17, 2022, 12:59pm

Not sure what the solution is here but I upgraded to a Raspberry Pi 4 as the 3B was running out of swap heavily given the devices. I also added USB extension cables and about 8 more routers and it is much more performant. I am going to chock this one up to not enough resources on the controller or in the network to support the quantity of devices.