Community help wanted to whitelist all compatible Zigbee and Z-Wave USB adapters for automatic discovery in Home Assistant OS

Hedda · October 6, 2021, 11:54am

Anyone have Zigbee or Z-Wave USB adapters that is not already whitelisted for USB discovery in HA?

Back-story: Home Assistant developer bdraco recently added a new user-friendly feature for automatically discovering USB adapters for Home Assistant core integrated components like ZHA (for Zigbee) and Z-Wave JS integrations as now noted in their documentation.

https://www.home-assistant.io/blog/2021/09/01/release-20219/#usb-discovery

Zigbee USB adapter discovery:

https://www.home-assistant.io/integrations/zha/#discovery-via-usb-or-zeroconf

ZWave USB adapter discovery

https://www.home-assistant.io/integrations/zwave_js/#discovery-via-usb

These integrations in turn rely on the USB Discovery integration which watches host for USB devices:

https://www.home-assistant.io/integrations/usb/

This new feature means that if a user plug-in a USB adapter with a whitelisted identifier it will be auto-detected by Home Assistant OS and notify users that new devices have been discovered, however, for that to work automagically we first need a list of some identifying information about compatible USB devices added to the whitelist in each integration component.

I think it would be much appreciated by new users if existing users in the HA community that already has a working ZHA compatible Zigbee USB adapter that is presented with a unique description could give feedback here on its IDs and description so that it can hopefully be auto-detected with a “friendly name” in the future.

What is currently needed to automatically detect a Zigbee USB adapter is its “Product Description String” (a.k.a. device descriptor/descriptors) in combination with its “Vendor ID” (a.k.a. vid hex) and “Product ID” (a.k.a. pid hex).

Once we know a device descriptor, vid and pid we can attempt to get those whitelist those in Home Assistant core and its ZHA or Z-Wave JS integration components.

Using this detection method it is only possible to detect USB adapter that are configured with a such unique custom, which is only possible to configure on Zigbee USB adapters that uses USB-to-UART converter chip with a writable EEPROM (like WCH CH340B, FTDI FT231 series, or Silicon Labs CP210x series).

This means that if the manufacturer of your USB adapter has been cheap and used a USB-to-Serial bridge chip without a configurable EEPROM or missed configuring a custom “Product Description String” then Home Assistant OS can probably not auto-detect it as a unique device.

Note that if you as a customer would ask manufacturers to add this as a feature for new USB devices it should be noted a manufacturer only need to configure the “Product Description String”, and device manufacturers should not need to change the original VID or PID (as doing so would require providing custom drivers for the operating system).

https://github.com/home-assistant/core/blob/dev/homeassistant/components/zha/manifest.json

  "usb": [
   {"vid":"10C4","pid":"EA60","description":"*2652*","known_devices":["slae.sh cc2652rb stick"]},
   {"vid":"10C4","pid":"EA60","description":"*tubeszb*","known_devices":["TubesZB Coordinator"]},
   {"vid":"1A86","pid":"7523","description":"*tubeszb*","known_devices":["TubesZB Coordinator"]},
   {"vid":"1A86","pid":"7523","description":"*zigstar*","known_devices":["ZigStar Coordinators"]},
   {"vid":"1CF1","pid":"0030","description":"*conbee*","known_devices":["Conbee II"]},
   {"vid":"10C4","pid":"8A2A","description":"*zigbee*","known_devices":["Nortek HUSBZB-1"]},
   {"vid":"10C4","pid":"8B34","description":"*bv 2010/10*","known_devices":["Bitron Video AV2010/10"]}
  ],

https://github.com/home-assistant/core/blob/dev/homeassistant/components/zwave_js/manifest.json

  "usb": [
    {"vid":"0658","pid":"0200","known_devices":["Aeotec Z-Stick Gen5+", "Z-WaveMe UZB"]},
    {"vid":"10C4","pid":"8A2A","description":"*z-wave*","known_devices":["Nortek HUSBZB-1"]},
    {"vid":"10C4","pid":"EA60","known_devices":["Aeotec Z-Stick 7", "Silicon Labs UZB-7", "Zooz ZST10 700"]}
  ]

https://github.com/home-assistant/core/blob/dev/homeassistant/generated/usb.py

# match https://github.com/home-assistant/core/blob/dev/homeassistant/components/zha/manifest.json and https://github.com/home-assistant/core/blob/dev/homeassistant/components/zwave_js/manifest.json

PS: For developer details se PR(s) #/54904, #54935, #54986, #56201, and #56719

Hedda · October 8, 2021, 7:31pm

FYI, omerk from Electrolama wrote that their upcoming zzhp and zzh2 USB dongles will have FT231X USB-to-UART bridge + come with customised “Product Description” strings so can be discovered.

The original zzh will however keep using the CH340E without EEPROM so can not use USB discovery.

Hedda · October 8, 2021, 7:40pm

FYI, users who got one of the first batch ITead Sonoff Zigbee 3.0 USB Dongle Plus (which has Silabs CP2102 USB-to-UART bridge with programable EEPROM) report that ITead has, unfortunately, forgot/missed or chosen not to configure their own custom “Product Description” string to it.

That means that those cannot use USB discovery even though their hardware supports it, which is a real bummer. IMHO potential buyers should really write to them and ask them to configure this on them.

This would not be hard for ITead to fix as all they need to do is write their own “description” to the EEPROM of the CP2102N USB-to-UART chip when they flash the firmware on it before packaging it.

Luke1 · October 19, 2021, 8:54pm

Hello, I was just wondering if anyone has experienced this issue before, Is set it up originally and it was all working correctly. Then I tried installing ZIgbee2MQTT and it failed, I have deleted Zigbee2MQTT and tried going back to ZHA but it keeps failing, I get the error Failed to set up check the logs

Logger: homeassistant.config_entries
Source: components/zha/core/gateway.py:152
First occurred: 14:14:25 (1 occurrences)
Last logged: 14:14:25

Error setting up entry /dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_Serial Number shows Up-if00-port0 for zha
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 304, in async_setup
    result = await component.async_setup_entry(hass, self)  # type: ignore
  File "/usr/src/homeassistant/homeassistant/components/zha/__init__.py", line 102, in async_setup_entry
    await zha_gateway.async_initialize()
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 152, in async_initialize
    self.application_controller = await app_controller_cls.new(
  File "/usr/local/lib/python3.9/site-packages/zigpy/application.py", line 69, in new
    await app.startup(auto_form)
  File "/usr/local/lib/python3.9/site-packages/zigpy_znp/zigbee/application.py", line 196, in startup
    return await self._startup(
  File "/usr/local/lib/python3.9/site-packages/zigpy_znp/zigbee/application.py", line 255, in _startup
    await self._write_stack_settings(reset_if_changed=True)
  File "/usr/local/lib/python3.9/site-packages/zigpy_znp/zigbee/application.py", line 1129, in _write_stack_settings
    current_value = await self._znp.nvram.osal_read(
  File "/usr/local/lib/python3.9/site-packages/zigpy_znp/nvram.py", line 173, in osal_read
    raise KeyError(f"NV item does not exist: {nv_id!r}")
KeyError: 'NV item does not exist: <OsalNvIds.CONCENTRATOR_ENABLE: 50>'

Logger: zigpy.application
Source: /usr/local/lib/python3.9/site-packages/zigpy/application.py:73
First occurred: 14:14:25 (1 occurrences)
Last logged: 14:14:25

Couldn't start application

Logger: zigpy_xbee.api
Source: /usr/local/lib/python3.9/site-packages/zigpy_xbee/api.py:390
First occurred: 14:13:28 (1 occurrences)
Last logged: 14:13:28

at: No response to AP command

Logger: zigpy_zigate.api
Source: /usr/local/lib/python3.9/site-packages/zigpy_zigate/api.py:140
First occurred: 14:13:26 (1 occurrences)
Last logged: 14:13:26

Retry command 0x0002

I have tried to uninstall it Remove the Zigbee database file, Go To a backup of home assistant before I installed Zigbee2Mqtt.

When I reinstall ZHA it finds the serial device path to the USB without any Issues,

Created configuration for CP2102N USB to UART Bridge Controller, s/n: Serial Number shows up - Silicon Labs.

But I get this error

Any Ideas What I am doing wrong?

francisp · October 20, 2021, 4:40am

You are sure Zigbee2MQTT is not running anymore ?

Luke1 · October 20, 2021, 7:02am

Yes It’s definitely not running or installed

Hedda · October 20, 2021, 12:52pm

@Luke1 off-topic for this subject so please suggest that you create a brand new thread for that instead.

Hedda · October 21, 2021, 1:09pm

FYI, got a reply from ITead that they will change default “CP2102 USB to UART Bridge Controller” text in the Product Description String value field to “Sonoff Zigbee 3.0 USB Dongle Plus” in their next batch.

They already tested that USB discovery works in Home Assistant on Linux, but the change will require them to generate and provide a custom device driver for Microsoft Windows-based operating systems.

Why it would require them to generate and provide a custom device driver I do not know as changing only changing the “Product Description” and leaving the original VID and PID should not require that.

javicalle · October 24, 2021, 7:08pm

BitronVideo AV2010/10 ZHA stick to be added here:

https://github.com/home-assistant/core/pull/58337

Hedda · November 12, 2021, 12:25pm

ZigStar Stick USB discovery to be added here (PR patch by @mercenaruss = the maker of ZigStar):

HA core pull request → https://github.com/home-assistant/core/pull/59559
- ZHA docs PR → Update zha.markdown with ZigStar USB Discovery by Hedda · Pull Request #20320 · home-assistant/home-assistant.io · GitHub

ZigStar is by the way an open source hardware (USB dongle is based on CC2652P + CH340B chip):

https://community.home-assistant.io/t/zigstar-zigbee-coordinators-and-routers/338586

https://zig-star.com/projects/zigbee-stick-v4/

https://github.com/mercenaruss/zigbee-stick-v4

https://www.tindie.com/stores/zigstar/

Hedda · December 7, 2021, 7:42am

Hedda:

Hedda:

FYI, users who got one of the first batch ITead Sonoff Zigbee 3.0 USB Dongle Plus (which has Silabs CP2102 USB-to-UART bridge with programable EEPROM) report that ITead has, unfortunately, forgot/missed or chosen not to configure their own custom “Product Description” string to it.

That means that those cannot use USB discovery even though their hardware supports it, which is a real bummer. IMHO potential buyers should really write to them and ask them to configure this on them.

FYI, got a reply from ITead that they will change default “CP2102 USB to UART Bridge Controller” text in the Product Description String value field to “Sonoff Zigbee 3.0 USB Dongle Plus” in their next batch.

They already tested that USB discovery works in Home Assistant on Linux, but the change will require them to generate and provide a custom device driver for Microsoft Windows-based operating systems.

Why it would require them to generate and provide a custom device driver I do not know as changing only changing the “Product Description” and leaving the original VID and PID should not require that.

@tteck reported in another thread that the very latest batch of Sonoff Zigbee 3.0 USB Dongle Plus that has shipped now looks to have a unique product description value written to CP2102N EEPROM.

The question that remains to answer is which exact product description that ITead chose to go with.

Perhaps people with new batch can verify product description and test changes from this pull request:

https://github.com/home-assistant/core/pull/58166

That uses wildcards so think it will work as long as product description contains “sonoff” somewhere(?).

Hedda · December 8, 2021, 12:25pm

@tteck posted serial path old and new batch shipped from ITead with updated Sonoff Zigbee 3.0 USB Dongle Plus product description in https://community.home-assistant.io/t/sonoff-zigbee-3-0-usb-dongle-plus-by-itead-is-based-on-texas-instruments-cc2652p-can-be-pre-ordered-for-10-99/340705/174

Old ID

/dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_a4766cbfb593eb11b7611b4f3d98b6d1-if00-port0

New ID

/dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_201e5if00-port0

Serial path adds “usb-” + vendor as prefix and some ID and port-number as suffix + added underscores.

So assume new product description is “Sonoff Zigbee 3.0 USB Dongle Plus” (without underscores).

Again, can someone with new batch HW test PR? → https://github.com/home-assistant/core/pull/58166

I do not own a dongle from the new batch with the new product description so can not test it myself.

Hedda · December 13, 2021, 8:30am

FYI, ITead now posted a script/program that should let users themselves update the product description on already shipped dongles to supposedly match description already written to latest batch of dongles:

https://www.facebook.com/SONOFF.official/posts/2927251720919807

Tested it myself on a Windows 10 computer and it did not work for me (it closed without any messages).

Sadly ITead did not just release a script tool based on this open-source cp210x-program by VCTLabs:

https://github.com/VCTLabs/cp210x-program

Again, the point of having a unique custom product description is to allow USB auto-discovery in ZHA:

https://community.home-assistant.io/t/community-help-wanted-to-whitelist-all-compatible-zigbee-and-z-wave-usb-adapters-for-automatic-discovery-in-home-assistant-os/344412

Hedda · December 17, 2021, 11:43am

FYI, OpenMyDog submitted a new PR for ZHA discovery of Sonoff Zigbee 3.0 USB Dongle Plus:

https://github.com/home-assistant/core/pull/62171

github.com/home-assistant/core

Add USB discover for Sonoff zigbee dongle plus

home-assistant:dev ← OpenMyDog:sonoff

opened 08:41AM - 17 Dec 21 UTC

OpenMyDog

+7 -0

## Breaking change  ## Proposed change  ## Type of change  - [ ] Dependency upgrade - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New integration (thank you!) - [x] New feature (which adds functionality to an existing integration) - [ ] Breaking change (fix/feature causing existing functionality to break) - [ ] Code quality improvements to existing code or addition of tests ## Additional information  - This PR fixes or closes issue: fixes # - This PR is related to issue: - Link to documentation pull request: ## Checklist  - [x] The code change is tested and works locally. - [x] Local tests pass. **Your PR cannot be merged unless tests pass** - [x] There is no commented out code in this PR. - [x] I have followed the [development checklist][dev-checklist] - [x] The code has been formatted using Black (`black --fast homeassistant tests`) - [x] Tests have been added to verify that the new code works. If user exposed functionality or configuration variables are added/changed: - [ ] Documentation added/updated for [www.home-assistant.io][docs-repository] If the code communicates with devices, web services, or third-party tools: - [x] The [manifest file][manifest-docs] has all fields filled out correctly. Updated and included derived files by running: `python3 -m script.hassfest`. - [ ] New or updated dependencies have been added to `requirements_all.txt`. Updated by running `python3 -m script.gen_requirements_all`. - [ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description. - [ ] Untested files have been added to `.coveragerc`. The integration reached or maintains the following [Integration Quality Scale][quality-scale]:  - [ ] No score or internal - [ ] 🥈 Silver - [ ] 🥇 Gold - [ ] 🏆 Platinum  To help with the load of incoming pull requests: - [ ] I have reviewed two other [open pull requests][prs] in this repository. [prs]: https://github.com/home-assistant/core/pulls?q=is%3Aopen+is%3Apr+-author%3A%40me+-draft%3Atrue+-label%3Awaiting-for-upstream+sort%3Acreated-desc+review%3Anone  [dev-checklist]: https://developers.home-assistant.io/docs/en/development_checklist.html [manifest-docs]: https://developers.home-assistant.io/docs/en/creating_integration_manifest.html [quality-scale]: https://developers.home-assistant.io/docs/en/next/integration_quality_scale_index.html [docs-repository]: https://github.com/home-assistant/home-assistant.io

Hedda · February 3, 2022, 9:33am

FYI, Home Assistant 2022.2 release notes now menation USB discover of Sonoff Zigbee Dongle Plus:

https://www.home-assistant.io/blog/2022/02/02/release-20222/#all-changes

“Add USB discover for Sonoff zigbee dongle plus (@OpenMyDog - #62171) (zha docs)”

Again, this only applies to newer batches of shipped dongles the initial batches missing ID in CP2102N, as if your dongle does not have this unique description then it will be discovered as dongle for Z-Wave.