ZHA automatic network discovery of Zigbee coordinator bridges/gateways (Ethernet/WiFi network adapters) that support Zeroconf or SSDP?

Feature Requests
,
1

Please consider adding automatic network discovery to Home Assistant’s ZHA (Zigbee) integration.

UPDATE! Support for Zeroconf discovery has now been added to ZHA but only Tube’s gateways so far:

https://www.home-assistant.io/integrations/zha#discovery-via-usb-or-zeroconf

Home Assistant OS/Supervisor already have some automatic network scanning + discovery features and functions needed for automatically discovering some types of network services and devices, and each integration for Home Assistant can choose to optionally opt-in to utilize these automatic network discovery features/functions by adding Zeroconf section and SSDP section, etc. to their manifest.json as described in their documentation:

https://www.home-assistant.io/integrations/zeroconf/

https://www.home-assistant.io/integrations/ssdp/

Reason for this “Service Discovery” / “Network Discover” request to Home Assistant’s ZHA developers that seeing more and more ser2net or com0com/com2tcp type network-attached Zigbee adapters (a.k.a. Zigbee bridge/gateway) that runs a serial-to-IP proxy-server application that basically only allows it to tunnel a serial connection over your local home network (a.k.a. ser2net type serial to TCP/IP bridges/proxy appliance server devices) to allow you to place the Zigbee coordinator for ZHA in the most optimal location.

From an end-user experience point-of-view, it would be very user-friendly if such devices could be automagically discovered by Home Assistant during the ZHA integration’s config flow for the initial installation and the adding of a network-attached Zigbee adapter.

As most ZHA users probably know about now, ZHA’s UI config flow today already allow to manually configure the serial device path to use a TCP/IP network socket (using socat) instead of a serial port:

  • ESPHome based DIY style bridge: socket://[IP]:[PORT] or socket://[FQDN]:[PORT] (if got local DNS), example socket://tube_zb_gw.local:6638
  • Sonoff ZBBridge if its ESP8266 hacked with Z2T (Zigbee2Tasmota): socket://[IP]:[PORT] for example socket://192.168.1.11:8888
  • ZiGate Gateway: socket://[IP]:[PORT] for example socket://192.168.1.10:9999
  • Xiaomi DGNWG05LM and ZHWG11LM gateways with Openlumi OpenWrt firmware: socket://[IP]:[PORT] for example socket://192.168.1.10:9999
  • Tuya TYGWZ-01 and Lidl Silvercrest gateways hacked for serial-to-ip: socket://[IP]:[PORT] for example socket://192.168.1.11:8888

I would think that the discovery features that is missing there and needs to be added are automatically discovered by the ZHA integration is the IP + TCP-port + radio-type + baud rate port speed + data flow control method, as only then the normal zigpy radio library probe methods could step in and take it from there to do its probing for the Zigbee radio firmware?

Anyway, such devices addressed via IP are already supported without auto-discovery by ZHA and zigpy and today include:

If support for Zeroconf and SSDP could be first added to ZHA inside Home Assistant then it might be more incentive to Zigbee gateway/bridge manufacturers to add needed discovering protocols to those?

mDNS and/or UPnP could be alternative network discovery methods but probably not as suited for Zigbee ser2net type bridges, or?

https://www.home-assistant.io/integrations/discovery/

Zeroconf and/or SSDP is presumable better suited and again ZiGate WiFi and Tube’s Zigbee Gateways already supports those, or?

https://developers.home-assistant.io/docs/creating_integration_manifest/#zeroconf

https://developers.home-assistant.io/docs/creating_integration_manifest/#ssdp

ESPHome which is very popular in the Home Assistant community is being used as ESP32 firmware by Tube’s Zigbee Gateways who sells both Texas Instruments and Silicon Labs based Zigbee gateways.

https://www.home-assistant.io/integrations/esphome/

“ESPHome can be auto-discovered by Home Assistant. If an instance was found, it will be shown in the top of the list of integrations as “Discovered”. If that is the case click on the Configure button to start setting up the discovered instance.”

Tasmota based gateways could similarly be automatically discovered if the Tasmota device is configured for native discovery by the user:

https://www.home-assistant.io/integrations/tasmota/

ZiGate Gateway firmware does feature Zeroconf and the old ZiGate integration by @doudz also had for Zeroconf discover as a feature:

https://github.com/doudz/zigate/blob/f42fbad6f677f67442e05c845ab4f3c2029b5431/zigate/transport.py#L385

https://github.com/zigpy/zigpy-zigate/issues/36

Note! See note stating; “If the integration supports zeroconf or ssdp , these should be preferred over dhcp as Zeroconf and SSDP generally offers a better user experience” at https://developers.home-assistant.io/docs/creating_integration_manifest/#dhcp

2

FYI; dmulcahey begun working on adding Zeroconf discovery for Tube’s Zigbee coordinators to ZHA:

Presumably, it could later also be extended to support other Zigbee bridges that also support Zeroconf.

@doudz Any chance you be willing to extend this to support for discovering of ZiGate WiFi Gateway?

EDIT: Pobably best now to see code used for Tube’s Zigbee Gateway as the reference implementation:

https://github.com/tube0013/tube_gateways/blob/main/V2_tube_zb_gw_cc2752p2/ESPHome/tube_zb_gw_cc2652p2v2.yml

zeroconf:
  - service: tubes_zb_gw
    protocol: tcp
    port: 6638
    txt:
      version: 1.0
      radio_type: znp
      baud_rate: 115200
      data_flow_control: software

To whitelist Zeroconf for more radio types like “zigate” in HA’s zeroconf and ZHA need to do PR for:

https://github.com/home-assistant/core/blob/dev/homeassistant/generated/zeroconf.py

 "_esphomelib._tcp.local.": [
        {
            "domain": "zha",
            "name": "tube*"
        }
    ],

and:

https://github.com/home-assistant/core/blob/dev/homeassistant/components/zha/manifest.json

"zeroconf": [
    {
      "type": "_esphomelib._tcp.local.",
      "name": "tube*"
    }
  ],

as well as:

https://github.com/home-assistant/core/blob/dev/homeassistant/components/zha/config_flow.py

async def async_step_zeroconf(self, discovery_info: DiscoveryInfoType):
        """Handle zeroconf discovery."""
        # Hostname is format: livingroom.local.
        local_name = discovery_info["hostname"][:-1]
        node_name = local_name[: -len(".local")]
        host = discovery_info[CONF_HOST]
        device_path = f"socket://{host}:6638"

        if current_entry := await self.async_set_unique_id(node_name):
            self._abort_if_unique_id_configured(
                updates={
                    CONF_DEVICE: {
                        **current_entry.data.get(CONF_DEVICE, {}),
                        CONF_DEVICE_PATH: device_path,
                    },
                }
            )

        # Check if already configured
        if self._async_current_entries():
            return self.async_abort(reason="single_instance_allowed")

        self.context["title_placeholders"] = {
            CONF_NAME: node_name,
        }

        self._device_path = device_path
        self._radio_type = (
            RadioType.ezsp.name if "efr32" in local_name else RadioType.znp.name
        )
3

I’ll take a look, it should be easy to add, but it requires Zigate Wifi firmware 2.x (firmware 1.3 doesn’t support mDns)

1 Like
4

Great! I believe that automatic discovery is another leap in making ZHA setup even more user-friendly.

But is mDNS support a prerequisite for this to work on ZHA side or is that only for the ZiGate firmware?

Reading the description by @bdraco in that PR #48420 it sounded as if it might work with IP too?

5

FYI, this PR for ZHA was merged 2-weeks ago but did not make it into Home Assistant 2021.4 release:

6

@bool2 Any chance you have time + interest to add auto-discovery to Tuya/Lidl/Silvercrest gateway?

7

Standard Zeroconf TCP broadcast with mDNS name should be supported as I understand, but then need to explicitly add it to ZHA:

https://en.wikipedia.org/wiki/Zero-configuration_networking#DNS-SD_with_multicast

“mDNS uses packets similar to unicast DNS to resolve hostnames except they are sent over a multicast link. Each host listens on the mDNS port, 5353, transmitted to a well-known multicast address and resolves requests for the DNS record of its .local hostname (e.g. the A, AAAA, CNAME) to its IP address. When an mDNS client needs to resolve a local hostname to an IP address, it sends a DNS request for that name to the well-known multicast address; the computer with the corresponding A/AAAA record replies with its IP address. The mDNS multicast address is 224.0.0.251 for IPv4 and ff02::fb for IPv6 link-local addressing. DNS Service Discovery (DNS-SD) requests can also be sent using mDNS to yield zero-configuration DNS-SD. This uses DNS PTR, SRV, TXT records to advertise instances of service types, domain names for those instances, and optional configuration parameters for connecting to those instances. But SRV records can now resolve to .local domain names, which mDNS can resolve to local IP addresses.”

Home Assistant documentation has this information on Zeroconf in HA:

https://www.home-assistant.io/integrations/zeroconf/

https://developers.home-assistant.io/docs/creating_integration_manifest/#zeroconf

So far ZHA developers only explicitly added Zeroconf discovery of Tube’s ESPHome based gateways → Add discovery for Tube's Zigbee coordinators to ZHA by dmulcahey · Pull Request #48420 · home-assistant/core · GitHub

If gateway developers/hackers could script/hack Zeroconf broadcast configuration parameter types to ZiGate gateway and Tuya/Lidl/Silvercrest gateways as the first step then in a second step could explicitly add your chosen local record name to Home Assistant and set “zha” as the domain for HA to make ZHA discover ít:

https://developers.home-assistant.io/docs/creating_integration_manifest/#zeroconf

Believe Zeroconf for ZHA needs to include hostname, port, baud rate, and local name which ZHA can translate into the radio type.

https://github.com/home-assistant/core/pull/48420/commits/6ba0629fcb68d12a34cd242f8ee70c5e9d89634c

PS: Automatic discovery via Zeroconf or SSDP should further simply hacking and adding Tuya/Lidl/Silvercrest gateway to ZHA:

https://paulbanks.org/projects/lidl-zigbee/ha.html

8

@bdraco Does Zeroconf support DNS-SD (Rendezvous) “TXT records” to pass values to integrations?

http://www.zeroconf.org/Rendezvous/txtrecords.html

Could Zeroconf DNS-SD (DNS Service Discovery) pass “TXT record” values to integration config flow?

Example of config parameters that Zeroconf could be passed along to ZHA integration via TXT records:

  • radio_type=ezsp
  • baud_rate_speed=115200
  • data_flow_control=software
  • port=8888
  • protocol=tcp

TXT records could also be used to pass along ex. hostname, versions, location, MAC address, etc…

https://en.wikipedia.org/wiki/Zero-configuration_networking#DNS-based_service_discovery

“Each service instance is described using a DNS SRV and DNS TXT record. A client discovers the list of available instances for a given service type by querying the DNS PTR record of that service type’s name; the server returns zero or more names of the form ., each corresponding to a SRV/TXT record pair. The SRV record resolves to the domain name providing the instance, while the TXT can contain service-specific configuration parameters.”

https://en.wikipedia.org/wiki/Zero-configuration_networking#DNS-SD_with_multicast

“DNS Service Discovery (DNS-SD) requests can also be sent using mDNS to yield zero-configuration DNS-SD. This uses DNS PTR, SRV, TXT records to advertise instances of service types, domain names for those instances, and optional configuration parameters for connecting to those instances. But SRV records can now resolve to .local domain names, which mDNS can resolve to local IP addresses.”

https://en.wikipedia.org/wiki/TXT_record

“TXT record (short for text record) is a type of resource record in the Domain name system (DNS) used to provide the ability to associate arbitrary text with a host or other name, such as human readable information about a server, network, data center, or other accounting information. It is also often used in a more structured fashion to record small amounts of machine-readable data into the DNS.”

9

Yes, example: core/config_flow.py at d4ed65e0f53fcab991b13061b1101470f24287a6 · home-assistant/core · GitHub

1 Like
10

FYI, thegroove working on Zeroconf custom component (with DNS TXT records support) for ESPHome:

https://github.com/thegroove/esphome-zeroconf

https://github.com/thegroove/esphome-zha-ezsp-zeroconf

More information on the idea of using DNS TXT records for passing along setting parameters to ZHA:

https://github.com/thegroove/esphome-zbbridge/issues/1

11

FYI, Tube’s Zigbee Gateways are now (only) devices listed for ZHA as discoverable via Zeroconf, see:

https://www.home-assistant.io/integrations/zha/#discovery-via-usb-or-zeroconf

12

Hi, we are working on firmware for ZigStar LAN Coordinator ,we included zerconf in the firmware.
But unfortunately till now I didnt understand how to submit a PR and what information should be provided.

13

@Hedda @doudz Can i get some help with this?

14

@mercenaruss I think to get your Texas Instruments “znp” based Zigbee coordinator adapter added to network discovery by both the Zeroconf integration in Home Assistant as well as the ZHA integration in Home Assistant you should look at how @tube0013 and ZHA developers did it for the competing Tube’s CC2652P2 USB-powered Zigbee to Ethernet Serial Coordinator) product which is now discovered by Home Assiatnt via mDNS using a combination of its “tube_zb_gw_cc2652p2.local.” DNS name as an unique identifier for the Zeroconf service an addition of Zigbee radio configuration settings for ZHA which he adds in his firmware via DNS TXT records an an extension of information sent via Zeroconf:

https://github.com/tube0013/tube_gateways/blob/7bde5f09852b95fd98d146f659464678bd6ce2f9/V2_tube_zb_gw_cc2752p2/ESPHome/tube_zb_gw_cc2652p2v2.yml

zeroconf:
  - service: tubes_zb_gw
    protocol: tcp
    port: 6638
    txt:
      version: 1.0
      radio_type: znp
      baud_rate: 115200
      data_flow_control: software

So you would want to use something similar but replace Zeroconf service name as a unique identifier to instead present like for example “zigstar_zb_gw_cc2652p2.local.” and include DNS TXT records in Zeroconf with similar config for radio_type, baud_rate and data_flow_control that can be used by the ZHA integration config flow, ex:

zeroconf:
  - service: zigstar_zb_gw
    protocol: tcp
    port: 6638
    txt:
      version: 1.0
      radio_type: znp
      baud_rate: 115200
      data_flow_control: software

Then once Zeroconf in the firmware presents both unique identifier and correct radio configuration settings for ZHA you need to submit a pull request to Home Assistant’s zeroconf and ZHA manifests that match your unique identifier:

https://github.com/home-assistant/core/blob/04f51e599a2ee8599143288a776caa365da41b0f/homeassistant/generated/zeroconf.py

There I think you need to extend:

    "_esphomelib._tcp.local.": [
        {
            "domain": "esphome"
        },
        {
            "domain": "zha",
            "name": "tube*"
        }
    ],

to

    "_esphomelib._tcp.local.": [
        {
            "domain": "esphome"
        },
        {
            "domain": "zha",
            "name": "tube*"
        }
    ],
    "_zb_gw_cc2652p2.local.": [
        {
            "domain": "zha",
            "name": "zigstar*"
        }
    ],

https://github.com/home-assistant/core/blob/04f51e599a2ee8599143288a776caa365da41b0f/homeassistant/components/zha/manifest.json

There I think you need to extend:

 "zeroconf": [
    {
      "type": "_esphomelib._tcp.local.",
      "name": "tube*"
    }

to in this example, for that Zeroconf discovery to be used by ZHA I believe something like this(?):

 "zeroconf": [
    {
      "type": "_esphomelib._tcp.local.",
      "name": "tube*"
    }
    {
      "type": "_zb_gw_cc2652p2.local.",
      "name": "zigstar*"
    }

Again, need to add it to both Zeroconf discovery as well as to ZHA integration in Home Assistant core.

Note! The DNS server in your network router at home of course needs to support mDNS (multicast DNS) so .local names can be used so be sure to enable/allow it if it is currently set to disabled.

15

@doudz Since are no other ZiGate radios you would also need to add it as a radio type to config flow?

https://github.com/home-assistant/core/blob/04f51e599a2ee8599143288a776caa365da41b0f/homeassistant/components/zha/config_flow.py

    async def async_step_zeroconf(self, discovery_info: DiscoveryInfoType):
        """Handle zeroconf discovery."""
        # Hostname is format: livingroom.local.
        local_name = discovery_info["hostname"][:-1]
        node_name = local_name[: -len(".local")]
        host = discovery_info[CONF_HOST]
        device_path = f"socket://{host}:6638"

        if current_entry := await self.async_set_unique_id(node_name):
            self._abort_if_unique_id_configured(
                updates={
                    CONF_DEVICE: {
                        **current_entry.data.get(CONF_DEVICE, {}),
                        CONF_DEVICE_PATH: device_path,
                    },
                }
            )

        # Check if already configured
        if self._async_current_entries():
            return self.async_abort(reason="single_instance_allowed")

        self.context["title_placeholders"] = {
            CONF_NAME: node_name,
        }

        self._device_path = device_path
        self._radio_type = (
            RadioType.ezsp.name if "efr32" in local_name else RadioType.znp.name
        )
16

It’s on my list to redo how this is done so as to not rely on the ESPhome mdns, and communicate all settings via the mDNS. I have ESP firmware doing it now just need to work with devs to get the change over to ZHA.

1 Like
17

@tube0013 Any news on making it communicate all configurations/settings to ZHA via DNS record(s)?

18

Still on my list. Newer V2 Ethernet uses the mdns component and sends out the information. I’m a bit hesitant though after the serial side got knocked out with the esphome updates. I know it’s been worked around up stream but it’s not been pushed into thegroove’s implantation which also has the binary sensor for serial connection which is broken now as well.

1 Like
19

FYI, Home Assistant founders explicitly mention they will prioritize focus on features like these this year:

https://www.home-assistant.io/blog/2022/01/19/streamlining-experiences/

Recommend check out this video at around 11 min 50 sec in where talk about that focus point is about making it easier to get started with integrations and features performing common in fewer steps, etc.

Hoping that this feature idea will peak the interest of even more network Zigbee coordinator developers.

20

Any ZHA users/devs with ZiGate Ethernet hardware out there willing to take another stab at adding Zeroconf support for Zigate?

https://www.home-assistant.io/integrations/zha#discovery-via-usb-or-zeroconf

I do not have ZiGate hardware myself but did a ZHA pull request blind for Home Assistant core here that could be used as a reference:

https://github.com/home-assistant/core/pull/58467

It is a relativly small change but it needs to be tested by someone with ZiGate Ethernet hardware before it will be merged into HA.

Anyway, the missing DNS records for Zeroconf discovery has now been merged to ZiGate Ethernet firmware by fairecasoimeme here:

https://github.com/fairecasoimeme/ZiGate-Ethernet/pull/8

For reference also see:

https://github.com/fairecasoimeme/ZiGate-Ethernet/issues/7