Notes on implementing Matter

A few notes about Matter and HA …

  1. Matter does not require Thread, it is optional. Matter simply requires IP networking, preferably IPv6. I have a working Wifi IPv6 Matter setup on my desk that has been running for a year. Ethernet or wifi makes no difference. My server is Linux and my devices are ESP32-C3 based.

  2. Thread is a transport layer similar to wifi. The Matter protocol is the same on all transport layers: Wifi, Thread and Ethernet. A border router is needed to bridge a Thread network to wifi/Ethernet. This border router is a form of a normal IPv6 router. Thread does support mesh similar to Zigbee. Thread support is optional. It is equally valid to network light bulbs with Wifi or Thread. Of course if you buy Thread based devices you’re going to need Thread in the server.

  3. Thread and Zigbee are not interoperable. They both use 802.15.4 but they won’t talk to each other. If you want a server to support both you need two radios. Zigbee smeared the protocol layers together where as with Matter they are clearly separated into Matter and Thread. Matter+Thread completely replaces Zigbee – Zigbee is a dead end. Some Zigbee devices are upgradable, many are not.

  4. Matter supports bridges. This is important for Home Assistant. Implement a Matter bridge on the Home Assistant server and then write a generic HA Matter proxy device driver. Now proxy each of the existing Home Assistant devices onto the Matter network. The mapping isn’t perfect because Matter is missing a lot of device types, but for sure you can proxy on/off, volume, inputs, channels, etc. By proxying you pick up voice support without needing the cloud.
    connectedhomeip/examples/bridge-app at master · project-chip/connectedhomeip · GitHub

  5. Matter has screwed up scene support. If you want scenes HA is going to have to implement them.

  6. It is possible to start the construction of a new software core for HA using the Matter proxy in #4. This new core would be Matter only, the proxy devices serve as a bridge to legacy. You can also define your own device types in Matter (example, cameras) as part of this. Then when Matter comes out with the real device type switch over to it.

  7. There is already a Python library for talking to Matter.
    connectedhomeip/python_chip_controller_building.md at d346617f388398b40a207c600fb0c0dee3321902 · project-chip/connectedhomeip · GitHub

  8. There are easily buildable demo devices: ESP32 light
    connectedhomeip/examples/lighting-app/esp32 at master · project-chip/connectedhomeip · GitHub

  9. Build a couple of esp32 demo lights and then you can control them from the Linux command line using connectedhomeip/examples/chip-tool at master · project-chip/connectedhomeip · GitHub
    There is a chiptool demo for IOS connectedhomeip/src/darwin/CHIPTool at master · project-chip/connectedhomeip · GitHub and Android connectedhomeip/src/android/CHIPTool at master · project-chip/connectedhomeip · GitHub

  10. The IOS/Android support is native. It would be useful if someone wrote a Javascript wrapper for Matter. Expo support for Matter | Voters | Expo

12 Likes

Internally Matter is using a very painful tool called Zap. Zap takes XML templates, connectedhomeip/src/app/zap-templates/zcl/data-model/chip at master · project-chip/connectedhomeip · GitHub and then expands them into tens of megabytes of static API code. This static code is exported by the Python library which is why it is 13MB of python plus a 40MB C library. Any time you change anything, you have to regenerate all of this == painful.

Python is a dynamic language and it doesn’t need a static API like C. So if you are willing to expend some effort there is another way to do this. The idea is to only use the core datatype code from Matter and ignore the other 90%. To do this HA would talk a device and retrieve its DeviceType. Use this device type to load the corresponding device XML file into memory. Now use that XML to DYNAMICALLY build the proxy object for the devices. In other words, make it work like how a browser dynamically builds a DOM tree. Next, take the browser analogy further. You have a DOM tree. Make a stylesheet language to display it. Note that it is also very easy to automate this DOM.

This is a very different model than how Matter is implemented, but it is still possible to build it using the provided code base. This model has one key feature — if you want to create a new device type, just make up a new XML file. As long as you don’t need new core datatypes (very rare) this will magically work. And later when Matter makes up new devices types – no coding – just edit the XML and HA will support them.

Consider a reimplementation of the HA core using this technique. HA would create local proxies between the core Matter Bridge support and the existing HA device drivers. As each proxy gets written provide an XML file describing it. That XML would create a proxy (DOM) of instantiated devices. These devices do not need to exist in the Matter standard since you used the extensible XML system to create them. By switching to a dynamic model you could see the Matter code size drop down to just a couple MB. You could even do a pure Python implementation if you have a lot of free time to burn.

By the way, the best solution would have been if the devices sent you the XML describing themselves, but alas that did not happen in the Matter standard. Then they could have told you about vendor extensions. Instead Matter half implemented an enumeration API that does not completely describe the device. So ignore this API and just use a local XML file on the HA server.

2 Likes

I know this is an old post, but I thought that Thread and ZigBee were going to use dotdot (||:) to communicate with each other:

Nope, Dotdot does not have anything to do with Matter. The only thing they got in common is at an architectural design level as both Matter’s data model and Dotdot’s data model are both inspired by the Zigbee Cluster Library (ZCL).

By the way, for the Matter implementation in Home Assistant you want for now to follow this other thread → Chip / Matter support (including Thread) and keep an eye on https://github.com/home-assistant-libs/python-matter-server + home-assistant/addons-development (though looks like they will soon move that from the development to the official repository as per https://github.com/home-assistant/addons/pull/2692 and https://github.com/home-assistant/core/pull/79372) as well as following developers https://github.com/agners + https://github.com/MartinHjelmare on GitHub

@Hedda : Yeah i know, I put Matter as opposed to Thread - amended

which was in relationship to point 3. above:Notes on implementing Matter

  1. Thread and Zigbee are not interoperable. They both use 802.15.4 but they won’t talk to each other. If you want a server to support both you need two radios. Zigbee smeared the protocol layers together where as with Matter they are clearly separated into Matter and Thread. Matter+Thread completely replaces Zigbee – Zigbee is a dead end. Some Zigbee devices are upgradable, many are not.

Why is getting 2 radios ( even 2 USB sticks) such a big deal with a HA server? Can’t you simply run two meshes?

There’s certainly nothing wrong with having different applications running on separate channels and being able to coexist, as we already have that with WiFi, ZigBee, Zwave etc.

Matter SDK is really painful to use, especially Python SDK. Adding feature is meraly impossible, as it has to call to many layers of C++ functions. Truly wish if any brave man can rewrite the framework in pure python.