[Proposal] A Paradigm Shift in Matter IoT Development

A Paradigm Shift in Matter IoT Development

From Selecting a Device Type to Defining an Action

A Proposed Shift in Design Thinking for Developers


1. The Structural Problems with Matter Development Today

Problem 0. The Entry Barrier: Matter Development Is Hard

Before writing a single line of actual behavior code, implementing a Matter device from scratch requires clearing a long series of hurdles.

ESP-IDF setup  →  Matter SDK build  →  Manual cluster mapping  →  Commissioning logic
NVS storage structure  →  GPIO conflict management  →  State management  →  … then, finally, behavior

Even when starting from a reference firmware, any deviation quickly requires digging into the SDK internals. Infrastructure code overwhelms actual behavior code.

Problem 1. The Core Issue: Actions Have No Names

The typical Matter/IoT development workflow looks like this:

  • Pick the closest match from the platform-defined Device Type list
  • Combine the features available within the clusters and attributes that Type supports
  • Fit the hardware (GPIO pins, sensors) into the Type’s structure
  • Copy an existing example or reference firmware and modify it

This is not designing behavior. It is fitting hardware into a pre-existing template.

When a new behavior is needed, developers pick the closest existing Type and force-fit it. If the platform does not define it, you cannot build it.


Additional Problems in the Age of AI

Problem 2 — AI Does Not Know What a Device Actually Does → It Guesses

Without named Actions, AI has no way to know a device’s intent. It reads the Device Type and cluster structure and infers meaning. This is not understanding — it is guessing.

In practice, diverse behaviors get reduced to simple on/off control patterns — not because they are the same, but because the system has no better way to describe them. A heater, a fan, and an electric blanket can all be controlled identically, not because they are equivalent, but because there is no structural way to express what they actually mean.

In the current Matter implementation, UserLabel (0x0041) is a user-editable display label. It is not used as a trusted system-level identifier. There is no official way to communicate to the system what an endpoint actually does.

Problem 3 — Actions Have No Safety Boundaries → AI Issues Commands Without Knowing the Limits

Constraints like maximum run time, risk level, and auto-shutoff conditions are hardcoded inside the firmware. They cannot be read from the outside.

AI issues commands without knowing what is allowed. It controls a kitchen light and an electric blanket in the same way. Safety conditions are guessed, not known.


2. The Proposal: Define the Action First

Instead of starting by selecting a Device Type, consider a structure where you think through everything about a single Action from the very beginning.

2-1. Nine Questions to Define an Action

① What is the intended action?
② What physical effect occurs?
③ What boundaries must never be crossed? ← newly added question
④ In what context is this Action valid?
⑤ What event triggers the action?
⑥ What is the goal value?
⑦ What is the maximum execution time?
⑧ What conflicts must be checked before starting?
⑨ What protection conditions apply before stopping?

Once you can answer these questions, the Action has a name. A collection of named Actions becomes a device.

Devices that do not belong to any existing Device Type become possible.

Where conventional development starts with “Which Type should I pick and fit this into?”, this approach starts with “What is this Action?”

2-2. Simplifying the Execution Model

To keep the execution model simple and explicit, every Action maps to one of two fundamental units.

Button (immediate execution) Switch (state maintained)
Instantaneous pulse action Persistent state maintained
e.g. doorbell, notification, one-shot trigger e.g. light ON/OFF, keeping a heater running

These two are the starting point. Additional clusters can be added as needed for more precise control.

This has a practical benefit as well: since all behaviors map to a simple ON/OFF model, only the OnOff cluster is needed on the Matter side as a baseline. This can significantly reduce certification costs and time, while preserving flexibility at the action level.


3. Implementation Within the Existing Matter Structure

Without any spec changes, reinterpreting existing Matter cluster structures is sufficient to address both AI-era problems.

3-1. Solving Problem 2 — The endpointLabel Pattern (UserLabel 0x0041)

The name of an Action needs to be communicated in a form the system can trust. UserLabel is user-editable, making it unsuitable as a system-level identifier.

Proposal: Introduce a pattern (endpointLabel) that embeds a manufacturer-defined identifier within the label field.

// UserLabel (0x0041) usage pattern
"UserLabel": "endpointLabel:keep-warm|display:Keep Warm"

// AI and automation systems can directly read the endpoint identity
// via the endpointLabel key -- no inference required
// -> No spec change needed; works within existing structure

This pattern has been proposed to the Matter SDK repository (issue #71521). It works within the existing structure without any spec changes.

3-2. Solving Problem 3 — Reinterpreting Fixed Label (0x0040)

The Fixed Label cluster (0x0040) is manufacturer-defined and read-only. Until now, it has only been used for physical orientation labels and UI labels.

By storing an Action’s safety boundaries declaratively in this cluster, the device itself can announce its own limits to the outside world.

// Fixed Label (0x0040) -- declaring safety boundaries
"FixedLabel": [
    { "label": "purpose", "value": "heating" },
    { "label": "risk",    "value": "high" },
    { "label": "auto-off","value": "300s" }
]

// Safety conditions hardcoded inside firmware -> unreadable from outside
// Conditions declared in Fixed Label -> AI, automation systems, other devices
//                                       can read them directly, without inference

Key point: Rather than safety conditions being hidden inside firmware, the action itself declares its own limits. Both of these work within the existing Matter structure, with no spec changes required.


4. Current Approach vs. Proposed Approach

Current Approach Proposed Approach
Starting point: Select a Device Type Starting point: Define an Action
Design direction: Hardware → fit into template Design direction: Intent → map to hardware
New behavior: Force-fit closest Type New behavior: Add an Action directly
AI recognition: Infers from cluster structure AI recognition: Reads endpointLabel directly
Safety boundaries: Hardcoded in firmware Safety boundaries: Declared in Fixed Label (read-only)
Behavior change: Rewrite firmware + reflash Behavior change: Update via configuration
Undefined device: Cannot be implemented Undefined device: Can be implemented

5. Why This Matters

The key outcomes of this proposal:

  • Behavior is defined in units of Actions, not Device Types
  • Safety boundaries stored declaratively via Fixed Label (0x0040) — readable by AI and automation systems without inference
  • Stable endpoint identification via the endpointLabel pattern in UserLabel (0x0041)
  • Action definitions are machine-readable — enabling AI/automation integration without hardcoding

This approach requires no Matter spec changes. Reinterpreting existing structures is sufficient.


Behavior is no longer code.
It becomes data that can be inspected and validated.


A Note on Execution

This is not about giving AI more control. AI does not decide actions. It validates predefined meaning and boundaries before execution.

Responsibility Model

To make this work in the physical world, responsibility must be clearly separated:

  • Manufacturers define the meaning of actions and safety boundaries
  • AI validates those definitions before execution
  • Users decide what is allowed based on those rules

Every device built so far was designed to be operated by humans.
Devices built from now on should be designed to be understood by AI.

A device is not a type.
A device is a composition of actions.

It starts with giving Actions a name.

1 Like

Wow, that a lot of LLM writing…
TA;DR

Anyway is seems to me to be a Feature Request, and if it actually is, I suggest Doing a little more human version and posting it in home-assistant · Discussions · GitHub

1 Like

… that seem irrelevant to HA or ESPHome as of today.
Neither allow to “create” a matter device, so why this is posted in this forum evades me, unless it’s spam.

1 Like

Besides that the proposal is also not really useful.
CSA controls the Matter standard and the standard is locked to the device types that they define, so the idea of starting with the action is not avoiding the fact that you might have to force-fit your product to a device type.

1 Like

While reviewing the discussion, I realized that one of the examples in the original document used an incorrect representation:

"UserLabel": "endpointLabel:keep-warm|display:Keep Warm"

This was a mistake during the drafting/editing process, and it also violates the Matter Value length constraint.

The intended structure is:

"UserLabel": {
  "LabelList": [
    { "Label": "endpointLabel", "Value": "Keep Warm" }
  ]
},

"FixedLabel": {
  "LabelList": [
    { "Label": "purpose",  "Value": "heating" },
    { "Label": "risk",     "Value": "high" },
    { "Label": "auto-off", "Value": "300s" }
  ]
}

The core idea of the document can be summarized very simply:

  1. If platforms define common identifiers for UserLabel usage, development of simple Matter devices becomes much easier, allowing many more products to be created without introducing new Device Types.

  2. If platforms encourage structured FixedLabel usage like the example above, it could contribute to safer AI-assisted control by exposing semantic intent and operational boundaries in a machine-readable form.

This proposal does not require any changes to the Matter specification.

Matter already defines User Label and Fixed Label structures. Today, these fields are often treated only as optional metadata or UI hints, but they may also provide a lightweight semantic layer for platforms and controllers.

For example:

  • UserLabel can expose user-facing action names
  • FixedLabel can expose manufacturer-defined semantic information such as purpose, risk level, or operational boundaries

This approach does not modify device capabilities, interoperability, certification behavior, or existing Device Types. It is simply an interpretation layer built on top of structures that already exist in Matter.

Most importantly, this can already be explored entirely at the platform level without requiring CSA specification changes — interpretation of these fields is ultimately a platform decision.

Home Assistant previously exposed UserLabel values ​​through the mobile UI for several months, but later discontinued this approach. The exact reason is not confirmed, but this experience itself illustrates the problem: without a shared convention for how UserLabel should be used, platforms and manufacturers will interpret it differently — and conflicts will follow.

As AI systems begin interacting more directly with physical devices, semantic interpretation becomes increasingly important. Matter may be one of the few ecosystems where this can be explored realistically today because it already provides:

  • standardized device capabilities
  • interoperable controller structures
  • endpoint-level metadata
  • existing label mechanisms

The goal is not to redesign Matter, but to explore whether these existing structures can support a lightweight semantic validation layer for AI-assisted physical control.

Why?

No.. Seriously, why.

HA already translates. And I have zero issues controlling things semantically. (see Friday’s Party, seriously I can even drive a hot tub like a car.)

The level you’re talking about isn’t where the ai needs to concern itself. It needs to understand the system of control (ha) not the transport. (matter)

AI systems will increasingly control physical devices through existing ecosystems like Matter.

The question is no longer whether AI control will happen, but which devices AI can understand and trust first.

Devices with structured semantic labels may enter the AI ecosystem much faster.

Semantic labels help AI systems interpret device intent and operational boundaries more reliably, while also creating clearer records when accidents or unexpected actions occur.

Manufacturers should prepare sooner rather than later.

You miss my point. Agentic AI systems ALREADY control systems. But they don't need that control Inside the matter spec it needs to be with the system that's actually using the thing that translates. And.

For that we already have semantic labeling and control. Through Homeassistant.

Matter defines what it is and how it works, those domains are set with specific services and actions that are documented. HA describes what's possible and if setup exposes context to the ai. Tools let it manipulate state. Done. Don't believe me I have over 500 posts ina thread where I'm showing people how. This isn't a device level problem. It's actually an abstract the transport and media FROM the semantic meaning problem..

I get what you are saying but this isn't that kind of problem and you're in the wrong arena with the wrong audience. If you want to impact the MATTER spec you need to discuss that with the matter board but TBH. At that hardware level I simply don't see the need.

I just finished reading the Friday's Party thread.
It's clear that a huge amount of experimentation and effort from many people has gone into it, and I appreciate that work.

I think approaches like this are extremely powerful for orchestration, tool usage, and AI-assisted automation.

But physical AI may eventually introduce a slightly different class of problems.

The more AI relies on inference, context accumulation, memory, and orchestration, the more important questions become:

  • where physical operational boundaries are defined
  • how device intent is represented
  • how accident analysis and responsibility tracing work when inference goes wrong

Those questions may require more than orchestration alone.

That layer may be worth exploring alongside the orchestration work already happening here.

I want to be clear about where I'm coming from. I'm not approaching this from an AI ethics perspective. I think about it the opposite way — considering safety boundaries earlier in the stack is what makes faster, broader deployment possible. Not a constraint. An enabler.