MQTT breaking changes RC 2022.6

Configuration
5

Looking at the beta documentation for mqtt MQTT - Home Assistant I really miss having a list of links to the many specific mqtt subtopics.

There are at least 25 of these “secret” topics. They are not linked from mqtt and they are not linked from the individual integration pages either. The only way to find them is to search integrations from the integrations search page.

I really think this should be done in connection with this major breaking change

I have read the PRs and understand why the change is done. Your can do anything in code and this change could have been avoided and I do not find it logical that a cover sensor is no longer a cover integration but an mqtt integration. Mqtt is protocol and software geek. A cover is a physical thing. I do not understand this decision. Just another breaking change … too many of them.

6 Likes
6

Sorry, I should have been more complete in my query.
I have in my configuration.yaml:

sensor: !include_dir_merge_list sensors

in the directory config/sensors, there are 25 different yaml files.
10 of them contain MQTT statements, the others - not.
I have no template files.
The statement suggested above:

mqtt:
  sensor: !include_dir_merge_list sensors/mqtt

won’t work - as I have other sensors that are not MQTT
The reason I broke up the files - If I introduce a new sensor, I only have to work on one file, instead of breaking everything.
How would you suggest I do this?
@Mike - yaml anchors???

2 Likes
7

I have the same problem. My HA system has 54 package files where each package contains the needed sensors and automations for a particular system in my home (ie. HVAC, locks, floor heat, shades, etc). With this change, for each package I would need to create a new package that contains only the MQTT sensors for the system and then include these yaml files under the mqtt integration. While it’s possible to do this, it unnecessarily breaks up the code into twice as many files and makes each package unreadable as I would then need to switch between files to lookup a sensor definition when working on the automation for it. I use package to easily allow me to drop in new capabilities and everything is self contained.

I will be holding off on upgrading to 2022.6 and hope that based on the feedback from this changes that things will stay the way they are.

2 Likes
8

you don’t need to make a new package. if your package contained…

<domain>:
- platform: mqtt
  name: xyz

you change it to

mqtt:
  <domain>:
  - name: xyz
1 Like
9

I’ll give it a try. I was thinking that having multiple mqtt declarations (one per package) would throw an error.

10

well, if you have multiple…

before:

sensor:
- platform: mqtt
  name: xyz

- platform: mqtt
  name: abc

binary_sensor:
- platform: mqtt
  name: ijk

- platform: mqtt
  name: lmn

after:

mqtt:
  sensor:
  - name: xyz

  - name: abc

  binary_sensor:
  - name: ijk

  - name: lmn

Basically, instead of being spread out across multiple domains (fields), they are now all inside the mqtt umbrella.

3 Likes
11

How can we !include your examples xyz and abc in individual files? I have 2022.6 working with a single mqtt.yaml file containing my sensors, binary_sensors and switches, but I’m not clear how to split these out with the “subdomains” under mqtt.

I guess we’d need something like:

mqtt:
  sensor: !include_dir_merge_list mqtt/sensors
  binary_sensor: !include_dir_merge_list mqtt/binary_sensors

with a file mqtt/sensors/xyz.yaml looking like:

- name: xyz
...

I’ll have to try that — feels like a new setup I’ve not come across before. UPDATE: yeah, that works fine.

2 Likes
12

What I have is:

hvac.yaml

sensor:
- platform: mqtt
  name: heat_status

- platform: mqtt
  name: heat_temp

- platform: mqtt
  name: fan_speed

locks.yaml

sensor:
- platform: mqtt
  name: front_door_lock_status

- platform: mqtt
  name: back_door_lock_status

Can I change this to:

hvac.yaml

mqtt:
  sensor:
    - name: heat_status

    - name: heat_temp

   -  name: fan_speed

locks.yaml

mqtt:
  sensor:
  - name: front_door_lock_status

  - name: back_door_lock_status
13

The lazy way I did it was:

mqtt: !include_dir_merge_named mqtt/

and then I made an mqtt folder and placed my files into that. The names of the files don’t matter, but I named them what they were adding…

config/mqtt/binary_sensor.yaml

binary_sensor:
- name: xyz

- name: abc

EDIT: If you do it the way @CentralCommand is doing it, you can make it granular. I.e. you’d make separate folders in the mqtt folder for each type, then you can name the files humidiy.yaml and it wouldn’t care but you’d have organization.

I’m just lazy and wanted the fastest way to a working config without a lot of copy/paste

2 Likes
14

Yep, that’s all you needed

15

Except it was always mqtt integration and always said MQTT. I’m not really understanding, the config went from this:

sensor:
  - platform: mqtt
    name: My MQTT sensor
    ...

cover:
  - platform: mqtt
    name: My MQTT cover
    ...

To this:

mqtt:
  sensor:
    - name: My MQTT sensor
      ...

  cover:
    - name: My MQTT cover
      ...

If anything it says MQTT less times now since you don’t have to put platform: mqtt in everything.

But more importantly here is this was never the cover integration, it was always the mqtt integration. Why does that matter? In the first part what wasn’t obvious to people who aren’t deeply familiar with the config is that your MQTT sensors, covers, selects, etc. all shared the same MQTT connection to the broker. Because they were all part of the MQTT integration despite being spread out all over your config. A user would likely be quite confused why in the UI all these things appeared under MQTT instead of under cover, sensor, etc. in the settings menus.

Now everything is grouped right under MQTT showing that they are clearly all part of that integration. All these entities share the same connection details and are based on topics on the same broker.

16

I do not think one way is better than another. You can see it both ways.

It is just another breaking change that 10000s of users need to deal with - every month.

It is too much now!

5 Likes
17

While I tend to agree with you, the new yaml format for MQTT looks cleaner and more logical to me. The repeated platform key appearing for every sensor was weird. I have a lot of MQTT stuff, so I really notice the difference.

That said, I also think that making this a breaking change was unnecessary. Both formats could have been kept at only minimal maintenance overhead. They made it a breaking change simply because it would make the config handling code cleaner.

2 Likes
18

The curious part is that this is the opposite of how entities in Home Assistant have traditionally been configured/organized.

For countless versions, entities have been organized according to their domain such as light, switch, sensor, binary_sensor, lock, etc and not the underlying integration that supports the entity. It’s been the responsibility of the platform option to indicate which integration is used for a given entity.

It was the recently revised Template integration that (I believe) was first to diverge from this long-standing organizational model. Now the MQTT integration has followed suit.

Would you happen to know the rationale for this change of organizational structure?

FWIW, I am unaffected by this change because I use MQTT Discovery (via scripts) to define all MQTT entities in my system (sensors, locks, binary_sensors, lights, switches, etc). According to the documentation, the new method of defining MQTT entities still doesn’t allow you to define devices (whereas you can with MQTT Discovery).

2 Likes
19

I’m not 100% on all the reasons. I looked up the first or in the series of them that rearranged Mqtt config to see if the author said anything:

Sounds like config entry set up prefers it. I cant say requires because Mqtt was using config entries before this but perhaps it was given an exception.

I think it may also have performance benefits. In many of the integrations that have done this I’ve seen centralized coordinators appear in the code to manage communication with the external service and update many entities with one response.

I think this was possible before but I’m not sure it was possible to have a centralized coordinator that updated entities of many different types. Because I believe in the new model the integration is handed the entire config to set itself up with all at once. Whereas before it was asked to set up sensors, covers, selects, etc all independently.

I’m less sure about this though since I don’t have a clear source to point to. Just a pattern I noticed in more recent code.

20

FWIW, someone asked the same question elsewhere (Reddit) and frenck replied that’s it’s a long overdue effort to bring MQTT-based entities into compliance with this ADR:

3 Likes
21

This ADR is interesting…

This is something that really confused me when I built my little integration: it’s a media player (i.e. configured under the media_player section in YAML), but it uses MQTT. So, the integration in this case is the media player, and the platform is MQTT, but originally it wasn’t clear to me which way around this had to be. Now I’m wondering whether I will need to update this at some point to sit under an mqtt: section. I think, the class/type/function of something should be treated differently to how it connects. I’m possibly missing something subtle here, but it seems like the concepts are getting mixed. Personally, I like to think of the domain as the top level concept, with the mechanism (templates, MQTT, REST) as secondary to that. I’ll deal with it either way, but thought I’d bring this up.

1 Like
22

HA is moving away from YAML to UI configuration to, I assume, become more accessible to new users. It would be madness to set up lights under “Lights”, unless they were MQTT lights, in which case you had to set them up under “MQTT” (or unless they were “Templates”, etc etc). Once it’s all done, you can’t imagine a new screen where you manage all of your lights in one place.

23

No, its actually neither of those. Your integration is called shairpoint_sync therefore if you are trying to follow that ADR then all config for it should be under the key sharepoint_sync in configuration.yaml. So something like this:

shairpoint_sync:
  name: Shairport Sync Player
  topic: your/mqtt/topic

If your integration supported multiple platforms and each had their own options then it might be more like this:

shairpoint_sync:
  media_player:
    name: Shairport Sync Player
    topic: your/mqtt/topic
  sensor:
    name: Shairport Sync Player sensor
    topic: your/mqtt/topic

The fact that your integration uses mqtt under the hood isn’t relevant. How your integration actually communicates with its external devices/services is an implementation detail. Ideally you hide those kinds of details from the user and simply expose options that make sense without requiring the user to actually know how it works under the hood.

MQTT makes this confusing since its a communication protocol rather then a specific service. But the thing to remember is that the only config that can be under the mqtt key in configuration.yaml is stuff actually handed to the mqtt integration. You need your config to be handed to the shairpoint_sync integration not the mqtt integration so it can’t be under mqtt. It either has to be under a shairpoint_sync key or using the old style with a platform key under media_player.

Also keep in mind that ADRs apply to core integrations. ADRs are enforced by code reviews and no one is reviewing custom integrations but the author(s). If you were thinking of contributing that back to core then yea you’d need to follow the ADRs. But if you just want to leave it in HACS then you can continue doing whichever approach you prefer here.

25

Yeah, I buggered up my own example…

It’s actually this:

So TIL that it’s the old way – I didn’t know that.

I agree 100% with that statement, but it seems to contradict the main point, which is to put things under a mqtt: key/section. It’s just a comms layer – like HTTP, etc. which I think is your point too:

And the above is what I’m kind of questioning or trying to understand better. I know I’ll probably not get the true answer here unless one of the devs chime in, but it’s good to hear all the explanations regardless.

True, but I personally try to follow best practices and keep the options open. It wasn’t really to talk about my contribution, but it’s an example I understand well, which was the reason I used it.