MQTT breaking changes RC 2022.6

[azerbinati]

Thank you very much!
Despite translating old to new format of a single mqtt sensor is a very simple thing, having a large configuration.yaml ( by large I mean a structured tree of included files ) to translate isn’t that straight forward.

Packages was exactly what I was missing in this forced configuration rewriting, and I think this is what many are missing too (judging from what is asked, or ranted ).

Using packages was optional, and still it is, but it’s gonna be mandatory for some way of organize things out.

In my case, with many dozen of devices, I have spitted my configuration.yaml into topics like that

switch laudry:
 - platform: mqtt
   bla: bla bla
sensor laudry:
 - platform: mqtt
   bla: bla bla

other: bla bla

switch living_room:
  - platform: mqtt 
    bla bla bla
sensor living_room:
  - platform: mqtt
    bla: bla bla

My first, failing, attempt was to convert that into something like

mqtt:
  switch laudry:
  bla bla bla
  sensor laudry:
  bla bla bla
  switch living_room:
  bla bla bla
  sensor living_room:
  bla bla bla

But while top integration like light or switch support key modifier, that’s not supported (yet?) by new MQTT syntax.

Using packages, I was able to keep my original organization into topics (actually improving it) with somethink like into the main configuration.yaml

homeassistant:
  packages: !include_dir_merge_named packages/

and many packages with mqtt in they top level like this

laudry:
  mqtt:
     switch:
        bla: bla bla
     sensor:
        bla: bla bla

I hope this will help other to migrate successfully their configuration as well.

[Farranco]

I had the exact same problem. Your YAML looks correct. After hours playing with my YAML and trying different configurations. I deleted my mosquito broker, both from integrations and the add-on pane. Then re-installed it and added in integrations it and all starting working. I could once again toggle the lights. I am using Shelly relays, but should be the same for you. Not sure what the original problem was, but fix stuck even after restarts.

The instructions for Mosquito Broker, say to create a user, this is not necessary. On adding the integration it added it’s own user named “homeassistant” to the configuration.

[jayw (Jay)]

Thanks @petro! Took me a while to get things sorted after updating today, but this helped me greatly.

[yaruslavm]

Hey guys , just did the entire mqtt sensors and switches move to different files
and i noticed that the documentation is not so clear about it (or maybe it is my shallow understanding)

my previous configuration.yaml

#camera for Vacuum
camera: !include_dir_merge_list camera/
#custom sensors
sensor: !include_dir_merge_list sensors/
#custom shelly switches
switch: !include_dir_merge_list switches/

inside old switch.yaml

- platform: mqtt
  name: "shelly1pm-boiler"
  unique_id: "shelly1pm-boiler"
  payload_on: "on"
  payload_off: "off"
  qos: 0
  state_topic: "shellies/shelly1pm-x/relay/0"
  command_topic: "shellies/shelly1pm-x/relay/0/command"
  json_attributes_topic: "shellies/shelly1pm-x/info"

new configuration.yaml

mqtt: 
  sensor: !include_dir_merge_list sensors/mqtt/
  switch: !include_dir_merge_list switches/mqtt/

and what i did in the new file

switch:
  - unique_id: "shelly1pm-boiler"
    name: "shelly1pm-boiler"
    state_topic: "shellies/shelly1pm-x/relay/0"
    command_topic: "shellies/shelly1pm-x/relay/0/command"
    json_attributes_topic: "shellies/shelly1pm-x/info"
    payload_on: "on"
    payload_off: "off"
    qos: 0

nothing worked !! had to google for several hours and after a while i stumbled upon the issue is that in the old file you had to declare
“- platform: mqtt”
BUT in the new file
you need to skip “switch:” declaration as it was already defined in the configuration.yaml

Looking back i think if i saw an example in the official docs for split yaml config (instead of keeping everything in configuration.yaml) it would save me lots of time.

working switch.yaml

- unique_id: "shelly1pm-boiler"
  name: "shelly1pm-boiler"
  state_topic: "shellies/shelly1pm-x/relay/0"
  command_topic: "shellies/shelly1pm-x/relay/0/command"
  json_attributes_topic: "shellies/shelly1pm-x/info"
  payload_on: "on"
  payload_off: "off"
  qos: 0

[PePeL (Piotr)]

Thank you @yaruslavm, thanks to you it took me 2 hours of wondering what is happening and 10 min to google your entry.

[jayjay (JJ)]

Well did happen to me too now, I am changing things but i break all my MQTT entries. Why are there so many puritan crack programmers not understanding that somethings do not need to be changed for it breaks configurations for many less gifted programmer!
After the new update I get this:

And basically I am stuck. Do I make a new file with MQTT as name, and make an include in configurations, do I chuck in all my lights, binary sensors and what not is MQTT in that new MQTT file which is less clear what is where now they are in sensors, lights and so forth.

Help!

[jayjay (JJ)]

So I startet to break my config on this new easier better mqtt thing, well I am going totally ape, the errors fly around my ears.

made a folder mqtt in config folder
made 4 new yamls there switch, sensor, light and binaries
went into configuration yaml made an entry there:

mqtt:
sensors: !include_dir_merge_list config/mqtt/sensors
switch: !include_dir_merge_list config/mqtt/switches
lights: !include_dir_merge_list config/mqtt/lights
binary: !include_dir_merge_list config/mqtt/binaries

Needless to say that does not work:

so what am I missing

[petro]

Spaces before each item under MQTT. Take a look at the examples posted in this thread. Then look at yours. If the spaces and words don’t match up, you’re doing it wrong.

also, it’s sensor not sensors, light not lights, binary_sensor not binary.

[jayjay (JJ)]

sorry that is the not using right option to enter text into here!
Well took me a day to alter all entries of my manual MQTT entities. but got them working again.
Hopefully all others will be able too.

tx

[KennethLavrsen]

You seem to layer switch within switch. In your configuration.yaml you have put mqtt: and below that switch: Then you cannot have switch: again in the included file

You end up with

mqtt:
  switch:
    switch:

You have to remove the extra switch layer

I personally use this in configuration.yaml

mqtt: !include_dir_merge_named mqtt

Then I have a directory called mqtt and inside of this I put a file for each type.

The top of the mqtt_switches.yaml I have

switch:
  - name: "Short Toilet Flush"
    unique_id: "Short Toilet Flush"
    state_topic: "toiletflushshort/state"
    command_topic: "toiletflushshort/set"
    payload_on: "ON"
    payload_off: "OFF"
    state_on: "ON"
    state_off: "OFF"
    optimistic: false
    qos: 0
    retain: false
  
  - name: "Long Toilet Flush"
    unique_id: "Long Toilet Flush"
    state_topic: "toiletflushlong/state"
    command_topic: "toiletflushlong/set"
    payload_on: "ON"
    payload_off: "OFF"
    state_on: "ON"
    state_off: "OFF"
    optimistic: false
    qos: 0
    retain: false

So I end up with

mqtt:
  switch:
    - list of switches

[Grijzekop (Gert)]

Hi, I am struggeling with the recent MQTT structure changes, so maybe someone can help me?

I went from the old structure to the new one and have something like this:

# PIR Ketelruimte (A-SE-01)
    - name: "Ketelruimte PIR"
      unique_id: "ketelruimte_pir"
      state_topic: "zigbee2mqtt/0x00158d0004239815"
      availability:
        - topic: "zigbee2mqtt/bridge/state"
      payload_on: true
      payload_off: false
      value_template: "{{ value_json.occupancy }}"
      device_class: "motion"

In this case the automation in Node-red isn’t working anymore because for example a ’ current state node’ isn’t working when I use the name as Identity-ID ’ Ketelruimte PIR’ . It is working well when I use the entity name ‘0x00158d0004239815 occupancy’. It worked well before the changes and I don’t want to change all relevant Node-Red nodes (+200!). Am I doing something wrong?

Old (not working):

New (Is working):

[petro]

you aren’t including the actual things that were changed.

You need to move your configuration from sensor.yaml and put it in the mqtt section under the sensor key.

the change from an overhead perspective is just:

before:

binary_sensor:
- platform: mqtt
  <YOUR OLD CONFIG>

to

mqtt:
  binary_sensor:
  - <YOUR OLD CONFIG>

[Grijzekop (Gert)]

Sorry, the information I gave was just a part.

I have this in my configuration.yaml:

mqtt: !include_dir_merge_list mqtt

and in the mqtt directory I have several files and one of them is binary_sensors_mqtt.yaml.
In that file I have:

  binary_sensor:
    # PIR Ketelruimte (A-SE-01)
    - name: "Ketelruimte PIR"
      unique_id: "ketelruimte_pir"
      state_topic: "zigbee2mqtt/0x00158d0004239815"
      availability:
        - topic: "zigbee2mqtt/bridge/state"
      payload_on: true
      payload_off: false
      value_template: "{{ value_json.occupancy }}"
      device_class: "motion"

And I have the same problem with switches that are in the switches_mqtt.yaml en lights that are in the lights_mqtt.yaml. I have no errors when checking and reloading the configuration.

I am clearly doing something wrong! Also have these problems in Home Assistant:

[petro]

@Grijzekop

mqtt: !include_dir_merge_list mqtt

it’s not a list, you can’t use that include.

The third post covers possible solutions

MQTT breaking changes RC 2022.6

You don’t say but this Makes me assume you have something this in configuration.yaml right now right? sensor: !include_dir_merge_list sensors So just make subfolders I guess. Something like this could work: sensors mqtt template … legacy_format And then add this to configuration.yaml: mqtt: sensor: !include_dir_merge_list sensors/mqtt template: - sensor: !include_dir_merge_list sensors/template sensor: !include_dir_merge_list sensors/legacy_format I assume you also needed to …

or this post

MQTT breaking changes RC 2022.6

The lazy way I did it was: mqtt: !include_dir_merge_named mqtt/ and then I made an mqtt folder and placed my files into that. The names of the files don’t matter, but I named them what they were adding… config/mqtt/binary_sensor.yaml binary_sensor: - name: xyz - name: abc EDIT: If you do it the way @CentralCommand is doing it, you can make it granular. I.e. you’d make separate folders in the mqtt folder for each type, then you can name the files humidiy.yaml and it wouldn’t care but you’…

or this

MQTT breaking changes RC 2022.6

I already had everything split out and really didn’t want to combine it again, so I did this… configuration.yaml --- mqtt: binary_sensor: !include_dir_merge_list mqtt/binary_sensor/ button: !include_dir_merge_list mqtt/button/ cover: !include_dir_merge_list mqtt/cover/ fan: !include_dir_merge_list mqtt/fan/ light: !include_dir_merge_list mqtt/light/ lock: !include_dir_merge_list mqtt/lock/ sensor: !include_dir_merge_list mqtt/sensor/ switch: !include_dir_merge_list mqtt/switch/ …

[Grijzekop (Gert)]

Got it! changed:

mqtt: !include_dir_merge_list mqtt

to

mqtt: !include_dir_merge_named mqtt

and now it’s working. Thanks!

[SomeRandomDude]

Well, I updated my HA in docker today and wanted to fix the messages:

Manually configured MQTT switch(s) found under platform key 'switch', please move to the mqtt integration key, see https://www.home-assistant.io/integrations/switch.mqtt/#new_format
Manually configured MQTT sensor(s) found under platform key 'sensor', please move to the mqtt integration key, see https://www.home-assistant.io/integrations/sensor.mqtt/#new_format

I think I did it all right when looking at the documentation and the examples, but my HA doesn’t line up with my expectations sadly. I’ve been fiddling with include files and later on moved just the mqtt switches directly in configuration.yaml to rule out wrong indenting and so on, but my switches and their automation is broken now.

I now have the following in configuration.yaml:

mqtt:                                                                                                                                                                             
  switch:                                                                                                                                                                         
  - name: "Shelly 1PM E67B86"                                                                                                                                                     
    unique_id: shelly_1pm_e67b86                                                                                                                                                  
    state_topic: "shellies/shelly1pm-E67B86/relay/0"                                                                                                                              
    command_topic: "shellies/shelly1pm-E67B86/relay/0/command"                                                                                                                    
    retain: true                                                                                                                                                                  
  - name: "Shelly Plug S 7AE6DF"                                                                                                                                                  
    unique_id: shelly_plug_s_7ae6df                                                                                                                                               
    state_topic: "shellies/shellyplug-s-7AE6DF/relay/0"                                                                                                                           
    command_topic: "shellies/shellyplug-s-7AE6DF/relay/0/command"                                                                                                                 
    retain: true                                                                                                                                                                  
  - name: "Shelly Plug S 0417EC"                                                                                                                                                  
    unique_id: shelly_plug_s_0417ec                                                                                                                                               
    state_topic: "shellies/shellyplug-s-0417EC/relay/0"                                                                                                                           
    command_topic: "shellies/shellyplug-s-0417EC/relay/0/command"                                                                                                                 
    retain: true                                                                                                                                                                  
  - name: "Shelly Plug S 79D7E8"                                                                                                                                                  
    unique_id: shelly_plug_s_79d7e8                                                                                                                                               
    state_topic: "shellies/shellyplug-s-79D7E8/relay/0"                                                                                                                           
    command_topic: "shellies/shellyplug-s-79D7E8/relay/0/command"                                                                                                                 
    retain: true                                                                                                                                                                  

Now I’ve been changing stuff and restarting HA, which I run in a docker container on Ubuntu. Based on the Shelly 1PM example above, I now see two similar entities in HA. I’ve added and deleted unique_id but the result is there are two devices. I can’t delete the one that’s marked as read only and the other one Buitenverlichting doesn’t work with the automation anymore.

the relevant part in automations.yaml:

- id: '1636216284999'                                                                                                                                                             
  alias: Buitenverlichting terras aan bij zonsondergang                                                                                                                           
  description: ''                                                                                                                                                                 
  trigger:                                                                                                                                                                        
  - platform: sun                                                                                                                                                                 
    event: sunset                                                                                                                                                                 
  condition: []                                                                                                                                                                   
  action:                                                                                                                                                                         
  - service: switch.turn_on                                                                                                                                                       
    target:                                                                                                                                                                       
      entity_id: switch.shelly_1pm_e67b86                                                                                                                                         
  mode: single                                                                                                                                                                    
- id: '1636216499389'                                                                                                                                                             
  alias: Buitenverlichting terras uit bij zonsopkomst                                                                                                                             
  description: ''                                                                                                                                                                 
  trigger:                                                                                                                                                                        
  - platform: sun                                                                                                                                                                 
    event: sunrise                                                                                                                                                                
  condition: []                                                                                                                                                                   
  action:                                                                                                                                                                         
  - service: switch.turn_off                                                                                                                                                      
    target:                                                                                                                                                                       
      entity_id: switch.shelly_1pm_e67b86                                                                                                                                         
  mode: single  

I don’t know if it is relevant, but I rename some entities to a more friendly name:

  switch.shelly_1pm_e67b86:                                                                                                                                                       
    friendly_name: Buitenverlichting   

Now, if I revert all my changes, everything should work again as I left it before I started changing configuration because I wanted to prevent the 2022.12 update to break my MQTT stuff. But I do realize I need to either fix the configuration before I update to 2022.12, or not update anymore.

I’m looking and staring and don’t know what’s wrong on my HA. I hope someone here with a fresh set of eyes can help me out?

[crazytok (Thomas)]

Im using this code below when I trying to update to the new mqtt format. But Im getting “Property Switch is not allowed” after “switch:”. Why is that?

mqtt:
  switch:
    - name: "NIBE Hot Water Activate/Deactivate" # Choose an easy-to-recognize name
      state_topic: "nibe/modbus/47387" # 48132 Topic to read the current state
      command_topic: "nibe/modbus/47387/set" # Topic to publish commands
      qos: 1
      payload_on: 1 # or "on", depending on your MQTT device
      payload_off: 0 # or "off", depending on your MQTT device
      retain: true # or false if you want to wait for changes

[CentralCommand (Mike)]

Is that snippet of yaml directly in your configuration.yaml file? Or is it in some other file which is used via !include? If it’s the latter please show how it is connected to configuration.yaml

[oldfart101]

my configuration.yaml file has this:

mqtt:
  sensor: !include_dir_merge_list mqtt/

my mqtt directory has loads of yaml sensor files in it

[CentralCommand (Mike)]

@crazytok

mqtt:
  switch:
    - name: "NIBE Hot Water Activate/Deactivate" # Choose an easy-to-recognize name
      state_topic: "nibe/modbus/47387" # 48132 Topic to read the current state
      command_topic: "nibe/modbus/47387/set" # Topic to publish commands
      qos: 1
      payload_on: 1 # or "on", depending on your MQTT device
      payload_off: 0 # or "off", depending on your MQTT device
      retain: true # or false if you want to wait for changes

Is this bit in a file in the mqtt folder? If so then that’s your problem. You’re merging everything in the mqtt folder into a list as the value for sensor.

But what you shared above is neither a list nor a sensor so that won’t work. You have to add the switch part to mqtt in your configuration.yaml file.