PSA: MQTT Name changes in 2023.8

Configuration
,
61

Presumably, whilst the state entity was being added the devs worked on the entity naming code, updated the integration API and Entity Naming Standards and found the existing MQTT integration result gave “Device Name Device Name” as they code now concatenates Device Name and Entity Name which were often set to the same value by MQTT device developers.

These fields were only loosely documented in the HASS MQTT Discovery documentation, so now every developer using the spec needs to update their code within 6 months.

The change makes perfect sense after several years of work improving Device and Entity naming, discovery, and logging. Sadly, the developer facing documentation is lagging so many software engineers read the (old) HASS source code directly, and tested the (old) behaviour to create devices that auto-discovered correctly. This includes me - my Python code (used to) create several sensors and actuators all with sensible names, MDI icons, and class/units supporting long term statistics.

There is a developer blog post which is factual and came out in advance of 2023.8, but three short paragraphs don’t really explain the impact of the changes, the user-facing warning message, nor what will happen in 6 months time.

Now this change has our attention, it would help a lot if the HASS MQTT Discovery page were changed to link to the new resources (dev blog post on MQTT, and the new entity naming standards), and the example tests were updated.

A possible updated developer MQTT Discovery test example:

mosquitto_pub -r -h 127.0.0.1 -p 1883 -t "homeassistant/binary_sensor/garden/config" -m '{"name": "Motion ENTITY", "device_class": "motion", "state_topic": "homeassistant/binary_sensor/garden/state", "unique_id": "motion01ad", "device": {"identifiers": ["01ad"], "name": "Garden Sensor DEVICE" }}'

Which on 2023.8 gives:
image

And YES, I’m currently logging in to GitHub to create a PR for just this documentation change…

EDIT: GitHub link so you can also contribute.

6 Likes
62

Note to developers - Whilst you’re re-factoring your MQTT Discovery code, can I suggest adding additional values to improve how users can interact with your hardware?

Here’s an example from my own test code (with some light changes) which adds:

  • The example Device is configured with optional extra information such as manufacturer, version, and the ‘suggested_location’ which can be used to set the Area.
  • The example Entity adds optional extra values such as an icon, the device class and unit of measurement which both allow long-term logging to give the user data graphs.
# Send a MQTT Discovery message to create a soil moisture sensor
# Note the difference between the DEVICE name and the moisture ENTITY (allowing several sensor channels)
# Note the additional DEVICE parameters such as manufacturer, version, and the 'suggested_location' which can be used to set the Area
# Note the additional ENTITY  parameters such as an icon, the device class and unit of measurement which both allow long-term logging to give the user data graphs.
mosquitto_pub -r -h 127.0.0.1 -p 1883 -t 'homeassistant/sensor/grow-1_chan1/config' --retain -m '{"name":"Channel 1 ENTITY", "stat_t":"tele/grow-1/SENSOR", "val_tpl":"{{value_json.Channel1}}", "avty_t":"tele/grow-1/LWT", "pl_avail":"Online", "pl_not_avail":"Offline", "uniq_id":"grow-1_chan1", "dev": {"ids":["grow-1"], "name":"Grow 1 DEVICE", "mf":"Great Devices", "mdl":"Grow Sensor", "sw":"0.2", "sa":"Conservatory"}, "icon":"mdi:sprout", "dev_cla":"humidity", "unit_of_meas":"%", "expire_after":"1200", "frc_upd": true}'

These additional values add quite a bit to the auto-discovery experience, and the time taken was really working out what the documentation really meant, and not the implementation!

The only light “hack” is using the device class of humidity for what is really a soil moisture sensor.

2 Likes
63

I don’t think core developers read this topic?

64

I see the Hisense AirCon add-on is impacted, or at least users of it are. That seems to get updated with some frequency, so hopefully this is accounted for in the next update.

EDIT 8/4 - yes, updated yesterday to v0.3.15.

65

We are now as the user reports of this error in everyone’s log are pouring in today.

66

Sorry, edited my post to ‘core developers’.

67

May I recommend for the first post adding:

TLDR: If all your MQTT devices are through Zigbee2MQTT, you can safely click on “Ignore” for the entity name warnings. An upcoming update for Z2M will fix everything.

5 Likes
68

Thanks, like someone mentioned above this simple explanation is worth more than the breaking changes message…
I need to fix two repos and now I know how :+1:

2 Likes
69

The problem is that im getting THOUSANDS of these warnings EVERY time that an mqtt message arrives in the broker but ONLY for the disabled entities. I’ve had more disabled sensors but enabled them to ensure that im not getting the warnings every few seconds.

For example one of the cameras.

image
image536×1092 52.3 KB 

2023-08-03 18:47:19.574 WARNING (MainThread) [homeassistant.components.mqtt.mixins] MQTT entity name starts with the device name in your config {'device': {'identifiers': ['mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3'], 'manufacturer': 'Xiaomi', 'model': 'Mi Home Security Camera 1080P (MJSXJ02HL)', 'name': 'mjsxj02hl_bedroomcam', 'sw_version': '1.5.0', 'connections': []}, 'availability': [{'topic': 'mjsxj02hl/bedroomcam/state', 'payload_not_available': 'offline', 'payload_available': 'online'}], 'availability_mode': 'all', 'name': 'mjsxj02hl_bedroomcam_info_volume_level', 'unique_id': 'mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3_info_volume_level', 'unit_of_measurement': '%', 'enabled_by_default': False, 'state_topic': 'mjsxj02hl/bedroomcam/info', 'value_template': Template<template=({{ value_json.volume_level }}) renders=0>, 'payload_available': 'online', 'force_update': False, 'encoding': 'utf-8', 'qos': 0, 'payload_not_available': 'offline'}, this is not expected. Please correct your configuration. The device name prefix will be stripped off the entity name and becomes '_info_volume_level'
2023-08-03 18:47:34.855 WARNING (MainThread) [homeassistant.components.mqtt.mixins] MQTT entity name starts with the device name in your config {'device': {'identifiers': ['mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3'], 'manufacturer': 'Xiaomi', 'model': 'Mi Home Security Camera 1080P (MJSXJ02HL)', 'name': 'mjsxj02hl_bedroomcam', 'sw_version': '1.5.0', 'connections': []}, 'availability': [{'topic': 'mjsxj02hl/bedroomcam/state', 'payload_not_available': 'offline', 'payload_available': 'online'}], 'availability_mode': 'all', 'name': 'mjsxj02hl_bedroomcam_info_volume_level', 'unique_id': 'mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3_info_volume_level', 'unit_of_measurement': '%', 'enabled_by_default': False, 'state_topic': 'mjsxj02hl/bedroomcam/info', 'value_template': Template<template=({{ value_json.volume_level }}) renders=0>, 'payload_available': 'online', 'force_update': False, 'encoding': 'utf-8', 'qos': 0, 'payload_not_available': 'offline'}, this is not expected. Please correct your configuration. The device name prefix will be stripped off the entity name and becomes '_info_volume_level'
2023-08-03 18:47:50.130 WARNING (MainThread) [homeassistant.components.mqtt.mixins] MQTT entity name starts with the device name in your config {'device': {'identifiers': ['mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3'], 'manufacturer': 'Xiaomi', 'model': 'Mi Home Security Camera 1080P (MJSXJ02HL)', 'name': 'mjsxj02hl_bedroomcam', 'sw_version': '1.5.0', 'connections': []}, 'availability': [{'topic': 'mjsxj02hl/bedroomcam/state', 'payload_not_available': 'offline', 'payload_available': 'online'}], 'availability_mode': 'all', 'name': 'mjsxj02hl_bedroomcam_info_volume_level', 'unique_id': 'mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3_info_volume_level', 'unit_of_measurement': '%', 'enabled_by_default': False, 'state_topic': 'mjsxj02hl/bedroomcam/info', 'value_template': Template<template=({{ value_json.volume_level }}) renders=0>, 'payload_available': 'online', 'force_update': False, 'encoding': 'utf-8', 'qos': 0, 'payload_not_available': 'offline'}, this is not expected. Please correct your configuration. The device name prefix will be stripped off the entity name and becomes '_info_volume_level'
2023-08-03 18:48:05.412 WARNING (MainThread) [homeassistant.components.mqtt.mixins] MQTT entity name starts with the device name in your config {'device': {'identifiers': ['mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3'], 'manufacturer': 'Xiaomi', 'model': 'Mi Home Security Camera 1080P (MJSXJ02HL)', 'name': 'mjsxj02hl_bedroomcam', 'sw_version': '1.5.0', 'connections': []}, 'availability': [{'topic': 'mjsxj02hl/bedroomcam/state', 'payload_not_available': 'offline', 'payload_available': 'online'}], 'availability_mode': 'all', 'name': 'mjsxj02hl_bedroomcam_info_volume_level', 'unique_id': 'mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3_info_volume_level', 'unit_of_measurement': '%', 'enabled_by_default': False, 'state_topic': 'mjsxj02hl/bedroomcam/info', 'value_template': Template<template=({{ value_json.volume_level }}) renders=0>, 'payload_available': 'online', 'force_update': False, 'encoding': 'utf-8', 'qos': 0, 'payload_not_available': 'offline'}, this is not expected. Please correct your configuration. The device name prefix will be stripped off the entity name and becomes '_info_volume_level'
2023-08-03 18:48:20.806 WARNING (MainThread) [homeassistant.components.mqtt.mixins] MQTT entity name starts with the device name in your config {'device': {'identifiers': ['mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3'], 'manufacturer': 'Xiaomi', 'model': 'Mi Home Security Camera 1080P (MJSXJ02HL)', 'name': 'mjsxj02hl_bedroomcam', 'sw_version': '1.5.0', 'connections': []}, 'availability': [{'topic': 'mjsxj02hl/bedroomcam/state', 'payload_not_available': 'offline', 'payload_available': 'online'}], 'availability_mode': 'all', 'name': 'mjsxj02hl_bedroomcam_info_volume_level', 'unique_id': 'mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3_info_volume_level', 'unit_of_measurement': '%', 'enabled_by_default': False, 'state_topic': 'mjsxj02hl/bedroomcam/info', 'value_template': Template<template=({{ value_json.volume_level }}) renders=0>, 'payload_available': 'online', 'force_update': False, 'encoding': 'utf-8', 'qos': 0, 'payload_not_available': 'offline'}, this is not expected. Please correct your configuration. The device name prefix will be stripped off the entity name and becomes '_info_volume_level'

ps. using ZHA

70

If you’d like to hide the warnings in your logs, add the following to configuration.yaml. (remove just this warning)

logger:
  default: info
  filters:
    homeassistant.components.mqtt.mixins:
    - "MQTT entity name starts with the device name in your config"

or (remove all warnings and info)

logger:
  default: info
  logs:
    homeassistant.components.mqtt.mixins: error
5 Likes
71

thanks @petro! This is much easier than disabling the entity_globs in the recorder since i don’t need most of these sensors !

1 Like
72

I noticed that after fixing some of the auto-discovered entities, that the name in the front-end has changed.

73

Yes, the default names themselves are the breaking change. So if you left your entities with a default name (I.e. you didn’t override the entity name or device name), they may have a new name shown around the UI.

74

Is this an attempt to migrate people from Z2M to ZHA?

Any way to list all unavailable entities and only these?

2 Likes
75

No…

1 Like
76

Then it’s the worst coordinated deprecation in Home Assistant history.

11 Likes
77

Thanks for your feedback. As a volunteer who organized this, your comments are deeply valued. I’ll make sure to continue doing this in my free time to help the community out in the future.

10 Likes
78

I was in FULL panic mode when I first reading the mgtt change.
Then Petro was kind enough to post the details of the change.
And my panic was curtailed.

Thanks Petro.

1 Like
79

Sorry for being honest, but this change will eat a lot of free time of people using Z2M + HA.

7 Likes
80

As one of the several volunteers who now have to clean up a huge mess as a result of this decision which impacts all my users, the snark isn’t helpful nor appreciated.

We’re all in this thread looking for solutions to a problem we didn’t create. Well… almost all of us.

8 Likes