PSA: MQTT Name changes in 2023.8

Configuration
,
56

But how about not having to do anything, and simply don’t change anything? If your device name is Some Smart Device and has light.smart_device_light as id; it would have that forever and nothing is wrong and nothing needs any change.

I agree that it is good with a warning if there is a change needed to be made. But it is not god to have a change that make stuff more complicated and does not really bring any benefits (that I see, at least) and just breaks existing behaviour.

To rephrase myself TL;DR:

  • It is working good today. Entities does not have names or ids like some_smart_device_smart_device_light today. Nothing needs to be fixed.
  • In the future, when both consumer and producer side has implemented changes, then that also will work and you will not have names or ids like some_smart_device_smart_device_light
  • So, why the need to change to a more complex solution with the same end result?

Simply put; why do a change to something that is not broken on the “consumer” side, forcing the “producer” side to also implement this change in order for things that was not broken also not to break in the future?

18 Likes
57

I totally agree with you Erik!

From a UX perspective, exposing this technical detail as a warning to the user without further explanation is not a smart move. A warning should always be a call to action. And as you can see some users did something unnecessary and now they are very disappointed.

I have to admit that I still don’t get it, which problem is actually supposed to be solved?

9 Likes
58

I fully support you, but the developers decided that if we can do something (and we are unlikely to be able to do this with a message in the forum), then I would support removing this behavior or at least adding mechanisms to influence this behavior.

59

As per my understanding, they are fixing the problem (just recently introduced!) of entities getting overly complex and long names automatically. However, the reason for that is, per my understanding, the new overly complex thing just introduced; i.e. to concat the device name and the entity name to become the name of the entity.

Before:

  • Device name was the name of the device
  • Entity name was the name of the entity

Future:

  • Device name is the name of the device
  • Entity name is the name of the device + name of the entity (!). <— this creates too long names, given that entity name is not changed. Thus the warning notifies you that the entity name has not (yet) been changed.

But if the change was not introduced, then you would not need to change the entity names to become shorter. End result (in terms of name and id) is the same, given that all entity providers implement shorter entity names. I don’t see the benefit of this.

1 Like
60

Thanks @petro for creating this post. You are right this change wil have 3087293749238438932423 users panicing. As always you invest a lot in helping people understand what is happening. I also understand why it is needed to get the upstream libraries attention if you do this.

The rest of this post is not for Petro but for the developers who thought this is a wise thing to change, and to do it this way:

  1. Asking 3087293749238438932423 scared people to contact dozens of unsuspecting upstream developers is not very nice towards those (often also community) devs. They will be overloaded with questions. Z2M is not the only integration using MQTT discovery and I’m guessing there’s also tons of discovery messages retained permanently in MQTT that will never get changed by upstream devs.
  2. There better be a good reason to do it then. But “make it more consistent” does not feel that way. How many people were bothered by the old behavior? So probably now some HA devs feel better about the code. But at the expense of point 1 and a lot of work to fix stuff?
  3. Here’s a thought: it is changed now, and there’s a workaround in place that allows for both the old behavior and the consistent one. Loose the message or make it an inconspicuous one in the logs and keep the workaround indefinitely? Only prepend the device name if it isn’t already there does not feel like a terrible thing to do.
  4. Wanting the workaround to go away: see point 2. Yes, keeping the workaround will make HA code more complicated, but it spares a lot of people a lot of aggravation. Isn’t that what software is all about? Spare users time and work at the expense of more hard work for programmers? This feels the other way around.
24 Likes
61

Presumably, whilst the state entity was being added the devs worked on the entity naming code, updated the integration API and Entity Naming Standards and found the existing MQTT integration result gave “Device Name Device Name” as they code now concatenates Device Name and Entity Name which were often set to the same value by MQTT device developers.

These fields were only loosely documented in the HASS MQTT Discovery documentation, so now every developer using the spec needs to update their code within 6 months.

The change makes perfect sense after several years of work improving Device and Entity naming, discovery, and logging. Sadly, the developer facing documentation is lagging so many software engineers read the (old) HASS source code directly, and tested the (old) behaviour to create devices that auto-discovered correctly. This includes me - my Python code (used to) create several sensors and actuators all with sensible names, MDI icons, and class/units supporting long term statistics.

There is a developer blog post which is factual and came out in advance of 2023.8, but three short paragraphs don’t really explain the impact of the changes, the user-facing warning message, nor what will happen in 6 months time.

Now this change has our attention, it would help a lot if the HASS MQTT Discovery page were changed to link to the new resources (dev blog post on MQTT, and the new entity naming standards), and the example tests were updated.

A possible updated developer MQTT Discovery test example:

mosquitto_pub -r -h 127.0.0.1 -p 1883 -t "homeassistant/binary_sensor/garden/config" -m '{"name": "Motion ENTITY", "device_class": "motion", "state_topic": "homeassistant/binary_sensor/garden/state", "unique_id": "motion01ad", "device": {"identifiers": ["01ad"], "name": "Garden Sensor DEVICE" }}'

Which on 2023.8 gives:
image

And YES, I’m currently logging in to GitHub to create a PR for just this documentation change…

EDIT: GitHub link so you can also contribute.

6 Likes
62

Note to developers - Whilst you’re re-factoring your MQTT Discovery code, can I suggest adding additional values to improve how users can interact with your hardware?

Here’s an example from my own test code (with some light changes) which adds:

  • The example Device is configured with optional extra information such as manufacturer, version, and the ‘suggested_location’ which can be used to set the Area.
  • The example Entity adds optional extra values such as an icon, the device class and unit of measurement which both allow long-term logging to give the user data graphs.
# Send a MQTT Discovery message to create a soil moisture sensor
# Note the difference between the DEVICE name and the moisture ENTITY (allowing several sensor channels)
# Note the additional DEVICE parameters such as manufacturer, version, and the 'suggested_location' which can be used to set the Area
# Note the additional ENTITY  parameters such as an icon, the device class and unit of measurement which both allow long-term logging to give the user data graphs.
mosquitto_pub -r -h 127.0.0.1 -p 1883 -t 'homeassistant/sensor/grow-1_chan1/config' --retain -m '{"name":"Channel 1 ENTITY", "stat_t":"tele/grow-1/SENSOR", "val_tpl":"{{value_json.Channel1}}", "avty_t":"tele/grow-1/LWT", "pl_avail":"Online", "pl_not_avail":"Offline", "uniq_id":"grow-1_chan1", "dev": {"ids":["grow-1"], "name":"Grow 1 DEVICE", "mf":"Great Devices", "mdl":"Grow Sensor", "sw":"0.2", "sa":"Conservatory"}, "icon":"mdi:sprout", "dev_cla":"humidity", "unit_of_meas":"%", "expire_after":"1200", "frc_upd": true}'

These additional values add quite a bit to the auto-discovery experience, and the time taken was really working out what the documentation really meant, and not the implementation!

The only light “hack” is using the device class of humidity for what is really a soil moisture sensor.

2 Likes
63

I don’t think core developers read this topic?

64

I see the Hisense AirCon add-on is impacted, or at least users of it are. That seems to get updated with some frequency, so hopefully this is accounted for in the next update.

EDIT 8/4 - yes, updated yesterday to v0.3.15.

65

We are now as the user reports of this error in everyone’s log are pouring in today.

66

Sorry, edited my post to ‘core developers’.

67

May I recommend for the first post adding:

TLDR: If all your MQTT devices are through Zigbee2MQTT, you can safely click on “Ignore” for the entity name warnings. An upcoming update for Z2M will fix everything.

5 Likes
68

Thanks, like someone mentioned above this simple explanation is worth more than the breaking changes message…
I need to fix two repos and now I know how :+1:

2 Likes
69

The problem is that im getting THOUSANDS of these warnings EVERY time that an mqtt message arrives in the broker but ONLY for the disabled entities. I’ve had more disabled sensors but enabled them to ensure that im not getting the warnings every few seconds.

For example one of the cameras.

image
image536×1092 52.3 KB 

2023-08-03 18:47:19.574 WARNING (MainThread) [homeassistant.components.mqtt.mixins] MQTT entity name starts with the device name in your config {'device': {'identifiers': ['mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3'], 'manufacturer': 'Xiaomi', 'model': 'Mi Home Security Camera 1080P (MJSXJ02HL)', 'name': 'mjsxj02hl_bedroomcam', 'sw_version': '1.5.0', 'connections': []}, 'availability': [{'topic': 'mjsxj02hl/bedroomcam/state', 'payload_not_available': 'offline', 'payload_available': 'online'}], 'availability_mode': 'all', 'name': 'mjsxj02hl_bedroomcam_info_volume_level', 'unique_id': 'mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3_info_volume_level', 'unit_of_measurement': '%', 'enabled_by_default': False, 'state_topic': 'mjsxj02hl/bedroomcam/info', 'value_template': Template<template=({{ value_json.volume_level }}) renders=0>, 'payload_available': 'online', 'force_update': False, 'encoding': 'utf-8', 'qos': 0, 'payload_not_available': 'offline'}, this is not expected. Please correct your configuration. The device name prefix will be stripped off the entity name and becomes '_info_volume_level'
2023-08-03 18:47:34.855 WARNING (MainThread) [homeassistant.components.mqtt.mixins] MQTT entity name starts with the device name in your config {'device': {'identifiers': ['mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3'], 'manufacturer': 'Xiaomi', 'model': 'Mi Home Security Camera 1080P (MJSXJ02HL)', 'name': 'mjsxj02hl_bedroomcam', 'sw_version': '1.5.0', 'connections': []}, 'availability': [{'topic': 'mjsxj02hl/bedroomcam/state', 'payload_not_available': 'offline', 'payload_available': 'online'}], 'availability_mode': 'all', 'name': 'mjsxj02hl_bedroomcam_info_volume_level', 'unique_id': 'mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3_info_volume_level', 'unit_of_measurement': '%', 'enabled_by_default': False, 'state_topic': 'mjsxj02hl/bedroomcam/info', 'value_template': Template<template=({{ value_json.volume_level }}) renders=0>, 'payload_available': 'online', 'force_update': False, 'encoding': 'utf-8', 'qos': 0, 'payload_not_available': 'offline'}, this is not expected. Please correct your configuration. The device name prefix will be stripped off the entity name and becomes '_info_volume_level'
2023-08-03 18:47:50.130 WARNING (MainThread) [homeassistant.components.mqtt.mixins] MQTT entity name starts with the device name in your config {'device': {'identifiers': ['mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3'], 'manufacturer': 'Xiaomi', 'model': 'Mi Home Security Camera 1080P (MJSXJ02HL)', 'name': 'mjsxj02hl_bedroomcam', 'sw_version': '1.5.0', 'connections': []}, 'availability': [{'topic': 'mjsxj02hl/bedroomcam/state', 'payload_not_available': 'offline', 'payload_available': 'online'}], 'availability_mode': 'all', 'name': 'mjsxj02hl_bedroomcam_info_volume_level', 'unique_id': 'mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3_info_volume_level', 'unit_of_measurement': '%', 'enabled_by_default': False, 'state_topic': 'mjsxj02hl/bedroomcam/info', 'value_template': Template<template=({{ value_json.volume_level }}) renders=0>, 'payload_available': 'online', 'force_update': False, 'encoding': 'utf-8', 'qos': 0, 'payload_not_available': 'offline'}, this is not expected. Please correct your configuration. The device name prefix will be stripped off the entity name and becomes '_info_volume_level'
2023-08-03 18:48:05.412 WARNING (MainThread) [homeassistant.components.mqtt.mixins] MQTT entity name starts with the device name in your config {'device': {'identifiers': ['mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3'], 'manufacturer': 'Xiaomi', 'model': 'Mi Home Security Camera 1080P (MJSXJ02HL)', 'name': 'mjsxj02hl_bedroomcam', 'sw_version': '1.5.0', 'connections': []}, 'availability': [{'topic': 'mjsxj02hl/bedroomcam/state', 'payload_not_available': 'offline', 'payload_available': 'online'}], 'availability_mode': 'all', 'name': 'mjsxj02hl_bedroomcam_info_volume_level', 'unique_id': 'mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3_info_volume_level', 'unit_of_measurement': '%', 'enabled_by_default': False, 'state_topic': 'mjsxj02hl/bedroomcam/info', 'value_template': Template<template=({{ value_json.volume_level }}) renders=0>, 'payload_available': 'online', 'force_update': False, 'encoding': 'utf-8', 'qos': 0, 'payload_not_available': 'offline'}, this is not expected. Please correct your configuration. The device name prefix will be stripped off the entity name and becomes '_info_volume_level'
2023-08-03 18:48:20.806 WARNING (MainThread) [homeassistant.components.mqtt.mixins] MQTT entity name starts with the device name in your config {'device': {'identifiers': ['mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3'], 'manufacturer': 'Xiaomi', 'model': 'Mi Home Security Camera 1080P (MJSXJ02HL)', 'name': 'mjsxj02hl_bedroomcam', 'sw_version': '1.5.0', 'connections': []}, 'availability': [{'topic': 'mjsxj02hl/bedroomcam/state', 'payload_not_available': 'offline', 'payload_available': 'online'}], 'availability_mode': 'all', 'name': 'mjsxj02hl_bedroomcam_info_volume_level', 'unique_id': 'mjsxj02hl_bedroomcam_02303d86fe0038c1a770030a46924d9421e94b18714f29e3_info_volume_level', 'unit_of_measurement': '%', 'enabled_by_default': False, 'state_topic': 'mjsxj02hl/bedroomcam/info', 'value_template': Template<template=({{ value_json.volume_level }}) renders=0>, 'payload_available': 'online', 'force_update': False, 'encoding': 'utf-8', 'qos': 0, 'payload_not_available': 'offline'}, this is not expected. Please correct your configuration. The device name prefix will be stripped off the entity name and becomes '_info_volume_level'

ps. using ZHA

70

If you’d like to hide the warnings in your logs, add the following to configuration.yaml. (remove just this warning)

logger:
  default: info
  filters:
    homeassistant.components.mqtt.mixins:
    - "MQTT entity name starts with the device name in your config"

or (remove all warnings and info)

logger:
  default: info
  logs:
    homeassistant.components.mqtt.mixins: error
5 Likes
71

thanks @petro! This is much easier than disabling the entity_globs in the recorder since i don’t need most of these sensors !

1 Like
72

I noticed that after fixing some of the auto-discovered entities, that the name in the front-end has changed.

73

Yes, the default names themselves are the breaking change. So if you left your entities with a default name (I.e. you didn’t override the entity name or device name), they may have a new name shown around the UI.

74

Is this an attempt to migrate people from Z2M to ZHA?

Any way to list all unavailable entities and only these?

2 Likes
75

No…

1 Like