MQTT discovery: Value of attribute "brightness_state_topic" is lost (set to None)

Configuration
1

I have created an MQTT device using an ESP8266 and can see in the serial console that the following payload is sent to Mosquitto:

{
  "uniq_id": "d9be9d",
  "name": "My ESP Dimmer",
  "dev_cla": "light",
  "stat_t": "homeassistant/light/d9be9d/state",
  "cmd_t": "homeassistant/light/d9be9d/state/set",
  "brightness_state_topic": "homeassistant/light/d9be9d/Brightness",
  "brightness_command_topic": "homeassistant/light/d9be9d/Brightness/Set",
  "brightness_scale": "12345",
  "optimistic": false,
  "retain": false
}

However, in the Home Assistant log I see the following:

2023-03-06 16:11:55.122 ERROR (MainThread) [homeassistant.util.logging] Exception in async_discover when dispatching 'mqtt_discovery_new_light_mqtt': ({'unique_id': 'd9be9d', 'name': 'My ESP Dimmer', 'device_class': 'light', 'state_topic': 'homeassistant/light/d9be9d/state', 'command_topic': 'homeassistant/light/d9be9d/state/set', 'brightness_state_topic': None, 'platform': 'mqtt'},)
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/mqtt/mixins.py", line 316, in async_discover
config: DiscoveryInfoType = discovery_schema(discovery_payload)
File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 232, in __call__
return self._exec((Schema(val) for val in self.validators), v)
File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 355, in _exec
raise e if self.msg is None else AllInvalid(self.msg, path=path)
File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 351, in _exec
v = func(v)
File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
return self._compiled([], data)
File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 818, in validate_callable
return schema(data)
File "/usr/src/homeassistant/homeassistant/components/mqtt/light/__init__.py", line 41, in validate_mqtt_light_discovery
config: ConfigType = schemas[config_value[CONF_SCHEMA]](config_value)
File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 232, in __call__
return self._exec((Schema(val) for val in self.validators), v)
File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 355, in _exec
raise e if self.msg is None else AllInvalid(self.msg, path=path)
File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 351, in _exec
v = func(v)
File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
return self._compiled([], data)
File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 818, in validate_callable
return schema(data)
File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
return self._compiled([], data)
File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 595, in validate_dict
return base_validate(path, iteritems(data), out)
File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 433, in validate_mapping
raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: string value is None for dictionary value @ data['brightness_state_topic']

So somewhere along the way, the value of the “brightness_state_topic” attribute is lost.
I see nothing useful in the Mosquitto logs.

Any idea what may be going on or how to further debug it?

2

I think I found the problem. My ESP published the JSON using something like this:

client.publish(TOPIC_CONF.c_str(), message, true);

The last argument means “retain”, which seems to cause Mosquitto to pretty much ignore the payload in follow-up requests.

Since I didn’t know how to delete retained messages, I just uninstalled and re-installed the Mosquitto addon, now the error disappeared and with Mosquitto in debug mode I can see that the device is connected. (Home Assistant still doesn’t show it for now, but that is another problem.)

3

…and the second problem is finally also resolved. For the record, what was wrong was the maximum packet size if PubSubClient on the ESP is 256 characters. After calling this:

client.setBufferSize(512);

The ESP sends the full packet, and it makes it’s way into Mosquitto and even into HA.