It seems mqtt entities manually defined in .yaml don’t show up there. Just removed a switch I created yesterday on one of my HA instances to help someone on the forum, and the number of entities was not changed.
Devices can only be created by auto-discovery, so that is normal in that case.
Just learned myself I have more than 506 mqtt entities 
The fact that francisp’s system subscribes to topics representing hundreds of entities in under 10 seconds suggests the process is quick and efficient. If it takes over a minute then either there are thousands of topics involved (rare) or there’s something else responsible for the delay (likely). I’d be interested to know what that ‘something else’ could be.
In my case, the broker is the Mosquitto Add-on hosted by the same machine as Home Assistant. All entities are configured via MQTT Discovery. I am also using the Tasmota integration which also employs MQTT for communication. That has 260 entities and takes 0.57 seconds to load. 
It would appear that there’s more to this puzzle than merely the number of entities involved.
The biggest difference between mine and your installation seems to be, that I don’t use solely MQTT Discovery, but have a bunch of MQTT sensors defined in sensor.yaml (140 to be exact). I’m not sure if I dare removing them for a test, to see if it loads faster without them (don’t want to break something, that works ).
Otherwise also my broker is an Addon runing on the same machine as HA. And besides the zigbe2mqtt (19 devices) I use MQTT for some other internal communication between HA and four Android phones and 11 ESP32 and ESP8266 microcontrolers.
OK, that’s one possible explanation. Does your Pi4 use an SD card or SSD?
In the “FWIW” department:
I have a test system hosted by an RPi3 with an SD card. Although it isn’t configured with MQTT, there are other integrations it has in common with my production system. The RPi3 is noticeably slower than the ancient Core2 Duo with SSD. Loading input_booleans takes over 10 seconds (which causes a warning message in the log noting the long delay).
SSD, much safer and faster :).
Btw, in the meantime I tried removing 80 out of 140 manualy defined entities in sensor.yaml… loading time reduced from 67 to 28 seconds… I guess I have my answer :).
Well, removing its workload is bound to reduce its loading time. What would be interesting to see is if you make them discoverable to MQTT Discovery and compare the loading time.
If you are interested, I explained how to use a script to create an entity via MQTT Discovery:
You can also define devices this way because manual MQTT configuration only allows you to define entities, not devices.
Tried turning on Autodiscovery and it found my 19 Aqara devices with corresponding 100 entities right away. Now, I’m sure it would reduce loading time and once day I might even try converting everything, but right now I’m not sure it’s really worth the effort as I would have to:
- Manually rename the entities, that autodiscovery found, to match the manualy defined entities - otherwise I must change all cards, automations etc…
- Make also other (the missing 40) MQTT entities autodiscoverable (not even sure how to do that on ESP and android machines) and then repeat the process from #1.
So, if I find myself one day having too much free time and/or long setup time bothering me too much, I might try it, but for now… well, one minute at rare restarts is not too annoying, is it? 
How much did it reduce loading time? What was reported for MQTT in Info?
Easily done using the script technique I described above.
That’s your call; you created this thread out of concern for the length of the loading time. Now you know it’s much longer than for users with far more MQTT-based entities.
Yes, I know, and I’m really thankfull for all your inputs. What I’m trying to say is, that at the moment I’m not sure if solving this CF I’ve created through time (as I didn’t know the right way of doing it) is worth the workload compared to waiting one minute at each restart (btw, I misscalculated before, autodiscovery found 100 entities, but I was only using 40 of them - so I would have to newly autodiscover the missing manually defined 100 of them and some of them are even templates, so I would have to have an additional sensor anyway).
I’ve learned a lot from this and from now on I’ll use autodiscovered entities when adding new stuff… but solving the legacy issues will wait for better times :).
Well, deep inside I know, I’ll have to deal with it someday… so if it’s not too much trouble for you, could you show me on my example, how would I make the esp9 autodiscoverable (the end result should be 6 entities - 4 from json and two separate ones)?
I’m afraid I’m unable to create the script myself from the example you provided…
Are your ESP-based devices flashed with Tasmota or ESPHome?
If not, you can create a script that employs MQTT Discovery to create entities. Post your existing MQTT sensor (s) configuration for the ESP9 device. That information is needed in order to create the script.
Neither… they were programmed using Arduino IDE. At the time of programming (when I just started my HA joruney) I had no knowledge about anything and therefore made some crappy decisions, which I can not easily correct now (re-programming of all ESP’s is not an option anymore). Therefore I assumed, that I can correct at least Autodiscovery in a way you suggested - by posting certain messages to “config” topics for existing devices.
So, there are six sensors for esp9, and they are:
- platform: mqtt
state_topic: 'home/esp9/dnevna'
name: 'Temperatura Dnevna'
unit_of_measurement: '°C'
value_template: "{{ value_json.temperatura }}"
- platform: mqtt
state_topic: 'home/esp9/dnevna'
name: 'Vlažnost Dnevna'
unit_of_measurement: '%'
value_template: "{{ value_json.vlaznost }}"
- platform: mqtt
state_topic: 'home/esp9/dnevna'
name: 'Datum Dnevna'
value_template: "{{ value_json.datum }}"
- platform: mqtt
state_topic: 'home/esp9/dnevna'
name: 'Čas Dnevna'
value_template: "{{ value_json.cas }}"
- platform: mqtt
state_topic: 'home/esp9/pinstatus'
name: 'Senzor Dnevna'
- platform: mqtt
state_topic: 'home/esp9/dummy'
name: 'Status Dnevna'
Please note, that (again, based on my begginer’s mistakes) the first four sensors are sent in one topic as json and the last two are sent in separate topics as raw text - if that is relevant for the script.
The following script (script.create_sensors) contains five service calls. Each service call is an mqtt.publish creating a sensor via MQTT Discovery. You can easily see all of the sensor information you provided is included in the payload of each mqtt.publish call.
The payloads are published as retained messages which means they will be stored on the MQTT broker; their configuration information will be instantly available to Home Assistant on startup. You don’t have to run the script more than once (unless you want to change the configuration of one or more of the five sensors).
In addition to creating the five sensors, it groups them together within a single device called ESP9. This is only possible with MQTT Discovery and cannot be done with manual MQTT configuration. I’ve included this in the script as a convenience so when you go to Configuration > Devices > ESP9, you will see all the sensors related to it.
IMPORTANT
Do not execute this script before you first remove your existing five MQTT sensors from your system. If you don’t remove them and run this script, you will create duplicated sensors.
I suggest you delete, or comment out, the existing 5 sensor configurations then execute Configuration > Server Controls > Reload Manually Configured MQTT Entities to purge them from Home Assistant. Then run the script and they should all re-appear but this time they will be created via MQTT Discovery.
For more information about MQTT Discovery, refer to its documentation.
Show script
script:
create_sensors:
alias: "Create sensors via MQTT Discovery"
sequence:
- service: mqtt.publish
data:
topic: homeassistant/sensor/temperatura_dnevna/config
retain: true
payload: >
{
"name": "Temperatura Dnevna",
"unique_id": "esp9_temperature_dneva",
"device_class": "temperature",
"unit_of_measurement": "°C",
"state_topic": "home/esp9/dnevna",
"value_template": "{{ value_json.temperatura }}",
"device": {
"identifiers": ["esp9"],
"name": "ESP9",
"model": "ESP",
"manufacturer": "Espressif",
"sw_version": "1.XX"
}
}
- service: mqtt.publish
data:
topic: homeassistant/sensor/vlaznost_dneva/config
retain: true
payload: >
{
"name": "Vlažnost Dnevna",
"unique_id": "esp9_vlaznost_dneva",
"unit_of_measurement": "%",
"state_topic": "home/esp9/dnevna",
"value_template": "{{ value_json.vlaznost }}",
"device": {
"identifiers": ["esp9"],
"name": "ESP9",
"model": "ESP",
"manufacturer": "Espressif",
"sw_version": "1.XX"
}
}
- service: mqtt.publish
data:
topic: homeassistant/sensor/datum_dneva/config
retain: true
payload: >
{
"name": "Datum Dnevna",
"unique_id": "esp9_datum_dneva",
"state_topic": "home/esp9/dnevna",
"value_template": "{{ value_json.datum }}",
"device": {
"identifiers": ["esp9"],
"name": "ESP9",
"model": "ESP",
"manufacturer": "Espressif",
"sw_version": "1.XX"
}
}
- service: mqtt.publish
data:
topic: homeassistant/sensor/cas_dneva/config
retain: true
payload: >
{
"name": "Čas Dnevna",
"unique_id": "esp9_cas_dneva",
"state_topic": "home/esp9/dnevna",
"value_template": "{{ value_json.cas }}",
"device": {
"identifiers": ["esp9"],
"name": "ESP9",
"model": "ESP",
"manufacturer": "Espressif",
"sw_version": "1.XX"
}
}
- service: mqtt.publish
data:
topic: homeassistant/sensor/senzor_dneva/config
retain: true
payload: >
{
"name": "Senzor Dnevna",
"unique_id": "esp9_senzor_dneva",
"state_topic": "home/esp9/pinstatus",
"device": {
"identifiers": ["esp9"],
"name": "ESP9",
"model": "ESP",
"manufacturer": "Espressif",
"sw_version": "1.XX"
}
}
- service: mqtt.publish
data:
topic: homeassistant/sensor/status_dneva/config
retain: true
payload: >
{
"name": "Status Dneva",
"unique_id": "esp9_status_dneva",
"state_topic": "home/esp9/dummy",
"device": {
"identifiers": ["esp9"],
"name": "ESP9",
"model": "ESP",
"manufacturer": "Espressif",
"sw_version": "1.XX"
}
}
3 Likes
Great, thx a lot…
I’ve tried it already with publishing a message to my broker (directly in MQTT Explorer) in this way:
Published to topic:
homeassistant/sensor/esp9/temperatura/config
json payload:
{
"availability": [
{
"topic": "zigbee2mqtt/bridge/state"
}
],
"device": {
"identifiers": [
"esp9"
],
"manufacturer": "Espresiff",
"model": "ESP8266",
"name": "Dnevna",
"sw_version": "1.0"
},
"enabled_by_default": true,
"icon": "mdi:gesture-double-tap",
"json_attributes_topic": "home/esp9/dnevna",
"name": "Temperatura Dnevna",
"state_topic": "home/esp9/dnevna",
"unique_id": "esp9_temperatura_dnevna",
"value_template": "{{ value_json.temperatura }}"
}
I based it on config topics of other devices, which have discovery enabled (IKEA and Aqara). But nothing happened… I could see the topic briefly appear in MQTT, but disappeared a few milliseconds later - no error returned. Obviously something was wrong, just no idea what.
So, I’ll try with your scripts and I’ll take it really slow, step by step and device by device, to make sure, everything works OK :). Will report results.
I want to caution you again about duplication. Your latest attempt to create a sensor via MQTT Discovery used the same name as one of your existing sensors. Unless you have already removed a sensor that’s manually configured, don’t attempt to create a duplicate of it via MQTT Discovery.
Although you can use MQTT Explorer to publish a payload to a topic, you can also use Developer Tools > Services:
Here’s a uniquely named sensor you can try to create via MQTT Discovery. Just paste the following YAML into Developer Tools > Services and click Call Service. Then check Configuration > Entities, or Developer Tools > States, to confirm it was created.
service: mqtt.publish
data:
topic: homeassistant/sensor/temperatura_dnevna_test/config
retain: true
payload: >
{
"name": "Temperatura Dnevna Test",
"unique_id": "esp9_temperatura_dnevna_test",
"state_topic": "home/esp9/dnevna",
"value_template": "{{ value_json.temperatura }}",
"device_class": "temperature",
"unit_of_measurement": "°C",
"icon": "mdi:gesture-double-tap",
"device": {
"identifiers": ["esp9"],
"name": "ESP9",
"model": "ESP8266",
"manufacturer": "Espressif",
"sw_version": "1.0"
}
}
1 Like
Well, this one doesn’t work for me…
When trying to publish, I get an error, that “value_json” is undefined. I guess the service understands, that it should pass the template {{ value_json.temperatura }} as a part of payload, which of course has no value in this context, so it returns an error.
So I’ll have to do it with scripts or in MQTT explorer…
Sorry about that; my mistake. Home Assistant’s Jinja2 interpreter attempts to evaluate this template
{{ value_json.temperatura }}
before the payload is published. We don’t want the jinja2 interpreter to evaluate it so we wrap the template in raw/endraw tags as shown below:
service: mqtt.publish
data:
topic: homeassistant/sensor/temperatura_dnevna_test/config
retain: false
payload: |
{
"name": "Temperatura Dnevna Test",
"unique_id": "esp9_temperatura_dnevna_test",
"state_topic": "home/esp9/dnevna",
"value_template": {% raw %}"{{ value_json.temperatura }}"{% endraw %},
"device_class": "temperature",
"unit_of_measurement": "°C",
"icon": "mdi:gesture-double-tap",
"device": {
"identifiers": ["esp9"],
"name": "ESP9",
"model": "ESP8266",
"manufacturer": "Espressif",
"sw_version": "1.0"
}
}
Here’s the result after I click Call Service and publish a JSON payload, containing the temperature, to home/esp9/dnevna.
The sensor’s appearance in the UI:
3 Likes
Yay, it works. Had to publish 130 pre-configured payloads (a couple of hours clicking, typing and double-checking, that the entity names will be the same, as they were before ), but now I have them all covered. Loading time for MQTT reduced from 68 to 5.9 seconds :).
Now, the new issue I have is, that some entities are missing as I somehow managed to delete two branches of existing config topics from Aqara and Ikea in MQTT Explorer while comparing existing and new entities and will most likely have to re-pair those devices (“only” 17 of them ) to have the entities discovered again. I’m not sure, but maybe they even got deleted by themselves when I enabled and again disabled discovery on the MQTT integration while testing (and additionally deleted all integration related devices and entities).
But apart from that, happy happy :).
Thx a lot once again @123 for your help.
1 Like
This is great. Thought that all this is only possible from “outside” but not already with mqtt.publish. With that it’s even more curios, that the device creations/assignments are not possible via yaml, neither old nor new way. Personally I like(d) the yaml way, but this is more or less the same, but with devices.
Is there anything, which I would not be able to do via the discovery way?
And secondly, for what are the identifiers of the device?
Very useful post! Thanks
1 Like