Restarts of HA are taking quite some time on my machine and I found out, that the main issue seems to be MQTT integration - it takes over 60 seconds to finish setup!
Granted, some integrations need more time than others (f.e. LocalTuya needs 15 secs, Mobile App needs 11 secs), but more than a minute seems a bit too much.
Anyone else with similar issue and/or advice hoiw to reduce it? Or is this normal?
Seems normal to me.
Hereās my startup time for mgtt:
Running on a I3 nuc. w/ssd.
EDIT:
After watching and reading this post evolve over time I remembered that I still had a vast majority of my Tasmota equipment still configured via yaml, wile running Tasmota FW 9.5 with the auto discovery on.
I guess it slipped my mind.
However after a few hours of reconfiguration my MQTT startup time is now down to 34.40 s
I still have a few hours of .yaml entries to go thru and remove and convert all automationās associated with the .yaml entries.
Intel NUC i3 with 128GB SSD, and my MQTT takes ~73 seconds to start up.
The actual restarts of HA itself are lightning quick though, but it does take a minute or so before it is fully up and running with all sensors reporting back correctly. The fact that is take a bit of time to fully display properly really hasnāt bothered me ā¦ but then I only restart when I need to (upgrades, I notice something is playing up).
Well, thatās weirdā¦ no devices and no entities.
Iām only using the integration (so it seems) to publish MQTT messages in certain cases - otherwise I use zigbee2mqtt for sensors (33 devices and 246 topics) and the broker runs on my Pi4 as a Supervisor addon.
It seems mqtt entities manually defined in .yaml donāt show up there. Just removed a switch I created yesterday on one of my HA instances to help someone on the forum, and the number of entities was not changed.
Devices can only be created by auto-discovery, so that is normal in that case.
Just learned myself I have more than 506 mqtt entities
The fact that francispās system subscribes to topics representing hundreds of entities in under 10 seconds suggests the process is quick and efficient. If it takes over a minute then either there are thousands of topics involved (rare) or thereās something else responsible for the delay (likely). Iād be interested to know what that āsomething elseā could be.
In my case, the broker is the Mosquitto Add-on hosted by the same machine as Home Assistant. All entities are configured via MQTT Discovery. I am also using the Tasmota integration which also employs MQTT for communication. That has 260 entities and takes 0.57 seconds to load. It would appear that thereās more to this puzzle than merely the number of entities involved.
The biggest difference between mine and your installation seems to be, that I donāt use solely MQTT Discovery, but have a bunch of MQTT sensors defined in sensor.yaml (140 to be exact). Iām not sure if I dare removing them for a test, to see if it loads faster without them (donāt want to break something, that works ).
Otherwise also my broker is an Addon runing on the same machine as HA. And besides the zigbe2mqtt (19 devices) I use MQTT for some other internal communication between HA and four Android phones and 11 ESP32 and ESP8266 microcontrolers.
OK, thatās one possible explanation. Does your Pi4 use an SD card or SSD?
In the āFWIWā department:
I have a test system hosted by an RPi3 with an SD card. Although it isnāt configured with MQTT, there are other integrations it has in common with my production system. The RPi3 is noticeably slower than the ancient Core2 Duo with SSD. Loading input_booleans takes over 10 seconds (which causes a warning message in the log noting the long delay).
Btw, in the meantime I tried removing 80 out of 140 manualy defined entities in sensor.yamlā¦ loading time reduced from 67 to 28 secondsā¦ I guess I have my answer :).
Well, removing its workload is bound to reduce its loading time. What would be interesting to see is if you make them discoverable to MQTT Discovery and compare the loading time.
If you are interested, I explained how to use a script to create an entity via MQTT Discovery:
You can also define devices this way because manual MQTT configuration only allows you to define entities, not devices.
Tried turning on Autodiscovery and it found my 19 Aqara devices with corresponding 100 entities right away. Now, Iām sure it would reduce loading time and once day I might even try converting everything, but right now Iām not sure itās really worth the effort as I would have to:
Manually rename the entities, that autodiscovery found, to match the manualy defined entities - otherwise I must change all cards, automations etcā¦
Make also other (the missing 40) MQTT entities autodiscoverable (not even sure how to do that on ESP and android machines) and then repeat the process from #1.
So, if I find myself one day having too much free time and/or long setup time bothering me too much, I might try it, but for nowā¦ well, one minute at rare restarts is not too annoying, is it?
How much did it reduce loading time? What was reported for MQTT in Info?
Easily done using the script technique I described above.
Thatās your call; you created this thread out of concern for the length of the loading time. Now you know itās much longer than for users with far more MQTT-based entities.
Yes, I know, and Iām really thankfull for all your inputs. What Iām trying to say is, that at the moment Iām not sure if solving this CF Iāve created through time (as I didnāt know the right way of doing it) is worth the workload compared to waiting one minute at each restart (btw, I misscalculated before, autodiscovery found 100 entities, but I was only using 40 of them - so I would have to newly autodiscover the missing manually defined 100 of them and some of them are even templates, so I would have to have an additional sensor anyway).
Iāve learned a lot from this and from now on Iāll use autodiscovered entities when adding new stuffā¦ but solving the legacy issues will wait for better times :).
Well, deep inside I know, Iāll have to deal with it somedayā¦ so if itās not too much trouble for you, could you show me on my example, how would I make the esp9 autodiscoverable (the end result should be 6 entities - 4 from json and two separate ones)?
Iām afraid Iām unable to create the script myself from the example you providedā¦
Are your ESP-based devices flashed with Tasmota or ESPHome?
If not, you can create a script that employs MQTT Discovery to create entities. Post your existing MQTT sensor (s) configuration for the ESP9 device. That information is needed in order to create the script.
Neitherā¦ they were programmed using Arduino IDE. At the time of programming (when I just started my HA joruney) I had no knowledge about anything and therefore made some crappy decisions, which I can not easily correct now (re-programming of all ESPās is not an option anymore). Therefore I assumed, that I can correct at least Autodiscovery in a way you suggested - by posting certain messages to āconfigā topics for existing devices.
Please note, that (again, based on my begginerās mistakes) the first four sensors are sent in one topic as json and the last two are sent in separate topics as raw text - if that is relevant for the script.