2022.6: Gaining new insights!

This is probably due to me not changing over MQTT sensors yet but I noticed that reloading manual mqtt configuration only updates the sensors not binary sensors. More testing to follow but as of now mqtt binary sensors are only reloaded on restart.

there will be a Reload option on the integration panel some time soon, so all entities can be reloaded from there.

for now I have severe issues with reloading the MQTT entities , see few posts above

maybe it is because I no longer have platform: mqtt sensors and moved them all to the new mqtt: format.

its nasty because I was experimenting with several availability options in the sensors configs, and reloaded, only to find out my whole instance was taken down…

too bad we dont have an availability_template option we can relate to the state topic, and need a special availability topic (which none of my sensors have)

would love to be able to do:

  binary_sensor:

    - unique_id: amp_left_auditorium_state_binary
      state_topic: '70:B3:D5:6F:31:F2/powerswitch-zwave/c5745b57/state'
      name: Amp left Auditorium state
      <<: &binary
        payload_on: 'on'
        payload_off: 'off'
        device_class: power
        availability_template: >
          {{ value in ['on','off'] }}

or

    - unique_id: temperatuur_living
      state_topic: '70:B3:D5:6F:31:F2/sensormultilevel-zwave-airtemp/0c926d01/airtemp'
      name: Temperatuur Living
      icon: mdi:thermometer
      unit_of_measurement: °C
      device_class: temperature
      value_template: >
        {{value|round(1,default=none)}}
      availability_template: >
        {{ is_number(value) }}

Yes, I did. I haven’t had time to dig deep but the problem seems to clear up if I compress the DB. Then a couple days later the average sensor gets results that are way off. Recently saw the average humidity = to 150% where the humidity sensor was around 20%. This just started with 2022.6 update.

Once it reports weird values it stays weird until I compress and then everything is normal again.

There were a number of suggestions:

  1. Indentation: Other comments have already verified this, but I did test the indentation before posting and discovered that it could go either way, which is nice because it saves the hassle of adding the indent to every line when converting to this new format!
  2. Included light. in the unique_id: I haven’t had any issues with that; there are no errors listed for that line and it worked fine when changing the id’s to include _mqtt for my Alexa globs. Even if this were an issue, it wouldn’t change the error on schema, which appears to only allow default.
  3. Change the way availability is formatted - same as above; either layout works fine and changing this would not affect the schema error.

Seems to me that this is likely a bug that needs to be fixed? EDIT TO ADD: I just tried schema: default and got no error and schema: template and got an error to include a required configurable variable, indicating that template is an accepted schema.

Hi,
I’ve just updated my configuration and seems everything is working fine.
My configuration.yaml file looks like this:

#sensor others: !include sensors.yaml
#switch: !include switch.yaml
mqtt:
    sensor: !include mqtt_sensors.yaml
    switch: !include mqtt_switches.yaml

The commented rows were my previous configuration.
An example of my mqtt_sensors.yaml file is:

  - name: "Corrente"
    state_topic: "tele/tasmota_consumi/SENSOR"
    value_template: "{{ value_json['ENERGY'].Current }}"
    unit_of_measurement: "A"
  - name: "Potenza Ist."
    state_topic: "tele/tasmota_consumi/SENSOR"
    value_template: "{{ value_json['ENERGY'].Power }}"
    unit_of_measurement: "W"

More details of MQTT consider_home changes here:

I thought I would provide you with my experience…I have moved all my MQTT YAML configs to the new format and running 2022.6.2, the MQTT reload works. I will add that after about 20-30 seconds I do get a lot of errors but they are related to cover groups which seems to be an issue with the group utility code and not the MQTT reload specifically. I find also that my MQTT entities take about the same 20-30 seconds to show up again after reload, so I deduce that it takes this long for the reload to actually process. During this 20-30 seconds I now get an occasional warning about some other integration taking more than 10 seconds to do something. But nevertheless all works when finished reloading.

You might be interested in an another user’s comments regarding loading time. I had helped them convert from defining MQTT entities via YAML to MQTT Discovery. Basically, you make a script that publishes the configuration information in the format accepted by MQTT Discovery and it creates the entities. It also lets you define devices which is something you can’t currently do in YAML.

The user converted 130 entities, from YAML to MQTT Discovery, and reported the following observation:

Loading time for MQTT reduced from 68 to 5.9 seconds

An example of a script, to create an MQTT entity via MQTT Discovery, is posted here and is followed by the user’s reply.

2 Likes

Thanks. I actually spent a fair bit of time reading that post a few months ago proving the concept out (to myself), but never took the time to convert mine over. I had not picked up on the time savings. Again thanks for the tip :slight_smile:

You’re welcome!

FWIW, I have about four dozen MQTT entities and all are configured via MQTT Discovery (so I am unaffected by the recent changes to the format of YAML-based MQTT configuration). According to the System Health page, the MQTT integration took 2.65 seconds to load.

That’s pretty neat. I always used Mqtt discovery for almost everything so I never had a perf comparison.

Another cool thing with that route is you could put that script in a separate folder that folder watcher watches. Then you could make an automation that triggers when that script is modified and does:

  1. check config
  2. reload scripts
  3. run the script

Then you’ve got automatic reloading of Mqtt entities after a config change without a restart as well (another issue others have faced above from what I’ve been reading). And fully automated!

Setting up from YAML is usually slower for any integration. Setting up a lot of yaml platforms can painfully slow (as observed above).

That’s partially driven by python overhead (will be much better in python 3.11), and the yaml parser being slow because of the enhanced error reporting (we have to keep that). The new python version will improve the yaml performance as well but we are still likely 16-20 months away from being able to upgrade since it’s not even released yet.

Some light reading Python 3.11 Performance Benchmarks Are Looking Fantastic - Phoronix

Generally anything you can move out of yaml will improve your startup time.

1 Like

I’ll give that a try for YAML-based helpers because they take surprisingly long to load.

My Mqtt was always atop of the integration loading times (over 450 sec…) and now has sunk to a measly 7th spot with

all in Yaml… so other improvements have been made too.

Problems here with mqtt integration. I have a dozen devices that are in the mqtt integration through discovery and work well. There are three that are not. They are all 3-way switches with rules and use of EVENT message instead of power1.

I posted on this site in another area. It shows my configuration.yaml, tasmota setup, and the rules.

The problem is that HA will not send ON payloads via the UI until HA wakes up (for lack of a better term). I have to use an mqtt tool (like HA’s mqtt integration publish page or mqtt_lens). Once this happens, HA ‘wakes’. As a fix for now, I have an automation at HA startup that sends ON and OFF payloads to wake up HA.
The set up was working well for a couple of years under the old platform: mqtt setup but is now broken. Control of the switches works as expected through an mqtt tool.

That might be the incentive I need to move my many helpers to the UI.

Have you looked in the database to see if that weird value is in there? I am not seeing any such issues.

Yes it is a compelling reason, there doesn’t seem to be a good “yaml is best” argument for not using UI based helpers.

But doesn’t this create a problem with sharing code?

I have a package that I share and as far as I can tell has quite a few (by which I mean probably at least ten, maybe a lot more) people using it and it relies on a lot of helpers.

I don’t believe I could sensibly share it if I moved to UI based helpers.

I also have a severe problem with reloading MQTT.

When I click the UI I get an error both in the UI and in the log.

I saw the bug report Reload MQTT entities takes down system · Issue #73275 · home-assistant/core · GitHub

And added my log entries. And MQTT stops working totally for everything. I have to restart HA to get it alive again.

This is a very recent bug. I have loaded manual MQTT entities so many times before and also just weeks ago.

The only way I can reload MQTT is to restart Home Assistant. This is really nasty

1 Like

I have 2 counters in my system, both UI created Helpers

so one would expect that to happen in the blink of an eye. And yet:

is indicative of an issue?

Id be glad to move all of my 130 input_booleans from yaml to UI, but given:

I wonder if that would be worth the trouble…