Templates broken after update 2022

They’re published with this command (not entirely sure the flags, I got it from the rtl_433 addon and some forums)

  echo $line | /usr/bin/mosquitto_pub -h $MQTT_HOST -u $MQTT_USER -P $MQTT_PASS -i RTL_433 -r -l -t $MQTT_PATH -V 5 --property publish message-expiry-interval 900

I think the expiry-interval was added because I had old data getting stuck in the MQTT broker for stuff that was only in range briefly but it’s been a while I can’t remember

Then they all follow the same copy-past pattern.

sensor:

  # Outside Front Sensor
  - platform: mqtt
    name: "Outside Front Temperature"
    unique_id: "outside_front_temperature"
    device_class: temperature
    force_update: true
    state_topic: "homeassistant/sensor/rtl433/Acurite-Tower/14084"
    unit_of_measurement: '°C'
    value_template: >-
      {% if value_json.temperature_C > -20 and value_json.temperature_C < 70 %}
        {{ value_json.temperature_C }}
      {% else %}
        {{ states.sensor.outside_front_temperature.state }}
      {% endif %}

And the intermediate computation is a max/min

  - platform: min_max
    name: main_floor_max_temperature
    type: max
    entity_ids:
      - sensor.sun_room_temperature_filtered
      - sensor.dining_room_temperature_filtered
      - sensor.office_temperature_filtered
      - sensor.livingroom_temperature_filtered

Yep they are retained messages:

-r,
--retain

If retain is given, the message will be retained as a “last known good” value on the broker.

Even if it is a race, and the subscription to the topic and publish of the retained message by the broker is missed, you still should get an update 15 seconds later (if that is your sensor update interval), and this will change the temperatures from unknown to some value, updating the template. At which point the delay should start. On expiration of that delay the template will report off or on.

the difference here being the state of the binary_sensor the delay is based on Is known. there’s no ‘unknown’ in the system related to this at all.

this feels like a fluke of the new design really and imho it should take the state of that base sensor, and then wait for the delay.

1 Like

Yeah, and that’s what I was expecting/hoping for when I first read the breaking-changes entry.

Yes there is. Consider this rather important template of mine that I dont want changing quickly as it is responsible for switching big loads:

- name: "Excess Solar"
  icon: "mdi:solar-power"
  state: "{{ ( states('sensor.lp20_inverter_power')|float(0) - states('sensor.hot_water_power')|float(0) ) > 2500 }}"
  delay_off:
    minutes: 10
  delay_on:
    minutes: 10

When Home Assistant starts, how long was it off for?

Was it a restart or did I just move house?

Is the previous state still valid?

Unknown.

I’m going to bow out of this discussion because I have already identified why mmiller7’s binary_sensor behaves differently now (the answer to the thread’s original question), I have no objection to the new behavior, and have no interest in debating it.

It’s whoever is unhappy with the new behavior that needs to debate this with the development team. Good luck.

1 Like

Maybe then there should be an initial-state value too?

I have some similar ones for lighting which are triggered by motion sensors and its (minimally) annoying that the lights used to default-off after restart but its also kinda dumb that it can’t pick up the state if it was on or off based on the state of physical devices.

In my light case, I turn on/off based on motion sensors but the switch on/off state would provide a ground truth of the last state.

It sounds like you would have a similar situation where you shouldn’t need to wait a long time “unknown” if there was a way to populate it on startup based on the state of the solar/inverter switch in real life, then you’d have your on/off to trigger on a delay from.

In that case (to me) the long “unknown” still seems like a very poor workaround at best.

Likewise.

That was suggested in the issue Marius raised.

Yep after the update my motion controlled lights would take two triggers to turn on. Drove me nuts until I realised it was because I was triggering from off to on, where as the state initially (after a restart) was changing unknown to on. Solution: drop the “from” in the triggers. Also as the motion sensors report via mqtt another solution was to retain the state in the broker (retain flag). This initialises all my movement sensors after a restart.

Nope. I want that sensor to take it’s time and calculate the first state over 10 minutes. The “lp20” in this sensor sensor.lp20_inverter_power stands for low pass filter 20 minutes. That should be given time to stabilise. In this case I don’t want any sudden changes.

For your specific sensor ( a template sensor with both delay for on and off ) that could be true .

A regular binary sensor with known state, like in my example: all known. At startup.

On or off.

Changing state should not be an issue for either delay.

This new approach has made sensors using longer delays rather useless, and we have to resort to automations with a for: option.

Btw, it’s not a matter of lack of understanding (nor the need for explanation ) of the new workings.

It’s more a matter of human versus computer intelligence…

I now have to explain to my fellow home mates the system isn’t ready for another hour, because it says it isn’t 100% sure, while we all know it should be considering the base considerations…

1 Like

There is another workaround sounds like the delay_ field takes a template someone mentioned, which sounds like a workaround to only have on-delay if its off and off-delay if its on.

But its going to be a mountain of work to come up with the correct permutations of if/else for every one of the many template sensors I have. And so many hours of hunting thru YAML and chances for typos. Seems like a flag yes/no would be more appropriate, or initial-state where you can duplicate the exact same thing as the value (or sub the target switch state)

To avoid so much complexity in my automation conditions I have most of the on/off stuff buffered thru templates to “break down” the intermediate steps, then automate based on those. But that also means there’s a HUGE number of things to fix whatever “fix” ends up meaning.

Oh, also, you can have more than one from in the automations BTW…so you could have

from:
    -off
    -unknown
to:
    -on

(I use something similar to restore the state of switches after they lose connectivity - such as a brief power-blip that reboots smart switches but doesn’t affect the sever on the UPS, when the switch goes from unavailable to off but should be on, I turn the switch on)

But changing EVERYTHING everywhere is also seemingly poor workaround when information exists to just have a proper value stored in the binary sensor to begin with in the majority of cases.

Disclaimer: I haven’t thought this through; just brainstorming.

Would a Trigger-based Template Binary Sensor be useful for this particular situation?

Not sure Taras, what were you thinking of? Have only a few trigger based sensors like that , and keep rebuilding those to regular sensors because the trigger based never really does what I seek…

For what its worth, I used to do stuff with a lot of for: options on automations and changed to intermediate binary-sensors with delays.

The reason I opted for so many template sensors goes back to my software engineering skills where good easy to maintain design is you want as little duplicated code as possible and as much code-reuse as possible. That means if you use the same thing in 2 places it ought to be simply referenced and the “logic” lives in a single place. By putting the delays in the template, it means if you decide you want to change it you can fix 1 place and its applied everywhere instead of having to find every instance. I’d hope many others apply that same “code re-use” logic to their designs too. The template-sensors are the closest thing we really have to “function calls” for state-logic.

This is another reason I find the “just fix all your automations everywhere” attitude really off-putting when it feels like the implementation could have been so much better with multiple ways to have it fixed that would not be such severe breaking changes. Similarly, when a lot of defaults changed for int/float/round/other math stuffs a while back my first thought is “why does this have to be set in every single cal, why can’t I specify a global-default for my system”.

The same “improvement” could have been made in a slightly different implementation to simultaneously fix the issue but keep everything easier to maintain.

1 Like

Merely commenting on the possibility of using a State Trigger, with a for option, in a Trigger-based Binary Sensor instead of in an automation (if the goal is to create a sensor whose value is based on a time duration).

hmm, well the goal is to have a grace period of an hour after the binary_sensor has changed state, and then execute the action. So, were I before had the binary sensor with the delay trigger the automation, I now have the binary sensor trigger the automation with a delay :wink:

I did give your suggestion some thought ofc, (thanks for that) but couldn’t find a way to do as described above.

I some how would have hoped that non template entities would even get their state restored, and the delay would survive a restart… but since this ia about last_changed, and not last_triggered that wont happen.
For that reason too I dont believe a fixed initial_state would be useful, because that would possibly interfere with the actual state the binary has at the time of restart.

why not a template initial-state? Then you could pre-populate it with anything you want (including another sensor or switch) depending on your need and cut the “unknown” time down from minutes/hours to a few seconds?

not sure about that, it would go against what a template sensor is really: a reflection of the states you manipulate in the template.

And, it’s a whole other matter than the delay issue, which I can understand when based on a template sensor, like Toms example.

My point in the issue is when the base state of the binary (in that case) is fully known, a delay should not be unknown, but but simply not yet fulfilled, so have the unchanged state of the base binary.

we do have other constructions in HA, like assumed state, though, even mentioning that here makes me fear the devs will add an unknown there too…

For this workaround, would it be ‘None’ or ‘unknown’?

When I put this in, I’m seeing erratic on/off behavior where stuff turns on when it shouldn’t be.

For me ‘None’ works. I made an update and skipped the ‘None’ or ‘unknown’ check.
Now I’m checking for the ‘on’ state in the delay_off template.

    delay_off: >
      {{ 15 if is_state('binary_sensor.xyz','on') else 0 }}
1 Like

None should not work, but we don’t know if bertrebs sensor is custom or not. Those tend to not follow the rules. The only states you’ll see work properly on a binary_sensor from a non-custom integration using is_state are: ‘on’, ‘off’, ‘unknown’, ‘unavailable’. If the binary_sensor comes from a custom integration, all bets are off but typically they should share the same states.

state meaning
on on
off off
unknown system does not know the state but the device is connected and working. You’ll usually only see this state after startup from battery devices because they only report in intervals (and they don’t have a way for HA to ask for the state).
unavailable system cannot reach the device at all