Heads up! Upcoming breaking change in the Template integration

123 · October 21, 2020, 2:34am

So is that a confirmation or are you guessing too?

You said no one will need to understand it because it will be so easy to use. However, I wager people will want to understand why one small change to their template causes it to change from updating every second to once a minute. Pretty sure they will notice near real-time updates to a relaxed per minute interval … and ask why.

petro · October 21, 2020, 3:00am

Nothing updates every second… it’s a rate limit described above. It updates when states change. See my diagram for clarification. Example: I have this template {{ states.light }}. If no light changes state in a 7 hour period… then the template never calculates. Conversely, if I get 15 state changes in 1 second, then you get a template update at the entrance and exit of that one second.

If you add now() to the example above, you’ll have a 1 second rate limit but you’ll get updates every 60 seconds if no update occurred over that duration.

123 · October 21, 2020, 3:24am

Thanks that helped to clear the cobwebs. Not a polling interval but a rate-limit.

If using states then no more than one update per minute.
If using states.DOMAIN, no more than once per second.
If now() is combined with states the maximum allowed is once per minute with once per minute assured.
If now() is combined with states.DOMAIN the maximum allowed is once per second with once per minute assured.

Did I get that right?

EDIT

Nope, I got the last two wrong. Corrected them to reflect the fact that the use of states ensures rate-limiting of per minute and the use of states.DOMAIN ensures a rate-limiit of per second.

tom_l · October 21, 2020, 6:30am

So what if I only what one update a day at midnight?

Mariusthvdb · October 21, 2020, 6:58am

then you should forget you want that: https://github.com/home-assistant/architecture/issues/450
because the dev’s say you don’t need that.

this home automation system now has developed to a system without user control. How s that for progress.

tom_l · October 21, 2020, 7:35am

This was a stupid post. I completely misinterpreted the issue.

amelchio · October 21, 2020, 8:06am

I was guessing at what you meant … states will cause per second evaluations is not quite right. It is not causing evaluations but it will activate the rate limit. And it’s not per second, it’s per minute.

I certainly did not say that no one will need to understand it. It may not apply to most people in this thread but people in this thread are a minority.

Also, “want to understand” is different from “need to understand”. Though some people will want to understand why one sensor is faster than another, I am quite sure that a lot of people will just shrug. A lot of integrations are polling in nature and take a little while to update.

You swapped “per second” and “per minute” in the last two bullets, otherwise it is correct.

There are now three exceptions to the general rule that templates update when referenced entities change state:

If using states it never updates more than once per minute
If using states.DOMAIN it never updates more than once per second (0.117)
If using now() it always updates at least once per minute (0.117)

amelchio · October 21, 2020, 8:10am

If the sensor is date related you may be able to use sensor.date rather than now(). For example, this example from Marius:

          {% set trigger = states('sensor.date') %}
          {% set this = now().replace(hour=0).replace(minute=0).replace(second=0).replace(microsecond=0) %}
          {% ... %}

can be changed like this:

          {% set this = strptime(states('sensor.date'), '%Y-%m-%d') %}
          {% ... %}

and it will only evaluate at midnight.

Mariusthvdb · October 21, 2020, 8:14am

the above seems to be very close to what would be a ‘general’ approach for the greater majority of simple operations and templates. So that’s good.

what I see missing are solutions for (call them power-)users that need:

updating on each state change, when using states
manual updating (or on demand), simply because they want 100% control of things

Could we at least try and go forward finding these solutions?
btw, let me also state here that I very much appreciate your last contribution on the architecture topic. I’ve replied there too, with a likewise cry of the heart.

anon43302295 · October 21, 2020, 8:21am

I’m still confused by all of this tbh, and the main issue I’m having is with this one…

sensor:
  - platform: template
    sensors:
      unavailable_entities:
        friendly_name: Unavailable Entities
        value_template: "{{states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
            |rejectattr('entity_id', 'in', state_attr('group.entity_blacklist', 'entity_id'))
            |rejectattr('entity_id', 'eq' , 'group.entity_blacklist')
            |map(attribute='entity_id')|list|length }}"
        attribute_templates:
          entities: "{{states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
              |rejectattr('entity_id', 'in', state_attr('group.entity_blacklist', 'entity_id'))
              |rejectattr('entity_id', 'eq' , 'group.entity_blacklist')
              |map(attribute='entity_id')|list|join(', ') }}"
        unit_of_measurement: items

What I want is for this to react to every state change in my system, set the sensors state to the number of unavailable entities, and set the attribute ‘entities’ to a comma separated list of the entity_ids that are unavailable.

If I must compromise on update frequency, then I would much prefer once every second. Currently it is once per minute.

I would rather not use an automation to do this as the current logic of my system is to wait until all entities are available before turning on my automations, and I would prefer not to make an exception just for this sensor (for OCD reasons if nothing else).

What would be the best way to adapt this sensor to update every second, or even better update with every state change as desired please?

(I clicked the reply arrow on petro’s post but this is meant for everyone, not a specific reply to him, sorry!)

Mariusthvdb · October 21, 2020, 8:24am

you could use the python script I posted and use an automation to trigger on event state_changed.

As far as I understand it all now, that wouldn’t only be the best, but also the only way… you might not want it, but following Amelchios schedule, there’s no other way of using states, and have factual updating

hence my fist bullet in the reply to that, was especially aimed at your needs

tom_l · October 21, 2020, 8:27am

Oh ignore me. I’m already doing it that way. I got confused. I thought that was being depreciated.

anon43302295 · October 21, 2020, 8:32am

Appreciate the suggestion, although I fear I’ve missed the python_script you’re talking about somewhere in the haze of all of this. Let’s call that a fallback option for now and see if anyone can think of a way to just adapt the template to fulfil the requirements

Mariusthvdb · October 21, 2020, 8:33am

but what about the timezone difference:

Mariusthvdb · October 21, 2020, 8:36am

+1!

btw here’s the link:

amelchio · October 21, 2020, 8:52am

There is no current way to fix the sensor, an automation is the only way to get what you want.

(For completeness, here is an example I posted earlier: Heads up! Upcoming breaking change in the Template integration - #434 by amelchio)

Mariusthvdb · October 21, 2020, 9:05am

this makes it update every 2 seconds:

  - alias: Trigger per 2 seconds
    id: Trigger per 2 seconds
    trigger:
      platform: time_pattern
      seconds: /2
    action:
      delay:
        seconds: 0

and this in place of now()

{{strptime(as_local(state_attr('automation.trigger_per_2_seconds','last_triggered')), '%Y-%m-%d') }}
{{state_attr('automation.trigger_per_2_seconds','last_triggered')}}

on a specially made automation. I guess we could also use an automatic state_change automation to trigger these per state change.

or will this ‘hack’ also be carved out?

edit

just stumbled on this quote, used for the introduction of the observer plugin in Supervisor 249:

says it all …

Burningstone · October 21, 2020, 9:07am

The same minority that is most active and provides a large part of the help here in the forums. Just saying.

anon43302295 · October 21, 2020, 9:35am

Thanks

Yeah, I think that automation is going to be my preference then (I was holding off moving to it whilst I was watching all the changes and then got confused), although the condition only looks for ‘new_state’, can I presume that the opposite of that is ‘old_state’ so that it also updates when the entities come back online?

I’ll implement this at the weekend.

petro · October 21, 2020, 11:47am

Do it with an automation, I know you don’t want to but it’s pretty easy.

- alias: Unavailable Group Updates
  trigger:
    platform: event
    event_type: state_changed
  action:
    service: group.set
    data:
      object_id: "unavailable_entities"
      entities: >
        {{states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
                |rejectattr('entity_id', 'in', state_attr('group.entity_blacklist', 'entity_id'))
                |rejectattr('entity_id', 'eq' , 'group.entity_blacklist')
                |map(attribute='entity_id')|list|join(',') }}

then your template sensor

sensor:
  - platform: template
    sensors:
      unavailable_entities:
        friendly_name: Unavailable Entities
        value_template: "{{expand('group.unavailable_entities) | count}}"
        attribute_templates:
          entities: "{{expand('group.unavailable_entities) | map(attribute='entity_id') | list | join(', ')}}"
        unit_of_measurement: items

But you probably don’t need the list in the sensor because you now have the group with the list…

sensor:
  - platform: template
    sensors:
      unavailable_entities:
        friendly_name: Unavailable Entities
        value_template: "{{expand('group.unavailable_entities) | count}}"
        unit_of_measurement: items