Heads up! Upcoming breaking change in the Template integration

[anon43302295]

@petro

It’s a throttle. When this PR is accepted, you can have at most 1 update per second when using states.

So now I can’t even predict when my template will update?

Might be every state change, might be every second, might be every 10 seconds, every half minute, every minute…

This is completely brain boggling.

[petro]

It’s updated when a state change occurs but throttled to at most once per second. So you’d rather have it only update on a period and not update on state changes? If that’s the case you’d have delays in your responses…

[Mariusthvdb (Marius)]

seems to be some confusion goin on indeed.

The current PR makes the templates update per second when using {{states.DOMAIN|xxx}}, and per minute when using {{states|xxx}}

[123 (Taras)]

@Mariusthvdb

update per second when using {{states.DOMAIN|xxx}} , and per minute when using {{states|xxx}}

…

Never mind; read it wrong the first time …

[anon43302295]

@petro

So you’d rather have it only update on a period and not update on state changes?

No, I would rather it updated on every state change without unpredictable throttling, I didn’t have any performance issues with that approach and it worked really well for me

I know lots of people did have performance issues so something needs to be done to accommodate everyone, I’m just saying that I’m getting a bit lost with all the changes in this particular area.

[petro]

It’s not unpredictable. Here’s some diagrams to help you predict them:

Using states.domain

Using states

Using anything with a predefined list, like expand, states(), state_attr(), etc.

If you use both states and states.domain in the same template, states rate_limit applies.

[123 (Taras)]

It’s not a bad compromise:

1. If the template contains states it behaves like in the “old days” when we used entity_id: sensor.time to make the template evaluate every minute.

2. If the template contains states.DOMAIN then the template is evaluated more frequently (every second).

Frankly, I’m fine with #1 but I’m less certain about #2. Polling at 1-second intervals hardly seems like ‘throttling’, unless you have a boatload of sensors that normally report at sub-second intervals. Anyway, 0.117 will be an interesting release.

[petro]

@123

Polling at 1-second intervals hardly seems like ‘throttling’, unless you have a boatload of sensors that normally report at sub-second intervals.

The throttle was put in because some people were using states and had upwards of 3 million updates a day, that’s somewhere around 35 updates a second.

[123 (Taras)]

That would definitely qualify as “boatload of sensors that normally report at sub-second intervals”

[Mariusthvdb (Marius)]

@123

sensors that normally report at sub-second intervals

How is this even possible to begin with? I’ve always learned the HA state machine only updates each second?

Secondly, since the most entities per domain will be in the sensor domain most likely, which is where most of the issues are from, I hope this now lowered threshold will not re-introduce the issues of 115. Or, was the main cause for that the use of {{states|xxx}} after all

@123

Anyway, 0.117 will be an interesting release.

+1

[123 (Taras)]

@Mariusthvdb

How is this even possible to begin with?

I may be wrong but I have a vague recollection of a recent release note indicating they reduced the duration of the system’s clock-tick (i.e. it’s now less than 1 second). As a result, you can now specify float values like 2.5 for time intervals (i.e. it now supports milliseconds).

---

EDIT

0.113 Automations: Scripts sub-second precision

… scroll down past the description of the new milliseconds option and you will find this:

Also, all time listeners now have microsecond precision as they are scheduled on the internal event loop, instead of the previous situation when it relied on the internal clock that triggered every second.

So it looks like I misspoke and the clock-tick is the same (1-second) but things now reference some other internal time standard to get microsecond resolution.

[Mariusthvdb (Marius)]

right, the internal event loop. Must be the same as what I now use here, upon Amelchio’s advice:

      - delay: >
          {% set delay = states('input_number.snooze_delay')|int * 60 %}
          {% set spent = (now() - state_attr('script.play_wakeup_radio','last_triggered')).total_seconds() %}
          {{ [0, delay-spent] | max }}

I was struggling to get an updating template (which this isn’t since it is using now() ) but this still works, because it is not ‘updated’ but uses the ‘internal timer’.

[Mariusthvdb (Marius)]

just stumbled on another thing I cant find the use for yet:
in my now commented automations using

    trigger:
      platform: event
      event_type: state_changed

if found some earlier experiments with the expand() functionality compared to the ‘old way’ of writing that:

{{trigger.event.data.entity_id in state_attr('group.iungo_switch_switches','entity_id')}}

compared to

{{trigger.event.data.entity_id in (expand('group.iungo_switch_switches')|map(attribute='entity_id'))}}

note the second line not only is longer… but it listens to way more states, at least entered in the template editor. As we now know, it listens to all states it finds in that group, while, entered as is, the top one only listens to the group. And still, that lists all the identical entities, and works as expected in the automation, checking if the event.data.entity_id is in the list.

Now, which of the 2 would be more efficient? For this automation, it wouldn’t even have to listen to the states, it would merely need to check if its in the list, no matter the state it has.

[petro]

@Mariusthvdb

trigger.event.data.entity_id

@Mariusthvdb

but it listens to way more states,

You’re in an automation, those rules don’t apply.

@Mariusthvdb

Now, which of the 2 would be more efficient?

the generator is more efficient, it also recursively searches your group for groups in groups.

[amelchio]

In related news, the upcoming 0.117 should have template sensors using now() update automatically at least once per minute: https://github.com/home-assistant/core/pull/41147

This is unfortunately another confusing change for those that just moved to {% set x = states('sensor.time') %} but I think it is a great simplification going forward. Now all templates will mostly “just work” and one only has to understand the nitty-gritty of the template engine in quite specialized situations (like those situations being discussed in this thread).

[Mariusthvdb (Marius)]

well that’s great for the

{% set x = states('sensor.time') %}

of course. so thanks for that. But what about all sensors/templates that would only need

{% set x = states('sensor.date') %}

will the now be updated 1439 times too many a day? Or can we still set that in the template, which would be very nice indeed.

I now see I missed Bdraco 's request to open an architecture issue, consider it done https://github.com/home-assistant/architecture/issues/450

[123 (Taras)]

I commented in your Architectural thread.

You’re right, we still won’t have the ability to restrict a Template Sensor’s evaluation to just once a day. If the sensor needs to reset its calculation at the start of every new day (like what we were able to do using sensor.date), that ability is lost.

Suggestions to use an automation with a Time Trigger (to force evaluation of the Template Sensor on a scheduled basis) overlook the fact the sensor’s template will continue to evaluated according to the new rules and not exclusively by the Time Trigger. In other words, it doesn’t matter if the Time Trigger is set to evaluate the Template Sensor at midnight, whatever entities present within the sensor’s template continue to cause evaluations, throughout the day, whenever they change state.

The new way now() will be handled is a welcome improvement. It’s a convenient replacement for including this in the template:
{% set x = states('sensor.time') %}
In fact, users with templates having both now() and the above will probably want to remove it to prevent doubling the number of evaluations per minute. In other words, the workaround suggested by this (now very long) thread will become redundant in 0.117.

To be clear, even with now() becoming a first-class citizen, we still won’t have the degree of control we had when entity_id was available. We won’t be able to constrain a template to be evaluated only once per minute (state-changes of other entities in the template will also cause evaluations) nor just once per day at the start of each new day. in addition, there’s no easy way to re-establish that behavior (short of creating a python_script).

[tom_l]

Well that’s not good news.

[Mariusthvdb (Marius)]

Yes, thanks for that. I’ve also just responded to Paulus and Frenck .

Somehow I get the feeling I simply can not get across what is the loss of the current situation, and the only thing that gets focus is the gain ( which is admittedly there).

[123 (Taras)]

@Mariusthvdb

I simply can not get across what is the loss of the current situation

FWIW, they are not ignorant people; they do get your point. However, they appear to be purposely ignoring/dismissing it.

The pattern has been:

There's no problem.

If there is a problem, it's not that bad.

The problem affects very few people, so it's not a big deal.

The people who have the problem should work around it.

This isn't the place to discuss it (topic closed).