Heads up! Upcoming breaking change in the Template integration

No, I would rather it updated on every state change without unpredictable throttling, I didn’t have any performance issues with that approach and it worked really well for me :man_shrugging:

I know lots of people did have performance issues so something needs to be done to accommodate everyone, I’m just saying that I’m getting a bit lost with all the changes in this particular area.

It’s not unpredictable. Here’s some diagrams to help you predict them:

Using states.domain

Using states

Using anything with a predefined list, like expand, states(), state_attr(), etc.

If you use both states and states.domain in the same template, states rate_limit applies.

It’s not a bad compromise:

  1. If the template contains states it behaves like in the “old days” when we used entity_id: sensor.time to make the template evaluate every minute.

  2. If the template contains states.DOMAIN then the template is evaluated more frequently (every second).

Frankly, I’m fine with #1 but I’m less certain about #2. Polling at 1-second intervals hardly seems like ‘throttling’, unless you have a boatload of sensors that normally report at sub-second intervals. Anyway, 0.117 will be an interesting release.

The throttle was put in because some people were using states and had upwards of 3 million updates a day, that’s somewhere around 35 updates a second.

That would definitely qualify as “boatload of sensors that normally report at sub-second intervals”

How is this even possible to begin with? I’ve always learned the HA state machine only updates each second?

Secondly, since the most entities per domain will be in the sensor domain most likely, which is where most of the issues are from, I hope this now lowered threshold will not re-introduce the issues of 115. Or, was the main cause for that the use of {{states|xxx}} after all

+1

I may be wrong but I have a vague recollection of a recent release note indicating they reduced the duration of the system’s clock-tick (i.e. it’s now less than 1 second). As a result, you can now specify float values like 2.5 for time intervals (i.e. it now supports milliseconds).


EDIT

0.113 Automations: Scripts sub-second precision

… scroll down past the description of the new milliseconds option and you will find this:

Also, all time listeners now have microsecond precision as they are scheduled on the internal event loop, instead of the previous situation when it relied on the internal clock that triggered every second.

So it looks like I misspoke and the clock-tick is the same (1-second) but things now reference some other internal time standard to get microsecond resolution.

right, the internal event loop. Must be the same as what I now use here, upon Amelchio’s advice:

      - delay: >
          {% set delay = states('input_number.snooze_delay')|int * 60 %}
          {% set spent = (now() - state_attr('script.play_wakeup_radio','last_triggered')).total_seconds() %}
          {{ [0, delay-spent] | max }}

I was struggling to get an updating template (which this isn’t since it is using now() ) but this still works, because it is not ‘updated’ but uses the ‘internal timer’.

just stumbled on another thing I cant find the use for yet:
in my now commented automations using

    trigger:
      platform: event
      event_type: state_changed

if found some earlier experiments with the expand() functionality compared to the ‘old way’ of writing that:

{{trigger.event.data.entity_id in state_attr('group.iungo_switch_switches','entity_id')}}

compared to

{{trigger.event.data.entity_id in (expand('group.iungo_switch_switches')|map(attribute='entity_id'))}}

note the second line not only is longer… but it listens to way more states, at least entered in the template editor. As we now know, it listens to all states it finds in that group, while, entered as is, the top one only listens to the group. And still, that lists all the identical entities, and works as expected in the automation, checking if the event.data.entity_id is in the list.

Now, which of the 2 would be more efficient? For this automation, it wouldn’t even have to listen to the states, it would merely need to check if its in the list, no matter the state it has.

You’re in an automation, those rules don’t apply.

the generator is more efficient, it also recursively searches your group for groups in groups.

In related news, the upcoming 0.117 should have template sensors using now() update automatically at least once per minute: https://github.com/home-assistant/core/pull/41147

This is unfortunately another confusing change for those that just moved to {% set x = states('sensor.time') %} but I think it is a great simplification going forward. Now all templates will mostly “just work” and one only has to understand the nitty-gritty of the template engine in quite specialized situations (like those situations being discussed in this thread).

1 Like

well that’s great for the

{% set x = states('sensor.time') %}

of course. so thanks for that. But what about all sensors/templates that would only need

{% set x = states('sensor.date') %}

will the now be updated 1439 times too many a day? Or can we still set that in the template, which would be very nice indeed.

I now see I missed Bdraco 's request to open an architecture issue, consider it done https://github.com/home-assistant/architecture/issues/450

I commented in your Architectural thread.

You’re right, we still won’t have the ability to restrict a Template Sensor’s evaluation to just once a day. If the sensor needs to reset its calculation at the start of every new day (like what we were able to do using sensor.date), that ability is lost.

Suggestions to use an automation with a Time Trigger (to force evaluation of the Template Sensor on a scheduled basis) overlook the fact the sensor’s template will continue to evaluated according to the new rules and not exclusively by the Time Trigger. In other words, it doesn’t matter if the Time Trigger is set to evaluate the Template Sensor at midnight, whatever entities present within the sensor’s template continue to cause evaluations, throughout the day, whenever they change state.

The new way now() will be handled is a welcome improvement. It’s a convenient replacement for including this in the template:
{% set x = states('sensor.time') %}
In fact, users with templates having both now() and the above will probably want to remove it to prevent doubling the number of evaluations per minute. In other words, the workaround suggested by this (now very long) thread will become redundant in 0.117.

To be clear, even with now() becoming a first-class citizen, we still won’t have the degree of control we had when entity_id was available. We won’t be able to constrain a template to be evaluated only once per minute (state-changes of other entities in the template will also cause evaluations) nor just once per day at the start of each new day. in addition, there’s no easy way to re-establish that behavior (short of creating a python_script).

Well that’s not good news.

Yes, thanks for that. I’ve also just responded to Paulus and Frenck .

Somehow I get the feeling I simply can not get across what is the loss of the current situation, and the only thing that gets focus is the gain ( which is admittedly there).

FWIW, they are not ignorant people; they do get your point. However, they appear to be purposely ignoring/dismissing it.

The pattern has been:

There's no problem.

If there is a problem, it's not that bad.

The problem affects very few people, so it's not a big deal.

The people who have the problem should work around it.

This isn't the place to discuss it (topic closed).

:raised_hand:

5 Likes

well, yeah, case is closed just now. ‘lacking concrete examples’

this is completely incomprehensible, and must be so discouraging, even for the dev that is exhausting himself trying to find options to reduce the amount of unnecessary updates as much as possible.

if decreasing with a ratio of 1439 to 1 isn’t sufficient enough, and pushed aside like this, I guess there’s no reason to try and discuss this any longer.

feels like being asked for proof of getting wet when stepping into the rain.

2 Likes

This should not be an issue; the now()-based update is more of a watchdog so it only kicks in if nothing else is updating the template for a minute. We actually set the delay to 60.5 seconds to handle this case without double updates.

The behavior you described plays directly to the point made by mf_social where it’s becoming more difficult (not less) to understand what influences a template’s evaluation.

The documentation will need to clarify the order of precedence. For example, if the template contains now() and states my best guess is that states will cause per second evaluations and now() plays no part (not even once a minute). Is that correct?

I agree it is no longer simple to understand. The good news is that it is simple to use so a lot less people will have to understand it now.

There is no conflict, one is a throttle (acting on a flood) and the other is a watchdog (acting on idle).

Your guess sounds right but if you have a very quiet system with more than one minute between any state change, now() would kick in.

Again, one should usually not have to think about this. It will always update within a minute.

1 Like