Heads up! Upcoming breaking change in the Template integration

Yes, sensor.time is among them.

Which makes me think: if what you say is correct, (not doubting you, it’s just I hadn’t realized that before) wouldn’t that require one to list all entity_id’s relevant to the template if sensor.time is listed?

If not it would only update once a minute?

Also, there is no change in behavior between either macro, so what would be the benefit using one or the other?

Yes (10 charachter minimum)

Edit: listing them tells it to “ONLY look at this list”

just as a fyi, Ive added the new macro in an _alt sensor, to be able to see if any difference exists: none… they both update exactly the same.

what’s more: this excercise has made clear that all sensors need both the triggering timestamp sensor, And sensor.time to update correctly. This can be easily established because the sensors in the screenshot show how long it will take for the next alarm to take place.
They don’t update without sensor.time, inspite of sensor.time being listed as trigging entity_id on sensor.next_alarm_timestamp, which has all needed entity_id’s and sensor.time listed…

This is obvious of course…sensor.time isn’t helpful on the timestamp sensor, because that is, well, a timestamp, which doesn’t change when time passes… all other sensors do need that time to shift…

leaving me with the question about the 2 macros: why the one or the other?

It looks small, but it feels “big”… I hope as_local() gets mentioned in the release notes so others outside this thread find it quickly (maybe a direct link to the example on the binary_sensor page you are adding).

Or maybe better yet, thread had lots of good info - perhaps release notes should just link to it?

You should be able to do a quick and dirty test using : -

Which will substitute until something better comes along

I just noticed this thread linked from the release notes and skimming through it, I didn’t really see this question answered. Forgive me if I missed it, it’s already a long thread.

A template like this (which I use to turn my amplifier on/off):

{{ is_state("media_player.samsungtv", "on") or is_state("media_player.apple_tv", "playing") }}

used to be recalculated whenever one of the two entities changed. However, if the TV is on, we know the template will always be true so updating the template when the Apple TV is (de)paused is a waste of effort. The Apple TV state cannot affect the result when the TV is on! With 0.115, the template is thus only recalculated when the TV changes state or if it is already off. The current state of the system is used to deduce which parts of the template can currently be ignored.

That is just a performance improvement (less template recalculations) but if we change it like this:

{{ is_state("media_player.samsungtv", "on") or is_state(state("input_select.media"), "playing") }}

the performance improvement becomes significant. We now watch the TV state, the media selector and the currently selected media player. The old way would always watch all possible media players.

Furthermore, previously you had to carefully list all possible input_select.media values as entity_id: on any template that used input_select.media as input like above. This is now completely automatic, a huge improvement in usability and much less error-prone.

Moving on, we can change the template like this:

{{ is_state("media_player.samsungtv", "on") or is_state(state("input_text.media"), "playing") }}

and we now have something that was impossible to do before. An input_text can contain any value so you cannot possibly list them all as entity_id: in advance.

So that’s the reasoning for the change. Everything is now faster, easier, smaller and it always works.

The downside is that we need to rework templates that rely on time to use a new workaround for periodic recalculation – which is what this thread was about.

(Another arguable downside is that interdependent templates will have to be untangled to not cause a CPU spike due to an essentially infinite loop. In essence, if template a is b+1 and template b is a+1 each template will now continuously update forever. This is actually correct but probably not very useful so I think code will be added to detect this and abort after a certain depth. This could not all be in place for the first release because we are still learning about such situations as people report templates that they use.)

4 Likes

Thank you for the explanation. It amazes me that this can be done at all, let alone improved.

So I’ve upgraded to 115 and I have a template sensor that calculates the difference between two dates. The way I achieved this previously was with the following:

platform: template
sensors:
  softener_salt_last_replaced:
    value_template: '{{ ((as_timestamp(now())-(states.input_datetime.softener_salt.attributes.timestamp)) | int /60/1440) | round(0) }}'
    unit_of_measurement: 'Days'
    entity_id: input_datetime.softener_salt, sensor.date
    friendly_name: 'Softener Salt'

So (if my understanding is correct) the above would only have updated when either of the listed entities changed state.
As I used as_timestamp(now(), which won’t auto-update in 115, and with the loss of listing the entity ids I have to find another way of doing this. Obviously the easy fix would be to change to something like the below (as already suggested in this thread) with the addition of reliability on sensor.date:

platform: template
sensors:
  softener_salt_last_replaced:
    value_template: >-
      {% set zilch = states('sensor.date') %}
      {{ ((as_timestamp(now())-(states.input_datetime.softener_salt.attributes.timestamp)) | int /60/1440) | round(0) }}
    unit_of_measurement: 'Days'
    friendly_name: 'Softener Salt'

However, as my overall intention was to calculate the difference between two dates I thought I could build sensor.date into the actual template like so:

platform: template
sensors:
  softener_salt_last_replaced:
    value_template: >-
      {{((as_timestamp(strptime(states('sensor.date'), '%d.%m.%Y'))-state_attr('input_datetime.softener_salt', 'timestamp')) /60/1440) | int}}      
    unit_of_measurement: 'Days'
    friendly_name: 'Softener Salt'

Is there a better way of doing the above? Or is it overkill? Would the addition of {% set zilch = states('sensor.date') %} be enough?

That is fine. Use what you like best, both will work equally well (and so will other variations).

more than that, would we need to set it only once in the value_template, to have it also trigger the other defined templates of the same template sensor? Ive asked in the release post here

or do we need to set that now in all individual sensors (since they are that, individual…)

eg:

      hour_icon:
        friendly_name: Hour icon
        entity_id: sensor.time
        value_template: >
          {% set word = ['skipthis','one','two','three','four','five','six','seven',
                         'eight','nine','ten','eleven','twelve','one'] %}
          {%- set hour = now().strftime('%-I')|int %}
          {%- set minute = now().strftime('%-M')|int %}
          {%- set index = hour if minute <= 30 else hour + 1 %}mdi:clock-time-{{word[index]}}
            {{- '-outline' if states('sun.sun') != 'above_horizon' }}

        icon_template: >
          {% set word = ['skipthis','one','two','three','four','five','six','seven',
                         'eight','nine','ten','eleven','twelve','one'] %}
          {%- set hour = now().strftime('%-I')|int %}
          {%- set minute = now().strftime('%-M')|int %}
          {%- set index = hour if minute <= 30 else hour + 1 %}mdi:clock-time-{{word[index]}}
            {{- '-outline' if states('sun.sun') != 'above_horizon' }}

        attribute_templates:
          Hour: >
            {{now().strftime('%-I')}}
          Minutes: >
            {{now().strftime('%-M')}}
          Addition: >
            {{0 if now().strftime('%-M')|int <= 30 else 1}}

would need the {% set x = states('sensor.time') %} in all templates…?

would this also go for a template like:

      entities_domains_counter:
#        entity_id:
#          - sensor.date
#          - sensor.count_persistent_notifications
        friendly_name: Entities per domain
        value_template: >
          {{states|count}}
        attribute_templates:
          headline: >
            {{states|count}} entities in {{states|groupby('domain')|count}} domains
          alert: >
            {{states.alert|count}}
          automation: >
            {{states.automation|count}}
          binary_sensor: >
            {{states.binary_sensor|count}}
          camera: >
            {{states.camera|count}}
          climate: >
            {{states.climate|count}}
          counter: >
            {{states.counter|count}}
# continue to list all domains

which, by adding entity_id: sensor.day was only re-evaluated once a day. Doesnt this trigger now on each and every state change?

Fair enough, I just wanted to make sure I wasn’t going about it in an overly complex way :slight_smile: Thank you!

You are right that {{states|count}} will trigger on each state change. For this simple example that is obviously redundant but for a slightly more complex example:

{{ states | map(attribute="last_updated") | max }}

we can see that each state change indeed must be considered when state is accessed. Incidentally, it is jaw-dropping to put this example into the template developer tool and see it update in real-time!

If this is actually a performance issue, it might be possible to make a special optimization for {{states|count}} where only life-cycle events are considered. I am not quite sure whether that is possible (or even desired), @bdraco could probably answer that. A workaround is to put the template into an input_number and update it manually as desired.

If you want to drive updates manually, automations are able to do that with much more flexibility than the templates were ever able to do. For example, your hour_icon value template seems to only need changes every hour and you can do that with a time_pattern which would make it 60x more efficient!

Please note, I am not saying that the current implementation is perfect. We will probably have to tweak it for a few releases. But the concerns I have seen are about things that can still be done so I think this will turn out fine once we get comfortable with the new ways

HI Amelchio,

thanks for replying. I admire the hugeness of the 115 update. And thank everyone for that. Its just that the deprecation of this entity_id in templates seems to have been rushed in, and not all consequences are thought out carefully enough.
I am not sure what you’re saying here is acknowledging the issues introduced by deprecating entity_id:

I do understand for a majority of template sensors, this is very nice to see happening, dynamic and constant updating of relevant entity_id’s even when not explicitly mentioned in the template.

However, suggesting even more convoluted solutions like creating timed automations for updating input_numbers or other domains which hold the current state of those templates is quite baffling tbh.

Especially since we had a perfect trigger/limiter for those updates!

Wouldn’t it be a viable scenario that the new way of evaluating the templates simply co-exist with the functionality of entity_id:

That way, we would have it all:

  • Automatic and dynamic template evaluation for entities in the templates
  • option for a global entity to be valid for the complete template sensor that limits updating to changes of that global entity (or entities in a list)

I don’t want to update manually, I simply don’t want to update on all state changes, which, now they are found automatically, drastically increase in number…

(btw my hour icon needs updating every minute, hence the sensor.time)

1 Like

In fairness to the changes, all of my concerns were because I didn’t realise how far reaching the changes went. I’ve removed entity_id from all of mine and patiently waited to see which broke and not a single one did :slightly_smiling_face:

It’s no secret that your configuration is particularly complicated, which of course is your perogative, but I suspect if you went through all of your templates you can probably define a lot of them in different ways that work more effectively and wouldn’t require any workarounds, then the few that are left may just need the ‘cheat’ defined above.

The downside for you, with which I sympathise, is that because your setup is complicated, going through it to achieve this is probably a lot of work.

2 Likes

The change is awesome tbh. One of the most annoying support issues (as you know) was explaining the updates on template sensors

well yeah, I am writing a few automations to get the next_alarm sensor up to speed with the new entity_id issue… so Im am conforming. no worries.

the counter template sensor above was not mine tbh, (I use a python script to do the works) but was posted because quite a few members here use that. If so, it is to be expected that they all see a rise in processor spikes.
What I am trying to say, is that I am not really speaking for myself, but for the community as a whole. Seeing the demise of a perfectly fine technique for limiting the updating of a sensor, and a global one at that (you must sympathize with that) feels just so unnecessary.

To be even more precise in describing my worries: it is not that I see these sensors break because they can’t find an entity_id any longer.
It is more of the opposite: I fear they will keep firing because they are constantly checking the state they are finding…

the few ones that need sensor.time are easily spotted and rewritten. It’s a more fundamental worry than that…

in fact it could be compared to the move custom button-card made some time ago. From updating on all state changes, to entering the triggers_update: configuration option. Don’t use that, and have the system update on all state changes mentioned. Do use it and have update only on that particular (set of) entity_id(s).

Best of both worlds. Would be great to have I core templates too…

We might be able put together an optimization for {{states|count}} , I’ll take a look later today.

I’ve helped a few people convert their templates, and by far the most common cases don’t actually need to iterate all states which is the better solution.

I’ve seen a lot of these

states | selectattr('entity_id', 'in', state_attr('group.family','entity_id'))

which can be replaced with:

expand('group.family')

Also for those who are struggling to find why an update happens, we have additional logging options coming to help track these down: https://github.com/home-assistant/core/pull/40180

Processing templates with states in them is already significantly faster in 0.115 after https://github.com/home-assistant/core/pull/39731

We have a change coming in 0.116 that will make iterating states twice as fast when you have a large number of states in the system https://github.com/home-assistant/core/pull/40012

I’ll see if we can make iterating states faster yet.

4 Likes

to understand fully: does this in fact change the burden on the backend, or is it simply simpler in the yaml config.
iow, does it limit the states checked for the evaluation of the template value?

I’m not 100% sure what you are asking so I’ll try to respond and hope I’ve hit your question.

The template developer tools will show you which state changes the template will listen for now:

3 Likes