Heads up! Upcoming breaking change in the Template integration

It’s implicit : -

  1. the new method sets states to observe based on the template itself
  2. if you have an entity_id: included - don’t do 1. just look at “these” instead

That’s two methods

Edit:
and examples : -
new method; check if light.21 os ‘on’, and light.23 is ‘off’ and the brightness of light.17 (3 things)
old method: check if light.21 has changed state OR ANY of it’s attributes (thats about 25 things straight away depending upon the integration)

Scale up to some of the complex templates we have been discussing

The new method presupposes there’s no longer a need for entity_id because it’s able to identify all entities far more effectively than the old technique. No question, it is better at that.

What it overlooks is that there was another purpose to entity_id. It wasn’t simply a crutch for the old entity-identification technique. It could also be used to constrain the entities that should be monitored.

In previous versions, I could create a complex template involving a boatload of entities, but by adding entity_id: sensor.whatever only one listener is assigned to sensor.whatever and no others (for the boatload of entities in the template).

That capability is gone in 0.115. Now it will assign listeners to each entity in that boatload. There’s no way to constrain it.

The suggested workaround is “don’t use a Template Sensor”; re-write it as an automation or a python_script. By definition, that’s a loss of functionality and probably unforeseen when the decision was made to deprecate entity_id.

3 Likes

Agreed (so don’t edit your post any further :crazy_face: )

But getting to Point “C” from Point “A” is no longer possible as we are currently counting the daisies from Point “B”

The ONLY way this will improve is if they optimise the current method (and entity_id: is not on the running list, it’s too confusing with enity_id: as used elsewhere, as they performed two completely different tasks) Trust in bdraco, it will get there.
Possible options will be : -

  • Constrain ALL templates to resolve just once in any second
  • allow a new (optional) parameter called (say) period: which takes seconds 1 to (say) 86400 (not 0 as that will allow sub-second updates)
  • allow it to run a few times, see that “this” template is absorbing too much processor time and throttle updates accordingly (i.e. it becomes self regulating)

I don’t know, I’m not bdraco and by god he’s done well enough in these last 6 months to merit some trust in this issues resolution

None listed are as flexible as simply re-instating entity_id.

I’m confused why you think it’s confusing. :slightly_smiling_face:

entity_id would be handled the way it has always been handled.

  • If entity_id not present, inspect the template and assign listeners to all identifiable entities.

  • If entity_id is present, don’t inspect the template and simply assign listeners to the entities listed in entity_id.

It’s not confusing to me, I grew up with HA using this key for two different purposes and never even considered that they were different until they got rid of it from template sensors and I had to explain it to a newbie who had a template which he’d obviously just been given with no understanding.
Automation Action / Script - Entity_ID : These are the list of items you will be acting upon
Sensor - Entity_ID: Look at THESE items (both states and attributes) and if ANY of them changes, update the current sensor
They are different purposes and newbies will have trouble putting it into context.
And the major push seems to be for the dumbing down of HA … and I can sort of see their point … that doesn’t stop me from being a hypocrite and whinging about certain “feature enhancements” (the latest being device triggers) AND I also had all my sensors ticking along nicely, I’ve chucked out entity_id:'s (that was a lot of total text to dump, so smaller files :+1: ) and they still do AND my processor use has fallen (I seem to be an exception I know, but look at the efficiency difference elicited by looking at specific states (and attributes) rather that EVERYTHING that this entity_id represents)

BUT that would require that the ‘old method’ of watching for changes be reinstated - hence two methods
UNLESS you are proposing something like : -

  • entiy_id: light.23.brightness, light.23.rgb.red_value

but that also is a change to the way the new method works, and can you see newbies getting their heads round that ?

In short the new method looks at specific states (as defined in the template)
The ‘old method’ looked at ANYTHING that changed in this list of entities (AND i also get that you are only thinking about things like sensor.time and sensor.date (maybe we should have a new sensor.hour ) - but it’s still different ! )

The meaning of the term entity_id is no different here compared to where it’s used in other places. It means “refer to these entities”. The fact you didn’t have to include it in a Template Sensor is what made Template Sensors differ from others (therefore somewhat anomalous).

The presence of entity_id simply signals the new entity-identification technique to “look here for entities” as opposed to looking for entities in value_template. Listeners are assigned to the entities in entity_id as opposed to inference/extraction from value_template.

But what ‘state’ are they listening for ???

New listeners are assigned to states (loose term covering specific attributes as dictated) not entities

Wut?

Listeners are created to monitor entities for state-changes.

If an entity’s state value changes, or the value of one of its attributes, the listener reports it and that’s cause for the template to be evaluated.

NOTE:
The one thing I have not confirmed in 0.115 is if the listener does actually monitor an entity’s attributes for state-changes.

No,
If I create a template.sensor that looks at say the battery level of my wife’s phone
Her moving / chaning zone, will not update the template.senor UNTIL the attribute of battery changes.
That’s what is bringing the increased performance

It does, and that’s why it will itterate through a template (to determine ‘day’ (say) ) and with the slug created with that ‘day’ only look if (say) input_boolean.sunday is ‘on’ rather than checking the rest (ie ALL of them)

What I suggest is that we create a sensor for an RGB light Green Value
check that it updates as we expect when you change the green (cos the template specifically looks for that)
Then check if we nudge the red up/down whether the green sensor is updated ?

Honestly, I can no longer follow the point you are trying to make.

If the loss of the ability to constrain a template’s listeners doesn’t affect you, that’s great.

So what is “entity_id: light.23” suposed to be listening to ???
It can’t assign a listener because there is no context (eg {{state_attr(‘light.23’, ‘brightness’) > 125 }} knows it’s looking at brightness only ! for this template
Hence two methods
I don’t get why you don’t get it

I’ll let someone else explain it because my attempts have gained no traction. Over and out.

1 Like

I’m not too sure about this one, I don’t think it monitors any entities, rather it monitors state change events that involve this entity, so the listener is attached to the events, not the entity itself or are the listeners attached to the entity? I don’t know how it works and I would be more than happy if someone would explain it to me.

Are you sure that it only attaches a listener to the specific attribute and not a listener to all state change events of this entity? I did a little test in the template editor

And here it shows that it’s listening to state change events of input_datetime.alarm_clock and not to state change events of the hour attribute of input_datetime.alarm_clock. I also quickly hovered over the template code at github and I only see there checking the template for entity_ids and not any attributes, see here.

maybe that w/could give us food for thought…if entering the sensor.time like this could prevent updating the full template sensor, maybe some other entity’s state could too… (thinking of a script to run the template sensor and using the scripts last_changed somehow…)

{% if states.sensor.time.last_changed 
    <= states.input_boolean.pause_template.last_changed %}
act # replace with actual template...?
{% else %} pause
{% endif %}

nah, this will not work, will give a timeframe of 1 minute runtime, not a single run.

Never said it did; I believe you quoted me out of context. I was referring to Mutt’s claim that listeners are assigned to states, not entities (whatever that means). Listeners are assigned to monitor an entity’s state-changes (i.e. changes in the values of its state and attributes).

Final attempt:

This example will use a very contrived Template Sensor in order to highlight the issue. I’ve tested these Template Sensors in 0.114.4 and 0.115 and they behave precisely as I expected they would. I ask that anyone who still doubts my explanation, to please try it for yourself first. No really, please test and compare their behavioral differences so you clearly understand the issue.

Let’s say there’s a need to count the number of input_boolean entities and add the result to the current timestamp.

          {% set x = states.input_boolean | count %}
          {{ now().timestamp() + x }}

We will use that in a Template Sensor but we have an additional design requirement:

  • The sensor must be updated only when light.kitchen undergoes a state-change (I told you it would be a contrived example).

In versions prior to 0.115, this Template Sensor can do that:

      demo:
        entity_id: light.kitchen
        value_template: >
          {% set x = states.input_boolean | count %}
          {{ now().timestamp() + x }}

Before version 0.115, no entities could be identified in that particular value_template because neither now() or states.input_boolean are entities. Therefore, the only listener assigned to this Template Sensor would be a single one that monitored light.kitchen. Whenever you turn that light on or off, or even change its brightness, the listener causes the Template Sensor to be refreshed.

In fact, you could have written the Template Sensor this way and it would continue to be refreshed only when light.kitchen experienced state-changes.

      demo:
        value_template: >
          {% set z = states('light.kitchen') %}
          {% set x = states.input_boolean | count %}
          {{ now().timestamp() + x }}

FYI: That second example works differently in 0.115.

A listener is still assigned to light.kitchen but now additional listeners are also assigned to each and every input_boolean in your system. That’s because the new entity-identification technique is more clever than the previous one and understands what states.input_boolean represents.

However, this means the Template Sensor is refreshed not only when light.kitchen has state-changes but also when any input_boolean undergoes a state-change. According to our original design requirement for this Template Sensor, that’s not the behavior we want. Moreover, given that entity_id is deprecated, there’s no way to force it to work the way we want.

Ideally, 0.115 would continue to permit the use of entity_id.

      demo:
        entity_id: light.kitchen
        value_template: >
          {% set x = states.input_boolean | count %}
          {{ now().timestamp() + x }}

Upon initial inspection of this Template Sensor (at startup or reload), it would detect the presence of entity_id and assign listeners to the listed entities as opposed to identifying entities in value_template. The Template Sensor would be refreshed only when light.kitchen undergoes state-changes, which is the behavior we wish to preserve for this example when using 0.115.

3 Likes

Wow … I have been sidetracked by real life for a couple of days but I see that the discussion could carry on without me :joy:. I cannot reply to all that, sorry.

However, I am trying to learn the use cases that have been broken. I have seen several templates counting unavailable entities or entities in a domain, etc. Is anyone using states for something that is not counting entities?

When you used entity_id: to decrease the update frequency, did you ever set it to anything but the time/date sensors?

I have been thinking about this a lot since I also discovered that limitation in a previous post. I no longer think that input_text is a suitable replacement. It is missing all the auxiliary templates that the template sensor has grown (attributes, availability, etc.). It is visually different in the UI. And it is a different domain which could give some trouble when doing things like listing all sensors.

heck yes, many config used other entities than sensor.time. As a matter of fact, most of the entity_id’s I used were other entities for triggering the update.
Also, I used scripts to update the template sensor that without an entity_id set in the config wouldn’t update by itself.

in the case of my earlier posted unavailable template sensor:

  - platform: template
    sensors:
      entities_unavailable:
#        entity_id:
#          - script.update_entities_uun
#          - automation.check_for_unavailable_entities
        friendly_name: Entities Unavailable
        value_template: >

making sure this only ran on demand, because it was only needed to run on demand, and not constantly.

or like this, including sensor.time, making sure it ran, once a minute, but also on demand by the state change of the boolean (which was automated, so notifications ran immediately:

      github_repo_updates:
        friendly_name: Github repo updates
#        entity_id: input_boolean.github_repo_update, sensor.time
        value_template: >
          {% set updates_on = states.input_boolean
             |selectattr('entity_id','in',expand('group.github_repo_updates'))
             |selectattr('state','eq','on')|map(attribute='name')|list|count %}
          {{updates_on}}
        attribute_templates:
          updates: >
            {% set updates_on = states.input_boolean
               |selectattr('entity_id','in',expand('group.github_repo_updates'))
               |selectattr('state','eq','on')|map(attribute='name')|list %}
            {% if updates_on|length == 0 %}
              No Updates..
            {% elif updates_on|length == 1 %}
              The {{updates_on[0]}} repo has an update available
            {% elif updates_on|length == 2 %}
              The {{updates_on[0]}} and {{updates_on[1]}} repos have updates available, please check
            {% else %}
              The {{updates_on[:-1]|join(', ')}}, and {{updates_on[-1]}} have updates available, please check
            {% endif %}

the above example shows something else, might I ask that here: would be really cool if we could have global templates (call them variables if must) per template entity, so we can use that for all templates in that sensor. As you can see we need to double up on the yaml code now, while it could be so much shorter (and less error prone because of that).

like:

      github_repo_updates:
        friendly_name: Github repo updates
        variable:
          update_on: >
            {% set updates_on = states.input_boolean
             |selectattr('entity_id','in',expand('group.github_repo_updates'))
             |selectattr('state','eq','on')|map(attribute='name')|list %}
#        entity_id: input_boolean.github_repo_update, sensor.time
        value_template: >
          {{updates_on|count}}
        attribute_templates:
          updates: >
            {% if updates_on|length == 0 %}
              No Updates..
            {% elif updates_on|length == 1 %}
              The {{updates_on[0]}} repo has an update available
            {% elif updates_on|length == 2 %}
              The {{updates_on[0]}} and {{updates_on[1]}} repos have updates available, please check
            {% else %}
              The {{updates_on[:-1]|join(', ')}}, and {{updates_on[-1]}} have updates available, please check
            {% endif %}

I have several templates for which I have created an extra backend sensor to use like this, but that is way more complicated, and of course creates yet another sensor …
check this:

      weather_icon_backend:
#        entity_id: sensor.dark_sky_icon, weather.dark_sky, weather.buienradar
#        friendly_name_template: >
#          {{states('sensor.weather_icon').split('weather-')[1]|title}}
        value_template: >
          {% set mapper_icon =
            {'partly-cloudy-night':'night-partly-cloudy'} %}
          {% set mapper_br =
            {'pouring':'pouring',
             'lightning-rainy':'lightning-rainy',
             'snowy-rainy':'snowy-rainy'} %}
          {% set mapper_ds =
            {'clear-night':'night',
             'partlycloudy':'partly-cloudy'} %}
          {% set icon = states('sensor.dark_sky_icon') %}
          {% set buienradar = states('weather.buienradar') %}
          {% set dark = states('weather.dark_sky') %}

          {% set weather = mapper_icon[icon] if icon in mapper_icon else
                           mapper_br[buienradar] if buienradar in mapper_br else
                           mapper_ds[dark] if dark in mapper_ds else
                           dark if dark in
                             ['sunny','rainy','snowy','snowy-rainy','windy','fog','cloudy','hail','lightning']
                           else 'sunny-alert' %}
            mdi:weather-{{weather}}
#        icon_template: >
#          {{states('sensor.weather_icon')}}

      weather_icon:
#        entity_id: sensor.dark_sky_icon, weather.dark_sky, weather.buienradar
        friendly_name_template: >
          {{states('sensor.weather_icon_backend').split('weather-')[1]|title}}
        value_template: >
          {{states('sensor.weather_icon_backend')}}
        icon_template: >
          {{states('sensor.weather_icon_backend')}}

which, btw, is another fine example of a sensor using more than sensor.time for entity_id’s :wink:

I said to decrease the update frequency. For the GitHub and the weather ones, it seems like you do want updates immediately?

I think this can be changed to:

     {% set updates_on = expand('group.github_repo_updates')
             |selectattr('state','eq','on')|map(attribute='name')|list|count %}

and it would work perfectly with the new engine, with less cruft and using no processing power at all except when the booleans toggle. Did I miss something?

as said and explained eloquently by @123, the entity_id was also used for templates that didn’t update by them selves on 114. In this case, I was trying to illustrate that this same template sensor (that under 115 would constantly update) could be limited ( == decreasing) to updating on adding that entity_id.
but yeah your right, I could have picked better examples…

right, its me that missed that one. thanks!

can we use the expand function also in the reject line here?

          {{states|selectattr('state','in',['unavailable','unknown','none'])
                  |rejectattr('entity_id','in',ignore_list)
                  |rejectattr('entity_id','in',state_attr('group.entity_blacklist','entity_id'))
                  |rejectattr('domain','in',['group','media_player'])
                  |list|length}}

only my question on the globale template variable?