Heads up! Upcoming breaking change in the Template integration

NOTE 2: Guidance for 0.116 and upcoming changes in 0.117

0.116 introduced improvements to mitigate performance problems reported by some users of 0.115 for templates containing states and states.DOMAIN. There are now built-in rate-limits that prevent excessive processing.

  • If your template refers to states, any entity in your entire system that changes state will cause the template to be evaluated (updated). 0.116 ensures evaluations cannot happen more than once a minute. In 0.117 the rate-limit for states will increase to once a second.

  • If your template refers to states.DOMAIN, any entity within DOMAIN that changes state will cause the template to be evaluated. 0.116 ensures evaluations cannot happen more than once a minute.

A significant improvement will be introduced in 0.117. In all previous versions, if a template contains the now() function, it doesnā€™t serve to periodically evaluate the template. In other words, a template with now() and no other entities, is evaluated once at startup and never again (until the next restart).

In 0.117, the now() function is promoted to being a first-class citizen and will cause the template to be evaluated once a minute. If your template currently uses a combination of now() and sensor.time, you will be able to safely remove the reference to sensor.time (and the template will continue to function nominally).

In other words, this particular bit of documentation will no longer be true (and is likely to be removed):

Templates without entities using now()

Note that templates that depend on time ( now() ) and do not use any entities will not be updated as it only happens on entity state changes. For more information and examples refer to template sensor documentation


NOTE 1: Revised with actual examples tested in 0.115.


There are many upcoming improvements to the Template integration (likely to be explained in a future releaseā€™s blog post). The improvements will help to simplify the templates you currently use in Template Sensors, Template Binary Sensors, etc. However, there is a breaking change that may complicate a few things.

Letā€™s say you have a Template Sensor with no identifiable entities. Hereā€™s one that determines if the current week number is odd or even.

      current_week:
        entity_id: sensor.date
        value_template: "{{ 'EVEN' if now().isocalendar()[1] % 2 == 0 else 'ODD'}}"

The value_template contains no entities so Home Assistant cannot assign a ā€˜listenerā€™ to anything in the template. That means the template will never be evaluated after startup. To fix this, we add an identifiable entity in entity_id like this:

        entity_id: sensor.date

On startup, Home Assistant identifies sensor.date and assigns a listener to it. Whenever sensor.date changes state, which is at the start of every new day, the value_template will be evaluated. Perfect; it now works like we want (template is evaluated once a day, at the start of the day).

The breaking change is that you will no longer be able to use entity_id (EDIT: 0.115 ignores it and reports a warning message). That means we can only do this:

      current_week:
        value_template: "{{ 'EVEN' if now().isocalendar()[1] % 2 == 0 else 'ODD'}}"

However, we know that wonā€™t get a listener and will not be evaluated on any periodic basis. How do we fix this?

The suggested workaround is to create an automation with a Time Trigger (or Time Pattern Trigger) that calls homeassistant.update_entity for the Template Sensor.

- alias: 'Update current_week sensor'
  trigger
    platform: time
    at: '00:00:01'
  action:
    service: homeassistant.update_entity
    entity_id: sensor.current_week

That will work but itā€™s not nearly as convenient as the old way where you simply added entity_id: sensor.date to the Template Sensor. In defense of this workaround, if you have many Template Sensors of this kind, that must be refreshed once a day, the same automation can be used for that purpose (i.e. you donā€™t need a separate automation for each sensor).

To avoid using an automation, it seems all one has to do is incorporate sensor.date into value_template. It will be identified by Home Assistant and assigned a listener. The trick is how do you include sensor.date into the template when it isnā€™t really needed to perform any calculations?

Hereā€™s a simple way to do that:

      current_week:
        value_template: >
          {% set x = states('sensor.date') %}
          {{ 'EVEN' if now().isocalendar()[1] % 2 == 0 else 'ODD'}}

Home Assistant can easily identify sensor.date within the template.

To prove this to yourself (using any version of Home Assistant) you can create this Template Sensor:

      test:
        value_template: >
          {% set x = states('sensor.time') %}
          {{ now().timestamp() | timestamp_local }}

It will be refreshed once a minute.

Here is another way to incorporate sensor.time into your template. Instead of using now() to get the current time, it uses last_updated from sensor.time.

      test:
        value_template: >
          {{ as_local(states.sensor.time.last_updated).timestamp() | timestamp_local }}

However, there is one caveat and thatā€™s last_updated is UTC time, not local time whereas the now() function reports local time. Therefore we use a new function introduced in 0.115 called as_local(). It converts a datetime object from UTC to local time.


EDIT

The original suggested technique was needlessly longer. All examples shown above have been replaced with petroā€™s suggested technique which is more compact: simply insert a line like {% set x = states('sensor.date') %} into your template.

19 Likes

Couldnā€™t we just add

{% set dump = states('sensor.time') %}
{{ now().timestamp() | timestamp_local }}

or

{% set dump = states('sensor.date') %}
{{ 'EVEN' if now().isocalendar()[1] % 2 == 0 else 'ODD'}}
2 Likes

Thanks for the warning. Seems like a huge step backwards to me. What was the reasoning / need for this change?

4 Likes

Yes, that would work too. All thatā€™s needed is to slap sensor.whatever somewhere so that it can be identified.

FWIW, I like your version better because thereā€™s no need for an if-else to wrap everything (like in my version) ā€¦ and itā€™s one line less. :slight_smile:

      current_week:
        value_template: >
          {% set zilch == states('sensor.date') %}
          {{ 'EVEN' if now().isocalendar()[1] % 2 == 0 else 'ODD'}}
1 Like

Yeah, I didnā€™t like the 2 extra lines. Iā€™m a minimalist if I can be.

Though reading the PR I was happy to see this:

we have made it easier by making template entities reloadable in the YAML configuration reloading section under Configure Home Assistant Server Controls .

I was surprised when I read the PR and scrambled to review all my Template Sensors to see if it would affect my configuration. I found only two instances. I feel the suggested workaround (an automation) is overkill and not nearly as tidy as the current method. However, inserting a ā€˜do-nothingā€™ one-liner into the template isnā€™t too bad.

The ā€˜goodā€™ in whatā€™s upcoming far exceeds the ā€˜mehā€™. :slight_smile:

Yeah not too bad, unless youā€™re a control freak like me. I have explicitly listed all my template entities. ā€œBelt and braces thinking.ā€ Though Iā€™m probably in the same boat as you, there are only a few that actually require a time or date listener all the rest Iā€™ll just delete the entity key and list.

What about the case where there are identifiable entities and a time/date listener is also required because the use of the now() function?

Including the one-liner should cover that.

{% set x = states('sensor.time') %}
#... the rest of the template containing the entities needed
# to perform the bulk of the work ...
1 Like

if you want to list entities, you just need the state objects

{% set entities = [
  states.sensor.foo,
  states.sensor.foo2,
  ] %}

More verbose.

I could see this change being argued against. I have a feeling that this will be all over the forums. Might want to get your WTHs readyā€¦

Iā€™m not really opposed to to the change if there are substantial benefits and the entity identification works.

Iā€™m hoping there will be no need to create an explicit ā€˜ingredientsā€™ list within the template.

The old technique for identifying entities used a complicated regex pattern. Thatā€™s been scrapped and replaced by something far more elegant (that I fully admit to not having researched its mechanism in great depth).

For example, it will understand how to properly handle states.sensor and the expand() function. In addition, I get the sense (but I may be wrong) that it will be more dynamic and be able to re-evaluate what needs listeners. In other words, listeners arenā€™t only assigned at startup.

Every time we evaluate the template we now know which states can affect it so we alter the listeners whenever a new state can be affected.

2 Likes

Would it be possible to have the parser create a listener for now()? Something that updates on the minute?

Or now().day where minute resolution is excessive.

Thanks for confirming it. I have to admit I havenā€™t tried any of this and only going on what Iā€™ve read (and potentially misinterpreted).

If Iā€™m not mistaken, this new ability to dynamically assign listeners means that if a Template Sensor contains

expand('group.my_group')

and I later add another member to my_group (followed by reloading groups), will the Template Sensor append another listener for the new group member? If it does, when does it do it? I imagine certainly after using the new reload service for the Template integration but can it also do it automagically? :slight_smile:

By providing entity_ids as well as the new template tracking, it means we would end up re-evaluating the templates once when anything that affects it changes, and a second time from tracked the entity ids when they change state. We currently donā€™t have a way of preventing duplicate updates.

Additionally, manually providing the entity ids meant we would evaluate the template regardless of the entity state change could affect the template (e.g., dependent states). The new method is smart enough to avoid re-evaluation only when the entities can affect the resulting change. We want to minimize template evaluation as much as possible because it is a performance bottleneck.

1 Like

It should pickup the new group member without having to reload. I havenā€™t tested that exact scenario.

2 Likes

It is likely possible to look for now(). Unfortunately we donā€™t know which now() change the template wants to get updates for. Since now() has microsecond precision it could we want an update every microsecond (unlikely), second, or minuteā€¦ or many only every hour

Yes, thatā€™s why it would just be a ā€˜minutelyā€™ update.

The other option would be to make the template platform support scan_interval.

Or a new template platform with periodic updates.