Heads up! Upcoming breaking change in the Template integration

Configuration
136

cool. heavily into checking the config, which is not so very nasty to do, given the new possibilities galore ! font get me wrong, I am very excited :wink:

please let me ask one specific config question:

        value_template: >
          {% set ns = namespace(total = 0) %}
          {% for state in states.sensor if '_actueel' in state.object_id %}
            {% set ns.total = ns.total + (state.state|int) %}
          {% endfor %}
          {{ns.total}}

can be written using the expand(group) function now to? there’s a group.energy_actueel which could be used… adding the values of all these sensors ?

137

You can rewrite it as

{{ expand('group.energy_actueel') | map(attribute='state') | map('int') | sum }}

1 Like
138

wow, Thanks!
that’s exactly what I was testing right now:

      sensors_actueel_verbruik_summed:
        friendly_name: Sensors actueel verbruik summed
#        entity_id: sensor.netto_verbruik
        unit_of_measurement: Watt
        value_template: >
          {{expand('group.sensors_actueel')
             |map(attribute='state')|map('int')|sum}}

I’ve made an automation to create the group:

  - alias: Create actueel group
    trigger:
      platform: homeassistant
      event: start
    action:
      service: group.set
      data_template:
        object_id: sensors_actueel
        entities: >-
          {%- for s in states.sensor
            if ('_actueel' in s.entity_id )%}
            {{s.entity_id}}{% if not loop.last %}, {% endif %}
          {%- endfor %}

beats writing the group verbosely.
Now only need to think of a way to get this restored. Or will the sensor be restored, even though the group is created at startup… ?

139

The sensor will update automatically as soon as the group is created because the group creation fires a state_changed event.

140

I’ve got a branch that will significantly reduce the number of re-renders when we are only counting states and not actually looking at what is in the state that builds on https://github.com/home-assistant/core/pull/40250

Edit: I’ve opened a draft PR for this https://github.com/home-assistant/core/pull/40272

2 Likes
141

Great !
ill check that out as soon as I get my HA instance back…

ive got this going on:

Invalid config for [automation]: 
  Integration 'sun' does not provide trigger support.

please see here:

what could I do to find that…

would it be an automation like this:

  - alias: Notify sun state change
    id: Notify sun state change
    trigger:
      platform: state
      entity_id: sun.sun
    condition:
      - condition: template
        value_template: >
          {{trigger.to_state.state is in ['above_horizon','below_horizon'] and
            trigger.to_state.state != trigger.from_state.state}}
    action:

which has worked just fine/hasnt caused havoc before?

142

The error you posted has nothing to do with the changes being discussed in this thread.

143

yeah, sorry, would have hoped it to be an erroneous error, pointing to a missing entity_id for sun in a template sensor somehow.
I’ve got no clue where to find elsewhere. if you have a suggestion, please don’t hold back in the other thread I opened for this?

144

I’m using this to monitor the sensors in error, how can I change this? it used to be updated using entity_id: sensor.time and being updated once per minute only, now it gets updated frantically…

  - platform: template
    sensors:
      issues:
        friendly_name: 'Sensors that need attention'
        value_template: >
          {{ states | selectattr('state', 'in', ['unavailable', 'unknown', 'none']) | list | length }}
        attribute_templates:
          entities: >
            {{ states | selectattr('state', 'in', ['unavailable', 'unknown', 'none']) | map(attribute='entity_id') | list | join('\n') }}
145

this is a new template problem:

2020-09-19 02:52:48 ERROR (MainThread) [homeassistant.helpers.event] Error while processing template: Template("{{states.sensor.birthday_marijn.attributes.type}}: {{states.sensor.birthday_marijn.attributes.original_date}}
")

which is quite impossible, since the package this sensor is configured in, is no longer in the config, and, along with quite some other packages, moved from my ‘put-aside’ folder on the HA card, to my Mac;s harddrive, so no longer physically there…

how can this still be referenced? Could it somehow be left in the DB, and the new templating engine, even after several ha host reboot s still want to read that??

146

The answer is at the bottom of the first post of this thread. Just add this line to the template:

{% set x = states('sensor.date') %}

In your case, use sensor.time instead of sensor.date.

The template starts with states which represents every entity in your system. That means Home Assistant has assigned a listener to every entity, so whenever any one of them changes state, the template is evaluated.

Minimally, change it to states.sensor so at least only all sensor entities are monitored as opposed to every entity.

2 Likes
147

Here is an attempt at a rework, using an automation with input_text and input_number helpers (you must create those in the UI).

This did reveal an issue because the input_text state might not be able to hold all the information you had in the attribute. To get around this I had to truncate the output.

automation:
  - alias: 'Update issues'
    trigger:
      - platform: time_pattern
        minutes: '*'
    action:
      - service: input_text.set_value
        data:
          entity_id: input_text.issues
          value: "{{ states | selectattr('state', 'in', ['unavailable', 'unknown', 'none']) | map(attribute='entity_id') | list | join('\n') | truncate(250) }}"
      - service: input_number.set_value
        data:
          entity_id: input_number.issues_count
          value: "{{ states | selectattr('state', 'in', ['unavailable', 'unknown', 'none']) | map(attribute='entity_id') | list | length }}"
148

This is an optimization of your template, it iterates states once instead of twice. I think this is valid even though it references itself now.

  - platform: template
    sensors:
      issues:
        friendly_name: 'Sensors that need attention'
        value_template: >
          {{ (state_attr('sensor.issues', 'entities') or '') | wordcount // 2 }}
        attribute_templates:
          entities: >
            {{ states | selectattr('state', 'in', ['unavailable', 'unknown', 'none']) | map(attribute='entity_id') | list | join('\n') }}
149

I just upgraded to HA 0.115.1 (from 0.114.4) and everything seems fine but I’m not too good with templates and fear that the below will need to be changed in order to keep working. Can someone please advise?

sensor:
  - platform: template    # determine if today is selected as a watering day for program 1
    sensors:
      retic_program1_watering_day:
        entity_id: input_boolean.retic_program1_monday, input_boolean.retic_program1_tuesday, input_boolean.retic_program1_wednesday, input_boolean.retic_program1_thursday, input_boolean.retic_program1_friday, input_boolean.retic_program1_saturday, input_boolean.retic_program1_sunday, sensor.time
        value_template:  >
          {% set sensor_names = [ 'monday', 'tuesday', 'wednesday','thursday','friday','saturday','sunday'] %}
          {% set today_name = sensor_names[now().weekday()] %}
          {% set entity_id = 'input_boolean.retic_program1_'+today_name %}
          {{ is_state(entity_id, 'on') }}

I have a few like these… (I didn’t create them, they were from forum help)

…and also these:

- platform: template
    sensors:
      green_waste_bin_tomorrow:
        entity_id:
          - sensor.time
        friendly_name: "Green Waste Bin Tomorrow"
        value_template: "{{ ((as_timestamp(now()) - as_timestamp('2019-01-09 00:00:00'))) / 86400 |int % 14 < 1 }}"
        entity_picture_template: >-
          {{ '/local/images/green_waste_bin_tomorrow.png'}}
  - platform: template
    sensors:
      recycling_bin_today:
        entity_id:
          - sensor.time
        friendly_name: "Recycling Bin Today"
        value_template: "{{ ((as_timestamp(now()) - as_timestamp('2019-01-03 00:00:00'))) / 86400 |int % 14 < 1 }}"
        entity_picture_template: >-
          {{ '/local/images/recycling_bin_today.png'}}
  - platform: template
    sensors:
      green_waste_bin_today:
        entity_id:
          - sensor.time
        friendly_name: "Green Waste Bin Today"
        value_template: "{{ ((as_timestamp(now()) - as_timestamp('2019-01-10 00:00:00'))) / 86400 |int % 14 < 1 }}"
        entity_picture_template: >-
          {{ '/local/images/green_waste_bin_today.png'}}

Help would be awesome. Thanks

EDIT: just looked at my logs and I do indeed have 12 occurrences that need fixing

150

Dave,
To maximise perfortmance I’d write it like this : -

sensor:
  - platform: template    # determine if today is selected as a watering day for program 1
    sensors:
      retic_program1_watering_day:
        value_template:  >
          {% set day =  (as_timestamp(as_local(states.sensor.date.last_changed)) | timestamp_custom ('%A')) | lower%}
          {% set ent_id = 'input_boolean.retic_program1_'~day %}
          {{ is_state(ent_id, 'on') }}

As this will only evaluate once per day. (Edit: AND whenever ‘today’s’ input_boolean changes )

IGNORE THIS : -
if you regularly (or even just occasionally) update your input_boolean’s then it might be wise to include them in the template (maybe @bdraco could advise, sorry to tag but …)

going to : -

sensor:
  - platform: template    # determine if today is selected as a watering day for program 1
    sensors:
      retic_program1_watering_day: # rubbish version
        value_template:  >
          {% set x = input_boolean.retic_program1_monday or input_boolean.retic_program1_tuesday or input_boolean.retic_program1_wednesday or input_boolean.retic_program1_thursday or input_boolean.retic_program1_friday or input_boolean.retic_program1_saturday or input_boolean.retic_program1_sunday %}
          {% set day =  (as_timestamp(as_local(states.sensor.date.last_changed)) | timestamp_custom ('%A')) | lower%}
          {% set ent_id = 'input_boolean.retic_program1_'~day %}
          {{ is_state(ent_id, 'on') }}

The top one correctly determins which sensor to look at on which day, it’s forced to re-evaluate at 00:00 everyday, so just moves to the next sensor
[my god bdraco, that’s a REALLY neat bit of processing]

I’m sure there’s a way to list just the states on those sensors, or expand the group but I hate creating a group just to expand upon it and the group listing is just the same as listing in the template, so unless it’s repeated …

Everyday’s a school day
:rofl:

Edit: try to avoid ‘+’ as a string concatenator, use ‘~’ instead

1 Like
151

For the rest (i.e. your second set), (I’ve just done the first done as a sample) : -

- platform: template
    sensors:
      green_waste_bin_tomorrow:
        friendly_name: "Green Waste Bin Tomorrow"
        value_template: "{{ ((as_timestamp(states.sensor.date.last_changed) - as_timestamp('2019-01-09 00:00:00')) / 86400) | int % 14 < 1 }}"
        entity_picture_template: >-
          {{ '/local/images/green_waste_bin_tomorrow.png'}}

I was going to simplify the fixed date timestamp substraction but leaving it makes the template more readable as to what and why. Again this updates only once per day, so ‘optimum’.

EDIT: Important See Post 3 below this one

1 Like
152

This has been a great thread and really helped me understand the change in 0.115 (and also the way it worked before which seems a mystery to me!) – thank you all contributors!

However, I have the opposite of the challenge that it seems we’re trying to overcome. I have a couple of sensors for monitoring stuff, previously I updated them when needed using homeassistant.update_entity.

Following this super-duper change they’re flying off left, right and centre which proves the strength of the change – but its causing havoc with my objective.

Before I start amending my automations, was looking for some insight to see if there’s a way to suppress this behavior and update it when I needed.

Example: this sensor checks for devices on my set up which are offline, got some flaky WiFi switches which drop off and come back every so often – however I like to do a check once a day to see who’s still offline and give them a kick……yes yes, I know I could update the automations associated to this activity but was curious… :blush:

      unavailable_devices:
        friendly_name: Unavailable Devices
        unit_of_measurement: Devices
        icon_template: "{{ 'mdi:checkbox-marked-circle-outline' if states('sensor.unavailable_devices')|int == 0 else 'mdi:alert-circle-outline' }}"
        value_template: >
          {{ states | selectattr('state', 'eq', 'unavailable') | list | length }}
        attribute_templates:
          matched_devices: >
            {%- for item in states -%}
            {%- if item.state == 'unavailable' -%} 
             - {{ item.name }}|
            {%- endif -%}
            {%- endfor -%}
153

yep, expressing the exact same thing. Hope Bdraco and Amelchio will bring back some way of limiting updating templates, the way entity_id did before.

have a challenge myself, which before simply and only was updated by an automation/script (see the commented entity_id)

now, it won’t stop and kill the instance…

  - platform: template
    sensors:
      entities_unavailable:
#        entity_id:
#          - script.update_entities_uun
#          - automation.check_for_unavailable_entities
        friendly_name: Entities Unavailable
        value_template: >
          {% set ignore_list = ['light.driveway_floodlight','light.garden_backyard_floodlight','light.garden_terrace_floodlight',
                                'light.porch_floodlight','light.parking_light'] if
                                 is_state('binary_sensor.outside_daylight_sensor','on') else [] %}
          {% set unavailable = states|selectattr('state','eq','unavailable')
                                     |rejectattr('entity_id','in',state_attr('group.entity_blacklist','entity_id'))
                                     |rejectattr('entity_id','in',ignore_list)
                                     |rejectattr('domain','eq','media_player')
                                     |map(attribute='entity_id')
                                     |list %}
          {{unavailable|count}}

#          {% if states('sensor.entities_unavailable')|int > 0 %} mdi:thumb-down
#          {% else %} mdi:thumb-up
#          {% endif %}

      entities_uun:
#        entity_id:
#          - script.update_entities_uun
#          - automation.check_for_unavailable_entities
        friendly_name: Entities U/U/N
        value_template: >
          {% set ignore_list = ['light.driveway_floodlight','light.garden_backyard_floodlight','light.garden_terrace_floodlight',
                                'light.porch_floodlight','light.parking_light'] if
                                 is_state('binary_sensor.outside_daylight_sensor','on') else [] %}

          {{states|selectattr('state','in',['unavailable','unknown','none'])
                  |rejectattr('entity_id','in',ignore_list)
                  |rejectattr('entity_id','in',state_attr('group.entity_blacklist','entity_id'))
                  |rejectattr('domain','in',['group','media_player'])
                  |map(attribute='entity_id')
                  |list|length}}
        attribute_templates:
          Unknown: >
            {% set unknown = states|selectattr('state','eq','unknown')
                                   |rejectattr('entity_id','in',state_attr('group.entity_blacklist','entity_id'))
                                   |rejectattr('domain','in',['group'])
                                   |map(attribute='entity_id')
                                   |list %}
            {% if unknown|count == 0 %} 0
            {% else %}
            {{unknown|count}}:
            {{'\n' + unknown|join(',\n')}}
            {% endif %}
          Unknown sensors: >
            {% set unknown = states.sensor|selectattr('state','eq','unknown')
                                          |rejectattr('entity_id','in',state_attr('group.entity_blacklist','entity_id'))
                                          |map(attribute='entity_id')
                                          |list %}
            {% if unknown|count == 0 %} 0
            {% else %}
            {{unknown|count}}:
            {{'\n' + unknown|join(',\n')}}
            {% endif %}
          Unavailable: >
            {% set ignore_list = ['light.driveway_floodlight','light.garden_backyard_floodlight','light.garden_terrace_floodlight',
                                  'light.porch__floodlight','light.parking_light'] if
                                   is_state('binary_sensor.outside_daylight_sensor','on') else [] %}
            {% set unavailable = states|selectattr('state','eq','unavailable')
                                       |rejectattr('entity_id','in',state_attr('group.entity_blacklist','entity_id'))
                                       |rejectattr('entity_id','in',ignore_list)
                                       |rejectattr('domain','eq','media_player')
                                       |map(attribute='entity_id')
                                       |list %}
            {% if unavailable|count == 0 %} 0
            {% else %}
            {{unavailable|count}}:
            {{'\n'}}{{unavailable|join(',\n')}}
            {% endif %}
          None: >
            {% set none_ = states|selectattr('state','eq','none')
                                 |map(attribute='entity_id')
                                 |list %}
            {% if none_|count == 0 %} 0
            {% else %}
            {{none_|count}}:
            {{'\n' + none_|join(',\n')}}
            {% endif %}
          Full: >
            {% set full = states|selectattr('state','in',['unavailable','unknown','none'])
                                |map(attribute='entity_id')
                                |list %}
            {{full|count}}:
            {{'\n' + full|join(',\n')}}
154

Okay, I did some thinking.

Dave is comparing (and I re-phrased) as midnight of a fixed date in the past to the timestamp generated when sensor.date “last_changed” And I’m doing that when LOCAL midnight occurs as that is when the date changes.
Now depending on when the ‘fixed’ date was it could be in DST or not.
Dave currently resides in the UK (until we kick his immigrant ass out again ( :stuck_out_tongue: just Joking about our current 'isolationist' policies)) The point is, Dave is in UTC for 5 months/yr and DST (BST) 7 months/yr.
If it wasn’t in DST then Dave’s offset to midnight last night (which was DST) when divided by 86400 will not yield an integer.
scenairios for different settings mean that for different start dates and different comparison dates you will be sometimes slightly ahead or sometimes slightly behind (never both, for the same fixed date) and sometimes spot on.

So @bdraco how do we compare a fixed date in the past with now() bearing in mind that if you are in TZ +5 then ‘local’ is 5 or 6 hours (depending on DST) out ?
I admit it’s a special circumstance for time comparison

How do we resolve this ?

Doing an addition of a couiple of hours and then | int the / 84600 should do it, but it seems VERY messy - inelegant !

@123 you may have ideas on making this less “kludgy”

155

Any template that starts with states:

{{ states | etc ... }}}

makes Home Assistant assign a listener to every entity in your system. So not just sensor and binary_sensor but other domains as well. My guess is that’s a lot more entities than one normally needs to monitor for the template’s purpose. (EDIT: It means your template is evaluated when just about anything else in your system changes state).

If the need is to monitor only certain domains, then you can specify those domains with the expand() function. This will creates listeners for only sensor and binary_sensor entities:

{{ expand(states.sensor, states.binary_sensor) | etc ... }}

If you know precisely which entities you wish to monitor, create a group containing those entities (if the entities have some sort of commonality, the group can even be created dynamically at startup via an automation). Then your template only needs to expand the group (and listeners will be created exclusively for the group’s members).

{{ expand('group.my_entities') | etc ... }}