Heads up! Upcoming breaking change in the Template integration

Configuration
158

For example, instead of using a ‘wildcard’ like states:

          {{states|selectattr('state','in',['unavailable','unknown','none'])
                  |rejectattr('entity_id','in',ignore_list)
                  |rejectattr('entity_id','in',state_attr('group.entity_blacklist','entity_id'))
                  |rejectattr('domain','in',['group','media_player'])
                  |map(attribute='entity_id')
                  |list|length}}

specify the domains of the entities you actually want to monitor for state-changes:

          {{expand(states.sensor, states.binary_sensor, states.light, states.switch, states.lock, states.fan)
                  |selectattr('state','in',['unavailable','unknown','none'])
                  |rejectattr('entity_id','in',ignore_list)
                  |rejectattr('entity_id','in',state_attr('group.entity_blacklist','entity_id'))
                  |map(attribute='entity_id')
                  |list|length}}

The eagle-eyed may have noticed that the second version no longer needs this line:

                  |rejectattr('domain','in',['group','media_player'])

because those domains are not listed in the first line.

EDIT

Observations:

Listeners will be created for every entity produced by the template’s first line. That means listeners will be created for the entities in ignore_list and group.entity_blacklist. In other words, this template will be evaluated even when one of those ignored/blacklisted entities changes state. That includes the sensor containing this template (i.e. self-referential: it is evaluated even when its own state changes).

159

Well, yeah, but not every second on all states on state change. Panic because it halts the system

I had it work ‘on demand’ exactly because it is an expensive template.

We can’t do that any longer.

@123 thanks those are very fine suggestions, of course I posted my ‘old’ sensor not yet optimized , let alone for 115.

Do you have a suggestion for the ‘on demand’ issue?

Maybe rewrite it as python script and use the same automation for calling it .

Now that’s a challenge :wink:

160

My system is running at 5% cpu use, which is exactly what it was at before these changes. This is my unavailable sensor:

sensor:
  - platform: template
    sensors:
      unavailable_entities:
        friendly_name: Unavailable Entities
        value_template: "{{states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
            |rejectattr('entity_id', 'in', state_attr('group.entity_ignore_list', 'entity_id'))
            |rejectattr('entity_id', 'eq' , 'group.entity_ignore_list')
            |map(attribute='entity_id')|list|length }}"
        attribute_templates:
          entities: "{{states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
              |rejectattr('entity_id', 'in', state_attr('group.entity_ignore_list', 'entity_id'))
              |rejectattr('entity_id', 'eq' , 'group.entity_ignore_list')
              |map(attribute='entity_id')|list|join(', ') }}"
        unit_of_measurement: items


group:
  entity_ignore_list:
    entities:
      - THINGS YOU ARE HAPPY TO BE UNAVAILABLE LISTED HERE

Updates on every state change, instant acknowledgment of unavailable items. No issues :man_shrugging:

3 Likes
161

Thanks Marc, will check after the time trial (TdF) :wink:

Response time might differ depending on the count of states of course …

1 Like
162

I believe Mariusthvdb summarized the issue nicely when he described it as being “expensive”.

Screenshot from 2020-09-19 11-22-13
Screenshot from 2020-09-19 11-22-131065×191 21.4 KB

As shown in the screenshot, the use of states means it now listens to everything (entities in all domains; automation, scene, script, zone, etc).

That’s different from how it worked prior to 0.115 (which created no listeners for states). Therefore for the people who are experiencing performance issues, they may wish to start their debugging adventure by removing any “expensive” Template Sensors they may have and seeing if it improves anything.

Whether it proves to be ‘expensive’ or not, from a design perspective it seems overreaching to have a Template Sensor refreshed when anything else changes state.

163

If you want to limit updates to only a specific interval, something like this will work:

- id: '1600355905751'
  alias: Update Unavailable Entities
  description: ''
  trigger:
  - platform: time_pattern
    hours: '*'
    minutes: '15'
    seconds: '0'
  condition: []
  action:
  - service: input_text.set_value
    data:
       value: "{{ states | selectattr('state', 'in', ['unavailable', 'unknown', 'none']) | list | length }}"
    entity_id: input_text.unavailable_entities
  mode: single
164

But that is kinda the point of the sensor in this case is it not? Monitor all state changes and record the ones that change to unavailable.

165 
{{states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
            |rejectattr('entity_id', 'in', state_attr('group.entity_ignore_list', 'entity_id'))
            |rejectattr('entity_id', 'eq' , 'group.entity_ignore_list')
            |list|length }}

Taking out the extra map will make this a bit faster

1 Like
166

I agree its purpose is to report when something is unavailable. It’s the “something” that I feel ought to have a narrower scope as opposed to its current scope which is “everything”.

For example, as a consequence of using states, the Template Sensor checks if any automation, zone, script, scene, etc becomes unavailable. Is that even a remote possibility? Nevertheless, listeners are created for all automations (and zones, scripts, etc) and the Template Sensor gets updated when any one of those ‘extraneous’ entities changes state. Seems like a very wide (global) net is being cast to catch just a few (local) fish.

167

the way I used it, was have a look a startup, see what got initiated and what not. was this expected? no? act. If expected, add the to the ignore list.

update the sensor. manually by clicking the update script.
all well? thumbs up. if not, have another look.

That did the trick to see for config errors, or integrations not being initiated at startup, mostly the weather integration.
For me, I don’t need the constant check of all states. Just a simple tool to reveal the issues in the current state of affairs.

update

even testing the above templates in the template field causes issues: my Hue integration has always been an indicator of internal issues, and becomes unavailable completely and remains like that, if I enter the template.
even after emptying the template editor, it wont come back.
so yeah, it is very expensive, and I am not yet very happy with the new way of handling these templates at all.
entering entity_id: was not so bad, and had the advantage of controlling the update process… now, the system keeps checking states, even if unnecessary and even worse, not desired.

at the moment. y system has more than 278 unavailable entities, so the list wouldn’t be anywhere near usefully listed using Nicks automation

seems any effort to control the evaluation of these templates (using an automation with an input_text, or maybe a python script) is futile if a single run of it already has such impact.

Cant help but feeling there is a bug somewhere, not giving back to the system, or staying in a loop somehow. I had a very swiftly responding system up to HA 114.4, even with all my many entities, without so much as a single Hue unavailability left.
This must be the muddiest iteration yet.

168

I think your system has a lot more going wrong than this sensor :man_shrugging:

1 Like
169

What would you say to the suggestion to undo the deprecation and permit entity_id to supercede the template?

As you and I know, that’s the way it works in versions prior to 0.115. If you specify entity_id you’re telling the system to "create listeners for these entities and not for any in value_template".

I have no idea what are the technical implications of allowing for this behavior now that there’s an entirely different way that the system identifies entities in a template.

1 Like
170

That’s what I suggested before. Would be best to have both options available.

So I would say: yes please!

1 Like
171

hi guys how could i fix this template to work with sensor.date:

{{ ['Monday','Tuesday','Wednesday','Thursday','Friday','Saturday','Sunday'][now().weekday()] }}

i tried this:

{% set day = states('sensor.date') %}
{{ ['Monday','Tuesday','Wednesday','Thursday','Friday','Saturday','Sunday'][now().weekday()] }}

In the template editor it validates it well, but then when checking the configuration it throws an error

greetings and thanks

172

Do you have the time & date integration enabled?

1 Like
173

Oops, my mistake, I was about to delete the post, I forgot to add ‘>’
Thanks for the quick reply

it might be a good suggestion for next month WTH
:crazy_face:

174

You can also include sensor.date in your template like this:

{{ ['Monday','Tuesday','Wednesday','Thursday','Friday','Saturday','Sunday'][as_local(states.sensor.date.last_updated).weekday()] }}

EDIT

Simply adding {% set day = states('sensor.date') %} to the template will work in the current (0.115) and previous versions of Home Assistant but adds another line to the template containing a seemingly unused variable called day (in your template). Nevertheless, it’s a quick way to compensate for the deprecation of entity_id.

The alternative way (shown above) will only work in 0.115+ because it uses a new function called as_local(). However, it doesn’t require adding another line and there’s no question that it’s an integral part of the template.

1 Like
175

Thank you very much for your wisdom, after having been reading the thread I had tried “as_local (states.sensor.date.last_updated)” but I have not been able to get it.
I actually had no idea about jinja until I started with home assistant, based on reading you and other gurus and doing my tests in the editor I am learning.
I love seeing its complex templates and I am really enjoying learning and understanding them, I am not satisfied with copy-paste, thank you very much to all gentlemen.

1 Like
176

How do feel about the idea of re-instating the use of entity_id in a Template Sensor? :thinking:

Its behavior would remain the same as in previous versions (i.e. it supersedes any entities within the template).

Would resistance to the idea be due to:

  1. Technical challenges given the significantly different way that entities are now identified and listeners are assigned.
  2. Philosophical disagreement about how it should work.
  3. A bit of both.
  4. None of the above.
177

I implemented a proof of concept a few weeks ago. However, after discussion, it was decided against moving it forward as there are better ways (ones that require less maintenance over time and that don’t require manually maintaining lists of entities) to accomplish what has been discussed in this thread. I certainly don’t feel comfortable maintaining two unique template implementations in the same integration, and it seems unlikely we would find someone who would sign up to that.

I rebased the commit to fix the conflicts with 0.115.1 if you want to play with it https://github.com/bdraco/home-assistant/commit/d30293e92609e222999b369b3454a1638c499997

I can’t speak to how well it works as it doesn’t have sufficient test coverage, and I haven’t tested it in production since I first wrote it a few weeks ago.

There is going to be some transition time as inefficient templates are adjusted. After that, we end up with a system that requires less manual configuration for the majority of cases.