Heads up! Upcoming breaking change in the Template integration

Ok then, downgrade and wait till all the features of templates are fully released.

It always did.

It has always examined the template to identify entities and assign listeners to them. Except it used to omit states and states.domain. That’s why entity_id was available, as a means to mitigate the deficiencies of the automatic entity detection.

The new entity identification system is better to the point of being “too good”. It assigns listeners to every entity represented by states. That results in a lot of template refreshing. 0.116 attempts to mitigate it by reverting to the technique users employed in the past, namely refresh once a minute.

Optimally, one could choose how states is handled. However, the throttling introduced in 0.116 is but the first step along that road to ‘optimal’.

no, I can live now. and dont want to lag to much behind… and about all features: are they still in the pipeline? As far as I read all Pr’s that were retracted from 116 shortly before release, they seem to be put on hold?

summarizing:

templates using

  • {{states.xxx}} or {{states|xx}} are updated once a minute

  • {{now()}} are not updated at all, and need the extra {% set trigger = states('sensor.time') %}, or any other entity_id that updates to a desired frequency

  • all other templates are updated continuously based on each state change of identified states in that template

So (I appreciate this is now contrary to other’s wishes in this thread), my template that I wanted to react to all state changes so I would be instantly notified of any unavailable entities is no longer useful? In fact I could lose connectivity with something for 59 seconds and never know?

If so what’s the solution for those of us that didn’t suffer any performance impact and do want every state change monitoring?

1 Like

There is resistance to adding a lot of knobs and complex rules that only a handful of people are asking for. That is why the manual rate control was reverted shortly before 0.116 final and I don’t think it will be returning without some real use cases (i.e. actual, useful templates that do not currently work).

1 Like

That is correct; if the template value changes forth and back within a minute, you might not know.

Explicit updates are not rate limited so maybe this can be used for inspiration?

automation:
- alias: Update template sensor for unavailable entities
  mode: parallel

  trigger:
    - platform: event
      event_type: state_changed

  condition:
    - condition: template
      value_template: "{{ trigger.event.data.new_state.state in ('unavailable', 'unknown', 'none') }}"

  action:
    - service: homeassistant.update_entity
      entity_id: sensor.unavailable_entities

You can leave out the condition but it should be quite an optimization because then we only take the expensive look at all states (i.e. render the sensor.unavailable_entities template) when something actually changes to a value that we care about.

Writing a condition that checks for entities becoming available again is left as an exercise, or you might be happy to have that delayed for (up to) a minute.

1 Like

I’ll check this out when I get a second, cheers :+1:

funny you should advise this now, because I have gone lengths to rewrite several automations that used that trigger, which is creating listeners on all states… and all state changes… causing the issues in the first place.

I now have a few automations that trigger on 48 listed states, rather than on the 2707 entities in the setup…

because isn’t that what this trigger does for a config like:


?

btw, if listing them now is the only/better option, I still have my hopes up to be able to use the expand() function for that in the near future:

      platform: state
      entity_id: >
        {{expand('group.github_repos_in_use', 'group.github_repos_not_in_use')
         |map(attribute='entity_id')|join(',')}}

It all depends. That is why I keep asking for actual templates.

With that trigger, I was responding to mf_social who does not have those issues and thus specifically asked how to look at all state changes.

1 Like

well, I am so glad you phrased it in your own words now.

This is what its all about isn’t it? This exactly proves my point different users could require different solutions for identical code in different circumstances.

Where Marc has no issues whatsoever with that code, my system grinds to a halt. As always one’s mileage may vary.

This is the exact reason I try to ask for end-user configurability, on frequency and entity_id update limiting. If all is ok, no extra need for limiting the auto-updating of the current template engine. If the situation arises, we need these 2 options (no more than that, so it’s not the world we ask here, and don’t forget, we had… ).

if so implemented, Marc would have this template run at all times, without further ado, and no extra limiters set (implemented the 115 way, not like it is now), and eg I would have to set a limiter (take sensor.date), and could run it only once, on demand.

But isn’t that in the eye of the beholder?

There have been several templates given in this thread (and I believe in that issue where the discussion was taking place) but apparently someone has deemed them “useless”.

I could be wrong but at least some people found them “useful” or they wouldn’t have spent time developing those templates in the first place. And others wouldn’t have adopted them.

So that said can anyone provide a definition of what the powers-that-be will deem “useful”? (To them? To us? :man_shrugging:)

1 Like

Not at all, the changes in 0.116 address most of those examples with the rate limiter (including your two sensors mentioned in the GitHub issue).

I am asking for examples that still have problems. They just have to be useful to you but it is much better if they real, not a theoretical invention just to prove a theoretical point. Also, showing the actual template helps so much in understanding the prose that describes a problem.

Marius has admitted that his examples were not the best ones but did not provide other examples. And a few posts ago he says that there are no performance problems left, so I am actually not sure what the problem is at this point.

maybe this is a communication of non-native language problem…
really sorry, but when I spoke of performance problems in that sense, I meant no other performance issues in the instance, besides the issues of the templating engine. I would have thought that to be clear.

Is it so difficult to understand what the posters in this community are trying to bring across? It all boils down to what you have summarized yourself:

At least @123, @anon43302295 @klogg @finity and I have provided more than 1 template, that each in their own instance fills to their needs, and not always behaves the same, depending on configuration and instance. It is not very clear what more you could need.

The bottomline problem is that we don’t have the option anymore to limit the update frequency of the templates.

(The fact the new templating engine is upping the processor usage too is purposely kept out of this discussion, since I don’t want to risk you might think I want the new behavior of identifying states to be reverted…)

The language barrier definitely exists but I do understand that you liked it just fine with entity_id.

However, that could be due to change resistance, Stockholm syndrome, nostalgia or just misunderstandings.

I am not saying that it is any of those things but it could be. Not providing a single template when I ask three times is not helping me discard the suspicion.

That is not a problem. Using 11% CPU rather than 7% CPU is also not a problem by itself. Especially not if the increase is due to automatically including entities that were unknowingly missed by the manual setup.

A crashing system is a problem. Light actions being delayed for a second is a problem.

1 Like

we clearly live in another world. I am lost why you would use these phrases at all, and in all honesty, feel really affronted.
Ill leave this be for now, since it is going nowhere.
especially if this is the feeling of the core team:

and this is not true:

unless you translate ‘unknowingly’ by ‘having not read the documentation’.

1 Like

But, unfortunately that is a ham-fisted approach to fix a problem imposed by the template system change. An approach that not all people would necessarily desire to use if given any alternative.

The really unfortunate thing is that there were solutions proposed to allow the user to “have their cake and eat it too” that was summarily nixed because, as you said, there is a resistance to adding “buttons & knobs”. I’m not sure if the person who pulled the plug noticed or not but this whole project is nothing but buttons & knobs. It’s baked in. Why is there a resistance to adding a couple more that would allow users the flexibility to “rate limit” or “not rate limit” or “set another type of rate limit”?

Especially since the solutions were already created and all the work was already done and ready to be incorporated…until it wasn’t…and now never will be. :man_shrugging:

2 Likes

My biggest concern here is why is this affecting some users really badly and others didn’t notice a single bit of an issue? @finity 's system is more powerful than mine, but his system is heavily impacted by these changes and mine hasn’t skipped a beat.

To my mind this shows that there is something else going on somewhere and we need to discover it and address it, otherwise the devs will be chasing their tails trying to find an ‘ideal solution’ for everyone in the wrong place.

2 Likes

Someone call the mods because I think amelchio’s account has been hijacked. Only thing missing is that we’re all “Democrats”. Otherwise, the amount of gaslighting and disingenuousness makes it a dead ringer for the Twitterer in Chief.

I’m not sure it’s true but I hear people saying that …

1 Like

ok so, im trying to keep up here, however I had a pretty big issue.

My irrigation didnt shut off and flooded my yard. Wife. Not. Happy.

here is what I have for shutting it off, that no longer is working. (worked fine for months) Unsure If related to these changes or not. Hoping someone can at least see the error. im on 0.115.6.

The automation:

- id: irrigation_zone_2_off
  alias: Irrigation Zone 2 Off
  initial_state: true
  trigger:
    - platform: numeric_state
      entity_id: sensor.zone_2_time_remaining
      below: 1
  condition:
    - condition: state
      entity_id: switch.drip_system_zone
      state: "on"
  action:
    - service: switch.turn_off
      entity_id: switch.drip_system_zone

the sensor:

- platform: template
  sensors:
    zone_2_time_remaining:
      friendly_name: "Time Remaining"
      value_template: >
        {% if is_state('switch.drip_system_zone', 'on') %}
          {{ [ (states('input_number.zone_2_run_time')|int - (as_timestamp(now()) - as_timestamp(states.switch.drip_system_zone.last_changed))/60)|round(0) ,0 ] | max }}
        {% else %}
          0
        {% endif %}
      unit_of_measurement: "min"

It does not shut off if the time remaining goes below the set 1 minute. just keeps on keeping on.

Anyone know whats going on here?

Screenshot_20201009_153219

Sorry, I was not intending to offend anyone. I was trying to explain why I keep asking for examples. Lamenting the loss of the old way is not quite sufficient to establish that the new way does not work.

Anyway, I agree that we are going nowhere so I will leave this thread as well.

1 Like