Heads up! Upcoming breaking change in the Template integration

Configuration
397

And for anyone interested here is the bug report I just submitted containing the relevant info (py-spy, screenshots)

EDIT:

And because I can’t post more than three consecutive posts in a row (silly rule…) I need to edit this post to add…

I finally ran a proper test to see what the expected latency “should be”.

I created a variable to store the last 10 results of the delta-T from the time HA sees the physical relay flip till the time that the light gets turned on.

Here are the results in v114.2:

ex4

And here are the results after upgrading back to v115.5:

ex3

Like you said - it’s pretty damning.

But I thought it would be good info to answer your question about expectations.

398

so, the option to control the update frequency is gone with this: https://github.com/home-assistant/core/pull/41147#issuecomment-703250497 ?

feels a bit odd that we had a fine way to control the updating of the templates, that was taken out explicitly, and adding it back is described as feature overload ? How could an option for user control be called feature overload at all? In general, but all the more now, when the system takes over (breaking) control so abundantly, since 114 versions…

3 Likes
399

agreed.

I wonder how many people need to report problems to meet the “until we can show that we have a larger set of affected users” requirement?

Alas, since most people don’t post those kinds of problems to the discord or github but only on the forums and the devs don’t monitor the forums very often to see the users reporting stuff (as observed by the huge number of “WTH?” threads) I don’t ever see this ever getting implemented.

1 Like
400

Edit: I think I was having issues because of this coupled with my old template below. The fix was to pipe the state_attr to float instead of using it as a function. See comment for old, uncomment for new.

I wonder if this change is causing the issues I am seeing with two of my templates? Although when I try them in the web-ui they say they are listening for any changes. After boot they are stuck on unknown.

My templates are:

    house_power_generation:
     friendly_name: "House Power Generation"
     unit_of_measurement: "W"
     #value_template: "{{ float(state_attr('sensor.aeon_labs_dsb28_home_energy_meter_2nd_edition_power_3', 'power_consumption'))*2.0 }}"
     value_template: "{{ (state_attr('sensor.aeon_labs_dsb28_home_energy_meter_2nd_edition_power_3', 'power_consumption')|float)*2.0 }}"
401

That’s an interesting observation. Theoretically, there should be no difference in behavior. You may want to post that as an Issue in the Core GitHub repo.

402

Yes, Frenck appears to have squelched development of what would be a substitute for entity_id: sensor.date (and offer even more scheduling flexibility). However, although one hand taketh, the other hand giveth. He’s currently working on a PR to allow templates to return value types other than just string.

That’s a game changer. It will make it possible to dynamically generate a list (such as for rgb_color). If implemented, it will probably be a Breaking Change but the benefit will be worth the trouble (of reviewing one’s templates).

403

I still want entity_id back to reduce needless processing.

3 Likes
404

I agree but based on the response from the development side, that’s never gonna happen.

The convenience and control provided by entity_id is gone. The currently suggested substitutes are less convenient and more complex. The proposed enhancement(s) still won’t be able to do what could be done with entity_id. It seems like a stiff price to pay for improved entity detection but it appears to be acceptable to the project’s gatekeepers.

1 Like
405

What’s really puzzling is that we could have had both worlds so easily. There’s been no true explanation why the current template engine cant exist with the former and suddenly abandoned entity_id. While at the same time, devs have gone into some real trouble and effort to replace it with newly created options that could not be named the same but aimed at the same goal.

Creating true python types might be mighty indeed, but that’s rather irrelevant to this issue isn’t it? Sure, it will bring us a lot of new options. But we still won’t be able to control the updating of our templates manually.

406

True. Just trying to brighten this gloomy discussion; some good news to look forward to (hopefully with no drawbacks)

I still believe all the consequences of the template changes were not foreseen during the design phase (it happens). With full confidence in the new method’s superiority, entity_id was probably deemed to be superfluous and so it was deprecated.

Then the proverbial $hit hit the fan and affected users’ system were crippled by excessive template evaluations. Suggestions to expediently fix the problem by restoring entity_id were dismissed. The reason offered (by Bdraco) is the new method doesn’t only touch upon Template Sensors but other things as well such as Template Triggers and Template Conditions. Therefore re-establishing entity_id for Template Sensors won’t help Template Triggers and Template Conditions.

So that brings us to the present where we await 0.116 to allow us to throttle the number of evaluations but not quite as comprehensively as with entity_id. A template that employs states and requires to be updated at the start of every day will need to be either redesigned, to exclude the use of states, or refreshed using an automation (that runs once a day, at the start of the day).

407

Agreed.
only hope the latest merges I saw fly by, don’t take away these options even more.

But yes, awaiting 116 eagerly, (Bdraco did a heck of a lot more than only try to accommodate need for the throttling options), and we’ll see what happens…

In the end, if we still need it, maybe a CC including the former entity_id: would be thinkable…

408

this the right place to thumbs up the reinstatement of the entity id? hope so. thanks.

seriously though. shouldn’t be an hour of reading to reinvent what isn’t broken.

409

I guess I don’t see how this will work.

If you have any entities listed in a template or all entities (by using states…) then the sensor will still update automatically on every state change that has a listener assigned.

So yes you can force an update to be a minimum of once a day by using a once-a-day automation but you can’t cause update at a maximum of no more than once a day by any automation unless the listeners themselves update less than once a day.

Unless I’m not getting something?

1 Like
410

Yeah I phrased that poorly. Your observation is what I have been saying all along about the deprecation of entity_id (loss of control of how and what determines template refreshes). If the template contains states an automation with a Time Trigger won’t prevent the template from being refreshed when any entity changes state. The automation will have to include the template outright and be used to set the value of a sensor (like how Marius has redesigned the “unavailable entities” sensor).

However, Bdraco is one of the most prolific contributors to the project, resolving long-standing performance problems and adding many enhancements. Based on bdraco’s investigation of this particular issue, I have every confidence it will eventually be resolved as well. It may take a version or three before it happens but the end-result is likely to be better than what we had. There’s a PR in 0.116 that performs rate-limiting for templates that use states so it should reduce/prevent the performance penalty users have reported for 0.115.

2 Likes
411

Yeah, I thought that was the case.

I just wanted to make sure I wasn’t thinking about it wrong since I’ve been up all night. :sleeping:

1 Like
412

Hello,
if I correctly understand this information, I have to use a time_pattern in automation with everyminute check to remplace the “entity_id” to update the sensor information… It’s not really to facilited the configuration. Don’t you think ?

413

Does your template use states?

If it doesn’t then there’s no reason to use an automation with a Time Pattern Trigger. All you have to do is described in the first post of this thread. Basically, just add this to your template:

  {% set x = states('sensor.time') %}
414

Oh, yes, I didn’t understand ( french guy :wink: ) it’s possible to do like that. THX

415

With 0.116, the template related performance improvements should improve performance quite a bit, especially for those who were seeing templates update too frequently.

If you still are seeing a performance issue, I’ve put together a profiler integration that can help track down any remaining issues.


Which will be available in core in 0.117.

In the mean time, I’ve uploaded the integration here so it can be installed as a custom repository though HACS

1 Like
416

How do we use the data generated by Profiler to investigate potential issues?

I’ve installed and run a scan to play around with it but I don’t know what to do to interact with the files generated.