yes probably so. I did go through and remove all the entity_id: like a week ago.
Without mitigating its removal? I’d warn you that’s a recipe for disaster but you already discovered it empirically:
I recommend you review every entity where you removed entity_id but overlooked to compensate for it. Those entities are now possibility compromised. See the first post of this thread for guidance.
1 Like
Lol. Yes that is a good idea and I did look over most of it and had to make a few changes such as adding the time or date sensor to a few others last week. Looking back now, I actually just commented out all the entity_id so I must not have had it in that template and it appears that the only other semi important stuff that might cause issues is the vacuum. but so far everything else has seemed to be okay except for this.
Great advice though! thanks.
hmmm, that’s not it, it’s looking forward and trying to get User control over updating, instead of having a brute force machine always update on state changes, or, as is the case now in 116, at a fixed rate of once per minute. That might even be too frequently, when it needs to be only once a day…
have a look here, we’re back to creating groups Template sensor stopped working on 0.116 · Issue #41582 · home-assistant/core · GitHub which is fine of course. But a long time ago, we were told to have as few groups as possible, because that would only create overhead in the state machine, updating on all states in those groups.
Seems that for all previous template sensors with entity_id: we now best create dedicated groups and use the expand feature? That would at least cover the entity limiting control we need.
If we could find a way to have these {{states|…}} template update only on request, and not each minute, we would have frequency control again.
wouldn’t that be the feature request we need now: have templates with {{states|...}} not update at all, adding a {% set trigger = states('sensor.time/date') %} to it to update automatically, or, don’t add anything, and create an automation/script with the timing and update.
update
this seems to have had a followup:
listing using expand(), with all single entities , where we previously used entity_id:
bit of a detour imho. and way less elegant than (sorry, dont want to look back too much now) entity_id: …
need some help on python here, as I’ve previously used a group.all_lights_only with all lights listed individually and use that like this
group = 'group.all_lights_only'
fname = 'Lights'
on_format = 'Lights on: {} -- {}'
off_format = 'No lights on since '
filter = 'on'
pic = 'lamp_off|lamp'
icon = 'mdi:lightbulb'
# min_show = 0
count = 0
desc = ''
for entity_id in hass.states.get(group).attributes['entity_id']:
state = hass.states.get(entity_id)
etcetc
to count and show the individual lights being ‘on’.
now with the nested groups being
all_lights_only:
name: All lights only
icon: mdi:lightbulb-outline
entities:
- group.all_inside_lights
- group.all_outside_lights
all_inside_lights:
name: All inside lights
icon: mdi:lightbulb-outline
entities:
- group.main_inside_lights
- group.guest_inside_lights
- group.living_ceiling_spots
all_outside_lights:
name: All outside lights
icon: mdi:flood-light
entities:
- group.outside_flood_lights
- group.terrace_outdoors_spots
etc
etc
this python script sees the group.all_inside_lights as being on, and displays and counts that group,(which is correct, as it is the entity_id…) instead of counting and listing the individual lights in that group.
Please help me change this python script to again iterate over these nested groups?
thanks!
(sorry if this is not exactly on topic, but it a follow-through of the new templated options using expand() in HA…)
and to be back on topic, I wonder if this could be more compact now:
sequence:
service: light.turn_off
data:
entity_id: >
{% set ns = namespace(group=[]) %}
{% if is_state('group.guests','not_home') %}
{% set ns.group = expand('group.all_inside_lights') %}
{% else %}
{% set ns.group = expand('group.main_inside_lights' , 'group.living_ceiling_spots') %}
{% endif %}
{% set lights_on =
ns.group|selectattr('state','eq','on')|map(attribute='entity_id')|list %}
{{lights_on|join(', ') if lights_on|length > 0 else 'none'}}
It’s off-topic (python code to emulate expand) but maybe someone else can help.
It’s choosing between which group(s) to use with an if-else. I suppose you could use an inline if to shorten it a bit. You don’t need to use a namespace with an if-else so that eliminates one line. 
Note that
expand(...) |
is not a substitute for
entity_id: ...
Rather, it is an alternative to the filtering:
states | selectattr('entity_id', 'in', [...]) |
To me it looks like a shortcut more than a detour.
sure, I understand that.
it’s just that the expand() function limits the updating of the template to the entities in the group that is expanded, while the states | filtering created listeners on all entities.
limiting the entities listened to is the important thing here, so that’s why I ‘compared’ it to the former entity_id:
so yeah, shortcut or detour, suppose getting there is most important 
you’ve got to admit, it’s not the most obvious way of doing things though.
hear you, will post in a dedicated thread, sorry.
not really sire what you mean with this? Are you saying I should template it all inside the template itself? Isnt the beauty of this, one creates a namespace, and handles that with the filter outside the template? Or do you mean something else.
You don’t need to use namespace.
{% if is_state('group.guests','not_home') %}
{% set x = expand('group.all_inside_lights') %}
{% else %}
{% set x = expand('group.main_inside_lights' , 'group.living_ceiling_spots') %}
{% endif %}
{% set lights_on = x | selectattr('state','eq','on')|map(attribute='entity_id')|list %}
{{lights_on|join(', ') if lights_on|length > 0 else 'none'}}
3 Likes
thanks!. you’re right, now why I had that I can’t remember… good catch.
since we’re talking efficiency too here, I wonder if there’s any preference using either of these:
{%- set ns = namespace(below=0) %}
{%- for s in expand('group.battery_sensors')
if s.state|int < states('input_number.battery_alert_level')|int %}
{%- set ns.below = ns.below + 1 %}
{%- endfor %}
{{ns.below}}
{% set alert_level = states('input_number.battery_alert_level')|int %}
{{expand('group.battery_sensors')
|map(attribute='state')|map('int')
|select('<',alert_level)
|list|count}}
would creating the namespace be more costly for the system?
since you asked for examples of templating frequency needs, please have a look here
Have a couple of situations at hand where I do need an update per second. Of course, without trying to overload the full system with that.
What I always wanted to know, but was afraid to ask… can’t we sense the systems heartbeat, ie have a sensor.second or sensor.heartbeat. Would be cool if that was supplied via core, so not causing any extra processor %, and use that in a template using now(). On a single occasion, where once a minute is way behind the needs for that single purpose.
thanks for having a look
Rate limit is dropped to 1s from 1 minute on {{ states.domain }} templates after bringing up this discussion.
FYI
Appreciate the tag, but I’ve got to be honest that this is getting really confusing. People are putting a lot of work into rewriting vast swathes of their configuration only for the goalposts to move again a couple of weeks later.
I can’t even tell now whether my templates are going to do what I want, whether they will update on a state change, every second or every minute, which ones I need to manually override with an automation to speed them up etc.
This puts me in a position where I basically have to wait for something to fail before I realise it needs fixing, which is the opposite of the proactive approach I usually like to take.
4 Likes
Good discussion.
FWIW, although the proposal to implement glob is appealing, the examples you provided illustrate a point I made elsewhere: an entity’s object_id shouldn’t be used for storing meta-data. Custom attributes are best suited for meta-data.
For example, by including the word “energy” in each entity’s name, it becomes a challenge to create a template that selects all energy-related entities. The proposal is to support glob so that the template is simplified to:
{{ new_glob('sensor.node_*_energy') | map(attribute='state') | map('float') | sum }}
where new_glob('sensor.node_*_energy') is the new feature.
Although I like the proposal, it encourages the practice of using the entity’s name (object_id) to store meta-data. I don’t think that’s a ‘best practice’.
What can be achieved now is to simply create a custom attribute for each energy-related entity, such as type: energy. In other words, instead of creating long-winded entity names, loaded with meta-data, keep them short and put the meta-data in custom attributes. Then the template can be:
{{ states.sensor | selectattr('attributes.type', 'eq', 'energy') | map(attribute='state') | map('float') | sum }}
Multiple attributes (standard and custom) can be employed to make it highly selective:
{{ states.sensor
| selectattr('attributes.device_class', 'eq', 'power')
| selectattr('attributes.location', 'eq', 'garage')
| map(attribute='state')
| map('float') | sum }}
It’s a throttle. When this PR is accepted, you can have at most 1 update per second when using states. The current implementation is at most 1 update per minute.
Well, that thread/issue was my compromise to the issue that @finity and @anon43302295 brought up: states.domain/states are throttled to update only once a minute. I was also trying to get a 2 birds one stone by ninja’ing in a poor mans "contains test for selectattr" through this new_glob method.
I appreciate the effort to rectify the throttling problem, I don’t think it ultimately fixes the overall issue.
as of before we had two choices - update when a monitored state changed or at most every minute.
Now we have two choices - update when a monitored state changed or at most every second.
What happens if we want a sensor to only update once a minute or once a day?
There’s no way to control that at all.
Even if you set up an automation to update the sensor at the desired periodicity the sensor will still update itself whenever one of the monitored entities change. If that happens before the automation triggers then it will update then in addition to when the automation triggers it to update.
Not to mention that even if the automation control worked as desired (which it won’t) then that still requires you to create an additional automation to control the update when before it was as simple as adding one line of code to the sensor config.
the change added (unforeseen?) complexity and minimized user control.
I still don’t see why the additional user controls were nixed especially when the work was already done to implement the functionality.
I wish someone could give a logical rationale for that decision besides “because I said so”.
the rationale is because the users have to specify an entity_id with every template for all integrations that need it. If the control done through the single template, the second field isn’t needed. Right now, there’s a ton of templates that do not have the entity_id control (prior to it’s deprecation). Those places all cannot update properly. The main goal of the change was to remove the need for entity_id while simultaneously providing a solution for all template fields.
The reason I received was that restoring entity_id would only address Template-based entities and not Template Triggers. Although I felt the urge to say “Yeah but that was the status quo up until 0.115!”, I didn’t. It’s clear to me that the driving force is to invent a comprehensive solution (wherever templates are used).
I should point out that this was, for me, an unsatisfactory reason for eliminating entity_id. I still believe it was removed with good intentions (simplification) but without a full appreciation of the consequences (loss of functionality).
2 Likes