Unleash the power of 'expand' for Template Sensors

[123 (Taras)]

Request

Home Assistant currently limits the usefulness of expand when used in Template Sensors.
As opposed to how it currently works, it should expand the group first and then create listeners for each entity in the group.

---

I believe the code responsible for identifying entities within a template is here. It’s a complex regex pattern that (if I understand it correctly) creates a capturing group containing all entities requiring listeners.

The regex pattern would need to be enhanced so that it recognizes the presence of expand() and then proceeds to extract its members. It’s a non-trivial challenge because determining a group’s membership is well beyond what the regex pattern currently does.

---

Background

The expand function reveals a group’s members. For example, this template:

• expands the my_lights group
• select all of the group’s entities that are on
• displays the result as a list of State objects.

{{ expand('group.my_lights') | selectattr('state', 'eq', 'on') | list }}

It’s the first step, expanding the group, that is so very useful because it simplifies code maintenance (i.e. makes your life easier). It lets you define a group then expand that group in automations, scripts, etc. If you add a new entity to the group, you do not have to revise all the automations, scripts, etc where the group was used. Excellent! Unfortunately, this does not currently work properly with a Template Sensor (or any other Template-based integration).

At startup, Home Assistant examines all Template Sensors and identifies the entities that must be monitored. Unfortunately, the only entities it will detect in the previous template are group.my_lights no entities at all. It will not expand the group and identify the entities within the group.

The current workaround is to list all of the group’s members in the entity_id option.

  - platform: template
    sensors:
      all_lights:
        friendly_name: 'Lights'
        entity_id:
          - light.kitchen
          - light.foyer
          - light.garage
          - light.hallway
          - light.bathroom
          - light.porch
          - light.patio
        value_template: >-
          {{ expand('group.my_lights') | selectattr('state', 'eq', 'on') | list | count }}

However, this workaround loses the advantage of having created a group because now the group’s members have to be repeated in entity_id. Effectively, you’ve defined the group in more than one place.

Ideally, when Home Assistant encounters expand('group.whatever') in a Template Sensor, it should expand the group and create listeners for each entity within the group (and not a listener for the group itself, as it currently does).

---

EDIT
Correction. As per pnbruckner’s explanation below, I’ve now indicated that the result of the template above is a list of State objects (i.e. each group member is represented by its full State object, not simply its entity_id). To get a comma-delimited list of entity_ids you would need to use a few more filters in the template:

[pnbruckner (Phil)]

@123

the only entities it will detect in the previous template is group.my_lights

Actually, it won’t even find group.my_lights in this case because of the regular expression it uses to extract entity_id’s from templates.

I totally agree. If the regular expression was enhanced it could find the group entity_id. And then if the function that uses the regular expression was enhanced to recognize groups and expand them, then I think the suggestion could be realized.

EDIT: FWIW, I looked into it a bit more and realized it would be a bit more involved. Still a good idea, just not as easy to implement as I originally assumed.

[klogg]

I’m going to vote for this as it is eminently sensible but before I do ….

I have learnt something new here i.e. expand.
However I might have misunderstood… When I do this:

{{ expand('group.all_window_sensors') | selectattr('state', 'eq', 'on') | list }}

I get this:

[<template state binary_sensor.shower_room_window=on; friendly_name=Shower Room Window, device_class=window @ 2019-09-12T22:12:03.178337+01:00>]

Which is right in the sense that that window is open but I thought it was supposed to give a list like this:

[binary_sensor.shower_room_window, another binary_sensor as appropriate]

Which would indeed be very useful (notwithstanding this feature request!).

[pnbruckner (Phil)]

With the template you wrote you’re getting a list of State Objects. (The expand function returns a list of State Objects, not entity_id’s.) Try:

{{ expand('group.all_window_sensors') | selectattr('state', 'eq', 'on') | map(attribute='entity_id') | list }}

[123 (Taras)]

What Phil said; the template as shown produces State objects, not entity_ids. In my haste to write the Feature Request, I provided an incomplete template. Sorry about that. I’ll fix the first post.

[123 (Taras)]

Good news!

The following PR intends to provide the functionality proposed by this Feature Request.

https://github.com/home-assistant/core/pull/35398

The PR will have a significant positive impact on how Template Sensors can be configured. The example I posted above can be reduced to this:

  - platform: template
    sensors:
      all_lights:
        friendly_name: 'Lights'
        value_template: >-
          {{ expand('group.my_lights') | selectattr('state', 'eq', 'on') | list | count }}

Any changes to the membership of the group will be handled by the value_template (although possibility only after a restart).

[inthehoose]

Great work @123! Will your PR make it in the next release (0.110)?

[123 (Taras)]

I had no part in the PR. All the credit goes to Tho85.

The PR was merged into the home-assistant:dev branch. I don’t know if it will appear in 0.110.

[123 (Taras)]

Update:

The blog post for 0.110.0 indicates that the PR was merged into the release. However, I tested it, using the example provided in the PR, and it did not work as advertized.

I have reported it as an Issue in the Core repo.

[ricreis394 (Ricardo Reis)]

I have a solution for not having to include every single entity_id, just create an automation that triggers when a light changes it’s state and force the update of the sensor
like this:

automation:
  - alias: current_lights_on
    initial_state: true
    trigger:
      - platform: event
        event_type: call_service
        event_data:
          domain: light
    action:
      - service: homeassistant.update_entity
        entity_id: sensor.current_lights_on

[123 (Taras)]

That’s an interesting workaround but it will trigger when any light’s state changes even lights that are not part of the desired group.

The goal is to update the sensor only when one of the group’s members changes state (and not just any light entity). The PR to implement this functionality is currently awaiting review and will hopefully be incorporated in the next major release.

[Mariusthvdb (Marius)]

been using this for a long time, now parked, but still functional:

  - alias: 'State changed Light'
    id: 'State changed Light'
    initial_state: 'off'
    trigger:
      platform: event
      event_type: state_changed
    condition:
      condition: template
      value_template: >
        {{trigger.event.data.entity_id in state_attr('group.all_lights_only','entity_id')}}
    action:

and this would be possible too:

  - alias: 'Call Service Event Light'
    id: 'Call Service Event Light'
    initial_state: 'off'
    trigger:
      platform: event
      event_type: call_service
      event_data:
        domain: light
    condition:
      condition: template
      value_template: >
        {{trigger.event.data.entity_id in state_attr('group.all_inside_lights','entity_id')}}

of course change the group to your needs.

or select more groups

      value_template: >
        {% for x in
             ['family',
              'hubs_binary_pinged',
              'critical_devices_state',
              'media_player_media',
              'device_tracker_media',
              'all_lights_only',
              'iungo_switch_switches_template',
              'iungo_switch_appliances_template',
              'binary_sensors_active_template']
          if trigger.event.data.entity_id in state_attr('group.'+x,'entity_id') %} true
        {% else %} false
        {% endfor %}

[123 (Taras)]

… and when the PR is merged these additional automations will be unnecessary.

[Mariusthvdb (Marius)]

yep, wanted to type:
‘and still would very much welcome the PR’ …

but you beat me

[ricreis394 (Ricardo Reis)]

Yes, it triggers every light, but that’s not a problem at all. If a light isn’t in the group the state remain the same

[123 (Taras)]

@ricreis394

it triggers every light, but that’s not a problem at all

It’s inefficient. Given a group of 5 lights out of 30 lights, the sensor requires updating only when one of the 4 lights changes state, not when one of the 25 other lights change.

In addition, the automation monitors all call_service events and then filters for domain = light. So the automation’s trigger is evaluated for service calls beyond just the ones for lights.

[ricreis394 (Ricardo Reis)]

Totally agree, but that’s a temporary workaround until something better is released
This way I don’t need to input every entity_id which is very great. I only need to maintain the group list

[Mariusthvdb (Marius)]

@123

In addition, the automation monitors all call_service events and then filters for domain = light. So the automation’s trigger is evaluated for service calls beyond just the ones for lights

yes that is of course true, and reason I went another way with these automations. Would you expect the unleashing of ‘expand’ to truly only see to those entities, or might it under the hood do the exact same thing…as it has to expand the group first, and then add some filtering, I would think it also has some overhead?

[123 (Taras)]

It appears that the problem that prevented this feature from working correctly (when it was first introduced in version 0.110) has been resolved.

Make sure groups are initialized before template sensors

So the new functionality should appear in 0.114 and a Template Sensor like this will work correctly:

  - platform: template
    sensors:
      lights_on:
        friendly_name: 'Lights ON'
        value_template: >-
          {{ expand('group.my_lights')
             | selectattr('state', 'eq', 'on')
             | list | count }}

The lights within group.my_lights will be monitored for changes.

If you add a light to the group, you won’t have to re-visit this Template Sensor to update any entities. However, you will need to restart Home Assistant first so that it re-evaluates the Template Sensor and detects the new light in the group. Currently, there’s no “reload” service for Template Sensors, so restarting Home Assistant is the only way to re-configure them.

… and, no, executing group.reload isn’t a substitute for restarting Home Assistant. On startup, Home Assistant examines a Template Sensor’s template, identifies entities, and assigns a listener to each entity. That operation occurs on startup and not when group.reload is executed.

[dave85stmn (David Mehoves)]

Do you know if this will work as a trigger for automations? Like, trigger off of a state change from any off the group? Let’s say motion sensors on the ground floor.

Could we still use trigger.to_state.entity_id? Or would it point to the group id instead of the individuals?