WTH can't I tag entities and use tags queries for service calls?

It is trivial to turn every entity in a particular domain off. E.G:

service: light.turn_off
data:
  transition: 30
target:
  entity_id: all

This ends up being super helpful but only if your intention was to turn everything off.

But what if I want to keep some lights on? To make my use case explicit:

  • I have some grow lights that are on their own schedule that really should not be disrupted… even if they’re off for just a second and some other automation turns them back on.
  • I have several multi-button light switches switches that have LEDs for each button. The LEDs come on at sunset and should remain on until sunrise even when my “going to bed now” automation turns the rest of the lights off. The LEDs are exposed to HA because I have other automations that indicates state by doing things with the LEDs (e.g. the light switch that turns the garage lights on/off will have the leds blink if the garage door has been open more than $duration) so I can’t reasonably “hide” the leds from HA by setting them to internal (the light switches use ESPHome <3).

Right now, the best “solution” is to build and maintain scenes or groups but this can quickly get untenable. You might end up adding each new device/entity to multiple groups.

The far better way would be to support some basic / templatable filtering based on device attributes.
This way you’d end up with something that looks like this:

service: light.turn_off
data:
  transition: 30
target:
  entity_id: all
  exclude_ids: >-
    {{ expand(states.light)  | selectattr("light_type", 'in' ['ambient', 'signage', 'garden_grow']) | join(', ') }}
  

Note: where is the selectattr() function documented? I had to guess about the supported args based on a few community posts!

But, essentially, this turns the “all lights off” statement into “all lights off except those that have an attribute called light_type with a value of ambient or signage or garden …”.

And ideally ESPHome would support adding arbitrary attributes to an entity so the amount of work required to add a new light to the exceptions list is minimal but that’s a different ask in a different thread; I would be OK with making one change to customize.yaml for each new light entity to make this work. That is certainly less than creating multiple groups (light.ambient and light.signage and light.garden_grow … etc) and having to manually add each new entity to the appropriate group(s).


This issue has been requested before, but I figured I’d bring it up again because there’s been a variety of proposed workarounds that have varying degrees of success.

One of the more ‘scalable’ solutions that comes up is something like this:

  - service: light.turn_off
    data_template:
      entity_id: >-
        {% set domain = 'light' %} 
        {% set state = 'on' %}    
        {% set compare = 'eq' %}    
        {% set filter =['nothing to exclude'] %} 
        
        {% set MyVal= states[domain] | selectattr('state',compare, state) 
        |  rejectattr('entity_id','in', filter) 
        |  rejectattr('attributes.is_hue_group', '==' , true) 
        |  map(attribute='entity_id') 
        | join(',') 
        %}

        {% if MyVal.count('.') >=1 %}
        {{ MyVal }}
        {% else %}
        light.dummy
        {% endif %}

Which is a more involved version of the above proposed solution because this builds an inclusive list rather than what would effectively be a entities_to_manipulate = set(allEntitiesInDomain) - set(entitiesFromTemplate) call.

TL;DR: Give use the ability to ‘tag’ entities for automatic and scalable configuration of entities. E.G.: I would like to turn off all lights except those tagged with ambient OR signage OR garden_grow. The ‘tags’ are essentially device attributes.

EDIT: I have created a feature request on the ESPhome side of things, too. Hopefully that provides a bit more context and shows how the capabilities in HA / ESPHome might be used to work together: Provide arbitrary attributes for entities 'announced' to HA via MQTT · Issue #1903 · esphome/feature-requests · GitHub

Why even bother with 2 fields when you can already do this with templates in the entity_id field.

service: light.turn_off
data:
  transition: 30
target:
  entity_id: >
    {{ expand(states.light) | rejectattr('entity_id', 'search', 'ambient|signage|garden_grow') | list }}

Its jinja syntax. It’s in the jinja documents which are linked in the template section.

Why even bother with 2 fields when you can already do this with templates in the entity_id field.

The two distinct fields are there to make it super clear what the ask was; set($all) - set($few).
A single field would be sufficient if there was a template expression that would suffice but part of the ask was for a more polished (read: intuitive) way of doing the subtraction/filter.

Similar to how the plumbing/iteration over some structure is part of stdlib and all I need to do is provide the ranking function to get the sorted results.


With respect to this query:

{{ expand(states.light) | rejectattr('entity_id', 'search', 'ambient|signage|garden_grow') | list }}

I’m a bit lost as to how/if this would work.
Reading between the lines, I’m getting an “any entity_id that contains any of these strings will be excluded” vibe or an “any entity_id that has any of these strings in any of its attributes will be excluded” vibe … and both/either are distinctly not what i’m asking for.


Without complicating things with a non-standard attributes (light_type for example) an equivalent task would be: Turn all lights off except those that support the Strobe effect.

Generating the entity IDs that support this effect:

{{ expand(states.light) | selectattr('attributes.effect_list', 'search', 'None|Strobe') | list }}

Yields

UndefinedError: 'homeassistant.util.read_only_dict.ReadOnlyDict object' has no attribute 'effect_list'

Changing the comparison:

{{ expand(states.light) | selectattr('attributes.effect_list', 'defined') | list | count }}

Yields a reasonable result but I am struggling to chain the {select,reject}attr() functions together.

The less elegant way is sufficient

{% for d in expand(states.light) | selectattr('attributes.effect_list', 'defined') | list -%}
  {%if 'Strobe' in d.attributes.effect_list %}
  {{d.entity_id}}
  {%endif%}
{%- endfor %}

Yields the expected result and can be further massaged into a format that data_template.entity_id should be able to accept.

I respectfully disagree. As soon as you add templates into the mix, you’re going to get glazed over eyes and people drooling. Adding a second field that says ‘all’ and filtering based on another list isn’t going to stop the drooling or make it any easier. As soon as we dive into filtering based on attributes, we are templating or handling complex yaml. Both unappetizing solutions that can’t be solved with glob or any ‘short’ syntax. It’s always going to be ugly and hard. No point in separating it into multiple hard to understand fields.

That 100% works, just might not do what you want. I based it off what I thought you were attempting to do before. However you had made up attributes, so I was guessing. It creates a list of lights that don’t have ambient, signage, or garden_grow in the entity id.

{{ states.light | map(attribute='entity_id') | reject('in', states.light | selectattr('attributes.effect_list', 'defined') | selectattr('attributes.effect_list', 'search', 'Strobe') | map(attribute='entity_id') | list) | list }}

or if you want to read it easier

{% set remove = states.light 
              | selectattr('attributes.effect_list', 'defined') 
              | selectattr('attributes.effect_list', 'search', 'Strobe') 
              | map(attribute='entity_id') | list %}
{{ states.light | map(attribute='entity_id') | reject('in', remove) | list }}

EDIT: So back to your original request now that I understand it

{% set remove = states.light 
              | selectattr('attributes.effect_list', 'defined') 
              | selectattr('attributes.effect_list', 'search', 'ambient|signage|garden_grow') 
              | map(attribute='entity_id') | list %}
{{ states.light | map(attribute='entity_id') | reject('in', remove) | list }}

Assuming thats in the effects_list attribute.


Also, that template of yours can be simplified a bit in your original post

        {% set domain = 'light' %} 
        {% set state = 'on' %}    
        {% set compare = 'eq' %}    
        {% set filter =['nothing to exclude'] %}
        {{ states[domain] | selectattr('state',compare, state) 
                          |  rejectattr('entity_id','in', filter) 
                          |  rejectattr('attributes.is_hue_group', '==' , true) 
                          |  map(attribute='entity_id') 
                          | list }}

and if you want to rid yourself of light.dummy…

- variables:
    my_entities: >
        {% set domain = 'light' %} 
        {% set state = 'on' %}    
        {% set compare = 'eq' %}    
        {% set filter =[] %}
        {{ states[domain] | selectattr('state', compare, state) 
                          | rejectattr('entity_id','in', filter) 
                          | rejectattr('attributes.is_hue_group', '==' , true) 
                          | map(attribute='entity_id') 
                          | list }}
- choose:
  - condition: "{{ my_entities | length > 0 }}"
    sequence:
    - service: light.turn_off
      target:
        entity_id: "{{ my_entities }}"
1 Like

No, we’re very much on the same page here. I’ll re-iterate

part of the ask was for a more polished (read: intuitive) way of doing the subtraction/filter.

This thread probably should have been titled “why the hell does HA not allow me to tag things”?

I was using attributes because they more or less are a superset of what people normally think of as ‘tags’… even if working with attributes isn’t quite as straightforward as working with tags is on most other platforms.

I - like many of the people that have asked versions of this question before on these threads - was hoping to avoid doing this:

service: light.turn_off
target:
  entity_id: light.a_manually_curated_group_of_all_the_primary_lights

Typically, the problem statement is something like “I want to turn off all the lights except the one in the bathroom and the nightlight in the hall”.

Logically, the algorithm is simple. Make a list of all the lights. Then subtract the two lights of interest.

This feels like it should be easy enough since Home Assistant already supports the shortcut all for entity ID.

Since HA already has a shorthand for specifying “all the entities in a domain” it feels like there should be a shorthand to say “except for …”

The ideal would be to not have to do this:

{% set remove = states.light 
              | selectattr('attributes.effect_list', 'defined') 
              | selectattr('attributes.effect_list', 'search', 'Strobe') 
              | map(attribute='entity_id') | list %}
{{ states.light | map(attribute='entity_id') | reject('in', remove) | list }}

And instead something like this:

service: light.turn_off
target:
  - tags: 
     - lightType: 
         # If I had to maintain `light.a_manually_curated_group_of_all_the_primary_lights`, this is what would be in it
         # But thankfully I don't need to maintain that in this ideal dream world!
         - contains: ['regular', 'exterior']

Assuming that the dozens of ESPHome light switches I have could be updated to advertise an attribute lightType, I would only need to set the classification once in the template that I use for the firmware builds.

All future light switches would seamlessly work with the automation and no manually curated groups!

You would see similar benefits on the UI side of things; an entities card that could be dynamic based on tags. As soon as a new light gets auto discovered, it’ll get added to the entities card for ‘exterior lights’ because the entities card is looking for that specific tag set.

Suggesting this:

service: light.turn_off
data:
  transition: 30
target:
  entity_id: all
  exclude_ids: >-
    {{ expand(states.light)  | selectattr("light_type", 'in' ['ambient', 'signage', 'garden_grow']) | join(', ') }}

was a compromise of sorts.

Home Assistant does not have tags as a first class citizen (yet?) but it does have templates and it does have the all shorthand.
Baking in support for subtracting something from all seemed like the cleanest approach; the user is still going to be writing a template but does not need to concern themselves with how subtracting two lists should work. As long as the user knows the all shorthand and can figure out a template that returns the IDs for the few lights they wish to treat differently, they’ll get the desired result.

But yeah. I guess if you’re already writing out a template, may as well do all the work yourself.


Thanks for your help w/ chaining the two selectattr() calls. I wasn’t groking what the pipe was feeding from one to the other.

Personally I would love to have something as simple as this, where tags could just be used in placed of an entity

service: light.turn_off
data:
  transition: 30
target:
  entity_id: tag.first_floor
  exclude_id:
   - tag.kitchen_island_lights
   - tag.kitchen_counter
   - light.floor_lamp_living_room

expanded on this concept could be the possibility to easy subtract or add tag results in templates, like this.

{{ tag.first_floor - tag.kitchen_counter }}
1 Like

You can sorta do this with expand(light.someGroupOfLights) but that means you’re back to managing groups and leveraging those “eye glazing, drool inducing” templates that @petro is such a huge fan of. /s.

Your proposed syntax was along the lines of what I was going for and is precisely why I think the “expand, then subtract” function should be baked in so the user only needs to supply the set(all) and set(few) expressions… but in the mean time @petro has provided a working ‘full’ template expression that will do the expand/filter/subtract work which does yield working - if not ideal - results.


The underlying motivation was to make this automagic; the user should declare a policy like “at 11 PM, all lights downstairs and all non-ambient lights upstairs should be turned off”.

As devices announce themselves to HA, they should be able to include ‘hints’ that tell HA if any existing policies should apply to the new entities as well. Doing this manually just seems like a missed opportunity to use computers for what they’re good at.

Tags just seemed like a more flexible way to implement those hints and attributes already exist so leveraging that seemed like the “lest amount of new code to introduce” path.

1 Like

If and whenever the following PR is implemented, it might provide the basis for what you’re requesting:

Add label support by frenck · Pull Request #69996 · home-assistant/core · GitHub

If I understood it correctly, it would allow you to assign one or more “labels” to an entity. Afterwards you can select/group entities based on their labels.

FWIW, I already do this with custom attributes. The only inconvenience is that you can’t add a custom attribute to an entity via the UI but only via Manual Customization (i.e. via YAML).

oh, this is interesting, If I’m reading/understanding this correctly it seems to cover everything except the exclude which makes sense since that is not really a part of the ‘label’.

So, the call would be more or less the same as the idea above (without the exclude)

service: light.turn_off
data:
  transition: 30
target:
  entity_id: label.first_floor_lights

and including them for entities seems to be as easy as writing

switch:
  - platform: template
    switches:
      kitchen_cabinet_light:
        value_template: "{{ is_state('sensor.cabinet_light', 'on') }}"
      labels:
        - label.kitchen_cabinet_lights
        - label.first_floor_lights

Oh cool / thanks for the link. subscribed :D.

That’s more or less what I was thinking of (w/r/t general use cases).

PS: The name “labels” is chosen instead of “tags”. This is prevent naming conflicts/confusion with the existing “tags”.

Ah, yeah… you know, that’d be why I wasn’t finding a ton of productive/helpful stuff.

Yep, this is what I’ve ended up doing as well for most of the devices on my network. A few devices run firmware that gives me direct/easy control over the JSON that is announced to HA so it was trivial to add in custom attributes for them.

If PR #69996 goes through then I would imagine that ESPHome support is not that far behind.