Template filtering of entities via the new labels feature - howto?

[bulkmail (Richard Gaywood)]

The docs say

Each of the label template functions can also be used as a filter

but I’ll be jiggered if can work out how and the docs don’t seem to say. I’m trying to build a template that will output the list of all the sensors that have stale values (usually because something has fallen of my zigbee network) and also have a specific label (because I don’t want to monitor everything, just some things I care about.)

So far, I have this, which can output all the stale sensors without the label filter (but has many things I do not want)

{{ states.sensor 
   | selectattr('last_changed', 'lt', now()-timedelta(seconds=3600)) 
   | map(attribute='entity_id') 
   | list }}

I feel like there must be some syntax where I can do something like | select(label("monitored")) in here, but I can’t figure it out.

It may also be possible to go the other way, start with a list of entity IDs and then filter down to those in the sensor domain:

{{ label_entities("monitored")   
   | list
}}

…but I am not sure how to do that either. I don’t know how to apply a filter to a list of entities that will only select ones in a certain domain.

(I know there’s other ways to solve the problem too, like I could have a special label just for sensors I want to monitor, or I could put them in a group or something. But I want to use this to learn about templating.)

---

EDIT TO ADD SOLUTIONS, FULL DETAILS BELOW

This works, but is inefficient, because it’s iterating over ever sensor you have:

{{ states.sensor 
   | selectattr('last_changed', 'lt', now()-timedelta(seconds=3600)) 
   | map(attribute='entity_id') 
   | select('in', label_entities("monitored"))
   | list }}

This is the better solution:

{{
  label_entities('monitored') 
    | select('match', 'sensor')
    | expand()
    | selectattr('last_changed', 'lt', now()-timedelta(seconds=3600)) 
    | map(attribute='entity_id') 
    | list
}}

[Didgeridrew]

https://www.home-assistant.io/docs/configuration/templating/#labels

https://jinja.palletsprojects.com/en/latest/templates/#builtin-tests

Or split the difference and use expand on the entity list.

{{ label_entities("monitored") | expand
| selectattr('last_changed', 'lt', now()-timedelta(seconds=3600)) 
| map(attribute='entity_id') | list }}

It’s more efficient to start from the smaller list of just the labelled entities instead of all sensors.

[bulkmail (Richard Gaywood)]

I mean, I know people not RTFMing is a real issue, and I get where you’re coming from. But at the same time, I both linked to and quoted that exact docs link, and I don’t see how it answers my question…? Any chance of maybe just one or two extra breadcrumbs here to help me join the dots?

EDIT - aha, you have ninja’d a code snippet since I wrote this, let me try that!

[Didgeridrew]

@bulkmail

I feel like there must be some syntax where I can do something like | select(label("monitored")) in here, but I can’t figure it out.

You use the in test if you’re doing that way… hence the link to Tests

The exact filter will depend on where you place it.

{{ states.sensor
| selectattr('entity_id', 'in', label_entities('monitored')) 
| selectattr('last_changed', 'lt', now()-timedelta(seconds=3600)) 
| map(attribute='entity_id') 
| list }}

or

{{ states.sensor
| selectattr('last_changed', 'lt', now()-timedelta(seconds=3600)) 
| map(attribute='entity_id')
| select('in', label_entities('monitored')) 
| list }}

[bulkmail (Richard Gaywood)]

Aahhhh thank you, now that is what I needed.

Recap for the record / people searching in future: what worked for me is

{{ states.sensor 
   | selectattr('last_changed', 'lt', now()-timedelta(seconds=3600)) 
   | map(attribute='entity_id') 
   | select('in', label_entities("monitored"))
   | list }}

In other words –

• get all sensors;
• filter to ones that haven’t changed their value in the last hour;
• change the data we’re working with from sensor names to entity IDs;
• filter down to only the entity IDs in the “monitored” label;
• and output this as a list.

Thanks again!

…and this has immediately been useful, because it just told me my sensor.truenas_max_hdd_temp template sensor is somehow broken, or my truenas server monitoring integration has died again. Which I didn’t know.

[123 (Taras)]

@bulkmail

do something like | select(label("monitored"))
…
I don’t know how to apply a filter to a list of entities that will only select ones in a certain domain.

There’s no need to filter a big list (all sensors) into a small list (only sensors with a monitored label) when label_entities('monitored') already gives you the small list.

Unless of course you have also labeled non-sensor entities with monitored in which case you can use a select with match to exclude them:

label_entities('monitored') | select('match', 'sensor')

So if your template starts by filtering states.sensor to get the ones with a desired label, that’s not the most efficient approach.

---

@bulkmail

In other words –

• get all sensors; <------ No
• filter to ones that haven’t changed their value in the last hour;
• change the data we’re working with from sensor names to entity IDs;
• filter down to only the entity IDs in the “monitored” label;
• and output this as a list.

Start with just the entities that have the monitored label, not all sensor entities. For your application, there’s no need to start with all sensors, plus it will cause the template to be subjected to Rate Limiting.

[bulkmail (Richard Gaywood)]

Ahhh thank you for that point! It had occurred to me that the solution above (while functional) was probably inefficient, but I neglected to followup.

For anyone reading this, I now have this code, that seems to work and be performant:

{{
  label_entities('monitored') 
    | select('match', 'sensor')
    | expand()
    | selectattr('last_changed', 'lt', now()-timedelta(seconds=3600)) 
    | map(attribute='entity_id') 
    | list
}}

One thing I’m not clear on exactly, if you can indulge me. While iterating to this solution, I briefly had

{{
  label_entities('monitored') 
    | select('match', 'sensor')
    | list
}}

At this point, if I am reading the Jinja correctly, the data being output is just a list of strings, each of which is an entity_id. It took me a minute to figure out that the next step I needed was expand() which would turn the list of strings into – I think – a list of entity objects that I could then call selectattr on.

Was expand() the right way to do this? It seems a little counter-intuitive to use it to map from strings to objects. The docs define it as being used to “…sort entities and expand groups.” Which is also more intuitive, based on the word “expand”.

Also – when debugging templates – is there any way to make the Developer Tools page output an object rather than a string? It’d be useful to see (say) which attributes exist on whatever object I am manipulating. But if I omit | list, I typcially get output like <generator object select_or_reject at 0x7ff7f75056c0>. It’d be really handy if I could get a full copy of each object there. Is there some other filter or map I can pass it through to do so?

[123 (Taras)]

You need to use expand if you want to convert an entity’s entity_id to its state object. You want the entity’s state object if you intend to reference its properties.

@bulkmail

if I omit | list, I typcially get output like <generator object

A generator is iterable but that’s how it’s displayed in the Template Editor. The result of some fiters is a list while for many others it’s a generator. For display purposes, you’ll need to convert a generator to a list.

[bulkmail (Richard Gaywood)]

@123

You need to use expand if you want to convert an entity’s entity_id to its state object. You want the entity’s state object if you intend to reference its properties.

That’s what I eventually realised, but as someone who’s (a) done a lot of coding but is (b) very new to the HA flavour of Jinja, I didn’t find it obvious that it would work that way. (In my head, the concept I was reaching for would be something like – in JS, for example – listOfIds.map(id => stateObjects.get(id)). But I dunno, maybe it’s just me, but the word “expand” doesn’t describe that very well.

What seems to be the other use of expand, to remap a group entity to the entities it contains – that makes more sense to me.

Do you think it’s worth a documentation PR to call that out more explicitly?

[123 (Taras)]

If you believe you can make its description clearer, submit a PR in the documentation repo and the dev team will decide if it should be merged.

[TheFes]

If you don’t use map(attriubite='entity_id') you will get the full state objects as output, and not only the enitty_ids

BTW, In the past there were issues with expand being used as filter. I’m not sure if that’s solved already.

It seems to work in devtools > templates, but it didn’t work when actually used in the config. To avoid that I always use it as a function.

[123 (Taras)]

It works as a filter now (in Template entities). I don’t know who fixed it or when. I had reported it as an Issue in 2022 but no one investigated it. I kept it open for over a year and then let it auto-close.

I don’t recall seeing the fix explicitly mentioned in Release Notes but it may be buried in the hundreds of small PRs. Anyway, glad it’s finally fixed.

[petro]

@TheFes

I’m not sure if that’s solved already.

It has

@123

I don’t know who fixed it or when

A refactor inadvertently fixed this, and many other issues with filters / functions that people didn’t notice.

[TheFes]

Good to know, that can improve readability and save on usage of parenthesis