Template with "list | count" - stuck with zero "count"

Ildar_Gabdullin · March 17, 2021, 12:37am

Testing in the HA -> Developer tools -> Template.

There is a list:

{%- set Z = expand('group.battery_sensors')|map(attribute='state') %}
{{Z|list}}

This gives this output - seems to be correct:

['100', '100', '26', '76', '37', '12', '100', '47', '90', '42', '6', '100']

Now if I try to get a count, I stuck.
Here is a full code:

{%- set Z = expand('group.battery_sensors')|map(attribute='state') %}
{{Z|list}}
{{Z|list|count}}
{{Z|list}}

The output:

['100', '100', '26', '76', '37', '12', '100', '47', '90', '42', '6', '100']
0
[]

I do not get it:

Why the 1st count is 0 ?
Why the 2nd list is empty?

OK, let’s consider that I can use some variable (for example, “Z”) for one template only - i.e. this is wrong:

{%- set Z = expand('group.battery_sensors')|map(attribute='state') %}
{{Z|list}}
{{Z|list|count}}
{{Z|list}}

it should be this:

{%- set Z = expand('group.battery_sensors')|map(attribute='state') %}
{{Z|list}}
{%- set Z = expand('group.battery_sensors')|map(attribute='state') %}
{{Z|list|count}}
{%- set Z = expand('group.battery_sensors')|map(attribute='state') %}
{{Z|list}}

And yes - this does work.

But for the same logic the following code should not work:

A0:
{%- set A0 = expand('group.battery_sensors')
            |selectattr('state','eq','100')
            |sort(reverse=false,attribute='name') %}
{{A0|list}}
{{A0|list|count}}
{{A0|list|count}}

But it DOES work:

A0:
[<template TemplateState(<state sensor.battery_life360_zemfira=100; unit_of_measurement=%, friendly_name=iPhone 5S (Земфира), icon=, device_class=battery @ 2021-03-16T23:49:53.166839+03:00>)>, <template TemplateState(<state sensor.battery_life360_mama=100; unit_of_measurement=%, friendly_name=Redmi 7A (мама), icon=, device_class=battery @ 2021-03-17T02:30:37.428513+03:00>)>, <template TemplateState(<state sensor.battery_cleargrass_1=100; unit_of_measurement=%, friendly_name=Xiaomi CG-1, icon=, device_class=battery @ 2021-03-16T16:15:56.819572+03:00>)>, <template TemplateState(<state sensor.battery_cleargrass_2=100; unit_of_measurement=%, friendly_name=Xiaomi CG-2, icon=, device_class=battery @ 2021-03-16T16:15:56.862384+03:00>)>]
4
4

Why do I get such different results?

finity · March 17, 2021, 3:13am

I think that looks like a bug in the template editor.

I tried it and got similar perplexing results as well.

%- set Z = expand('group.alexa_all_basement_lights')|map(attribute='state') 
%}
{{Z|list|count}}
{{Z|list}}
{{Z|list}}
{{Z|list|length}}
{{Z|list|count}}
{{Z|list}}

6
[]
[]
0
0
[]

it seems to process the first item in the list correctly then it fails after that.

you can test it by moving the return templates around in the order they are processed and the first one will always work but the subsequent ones never do.

I would file bug report.

If you link it here I can add my input if needed.

Ildar_Gabdullin · March 17, 2021, 3:32am

When editing "configuration.yaml" file (or similar yamls) once I realized that for templates I can use only this structure:

{% set XYZ = XXXXX %}
{{ use XYZ for something }}

but not this one:

{% set XYZ = XXXXX %}
{{ use XYZ for something }}
{{ use XYZ for something }}

Otherwise I would have been able to declare at the beginning of some file something like

{% set GLOBAL_VARIABLE = XXXXX %}

and then use this variable many times in the file.
But in a real life I have to declare this variable every time before every template in which this variable is used:

{% set XYZ = XXXXX %}
{{ use XYZ for something }}
....
{% set XYZ = XXXXX %}
{{ use XYZ for something }}

So I think that we cannot use

{% set XYZ = XXXXX %}
{{ use XYZ for something }}
{{ use XYZ for something }}

in the HA → Development tools → Templates because of the same reasons (some yaml / jinjia2 / ??? rules).
That is why I am very surprised why this DID work in some cases…

finity · March 17, 2021, 3:42am

That is true for templates in yaml because the variable scope is only internal to the current section.

But for the developer tools template editor that isn’t the case. the variable scope is the entire template editor.

you can see that here:

{%- set Z = expand('group.alexa_all_basement_lights')|map(attribute='state')|list 
%}
{{Z}}
{{Z|count}}
{{Z|list}}

results in:

['off', 'off', 'off', 'off', 'off', 'off']
6
['off', 'off', 'off', 'off', 'off', 'off']

with results as expected.

I’m not sure why the template without “|list” didn’t work as well.

Ildar_Gabdullin · March 17, 2021, 3:10pm

Well, I can see that this does not work:

{%- set Z = expand('group.battery_sensors')
           |map(attribute='state') %}
{{ Z|list }}
{{ Z|list|count }}

This works:

{%- set Z = expand('group.battery_sensors')
           |map(attribute='state')
           |list %}
{{ Z }}
{{ Z|list|count }}

This works too:

{%- set Z = expand('group.battery_sensors') %}
{{ Z|list }}
{{ Z|list|count }}

Cannot explain it, cannot understand it - so I just have to memorize it…

123 · March 17, 2021, 3:36pm

I have a foggy recollection of encountering this issue and petro explained it has to do with the fact your first template produces a generator object which has a limited scope.

{%- set Z = expand('group.battery_sensors') | map(attribute='state') %}

Paste each one of these separately into the Template Editor and it will report the first one produces a list whereas the second one produces a generator object.

{{ expand('group.battery_sensors') }}

{{ expand('group.battery_sensors') | map(attribute='state') }}

A list’s scope isn’t as limited as a generator and explains why it can be referenced several times.

Ildar_Gabdullin · March 17, 2021, 5:01pm

Does it mean that this - also a list, not a generator?

{%- set A0 = expand('group.battery_sensors')
            |selectattr('state','eq','100') %}

123 · March 17, 2021, 5:13pm

That produces a generator object, not a list.

Paste this into the Template Editor and it will tell you what is produced.

{{ expand('group.battery_sensors')
            |selectattr('state','eq','100') }}

Ildar_Gabdullin · March 17, 2021, 5:28pm

But if I add a “sort” method - it produces a list…
Very complicated for me.

A0:
{%- set A0 = expand('group.battery_sensors')
            |selectattr('state','eq','100') %}
{{A0}}
{{A0|list|count}}

A1:
{%- set A1 = expand('group.battery_sensors')
            |selectattr('state','eq','100')
            |sort(reverse=false,attribute='name') %}
{{A1}}
{{A1|list}}
{{A1|list|count}}

A0:
<generator object select_or_reject at 0x7199ff08>
2

A1:
[<template TemplateState(<state sensor.battery_cleargrass_1=100; unit_of_measurement=%, friendly_name=Xiaomi CG-1, icon=, device_class=battery @ 2021-03-16T16:15:56.819572+03:00>)>, <template TemplateState(<state sensor.battery_cleargrass_2=100; unit_of_measurement=%, friendly_name=Xiaomi CG-2, icon=, device_class=battery @ 2021-03-16T16:15:56.862384+03:00>)>]
[<template TemplateState(<state sensor.battery_cleargrass_1=100; unit_of_measurement=%, friendly_name=Xiaomi CG-1, icon=, device_class=battery @ 2021-03-16T16:15:56.819572+03:00>)>, <template TemplateState(<state sensor.battery_cleargrass_2=100; unit_of_measurement=%, friendly_name=Xiaomi CG-2, icon=, device_class=battery @ 2021-03-16T16:15:56.862384+03:00>)>]
2

123 · March 17, 2021, 5:41pm

Your original question has been answered. The results you couldn’t explain were because you assumed the result had to be a list when, in fact, it can be a generator (whose scope isn’t like a list).

Depending on the combination of filters used in the template, the result is likely to be a generator but can be a list if the final filter outputs a list. In your latest example, sort produces a list.

The upshot is that if you want to use the template’s result as a variable, to be referenced several times later, ensure it isn’t a generator object.

petro · March 17, 2021, 5:42pm

A generator can only be iterated once. Once you ‘use’ it, it’s gone. Much like what @123 said with it being ‘out of scope’.

petro · March 17, 2021, 5:47pm

This is because sort requires the item to be a list in order to sort it. Generators don’t keep things in memory, if it’s not in memory you can’t sort it. It’s like of like removing an item from a collection. With a generator, it hands you everything one at a time, but it doesn’t keep a copy of what it handed you. With a list, it’ll hand you things one at a time, but it sill has a copy of it. Technically it’s not a copy, but that a different discussion.

Ildar_Gabdullin · March 17, 2021, 5:49pm

Thank very much for your answers to everyone!