Binary_sensor, Template trigger and expand()

[123 (Taras)]

Here’s why your Trigger-based Template Binary Sensor fails to work the way you think it should.

A Template Trigger will trigger when the result produced by its template changes from false to true. It does not trigger when it changes from true to false.

Given that information, when will this template’s result change from false to true?

- trigger:
    - platform: template
      value_template: >-
        {{ expand('group.cucina_attiva') 
                | rejectattr("state", "in", ["undefined", "unavailable"]) 
                | rejectattr("entity_id", "in", ['switch.caldaia_on_off'])
                | selectattr('state', 'eq', 'on')
                | list | count > 0 }}

It will happen when none of the lights are on and then one of them is turned on. When that happens the number of illuminated lights is greater than 0 and the template’s result will change from false to true. That’s the only time this template will trigger the Template Trigger.

When all lights are turned off, the quantity of illuminated lights is no longer greater than 0 so the template’s result changes from true to false. That change does not trigger the Template Trigger. That’s why your Trigger-based Template Binary Sensor’s state changes to on but never changes to off.

The fact is that there is no need to create a Trigger-based Template Binary Sensor for your stated application. It adds additional complexity, provides no advantages over a Template Binary Sensor (in this case), and even introduces a major functional disadvantage.

I highly recommend you use the Template Binary Sensor I posted earlier.

@emandt

UPDATE: adding a “time_pattern” Trigger to the Binary seems fix this

The Time Pattern Trigger is effectively bypassing the Template Trigger and forcing the state template to be evaluated every 30 seconds.

There’s no need for the Time Pattern Trigger or the Template Trigger. A Template Binary Sensor, having no explicit triggers, will report the correct state using its template and nothing else. It works efficiently because it only updates when necessary (when one of its monitored entities changes state) and not unnecessarily according to a time pattern.

[emandt]

Thx for your explanation.

Triggers are ALWAYS useful to avoid triggering unwanted-events so I’m usually use them in all my template sensors.
Sincerily I never created a template sensor without any Trigger declared…even the most easy/stupid sensor I created has a Trigger section.
I think this is the only good way to declare a template sensors. Not using any Trigger would have unpredicted behaviours and/or reduce overall performance.

My entities have a lot of Attributes and I don’t want HA wastes CPU cycles to trigger something that I don’t want like at each Attribute changes.
For example I have a smart plug that has its Status On/Off but also Attributes Voltage and Temperature that changes avery 5 seconds…and I don’t want my template sensor will be triggered all the day every 5 seconds even if the status is always Off.
This moreover in an expanded group where a lot of entities could be present into it and triggers the Sensor.

Then a Template Triggered Sensor seems unuseful because ALL triggers should be processed for both Trute-False and vice-versa cases: a light or a switch could became ON from its OFF status and then (sometime in the future) it have to return to the prev one but if the trigger won’t triggers it’s useless.
Other cases could be like mine where the Template have to listen for different kind of “things” but there isn’t any only one event to make it Off again. This because if the ON was a complex “IF” statement, the OFF will need the same complexity and not just an Off state trigger like a simple “platform: state”…

This is not necessary in your case

The case I showed here is a simplified version.
My group has 37 entities:

• most of them have Attributes that changes every 1-5 seconds _{(i.e: Tuya presence sensor sends an MQTT message every single second even if the State is always “not_home”)}
• few of them are always ON and I want to read only some of their Attributes inside the Trigger _{(of course I could create a separated sensor from that Attribute, but I tried to do in a more cleaned way by using Template Triggers first)}
• many of them are from different Domains (I grouped switches, lights, numbers, input_booleans and input_text) so I have to exclude someone of them and/or read different things from each of them

Declaring just a template sensor without any trigger would take too much CPU resources and I don’t want this.

[…] This would be a use case for a trigger-based TBS […]

I don’t figure it out because that sensor you imagine have to return (in the future) to its OFF status, but using a TBS seems not possibile if it the Off status would not be triggered. Each sensor needs two-ways trigger…I really don’t understand why a Trigger would have to work only one-way…
Then if I made a very complex Template Trigger from False-to-True status then have I to make the same complex status (but at opposite logic) in a second Trigger? This implementation seems stupid to me…and I’m a 40years programmer worked with dozen of programming languages in all those years…

UPDATE:
adding a second “platform: template” trigger using same Jinja statement but at inversed logic (“… | count <= 0”) seems works…however this is crazy because now the expand() and all those filters would be executed three times: at Trigger 1, at this new Trigger 2 and in the State…
An in my full-version I will have at least 5 expand() function to call and 3-5 selectattr/rejectattr each one…
Very very bad.

[123 (Taras)]

@emandt

Declaring just a template sensor without any trigger would take too much CPU resources and I don’t want this.

No, that’s false.

That’s at least the second time you have misunderstood how a template in a Template entity is processed. As a consequence, you are making unnecessary and self-defeating design decisions.

@emandt

Triggers are ALWAYS useful to avoid triggering unwanted-events so I’m usually use them in all my template sensors.
Sincerily I never created a template sensor without any Trigger declared…even the most easy/stupid sensor I created has a Trigger section.

That sounds like dogma and not an engineering decision.

@emandt

Not using any Trigger would have unpredicted behaviours and/or reduce overall performance.

No, that’s false. You’re repeating the same falsehood you made earlier and it implies you have not read, or understood, the documentation explaining how a Template entity’s template is evaluated.

@emandt

This implementation seems stupid to me…and I’m a 40years programmer worked with dozen of programming languages in all those years…

It seems stupid because your understanding of its operation is not how it actually works. Respectfully, your extensive experience with other programming languages isn’t serving you well in this situation.

You have designed an entity that does not detect “no lights on” because you used a trigger that cannot detect it. Let that sink in for a minute. Its failure to work the way you want is not due to an internal shortcoming but to your misunderstanding of its operation.

Your intended application isn’t best served by the use of a trigger. Yet you persist to add more triggers in a valiant attempt to force it to work the way you think it should. Then you conclude that the overly complicated entity you designed indicates the “implementation seems stupid” and “very very bad”.

You’re simply using the wrong tool for the job. Your application doesn’t need or benefit from a trigger. Use the Template Binary Sensor I posted above. As previously explained, its template will be evaluated only when one of its monitored entities undergoes a state-change (to its state value, not one of its attributes).

[Didgeridrew]

@emandt

An in my full-version I will have at least 5 expand() function to call and 3-5 selectattr/rejectattr each one…
Very very bad.

Why are you using expand() at all… it’s not necessary any more for the type of use you have shown.

  - binary_sensor:
      - name: Cucina attiva
        state: >-
          {{ state_attr('group.cucina_attiva', 'entity_id') 
          | reject('eq', 'switch.caldaia_on_off') 
          | select('has_value')
          | select('is_state', 'on')
          | list | count > 0 }}

[Mariusthvdb (Marius)]

yes, or use label and {{label_entities()}} so you don’t even have to exclude, and can leave out the has_value() check…

{{label_entities('cucina_attiva')|select('is_state','on')|list|count >0}}

[123 (Taras)]

@didgeridrew @Mariusthvdb

emandt is likely to use your template suggestions in the same way he currently insists and that’s in a Template Trigger of a Trigger-based Template Binary Sensor. As a result, it will fail to create a proper binary_sensor because a Template Trigger will not trigger when its result changes from true to false. Therefore the entity will only be able to report on but never off.

If emandt were to simply use a Template Binary Sensor (i.e. not Trigger-based) it will correctly report on and off using the template I had suggested (and proven to work) or your suggested templates. However, convincing emandt that he is using the wrong tool for the job has proven to be challenging.

Emandt is under the mistake impression that his binary_sensor must have a trigger in order to avoid “unpredicted behaviours and/or reduce overall performance”.

I have explained, more than once, that the template I suggested will be evaluated only when there’s a state-change in one of its monitored entities. It’s effective, efficient, and predictable. However, he insists the binary_sensor must have a trigger and so has employed a Template Trigger incapable of reporting off and so has attempted to compensate for it by adding more triggers.

At this point, emandt should be realizing that the over-complicated, under-performing thing he has created is flawed because it’s the wrong path to the solution. Yet he remains convinced it’s all due to flaws in the system and not because he is using the wrong tool for the job.

---

EDIT

tl;dr
It’s not what is in the template that makes the Template Trigger, of this Trigger-based Template Sensor, unable to report off, it’s the fact that it should not be using a trigger. For this particular application, all it needs is a vanilla Template Binary Sensor.

[petro]

@emandt

I think this is the only good way to declare a template sensors. Not using any Trigger would have unpredicted behaviours and/or reduce overall performance.

No, both a template trigger and template entities without triggers use the same code. The CPU usage is identical between the two. Template entities and template triggers are rate limited the exact same way.

@123’s proposed solution is the correct template that should be used. I’d revisit all your templates understanding what 123 has described to you.

@123

template:
  - binary_sensor:
      - name: Cucina attiva
        unique_id: cucina_attiva
        state: >
          {{ expand('group.lights_exterior_house')
            | rejectattr('entity_id', 'in', ['light.shed_lamp'])
            | rejectattr('state', 'in', ['unknown', 'unavailable'])
            | selectattr('state', 'eq', 'on')
            | list | count > 0 }}

---

EDIT: You’re actually using more resources by using the exact same template as a trigger. If you do a template trigger with a sensor that contains the same template, you’re rendering the template twice for each new state change the sensor obtains.

[emandt]

That’s at least the second time you have misunderstood how a template in a Template entity is processed.
[…]
its template will be evaluated only when one of its monitored entities undergoes a state-change (to its state value, not one of its attributes)

Guides are extremly precise about triggers:

" To trigger on all state changes, but not on changed attributes, set at least one of from, to, not_from, or not_to to null"

Then Triggers are NECESSARY if I don’t want to trigger even when one of Attributes changes, that’s why I used them also in stupid/easy sensors.
However I also explained that THIS simple test was just a tiny part of what I want to do where I would like to read some entities States and ALSO some other entitites Attributes, so the trigger have to react by some State and some Attributes.
This specific test doesn’t covers my case I’ve explained before, so maybe only THIS test could be solved in the way you have said.

Your application doesn’t need or benefit from a trigger

I though to have already well explained why I don’t want the sensor be triggered at each Groups entities attribute changes.
37 entities could change their attributes every second and then (in the worst scenario) trigger the Sensor every 27 milliseconds and this is not what I want for my HA.
I’m used to do something performant and not “just that works” as maybe usual ppl that are not programmers and doesn’y have enough skills to do things in the right way.

However thx for your time and explanations.
I’ll try to solve this issue in another way without the non-sense Template Trigger Sensor.

[emandt]

Unfortunately this does not listen for entitites change but only to Group state changes as you can show in Development Template page.
Using “expand()” a listener is created for each entity but in your example (that I didn’t know, so thx to let me that it exists) only a group listener is created.

However thx for your suggestion

[petro]

@emandt

37 entities could change their attributes every second and then (in the worst scenario) trigger the Sensor every 27 milliseconds and this is not what I want for my HA.

You have a fundamental miss understanding on how things are working under the hood.

When you create a trigger, it generates 1 or (more listeners depending on the trigger) for your trigger. In your hypothetical situation you have 1 template trigger with 37 entities being used.

This creates 1 listener that pays attention to 37 entities. With each state change on 37 entities the template is executed. If you have a state change or attribute change every 27 milliseconds, your listener is executing the template every 27 milliseconds. Then once your template resolves from false to true, your trigger then causes the sensor to resolve your state template. You are not bypassing the 27 milliseconds state changes, you are merely hiding them.

In your original question, it’s evident that you will end up needing a second trigger to capture the on → off template change. This would create a second listener, and an additional template resolution when that trigger fires. To recap, you are now at 2 listeners listening to the same 37 entities firing every 27 milliseconds that when triggered cause the template entity to execute an additional template.

Now, if you forgo the trigger for the template entity, you actually reduce the number of template executions and the number of listeners. It goes from: 2 listeners executing templates every 27 ms triggering sensor template executions, down to: 1 listener executing a template every 27 ms.

You literally reduce the HA overhead by a factor of 2*x + 2.

Now lets look at other simple templates that you probably have. Lets say you have a simple template {{ is_state('sensor.a', 'foo') and is_state(sensor.b', 'bar') }}. Without triggers, this would have 1 listener. If you add a state change trigger for each entity, you now have 2 listeners, effectively doubling the load again. It’s very easy to actually triple or quadruple the load when you add multiple triggers.

So the take away from this, you should utilize template entities without triggers inn most cases. At a maximum, templates will generate 2 listeners: 1 for time (if you use one of the datetime functions) and 1 for state changes (if you access the state machine) 1 listener for both time and state changes. So if you have 3 or more triggers on a template, it will likely create more load. There are exceptions to this, however I will not cover them here.

The take away from my post is that you should not use triggers for all template entities just because you think that it reduces the load or increases performance.

@emandt

Using “expand()” a listener is created for each entity but in your example

No, a single listener.

separate triggers → separate listeners.

single template → at most 2 listeners. 1 for time, 1 for entities. 1 listener.

You have to understand that templates have been optimized under the hood to reduce load. There’s even throttles in place to ensure your calculations don’t bog down your system.

---

EDIT: It’s been awhile since I’ve been in that part of the code. I just checked it out and templates only generate a single listener.

[123 (Taras)]

@emandt

Guides are extremly precise about triggers:

You referenced the description for a State Trigger. We’re discussing your use of a Template Trigger.

A Template Trigger monitors whatever you design its template to monitor. Its behavior is controlled by the template you design. It does not have a State Trigger’s from, to, not_from, or not_to to null options because the template offers complete control over what changes it should detect. You design the template to listen for what you want and to report true when it is detected.

@emandt

I though to have already well explained why I don’t want the sensor be triggered at each Groups entities attribute changes.

Yes, you have explained that because you mistakenly continue to believe a Template Trigger will automatically listen for attribute changes. It listens for whatever changes you choose to have monitored in the Template Trigger’s template. The template I suggested listens for changes to the entity’s state, not its attributes.

@emandt

I’m used to do something performant and not “just that works” as maybe usual ppl that are not programmers and doesn’y have enough skills to do things in the right way.

Since you are a conscientious programmer, it may come as a shock for you to learn that your latest creation is less performant than a Template Sensor. See petro’s post for the details.

Unfortunately, that’s what can happen when design decisions are based on an incorrect understanding of how the system works. In this case, a Template Binary Sensor not only “just works” it works better than a Trigger-based variety. For this application, it doesn’t represent a compromised solution but a superior one.

[emandt]

And I’ve already told you that your way is wrong.
This is the minimum extract of my big/long Triggered Template Sensor from where I removed the Trigger section and it doesn’t works without it even if in Development Template page the same Jinja is evaluated well becoming TRUE and FALSE according to it and also according to events of the group sensors.

The Sensor, instead, stays always OFF even if in Development Template page I can see a “TRUE” evaluation of the same Jinja.

- trigger:
  binary_sensor:
    - name: Cucina attiva
      unique_id: cucina_attiva
      state: >-
        {% set excluded_due_to_always_on = ['switch.caldaia_on_off'] %}
        {% set entita_gruppo = expand('group.cucina_attiva') %}
        {% set entita_on = entita_gruppo | rejectattr("state", "in", ["undefined", "unavailable"]) | rejectattr("entity_id", "in", excluded_due_to_always_on) | selectattr('state', 'eq', 'on') | map(attribute="entity_id") | list %}
        {% set excluded_due_to_always_greather_than = ['sensor.caldaia_potenza'] %}
        {% set entita_greather_than = entita_gruppo | rejectattr("state", "in", ["undefined", "unavailable"]) 
            | rejectattr("entity_id", "in", entita_on)
            | rejectattr("entity_id", "in", excluded_due_to_always_greather_than)
            | map(attribute='state') | map('float', 0) | select('>', 0) | list %}
        {{ ((entita_on | count) + (entita_greather_than | count)) > 0 }}

So what I have solved using your method? Nothing…

[Troon]

@emandt

This is the minimum extract of my big/long Triggered Template Sensor from where I removed the Trigger section

@emandt

- trigger:
  binary_sensor:

That’s not removing the trigger section, it’s declaring an empty one that will never trigger, which is why it’s not working for you. It should (as advised multiple times above) start with:

- binary_sensor:

[petro]

@emandt

So what I have solved using your method? Nothing…

It’s the attitude like this that makes this whole thread a head scratcher. You’re over here acting like you know HA more than multiple HA experts, while simultaneously posting incorrect configurations & conceptions.

---

The reason your template doesn’t update is because what @troon (another template expert) highlighted. Remove the trigger section from your YAML.

- binary_sensor:
    - name: Cucina attiva
      unique_id: cucina_attiva
      state: >-
        {% set excluded_due_to_always_on = ['switch.caldaia_on_off'] %}
        {% set entita_gruppo = expand('group.cucina_attiva') %}
        {% set entita_on = entita_gruppo | rejectattr("state", "in", ["undefined", "unavailable"]) | rejectattr("entity_id", "in", excluded_due_to_always_on) | selectattr('state', 'eq', 'on') | map(attribute="entity_id") | list %}
        {% set excluded_due_to_always_greather_than = ['sensor.caldaia_potenza'] %}
        {% set entita_greather_than = entita_gruppo | rejectattr("state", "in", ["undefined", "unavailable"]) 
            | rejectattr("entity_id", "in", entita_on)
            | rejectattr("entity_id", "in", excluded_due_to_always_greather_than)
            | map(attribute='state') | map('float', 0) | select('>', 0) | list %}
        {{ ((entita_on | count) + (entita_greather_than | count)) > 0 }}

[123 (Taras)]

@emandt

And I’ve already told you that your way is wrong.

The problem is you’re NOT doing it the way I suggested.

You persist to include a trigger section, an empty one at that, and that’s why it fails to work correctly.

By definition, only a Trigger-based Template Binary Sensor uses a trigger section. The correct format of a Template Binary Sensor (what I suggested one week ago) does not include a trigger section.

Reference

Template integration.

@emandt

So what I have solved using your method? Nothing…

You didn’t use my method. By adding an empty trigger section, you used your method. That’s why you solved nothing.

[Mariusthvdb (Marius)]

@emandt

I’m used to do something performant and not “just that works” as maybe usual ppl that are not programmers and doesn’y have enough skills to do things in the right way.

my bold accents

actually. wow.
personally, I find this so dismissive, I am tempted to flag this post, and even suggest the discussion to be closed.

Man, you’ve got all experts in this community spoon-feeding explaining you your issue and helping you out with the solution, even explain in length as to the why, and yet you remain projecting this attitude?

I won’t even mention the expertise of some well known developers in the background that has made the template integration so efficient as it is today.

[emandt]

Yes, ppl that normally uses HA are not programmers so if their HA drains 1% of CPU or 25% they they don’t even pay attention to it.
If a Trigger is wrong and eats a lot of CPU than the normal but the result is acceptable, they’re fine

I’m in Homeassitant FB groups (international and italian) where I saw thousand of bad and/or not_optimized YAML but that reach user achievement at the end…and they didn’t care if they have created a Trigger that listens for the WHOLE “sensors” domain or not.

So yes the BOLD section you highlighted.

If the solution (after explained my need and reasons) is not right for me and it doesn’t work, why the discussion should be closed?

[emandt]

Yes I changed the Sensor without remove trigger section and this was my fault.
I confirm that I NEVER declared a template section without a Trigger

The easy example we have used until now seems works quite fine if just few entities are declared in the Group, but became heavy if all 37 entities are included in it.
I already told that I have to listen for some entity States and some Attributes because there are sensors that have constant State but changes their Attributes and vice-versa.

This “solution” works only where States changes but not on Attributes.
So the State section is well executed every entity State is changed, but if an Attribute changes there isn’t anything that triggers the State section again to evaluate it again according to the long Jinja inside it…
So I do not solved anything if all Attribute changes doesn’t trigger anything.
And there isn’t any 1:1 relation between an Attribute and the final Sensor state because an Attribute change could not change anything if Jinja does not deem it necessary to change its status.
This was why I need a Trigger template: a sensor is triggered ONLY when really needs by the Jinja according to a complex State and Attribute interoperability.

OR I have to “export” all needed Attributes in standalone sensors, add them to the Group, find a way to mark them (in the Jinja inside State section) as “same entity X” to distinghuish which Real sensor is updated (because I could need to verify State AND Attributes from the SAME sensor), change the Jinja to handle this “entity mark” and then it could be work, but…

[petro]

@emandt

This “solution” works only where States changes but not on Attributes.
So the State section is well executed every entity State is changed, but if an Attribute changes there isn’t anything that triggers the State section again to evaluate it again according to the long Jinja inside it…

No… It will update when any aspect of the state object changes. That includes attributes.

State changes on the entity themselves are suppressed when it evaluates to the same result. If your template reaches the same result between 2 updates, you won’t get a state change event.

[123 (Taras)]

@emandt

This “solution” works only where States changes but not on Attributes.

Your “observation” indicates you haven’t remembered what has been explained.

I recommend you review all the explanations in this topic.