Template sensor(s) using "integration_entities" not updating automatically

e-raser · February 12, 2024, 11:04am

I’m using

template:
  - sensor:
      - name: Statistik Entitäten Integration HACS
        # entity_id: statistics_entities_integration_hacs
        unique_id: b5cb40e7-cb6c-4a34-8e3f-54e481ad1d0d
        state_class: total
        icon: mdi:numeric
        state: "{{ integration_entities('hacs') | list | length }}"

which renders fine and gives

Unfortunately, that sensor does not update.
At the same time

a) checking with /developer-tools/template actually DOES update.
b) running this service updates it just fine:

service: homeassistant.update_entity
data: {}
target:
  entity_id: sensor.statistics_entities_integration_hacs

So why does the template sensor not get updated automatically?

I have many statistics sensors of that kind and afaics others update just fine. Just those using integration_entities don’t. Complete miracle to me.

123 · February 12, 2024, 12:56pm

Copy-paste this template into the Template Editor. Ensure that it’s the only template displayed in the editing window.

 {{ integration_entities('hacs') | list | length }}

What are the entities listed under “This template listens for the following state changed events”?

e-raser · February 12, 2024, 1:54pm

Translates to:

“This template does not monitor events and is not updated automatically.”

123 · February 12, 2024, 2:05pm

There’s your answer why the Template Sensor is not updating automatically. It’s not listening to anything for state-changes.

It’s designed to monitor entities for state-changes. It knows how to extract entities from some filters, such as expand, but not for others like integration_entities.

I suggest you consider creating a Trigger-based Template Sensor. Ideally the trigger should detect when a new integration is added or an existing integration is removed (~~and another trigger to detect when Home Assistant starts~~ EDIT probably unnecessary because the core_config_updated event is likely to occur on startup; see below). It would then trigger and evaluate your template.

I don’t recall the exact event that occurs when integrations are added/removed. It might be core_config_updated.

e-raser · February 12, 2024, 2:52pm

Interesting, never ever had a deeper look at events which seem to be the precondition for some template filters to work (aiming at automatically updating template sensors).

I can confirm those template sensors are updated once HA starts.
Trigger based template sensors won’t fix anything for me (I’m not going to walk that path) as I’m looking for active entities for several integrations.
I want the template sensor to update not only when integrations get removed or added, but only once single entities get added or removed (happens by enabling/disabling them or - specifically for HACS with experimental features enabled - by simply adding/removing integrations)
As a workaround I implemented a time based trigger automation which runs the homeassistant.update_entity service every now and then (I think every 15 minutes) for all my template sensors using the integration_entities.

That’s a pretty ugly and dumb workaround to be honest, like a hammer smashing every 15 minutes no matter if needed or not.

But I (still) don’t get why this is not working as expected. No idea when Templating - Home Assistant got added, but based on what you uncovered with the missing events to properly update template sensors using integration_entities function, it looks like it has only been “half integrated” and something is missing in core - due to the missing events part. Isn’t that the case?

Looking ahead: what’s the best way to raise and transfer this into a feature request or proposal? Or is it even a bug? Because it works different from many (all?) other functions used in templating.

123 · February 12, 2024, 3:30pm

I agree.

Then trigger on that event. I believe it’s entity_registry_updated. A Trigger-based Template Sensor is the most efficient way to realize this application.

The template is evaluated when one of the entities listed in the template changes state. Your template contains no obvious entities (that’s not just my opinion, it’s confirmed by the Template Editor). The entities are listed only after integration_entities is executed and Home Assistant doesn’t do that in advance (with the exception of expand).

There’s more about the template updating system here: Rate Limiting Updates

e-raser · February 12, 2024, 4:08pm

Well… I think you delivered an actual solution which may act like an appropriate QUICK FIX - and is therefore way more attractive than changing my dozens of template sensors to act as a trigger-based template sensor:

magic word expand.

Now have a look at this:

So why not just add expand after using integration_entities('something') to all my template sensors? I think I’ll give that a try…

Edit 1:

- Adjusted 98 % (2 % gave a different result when using expand, e. g. a higher number than without) of all my template sensors using integration_entities() to use expand immediately after, tested few of those template sensors, confirmed working: updating instantly.
~~- Time trigger automation based workaround removed.~~

Awaiting some theory on potential side effects, otherwise this topic can be marked as solved (even I’ll monitor the adjusted template sensors for some days).

Edit 2:

OK there actually IS a (big) downside of using this… here’s what I tested:

Template sensor:

      - name: Statistik Entitäten Integration HACS
        # entity_id: statistics_entities_integration_hacs
        unique_id: b5cb40e7-cb6c-4a34-8e3f-54e481ad1d0d
        state_class: total
        icon: mdi:numeric
        state: "{{ integration_entities('hacs') | expand | list | length }}"
        availability: "{{ (integration_entities('hacs') | count) > 0 }}"

Once I add or remove an integration in HACS, the template sensor renders to 0 !

I played with the state and changed it to

        state: >-
          {% if (integration_entities('hacs') | expand | list | length) > 0 %}
            {{ integration_entities('hacs') | expand | list | length }}
          {% endif %}

but now instead of 0 actually nothing is shown as value:

Adjusting the availability_template, the entity renders unavailable. So there must happen something which breaks the proper calculation. A new miracle!

Why? Something must get confused when using the expand and the (HACS) integration changes…

So update: still not a proper solution!

For now I reverted adding the | expand to my template sensors and reactivated the dumb automation workaround firing the update service every 15 minutes… dumb but bullet-proof. Templating is pure voodoo stuff, so annoying.

123 · February 12, 2024, 6:57pm

FWIW, if the dozens of others require exactly the same Event Trigger, then you define them all under that one trigger. In other words, converting them to Trigger-based would be trivial.

e-raser · February 12, 2024, 7:01pm

As already mentioned, transformation to trigger-based template sensors is not the right way. What would be the actual trigger? I specified that already:

123 · February 12, 2024, 7:07pm

Have you tried the alternative event I suggested?

e-raser · February 12, 2024, 7:20pm

You mean

?

Well, not tried as trigger for template sensors, but I had a look at it in the /developer-tools/event section:

First I installed a HACS integration, then I removed it.

As I think entity_registry_updated events are fired way more often than needed for my template sensors, that would “over-trigger” those sensors. So it’s not really smarter than the current workaround (run the service: homeassistant.update_entity).

123 · February 12, 2024, 9:09pm

Should fire far less frequently than every 15 minutes (unless your system is creating/deleting entities every few minutes. … which would be unusual).

Anyway, you don’t appear to be willing to actually try my suggestion (and get actual performance data) so I won’t press the point any further.

e-raser · February 12, 2024, 11:55pm

Good point. Indeed running the update entity service 4 times an hour is something. BUT (next to the fact that‘s nothing in relation to power consumption entities etc. which update sometimes multiple times or at least once a second):

4 times an hour (maybe I’ll lower that to once an hour or even once a day) only a specific set of entities are requested to update (which they don’t if there is no need to change/update them)
vs. on creation/removal/??? (edit? what other actions?) of ANY entity (I got almost 2.000) the trigger is pulled to update my specific small set of entities

I don‘t know. Both options are not great (we’re only looking at the 2nd best possibilities in this thread so far) and definitely not what I initially expected, and that still is: this should happen automatically. Period

123 · February 13, 2024, 12:00am

If that’s your impression of how it works then I failed at explaining it. Hopefully someone else has more time than I do to help you understand how it would work. Good luck.

jazzyisj · February 13, 2024, 12:19am

I have a couple dozen template sensors using integration_entities that monitor cloud integrations to notify me when they are offline. This works. The sensors update once per minute using sensor.time as a trigger.

- trigger:
    - platform: homeassistant
      event: start

    - platform: event
      event_type: event_template_reloaded

    - platform: state
      entity_id: sensor.time
  binary_sensor:
    - name: "Nest Connected"
      unique_id: nest_connected
      picture: !secret NEST_ICON
      device_class: connectivity
      state: "{{ integration_entities('nest') | select('has_value') | list | count > 0 }}"