How can i get this template sensor to update in a more timely manner?

[famous.bulb]

@koying

It works for me in the template debugger, at least.

Yeah, it’s very responsive there, but try this:

• Open a first HA tab in your browser, navigate to dev tools/template
• Open a second HA tab in your browser, navigate to dev tools/service
• using the first tab, clear the template editor, fill with just {{states.persistent_notification|length}}
• using the second tab, call service w/ this YAML:

service: notify.persistent_notification
data:
  message: test

• Observe that the first tab shows Result type: number and 1
• use either tab to open the notifications drawer and dismiss the test notification
• Observe that the first tab STILL shows Result type: number and 1.

I can eventually get the first tab to show the correct results, but i need to effectively type into the template editor which forces evaluation… which brings me back to the root of my question: how can i get the template to be re-evaluated automatically!?

@koying

What does {{states.persistent_notification|list}} return?

It returns:

[]

(as expected).

[123 (Taras)]

@famous.bulb

the above template sensor does NOT update in real time

I cannot replicate the problem you described. It updates in real-time for me.

I created the following Template Sensor:

  - platform: template
    sensors:
      notifications:
        value_template: "{{ states.persistent_notification | length }}"

After reloading Template Entities, the newly created sensor.notifications immediately reported 3 because there were three existing persistent notifications. As I deleted the notifications, the sensor’s value decreased accordingly.

I opened a second browser tab for Home Assistant and used Developer Tools > Services to generate additional persistent notifications. Regardless of which browser tab I viewed, the total number of active notifications displayed in the menu and by sensor.notifications was identical.

The test was performed using Home Assistant version 2021.7.2 with Firefox 89.0.2 (on Ubuntu 20.04). What version of Home Assistant (and browser) are you using?

[famous.bulb]

@123

As I deleted the notifications, the sensor’s value decreased accordingly.

Where were you monitoring that value? Dev tools > States ?

I was able to get the template sensor value to update in near real time only after i fired off more than two test notifications. That is, i pulled up my two tabs - side by side - and fired a single notification… values was still 0. fired off two more back to back (i accidentally clicked the call service button 2x) another the sensor value jumped to 3 (expected).

Once i dismissed the notifications, the template sensor decremented… with a bit of lag.
I repeated the test but i clicked the call service button at least 15 times. The sensor value did catch up. I dismissed each notification and the template sensor did catch up… eventually. It went from 13 → 12 → 11 → (lag) → 8 → 4 (lag) → 2 → 0.

The lag was at most a second or two… perfectly reasonable for multiple template evaluations.

@123

The test was performed using Home Assistant version 2021.7.2 with Firefox 89.0.2 (on Ubuntu 20.04). What version of Home Assistant (and browser) are you using?

I am running HA in K8s.
Firefox is 89.0.2 (64-bit) on a few different OSs.

System Health

version	core-2021.7.2
installation_type	Unknown
dev	false
hassio	false
docker	false
virtualenv	false
python_version	3.9.5
os_name	Linux
os_version	5.4.0-77-generic
arch	x86_64
timezone	America/Los_Angeles

At this point, i believe that the template sensor works, but still don’t believe that it’s triggered at the appropriate time the first time a notification is fired off. After whatever code-path leads to the template sensor being re-evaluated is ‘warmed up’ so to speak, the template sensor performs reliably.

Either that or the notification that is fired off when the withings integration goes wrong is not counted as a persistent notification (some how…).

[petro]

templates with states.domain are throttled to 1 update per second max. That includes persistent_notification, which is the domain. So, you’ll get at most 1 update per second. If 4 notifications fire in 1 second, it’ll jump from 0 to 1 on the first notification, then 1 second later it will jump to 4.

[123 (Taras)]

@famous.bulb

I was able to get the template sensor value to update in near real time only after i fired off more than two test notifications.
…
still don’t believe that it’s triggered at the appropriate time the first time a notification is fired of

I cannot replicate those results.

• I started from scratch (no sensor, no persistent notifications).
• I created the Template Sensor and reloaded Template Entities.
• sensor.notifications reports 0 (in Lovelace UI and in Developer Tools > States).
• Created a new persistent notification (using Developer Tools > Services)
• sensor.notifications promptly reports 1 (in Lovelace UI and in Developer Tools > States). When I say “promptly” I mean the teeny-tiny amount of time it takes me to switch from one browser tab to another.

[petro]

He’s clicking the notifications like mad and hitting the rate limit described in that huge thread you created https://community.home-assistant.io/t/heads-up-upcoming-breaking-change-in-the-template-integration/223715

[123 (Taras)]

@petro

that huge thread you created

I created the topic, everyone else created the ‘huge thread’.

[petro]

Templates with now() update at least once per minute (on the minute). Templates with states will update at most once per minute. Templates with states.<domain> will update at most once per second.

[petro]

Yah, it wasn’t a ‘bash’, I was just pointing out where the discussion took place

[123 (Taras)]

FWIW, I just opened two separate browser windows, side-by-side.

• Left window displays sensor.notifications in Developer Tools > States
• Right window displays Developer Tools > Services, setup for creating a persistent notification.

I mashed the right window’s “Call Service” button repeatedly and the left window’s count for sensor.notifications kept up with the button-mashing.

All this to say, there’s something unique about your situation because it works flawlessly for me. I realize that’s cold comfort because it means what you’re experiencing isn’t nominal.

---

By “mashing” I mean I banged out about ten notifications in under 3 seconds.

[petro]

Here’s a post with a diagram that explains the rate limits.

Heads up! Upcoming breaking change in the Template integration

It’s not unpredictable. Here’s some diagrams to help you predict them: Using states.domain [image] Using states [image] Using anything with a predefined list, like expand, states(), state_attr(), etc. [image] If you use both states and states.domain in the same template, states rate_limit applies.

[famous.bulb]

@petro

templates with states.domain are throttled to 1 update per second max.

Good to know / that explains the small lag… which is not what I was concerned about, really.

I have a very simple automation that boils down to “when there is a persistent notification, set $someLight to blue”.

The whole reason why I started this thread is because one day I noticed two persistent notifications (withings error and a new HA release) and was puzzled to see why the light in question was not blue. I checked the trace on the automation that turns the light blue when the sensor > 0 and discovered thatthe template sensor still said 0… which is why the light was off and not the expected blue.

At least for now the template seems to be behaving. I’ll wait until the withings integration fires off the bogus error again and see what happens…

[famous.bulb]

An update!

There was an issue with HACS starting up which generated a persisnten notification saying as much. The notification language was something along the lines of “an integration had trouble setting up…”.

As soon as that notificaiotn fired, the template sensor went to a state greater than 0 which caused my automation to fire which resulted in a blue light.

However, dismissing the notification did not return the template sensor down to 0!

I opened dev tools / template and noticed that this:

 {{states.persistent_notification|length}}

evaluated to 7 … which is more than 0 so that would explain why the light was still on. But there are zero persistent notifications… so the value should be 0, right!? Another test:

 {{states.persistent_notification}}

evaluated to unknown which does have a length of 7.

So with that, the template sensor is revised:

# get a count of the number of persistent notifications
##
-  sensor:
    - name: "Active Persistent Notifications"
      # Unique ID is required for mgmt through the web UI
      unique_id: tmpl-active-persistent-notifications
      # I don't understand why, but from time to time the state of persistent notifications can be 'unknown' rather than 0.
      # This means we need to explicitly test for this unique condition; we can't just pipe to len() otherwise we'll get back
      #     the number of letters in the word unknown!
      ##
      icon: >-
        {% if states.persistent_notification == 'unknown' %}
        mdi:bell-badge
        {% else %}
        mdi:bell
        {% endif %}
      state: >
        {% if states.persistent_notification == 'unknown' %}
        0
        {% else %}
        {{states.persistent_notification|length}}
        {% endif %}

[123 (Taras)]

You may believe it to be a solution but it reinforces the hypothesis that your system contains a fault that cannot be replicated. The template you posted serves as a means to hide that unexplained fault.

You stated:

@famous.bulb

I opened dev tools / template and noticed that this:

 {{states.persistent_notification|length}}

evaluated to 7
…
But there are zero persistent notifications… so the value should be 0

When there are no persistent notifications, it evaluates to 0 for me.

In addition, the following doesn’t evaluate to unknown for me like you’ve reported:

The template you’ve created works around the fault so it’s possible it may manifest itself in some other way at a later date.

[petro]

Something is not correct on your system. states.<ANYVALUE> always resolves to a domain object.

Also, to expand on 123s suggestions, your template checking against unknown will always hit the else state.

What’s actually happening is states.persistent_notification is none and the template editor changes none to "unknown". So your template should actually be:

        {% if states.persistent_notification is none %}
        0
        {% else %}
        {{states.persistent_notification|length}}
        {% endif %}

However, you shouldn’t be getting none from states.xyz, so something is very wrong.

[famous.bulb]

Another update! This time with the clearest possible depiction of the issue.

I am convinced that there is a missing code path or other bug that prevents the template sensor from being evaluated every time it should.

Background

Some time shortly after midnight, two things happened:

• The withing integration bug fired off again
• At least one of the Shelly devices on my network determined that there was a new firmware update available.

Here is the automation that manages the notification when there is a shelly device firmware update.

alias: Notify when Shelly Devices have new FW updates to apply
description: ''
trigger:
  - type: turned_on
    platform: device
    device_id: 6<...>2a
    entity_id: binary_sensor.shelly_2_5_<...>_firmware_update
    domain: binary_sensor
    id: some_id
    # <... this repeats for each shelly device ...>
condition: []
action:
  - service: persistent_notification.create
    data:
      message: Please check shelly devices
      title: Shelly device has a pending FW update  # Yes, i fixed the typo; you will see SHelly in screenshot.
  - service: todoist.new_task
    data:
      content: Update Shelly Firmware
      # ...
mode: single

And here is the automation in question that I keep referencing w/r/t the blue light.

alias: Toggle status light blue when HA has notifications pending
description: >-
  A visual queue that something needs my attention
trigger:
  - platform: state
    entity_id: sensor.active_persistent_notifications
    from: '0'
    id: some-pending
  - platform: state
    entity_id: sensor.active_persistent_notifications
    id: none-pending
    to: '0'
condition: []
action:
  - choose:
      - conditions:
          - condition: trigger
            id: none-pending
        sequence:
          - type: turn_off
            device_id: d<...>b
            entity_id: light.blue
            domain: light
      - conditions:
          - condition: trigger
            id: some-pending
        sequence:
          - type: turn_on
            device_id: d<...>b
            entity_id: light.blue
            domain: light
    default: []
mode: single

SO with that in mind, lets get on with the show!

What I Woke Up to

After waking up and making my way into the kitchen for some coffee, I noticed the indicator light was blue. Since my computer was still asleep, I used my phone to check what the notifications were:

Before dismissing any notification, I woke up my desktop and prepared to start taking screenshots / documenting.
I pulled up three instances of Lovelace and then dismissed went back to my phone to dismiss the “withings failure” notification.
I used my phone to dismiss because I wanted to rule out any websocket related issues. As soon as I dismissed the notification on my phone, the notification badge count dropped by one on my mobile and desktop. It is safe to say that there is no problem with websockets here; a button press on one device is reflected in real time on the other.

Look above, I want to call your attention to a few things:

• The notification badges are correct.
• The template sensor is not correct
• The template evaluation in dev tools is correct

After collecting and annotating that screenshot, I dismissed the only remaining notification; the shelly fw notification:

This time:

• The notification badges are correct; now reads 0.
• The template sensor is not correct. It isn’t an ‘off by one’ error… it straight up did not get evaluated!
• The template evaluation in dev tools is correct

With that, I wondered if there was some sort of code path that either the shelly or the withings notification were created through that does not result in the template sensor evaluation. I fired off a new test notification:

This time:

• The notification badges are correct; now reads 1. The notification drawer has the expected value.
• The template sensor is correct. It has changed from 2 to 1.
• The template evaluation in dev tools is correct

So then dismissing the test notification absolutely should result in all counts reading 0, right?! Well, i thought so, too.

This time:

• The notification drawer and badge both show 0. The expected result.
• The template sensor is not correct. It still appears as though it was never re-evaluated.
• The template evaluation in dev tools is correct

So then what would it take to force the template sensor to be re-evaluated!?
I went into server controls and reloaded the template entities. The template sensor was updated and the notification light went from blue to off. Success!

I went to update the firmware on all but one of the Shelly devices and to make another cup of coffee.
While away, the shelly discovery script fired and the one device that didn’t get a FW update caused a notification to fire off again as expected / intended.

This time:

• The notification drawer and badge both show 1; the single notification created by the automation shown @ the top of this post.
• The template sensor is correct.
• The template evaluation in dev tools is correct

So what happens when I dismiss the shelly notification? If you’ve been paying attention, you know that the answer is not “the template sensor worked as it should have”

This time:

• The notification drawer and badge both show 0 as expected.
• The template sensor is ** not correct**; it is still behaving as though it was never evaluated. The indicator light is still blue / on.
• The template evaluation in dev tools is correct

So with that, I am convinced that there is an issue with the logic that schedules template sensors for (re)evaluation. In at least some (not yet fully understood) cases, dismissing a notification does not trigger re-evaluation…

[123 (Taras)]

If you’re convinced it’s a bug, I suggest you report it as an Issue in the Core repository on GitHub. However, your challenge is to provide a means of reproducing the error. Without that, a software developer is less likely to take on the challenge when there’s no way to observe the bug in action first hand.

Good luck!

[petro]

Just to throw a wrench in your hypothesis, both the template tester and template sensors use the same exact code to update.

How about you post your entire template section instead of focusing on the automation.

[famous.bulb]

@123

If you’re convinced it’s a bug, I suggest you report it as an Issue in the Core repository on GitHub. However, your challenge is to provide a means of reproducing the error.

This is precisely why I posted here and not directly to a new GH issue. If i can figure out what condition(s) are required to reproduce then i’ll open an issue there.

[famous.bulb]

@petro

Just to throw a wrench in your hypothesis, both the template tester and template sensors use the same exact code to update.

This is precisely why i used the word “schedules”. At the end of the day, some code path is triggered; that path ends up calling out to the jinja2 library for evaluation. I do not have an issue with the result of the template… the issue is that the result is not being computed in a timely or consistent manner… hence ‘schedule’.

While i am very certain that the ephemeral template expression via the webUI and template entities use the same evaluation engine, I am less certain that the triggers that lead to that eval are the same… which is why ephemeral expression seems to do what I want, but the template entity does not appear to be evaluated when i’d expect it to.

@petro

How about you post your entire template section instead of focusing on the automation.

I don’t understand what you’re asking for?
The template sensor in question is detailed a few posts up in this thread:

How can i get this template sensor to update in a more timely manner?

An update! There was an issue with HACS starting up which generated a persisnten notification saying as much. The notification language was something along the lines of “an integration had trouble setting up…”. As soon as that notificaiotn fired, the template sensor went to a state greater than 0 which caused my automation to fire which resulted in a blue light. However, dismissing the notification did not return the template sensor down to 0! I opened dev tools / template and noticed that t…

Hopefully this is enough to explain things / satisfy your ask?

me@some-host # grep -C2 "template" configuration.yaml

# Template sensors are now a specific integration
# See: https://www.home-assistant.io/integrations/template/
##
template: !include_dir_merge_list devices/template/


me@some-host # cat devices/template/notifications.yaml 
# get a count of the number of persistent notifications
##
-  sensor:
    - name: "Active Persistent Notifications"
      # Unique ID is required for mgmt through the web UI
      unique_id: tmpl-active-persistent-notifications
      icon: >-
        {% if states.persistent_notification|length > 0 %}
          mdi:bell-badge
        {% else %}
          mdi:bell
        {% endif %}
      state: >
        {{states.persistent_notification|length}}