Detect when a sensor has not updated for x minutes

I have an average temperature sensor but I want to exclude any that have not changed state for x minutes (they sometimes stop reporting via Bluetooth).

My average temp sensor loops through the temp sensors so I need to exclude them in the loop when appropriate.

The main reason I’m asking is because I have recently learnt that the system treats states changing and attributes changing differently and I want to make sure I get it right.

I don’t necessarily need a full example answer, just a pointer to the relevant facts.
(Although I am still pretty hopeless with time calculations :wink: )

Thanks.


For completeness here is my average temperature sensor currently:

sensor:
  - platform: template
    sensors:
      average_downstairs_temperature:
        friendly_name: Average Downstairs Temperature
        entity_id:
          - sensor.hall_temperature
          - sensor.kitchen_temperature
          - sensor.sitting_room_temperature
          - sensor.back_bedroom_temperature
        value_template: >
          {% set entities = ['sensor.hall_temperature',
                             'sensor.kitchen_temperature',
                             'sensor.sitting_room_temperature',
                             'sensor.back_bedroom_temperature'] %}
          {% set ns = namespace(count=0, value=0) %}
          {% for e in entities %}
            {% set s = states(e) %}
            {% if s != 'unknown' and
                  s != 'unavailable' and
                  s != '' %}
              {% set ns.count = ns.count + 1 %}
              {% set ns.value = ns.value + s | float %}
            {% endif %}
          {% endfor %}
          {% if ns.count == 0 %}
            'unavailable'
          {% else %}
            {{ (ns.value / ns.count) | round (1) }}
          {% endif %}
        unit_of_measurement: °C

1 Like

Without optimizing so you understand what’s going on:

sensor:
  - platform: template
    sensors:
      average_downstairs_temperature:
        friendly_name: Average Downstairs Temperature
        entity_id:
          - sensor.hall_temperature
          - sensor.kitchen_temperature
          - sensor.sitting_room_temperature
          - sensor.back_bedroom_temperature
        value_template: >
          {% set entities = ['sensor.hall_temperature',
                             'sensor.kitchen_temperature',
                             'sensor.sitting_room_temperature',
                             'sensor.back_bedroom_temperature'] %}
          {% set time = as_timestamp(now()) %}
          {# anything older than x seconds will be ignored #}
          {% set ignore_time_seconds = 300 %}
          {% set ns = namespace(count=0, value=0) %}
          {% for e in entities %}
            {# get the state object instead of the state #}
            {% set domain, object_id = e.split('.') %}
            {% set s = states[domain][object_id] %}
            {% if s.state is defined and
                  s.state != 'unknown' and
                  s.state != 'unavailable' and
                  s.state != '' and
                  (time - as_timestamp(s.last_updated)) < ignore_time_seconds %}
              {% set ns.count = ns.count + 1 %}
              {% set ns.value = ns.value + s | float %}
            {% endif %}
          {% endfor %}
          {% if ns.count == 0 %}
            'unavailable'
          {% else %}
            {{ (ns.value / ns.count) | round (1) }}
          {% endif %}
        unit_of_measurement: °C
1 Like

I was trying to be too clever by half and ended up painting myself into a corner with this template. It replicates what your original Template Sensor achieved, with less code, but now it’s a misery to fulfill the desired ‘no updated for X minutes’ requirement. Anyway, I’ll leave this here in case it helps someone else:

sensor:
  - platform: template
    sensors:
      average_downstairs_temperature:
        friendly_name: Average Downstairs Temperature
        value_template: >
          {% set entities = [states('sensor.hall_temperature') | float,
                             states('sensor.kitchen_temperature') | float,
                             states('sensor.sitting_room_temperature') | float,
                             states('sensor.back_bedroom_temperature') | float] %}
          {% set values = entities | reject('eq', 0) | list %}
          {% set qty = values | count %}
          {{ 'unavailable' if qty == 0 else ((values|sum)/qty) | round (1) }}
        unit_of_measurement: °C

Some evidence to demonstrate that it works:

1 Like

@petro Thanks, now to work out your tantalising

:slight_smile:

@123 That makes me feel a bit better! If you can paint yourself into a corner over it (albeit probably not in trying to solve it but in trying to improve on it) I won’t feel so bad about asking a question that I initially thought I should have known the answer to.

And I just read today an article about how important failure is (and was for our evolution as a species). How else would we learn without failure?

Yah, that’s why i didn’t want to optimize it… filters suck when you want to perform an operation.

Your question made me think about human nature:

  • We would prefer to learn from other people’s failures (based on Jim’s reaction, that plant is poisonous).
  • Sometimes we take pleasure in other people’s failures (schadenfreude).

This took some effort (plus a kludge) but it seems to work (for me). Acid test is if it will work for you.

sensor:
  - platform: template
    sensors:
      average_downstairs_temperature:
        friendly_name: Average Downstairs Temperature
        value_template: >
          {% set t = strptime(((now().timestamp() - 300) | timestamp_utc) ~ '+00:00', '%Y-%m-%d %H:%M:%S%z') %}
          {% set sensors = [states.sensor.hall_temperature,
                            states.sensor.kitchen_temperature,
                            states.sensor.sitting_room_temperature,
                            states.sensor.back_bedroom_temperature]
                            | rejectattr('state', 'in', ['unavailable','unknown',''])
                            | rejectattr('last_changed', '<', t)
                            | list %}
          {% set qty = sensors | count %}
          {% set x = sensors | map(attribute='state') | list | string %}
          {% set total = x.replace("'","") | from_json | sum %}
          {{ 'unavailable' if qty == 0 else (total/qty) | round (1) }}
        unit_of_measurement: °C
  • The variable t contains a datetime object representing the current UTC time less 300 seconds.
  • The variable sensors takes a list, containing your four sensors, and rejects any whose state is unavailable, unknown, or blank. It also rejects any whose last_changed is less than variable t. Whatever is left is reported as a list.
  • The variable qty counts the number of items in the sensors list.
  • The next two lines are a kludge. If anything is bound to break for you, I imagine it might be here. The challenge is that states are reported as strings which prevents using the sum filter. This kludge converts the list to a string, removes all single-quotes, then converts it back to a list.
  • Variable x extracts state from all entities in the sensors list and then converts the list into a string. Example (as string): ['23.4', '19.1', '21.6']
  • Variable total removes all single quotes from the string then uses from_json to convert it back to a list, then calculates the sum. Example (as list of floating point numbers): [23.4, 19.1, 21.6]
  • The last line simply reports unavailable if qty is zero otherwise it reports the average value.

EDIT

Applied petro’s suggestion to streamline the template. Replaced 3 lines of rejectattr with this one line version:

rejectattr('state', 'in', ['unavailable','unknown',''])
1 Like

@petro A small typo, only correcting for future visitors.

{% set ns.value = ns.value + s | float %} should be
{% set ns.value = ns.value + s.state | float %}

1 Like

@123 I’m going to need some time to go through that and work out what it is doing!!

There must be a German word like ‘schadenfreude’ that means something like ‘taking pleasure from seeing others put in a lot of effort to solve a question you originally thought was trivial’.

1 Like

Even the example doesn’t work for you, I learned a lot while writing it.

  • It would’ve been easier if there was a fromtimestamp filter available (in python, it’s a method of a datetime object). Without it, I felt I had to jump through several hoops to subtract 5 minutes from a datetime object. Maybe I just overlooked some simpler solution.
  • The rejectattr filter is pretty handy.
  • A shoutout to @SteveDinn for adding the from_json filter. It’s the second time today I found it helped to simplify a previously difficult task.
1 Like

So there’s really no easy way to go through the list and filter each string with “| float”? You really have to search and replace the single quotes and then convert it back from JSON? That’s nuts.

Other that iterating through the list with a for-loop (or the kludge I posted) I don’t know of a filter that can first convert each list item to float before summing them all up.

FWIW, python has ‘list comprehensions’ but Jinja2 doesn’t support them. If it did support it then I could create a new list of floats from the list of strings.

Example in python:
Screenshot%20from%202019-11-20%2015-28-16

Yah, i missed that, does it work for you?

@petro It seems to. I haven’t had a lot of time since posting the question to check thoroughly.

And yes, I have used the new @SteveDinn json filters too. They got me out of a real hole elsewhere.

I’d just like to say to Klogg (for asking the qestion), Petro and Teras (for your informative and entertaining answers)

Thank You

That must be the most I’ve learned in any 20 minute period for at least the last 5 years

FYI

can be simplified to

                            | rejectattr('state', 'in', ['unavailable','unknown',''])
1 Like

Funnily enough, I had used that in my ‘Mark 1’ version, erased it, kept hacking away, exploring new avenues, and by the ‘Mark 99’ version I was so enamored with rejectattr that I forgot all about how I had used it more efficiently in Mark 1. :man_shrugging:

Thank you and I’ll use it to prune my example.

I figured it must have skipped your mind. When I got home I attempted many iterations to try to optimize it even further but failed. Maybe i’ll create a PR to add a few [str] -> [type] filters, because this should be that easy.

I had considered suggesting creating a group for the 4 sensors and then using expand in the template. However, it being a Template Sensor, Home Assistant won’t create listeners for the 4 sensors so they’ll have to be explicitly listed under entity_id. I figured if I have to list them there then I might as well just list them within the filter and skip the use of a group entirely.

I’m certain any additional filters you add will prove to be very useful. If I had the requisite skills, I’d be adding support for easier date math. There really ought to be an easy means of converting a timestamp into a datetime object. The way I did it doesn’t seem very elegant.