Sensor - Unavailable/Offline Detection

Tags: #<Tag:0x00007f73ae16a2c8>

OK - the sensor -

sensor:
  - platform: template
    sensors:
      unavailable_entities:
        entity_id: sensor.time
        friendly_name: Unavailable Entities
        value_template: >
          {% set count = states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
            |rejectattr('entity_id', 'in', state_attr('group.entity_blacklist', 'entity_id'))
            |rejectattr('entity_id', 'eq' , 'group.entity_blacklist')
            |rejectattr('entity_id', 'eq' , 'group.entity_whitelist')
            |map(attribute='entity_id')|list|length %}
          {%- set devices = state_attr('group.entity_whitelist', 'entity_id') -%}
          {%- set dev = namespace(value=0) -%}
            {%- for entity_id in devices -%}
              {%- if states(entity_id) in ['unavailable', 'unknown', 'none'] -%}
              {%- set dev.value = dev.value + 1 -%}
              {%- endif -%}
            {%- endfor -%}
          {{ count + dev.value }}
        attribute_templates:
          entities: >
            {% set entities = states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
              |rejectattr('entity_id', 'in', state_attr('group.entity_blacklist', 'entity_id'))
              |rejectattr('entity_id', 'eq' , 'group.entity_blacklist')
              |rejectattr('entity_id', 'eq' , 'group.entity_whitelist')
              |map(attribute='entity_id')|list|join(', ') %}
            {%- set whitelisted = state_attr('group.entity_whitelist', 'entity_id') -%}
            {%- set dev = namespace(value=0) -%}
              {%- for entity_id in whitelisted -%}
                {% if states(entity_id) in ['unavailable', 'unknown', 'none'] %}
                  {%- if not dev.value == 0 %}{{- ', ' -}}
                {% endif %}
                {%- set dev.value = dev.value + 1 -%}
                {{- entity_id -}}
              {% endif -%}
            {% endfor %}
            {%- if not dev.value == 0 %}{{- ', ' -}}{% endif %}
            {{- entities -}}

It currently updates every minute (with sensor.time) but I’m thinking of refreshing it with an automation that runs every second instead just to be sure it captures straight away when things go offline.

Obviously you’ll want

group:
  entity_whitelist:
    # Entities you want to check are there that might not be in the states list at startrup
  entity_blacklist:
    - sensor.unavailable_entities
    # any other entities that you don't mind being unavailable/unknown/none

As I say, aside from the duplication of whitelisted items that get in to the states list and then go offline, it’s pretty ticketyboo.

2 Likes

:rofl:

I have no idea what that means (but I can make an educated guess…) but for some reason I found that hilarious.

1 Like

Lol - hopefully you guessed right…

:slightly_smiling_face:

I did! :+1:

:slightly_smiling_face:

1 Like

There is something new in 0.104 that might influence the way you detect unavailable sensors:

Entity handling
Draft: One major change: we’re restoring all entity registry entities on Home Assistant startup this allows Homekit/Google/Alexa to always find all entities to expose and Lovelace will no longer show yellow/red boxes. All entities are being restored as “unavailable” they can be differentiated from normal unavailable entities by the “restored” state attribute


Now officially part of release version 0.104.
Improved entity handling on startup.

2 Likes

After going through this thread and then looking at the changes for 0.104 I think this will render the whole whitelist concept unnecessary? Am I interpreting that correctly? I’ve updated to 0.104 now and my original sensors were not affected by the change. They function as they did before but I never utilized the whitelist concept that @mf_social came up with.

I was able to scoop a couple techniques from this thread to simplify my own templates and make them a little more robust so thanks guys!

1 Like

Can confirm that whitelist is unnecessary in 104 - will post my revised sensor when I get chance.

I think it basically boils down to this…

    unavailable_entities:
      entity_id: sensor.time
      friendly_name: Unavailable Entities
      value_template: >
        {% set count = states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
          |rejectattr('entity_id', 'in', state_attr('group.ignored_sensors', 'entity_id'))
          |rejectattr('domain', 'eq', 'group')
          |map(attribute='entity_id')|list|length %}
        {{ count }}
      attribute_templates:
        entities: >
          {% set entities = states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
            |rejectattr('entity_id', 'in', state_attr('group.ignored_sensors', 'entity_id'))
            |rejectattr('domain', 'eq', 'group')
            |map(attribute='entity_id')|list|join(', ') %}
          {{- entities -}} 

This is how unavailable entities show up in lovelace now!

image

Indeed - I ended up with:

sensor:
  - platform: template
    sensors:
      unavailable_entities:
        friendly_name: Unavailable Entities
        value_template: "{{states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
            |rejectattr('entity_id', 'in', state_attr('group.entity_blacklist', 'entity_id'))
            |rejectattr('entity_id', 'eq' , 'group.entity_blacklist')
            |map(attribute='entity_id')|list|length }}"
        attribute_templates:
          entities: "{{states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
              |rejectattr('entity_id', 'in', state_attr('group.entity_blacklist', 'entity_id'))
              |rejectattr('entity_id', 'eq' , 'group.entity_blacklist')
              |map(attribute='entity_id')|list|join(', ') }}"
        unit_of_measurement: items

…And using an automation to update the sensor every second. Works well :slight_smile:

I never did get around to resolving the duplicates. However, it looks like procrastination paid off because 0.104 has obviated the need for it. Nevertheless, I don’t wish to appear as a slacker so I offer you some minor streamlining.

According to my tests, you can replace this:

            |rejectattr('entity_id', 'in', state_attr('group.entity_blacklist', 'entity_id'))
            |rejectattr('entity_id', 'eq' , 'group.entity_blacklist')

with this:

            |reject('in', expand('group.entity_blacklist'))
            |reject('eq', states.group.entity_blacklist)

You can also remove this filter from the value_template:

map(attribute='entity_id')

Here’s the streamlined version:

sensor:
  - platform: template
    sensors:
      unavailable_entities:
        friendly_name: Unavailable Entities
        value_template: >
          {{states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
            |reject('in', expand('group.entity_blacklist'))
            |reject('eq', states.group.entity_blacklist)
            |list|length}}
        attribute_templates:
          entities: >
            {{states|selectattr('state', 'in', ['unavailable', 'unknown', 'none'])
              |reject('in', expand('group.entity_blacklist'))
              |reject('eq' , states.group.entity_blacklist)
              |map(attribute='entity_id')|list|join(', ')}}
        unit_of_measurement: items
4 Likes

Thanks for putting this together, so does this streamlined version just replace the sensor section of @jazzyisj unavail_sensor.yaml? I’m not really certain what to change?

Where does this card get generated from? I don’t see anything in the github code for a lovelace card?

image

That is just one of my basic lovelace entities cards in my config. I’d have to go back through the thread to remember why it was referenced here. It has nothing to do with the unavailable entities sensor.

Not sure who you’re replying to but I think this was the final revised version of the sensor we came up with. Is this what you’re looking for?

sensor:
  - platform: template
    sensors:
      unavailable_entities:
        entity_id: sensor.time
        friendly_name: Unavailable Entities
        unit_of_measurement: items
        icon_template: >
          {% if states('sensor.unavailable_entities')|int == 0 %} mdi:check-circle
          {% else %} mdi:alert-circle
          {% endif %}
        value_template: >
          {% set count = states|selectattr('state','in',['unavailable','unknown','none'])
            |rejectattr('entity_id','in',state_attr('group.ignored_sensors', 'entity_id'))
            |rejectattr('domain','eq','group')|list|length %}
          {{ count }}
        attribute_templates:
          entities: >
            {% set entities = states|selectattr('state','in',['unavailable','unknown','none'])
              |rejectattr('entity_id', 'in', state_attr('group.ignored_sensors', 'entity_id'))
              |rejectattr('domain','eq','group')|list|join(', ') %}
            {{- entities -}}
3 Likes

@jazzyisj, thanks for providing the latest sensor for detecting unavailable entities! I’m trying to use this as a way of detecting when I’ve lost power and shut my instance of HA down gracefully to avoid data corruption. The problem I’m having is that the majority of my sensors are Z-wave dimmers (JASCO 28167) and even after 18 mins (and counting) after unplugging one of these devices, it’s still not showing as unavailable. It’s not in my ignored_sensors group. Any ideas on how to improve status reporting of the Z-wave hub (I’m using a GoControls HUSBZB-1)?

Using something like @ptdalen as shown below with polling intensity = 1 at least gets the device entity to show that the device is “Dead” pretty soon after unplugging it, but the status of the entity still shows blank as if the device is still available.

zwave:
  usb_path: /dev/ttyACM0
  config_path: /srv/homeassistant/src/python-openzwave/openzwave/config
  polling_interval: 10000
  customize:

    light.office_level_3_0: 
      polling_intensity: 1
      refresh_value: 2
      delay: 3

Okay… I think I found the solution. Along with the polling intensity set to 1 as shown above, add the status of ‘dead’ to the states being checked for in to the sensor code @jazzisj provided directly above my remarks.

I’ve been trying to get this to work for 2 weeks with no success. I have Zigbee contact sensors that report as unavailable but the script never notify’s me. Can you test it with a zigbee device?

@mboarman: when you say that your zigbee contact sensors “report as unavailable”, do you mean that you are using the sensor provided by jazzyisj and the the sensor is reporting your zigbee contact sensors as unavailable? If so, not sure how my testing is going to help. I’ve got a zigbee contact sensor, so I can test for you in the next day or so, but need you to confirm what you mean by yours is “reporting as unavailable.”

@chad01 Sorry I should have been more specific, I have a couple sensors that the battery has died and are reporting as unavailable to HA. I assumed when I installed this script I was hoping it would give me feedback on any sensors that went missing from the network or the battery has died. So far it hasn’t done either. Appreciate any help! Thanks!