A sensor to list unavailable devices

Add this line to your existing template:

  | map('device_attr', 'name') | unique

Example

{{ states
  | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch'])
  | selectattr('state', 'in', ['unavailable'])
  | map(attribute='entity_id')
  | map('device_attr', 'name') | unique
  | list }}

Reference

From Templating - Devices:

1 Like

Intriguing. Can you explain a bit what ‘device_attr’ is? Is it an attribute of State that is only set to a name when the entity is the main one for the device??

I am aware of the document you quoted but there is nothing there to say, and nothing in my experiments to show, that device attributes are not available from any entity associated with the device. Filtering with them does not therefore reduce the list. I need a test that identifies exactly one instance of States (an entity) for each physical device that shows the availability of the device as a whole. It is equivalent to searching for Devices rather than Entities in the UI.

See my previous post.

Updated in parallel to your post! :grinning:

I am aware of the document you quoted but there is nothing there to say, and nothing in my experiments to show, that device attributes are not available from any entity associated with the device. Filtering with them does not therefore reduce the list. I need a test that identifies exactly one instance of States (an entity) for each physical device that shows the availability of the device as a whole. It is equivalent to searching for Devices rather than Entities in the UI.

Many thanks. I think I get it now! You have given me a more elegant way of programming my template. With the modification of using the ‘name_by_user’ and adding a filter to remove the None entries, the following produces the same result as my original code above for the device name list

{{ states
  | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch'])
  | selectattr('state', 'in', ['unavailable'])
  | map(attribute='entity_id')
  | map('device_attr', 'name_by_user') 
  | reject('match', 'None')
  | unique
  | list 
  | sort 
  | replace("'","") | replace("[","") | replace("]","") | replace(", ","\n") 
}}

Many thanks :+1:

You miss understood me.
There are physical devices that do not have any devices in HA, but just a list of entities.

I do not think it is possible to get the Device name programmatically.
In the Spook documentation it has this advice to find a device ID:

Not sure what the device_id of an your device is? There are a few ways to find it:

Use this service in the developer tools, in the UI select the device you want to add and
select the **Go to YAML mode** button. This will show you the device ID in the YAML code.

Alternatively, you can visit the device page in the UI and look at the URL. The device ID is
the last part of the URL, and will look something like this: dc23e666e6100f184e642a0ac345d3eb.

Sorry, yes I did, although I am not aware of any such devices in my installation. If you scan the entity lists above, it is easy to see which one is the main one. I can find all my devices in the Devices page of the UI.

The code I am now using gives the result I want at the moment – the device name give by the user.

My only concern with this code is that there is a theoretical possibility that one of the sub-entities of a device will go unavailable while the main entity (an the device itself) is really still available. That would put the device on the list erroneously. Whether that actually happens in practice remains to be seen! I have not tested it by turning all devices off yet.

You’re welcome.

About that last line in your template, doing multiple replaces, what’s that all about?

None of my devices have a name_by_user attribute (it’s not one of the usual device attributes) so I can’t replicate the result that required you to do all of that replacement. I’m guessing you got a list, or list of lists, or something like that, and you are trying to ‘flatten’ it.

Without that line the output is a raw list…

[
  "Back door closure sensor",
  "Conservatory inner door",
  "Guest bedroom TRV",
  "Landing socket",
  "Sophias bedroom east TRV",
  "Studio motion sensor",
  "Studio temperature and humidity"
]

That line just tidies it up by removing the unwanted characters, so we get:

Back door closure sensor
Conservatory inner door
Guest bedroom TRV
Landing socket
Sophias bedroom east TRV
Studio motion sensor
Studio temperature and humidity

Of course, you can change name_by_user back to name, if that is what you want to report. You then get the system name for the devices, which in my case is:

_TZ3000_msl6wxk9 TS0202
_TZ3000_xr3htd96 TS0201
eWeLink WB01
IKEA of Sweden TRADFRI control outlet
LUMI lumi.airrtc.agl001
LUMI lumi.sensor_magnet.aq2

In my case the list is intended for a householder with little technical knowledge. If I tell him lumi.airrtc.agl001 is offline and needs attention, that will mean nothing to him. He cannot find it easily by searching Devices in the UI, either. If, however I say that Guest bedroom TRV is offline and needs attention, he can go straight to it, check the batteries and reconnect it to ZigBee if necessary. It is easy to find in the UI under that name.

Note too that the system name is not unique. In my case, Back door closure sensor, Conservatory inner door and about 20 similar devices have the name LUMI lumi.sensor_magnet.aq2, so reporting that that ‘device’ is unavailable is not very helpful, even if you recognise it as a closure sensor.

Use join to convert the list of strings to strings delimited by newlines.

{{ states
  | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch'])
  | selectattr('state', 'in', ['unavailable'])
  | map(attribute='entity_id')
  | map('device_attr', 'name_by_user') 
  | reject('match', 'None')
  | unique
  | list 
  | sort
  | join('\n')
}}

Yep! That’s another improvement! Thanks again :bouquet:

So, in case anyone else wants it, here is the final full code – tested, tidied up and annotated.

Thanks to all of you for your help and to others on the forum and elsewhere on the internet who made contributions to this subject.

##----------------------------------------------------------------------------------------------------------------------
##
## Unavailable Entities Sensor
##
## 05-Feb-2024 | Andy Symons | created
##
## Credit: based loosely on https://github.com/jazzyisj/unavailable-entities-sensor/blob/main/README.md
##
## The sensor provides lists related to real devices, not internal entities, helpers, automations etc.,
## Entities with state 'unknown' are not counted, because it is possible for a device to have a sub-entity that is
##    unknown while the device itself is available.
##
## The STATE simply gives the count of unavailable entities.
## The long results have to be attributes because the state cannot contain more than 255 characters:
##   ATTRIBUTE 'entity_id_list' contains a list of unavailable entities using their entity ids, which may or may not have been set by the user.
##   ATTRIBUTE 'entity_name_list' contains a list of unavailable entities using their friendly names as assigned by the user.
##   ATTRIBUTE 'device_name_list contains a list of the devices that are unavailable, which is to say having one or more entities that are unavailable,
##      using their friendly names as assigned by the user.
##
##----------------------------------------------------------------------------------------------------------------------


template:
  - sensor:
      name: "Unavailable Entities"
      unique_id: unavailable_entities
      icon: "{{ iif(states(this.entity_id)|int(-1) > 0,'mdi:alert-circle','mdi:check-circle') }}"
      state_class: measurement
      unit_of_measurement: entities

      # The entity state is the count of unavailable entites
      state: >
        {{ states
        | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch'])
        | selectattr('state', 'in', ['unavailable'])
        | map(attribute='entity_id')
        | unique
        | list 
        | count
        }}

      # The long results have to be attributes because the state cannot contain more than 255 characters.
      attributes:
        ## A list of unavailable entities using their entity ids (which mnay or may not have been set by the user).
        entity_id_list: >-
          {{ states
          | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch'])
          | selectattr('state', 'in', ['unavailable'])
          | map(attribute='entity_id')
          | reject('match', 'None')
          | list 
          | sort 
          | join('\n')
          }}

        ## A list of unavailable entities using their friendly names as assigned by the user.
        entity_name_list: >-
          {{ states
          | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch']) 
          | selectattr('state', 'in', ['unavailable'])
          | map(attribute='entity_id')
          | map('state_attr', 'friendly_name') 
          | reject('match', 'None')
          | list 
          | sort 
          | join('\n')
          }}

        ## A list of the devices that are unavailable, using their friendly names as assigned by the user.
        device_name_list: >-
          {{ states
          | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch'])
          | selectattr('state', 'in', ['unavailable'])
          | map(attribute='entity_id')
          | map('device_attr', 'name_by_user') 
          | reject('match', 'None')
          | unique
          | list 
          | sort 
          | join('\n')
          }}

# HOW THE ATTRIBUTE TEMPLATES WORK
#  -- Taking device_name_list as an example...
#
# {{ states                                                        -- all the states (entities) in the system
#    | selectattr('domain','in',['binary_sensor', 'climate', etc.  -- filter only the entities for real devices
#    | selectattr('state', 'in', ['unavailable'])                  -- filter only entities that are unavailable
#    | map(attribute='entity_id')                                  -- get the entity id from the record
#    | map('device_attr', 'name_by_user')                          -- map the entity id onto the device name
#    | reject('match', 'None')                                     -- take out names 'None' (meaning there is no name, so not a device)
#    | unique                                                      -- take out duplicates (devices usually have several entities)
#    | list                                                        -- make into a list (in the template sense)
#    | sort                                                        -- put them in alphabetical order
#    | join('\n')                                                  -- take out extraneous punctuation for a tidy output
#  }}

This web page is colouring the last few lines incorrectly; it’s just comment and will be fine when you paste it into a YAML file.

4 Likes

FWIW, there are one or two other templating shortcuts that are possible but, seeing that you have already have your own Solution, it’s good enough.

I would like to see them anyway!
I readily confess that my solution was achieved more by trial-and-error than by a deep understanding of how templates work :person_shrugging:

I’m running into an issue using this because I have a separate sensors.yaml and when I paste this in, I think the indentation is wrong. Below are the non-critical errors that only exist when I paste the code in to my YAML file and attempt to check the HA config.

  - platform: template
    sensors:
        unavailable_entities:
            name: "Unavailable Entities"
            unique_id: unavailable_entities
            icon: "{{ iif(states(this.entity_id)|int(-1) > 0,'mdi:alert-circle','mdi:check-circle') }}"
            state_class: measurement
            unit_of_measurement: entities

      # The entity state is the count of unavailable entites
    state: >
        {{ states
        | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch'])
        | selectattr('state', 'in', ['unavailable'])
        | map(attribute='entity_id')
        | unique
        | list 
        | count
        }}

      # The long results have to be attributes because the state cannot contain more than 255 characters.
    attributes:
        ## A list of unavailable entities using their entity ids (which mnay or may not have been set by the user).
        entity_id_list: >-
          {{ states
          | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch'])
          | selectattr('state', 'in', ['unavailable'])
          | map(attribute='entity_id')
          | reject('match', 'None')
          | list 
          | sort 
          | join('\n')
          }}

        ## A list of unavailable entities using their friendly names as assigned by the user.
    entity_name_list: >-
          {{ states
          | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch']) 
          | selectattr('state', 'in', ['unavailable'])
          | map(attribute='entity_id')
          | map('state_attr', 'friendly_name') 
          | reject('match', 'None')
          | list 
          | sort 
          | join('\n')
          }}

        ## A list of the devices that are unavailable, using their friendly names as assigned by the user.
    device_name_list: >-
          {{ states
          | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch'])
          | selectattr('state', 'in', ['unavailable'])
          | map(attribute='entity_id')
          | map('device_attr', 'name_by_user') 
          | reject('match', 'None')
          | unique
          | list 
          | sort 
          | join('\n')
          }}


I don’t see your log with errors — only YAML config.

state and attributes should be indented more, but if you learn to interpret the errors and check the docs, you can fix it yourself.

If you have a separate sensors.yaml, then the key sensor is already in your configurations.yaml, presumably (as sensor: !include sensor.yaml) so you should not have the sensors keyword again in sensors.yaml

I have labeled mine, so it is written like this:

sensor sun2: !include sun2.yaml
sensor climate: !include climate.yaml
...

I managed to get it down to one error.

Invalid config for 'template' from integration 'sensor' at sensors.yaml, line 394: 'attributes' is an invalid option for 'sensor.template', check: sensors->unavailable_entities->attributes

I had to replace a bunch of the items like “state” with “value_template”, etc. I’m not sure what to do with the attributes section. I tried various indents. See below for updated YAML. I know the beginning is correct because all the other items in my sensors.yaml are formatted the same way.

  - platform: template
    sensors:
        unavailable_entities:
            friendly_name: "Unavailable Entities"
            unique_id: unavailable_entities
            icon_template: "{{ iif(states(this.entity_id)|int(-1) > 0,'mdi:alert-circle','mdi:check-circle') }}"
            unit_of_measurement: entities

# The entity state is the count of unavailable entites
            value_template: >-
                {{ states
                | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch'])
                | selectattr('state', 'in', ['unavailable'])
                | map(attribute='entity_id')
                | unique
                | list 
                | count
                }}

# The long results have to be attributes because the state cannot contain more than 255 characters.
            attributes:
# A list of unavailable entities using their entity ids (which mnay or may not have been set by the user).
                entity_id_list: >-
                    {{ states
                    | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch'])
                    | selectattr('state', 'in', ['unavailable'])
                    | map(attribute='entity_id')
                    | reject('match', 'None')
                    | list 
                    | sort 
                    | join('\n')
                    }}

# A list of unavailable entities using their friendly names as assigned by the user.
                entity_name_list: >-
                    {{ states
                    | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch']) 
                    | selectattr('state', 'in', ['unavailable'])
                    | map(attribute='entity_id')
                    | map('state_attr', 'friendly_name') 
                    | reject('match', 'None')
                    | list 
                    | sort 
                    | join('\n')
                    }}

# A list of the devices that are unavailable, using their friendly names as assigned by the user.
                device_name_list: >-
                    {{ states
                    | selectattr('domain','in',['binary_sensor', 'climate', 'light', 'sensor', 'switch'])
                    | selectattr('state', 'in', ['unavailable'])
                    | map(attribute='entity_id')
                    | map('device_attr', 'name_by_user') 
                    | reject('match', 'None')
                    | unique
                    | list 
                    | sort 
                    | join('\n')
                    }}

Thanks a million for sharing your final code! Really helpful!

This is for those who are having the same problem with an empty device_name_list:
Both attributes entity_id_list and entity_name_list work fine at my end.
However, I had a problem that the attribute device_name_list produced just an empty value. Reason was that the list is based on name_by_user which needs to be manually defined for each device which I never did.

My workaround was to replace name_by_user by just name.
Accordingly, my corresponding line in the device_name_list section is the following:
map('device_attr', 'name') instead of map('device_attr', 'name_by_user')