Unavailable / Unknown Entity Monitoring - Template Sensor

[jazzyisj (Jason)]

You have placed your additions to the template after the closing bracket for the statement in the template that should have contained them.

Had you put your actual code in your comment instead of a screenshot I could have put a corrected version up for you to use.

Instead here is an example from my config for you.

          {% set ignore_seconds = 60 %}
          {% set ignored = state_attr('group.ignored_unavailable_entities', 'entity_id') %}
          {% set ignore_ts = (now().timestamp() - ignore_seconds) | as_datetime %}
          {% set entities = states
              | rejectattr('domain', 'in', ['button', 'device_tracker', 'event', 'group', 'input_button', 'input_text', 'scene'])
              | rejectattr('entity_id', 'in', integration_entities('browser_mod'))
              | rejectattr('entity_id', 'in', integration_entities('hassio'))
              | rejectattr('entity_id', 'in', integration_entities('dlna_dmr'))
              | rejectattr('entity_id', 'contains', 'jason_laptop_media_player')
              | rejectattr('entity_id', 'contains', 'jason_phone_media_player')
              | rejectattr('entity_id', 'contains', 'jason_tablet_media_player')
              | rejectattr('entity_id', 'contains', 'charlie_crate_camera')
              | rejectattr('entity_id', 'contains', 'living_room_camera')
              | rejectattr('entity_id', 'search', 'jphone_app|jtablet_app|jlaptop_chrome|jwork_chrome
                |_dlna|_timers|_alarms|_next_alarm|sensor.daily_energy_consumption')
              | rejectattr('last_changed', 'ge', ignore_ts) %}
          {% set entities =  entities | rejectattr('entity_id', 'in', ignored) if ignored != none else entities %}
          {{ entities | map(attribute='entity_id') | reject('has_value') | list | sort }}

You also should test your templates in Developer Tools template editor to see if they evaluate as expected before putting them in your configuration.

Additionally, if you are only editing a template, only need to reload templates. You don’t need to reload your entire YAML config.

Hope this all helps!

[Mar1us]

Currently i use a group of entities i want to ignore. Is there also a way to ignore all entities of a device in a different defined group?

[jazzyisj (Jason)]

Excluding all entities from a device.

[Mar1us]

thats i already red but here i need one line per device or? Thats why i asked to use a group of devices?

[jazzyisj (Jason)]

You’ll need to give more details then. How many devices are you talking about here? Are all the devices you want to exclude part of the same integration? If so, you can just exclude the entire integration.

[Mar1us]

I have a bunch of devices (from different integrations) i want to exclude these. So my idea was to create a group with these device ids that should be excluded.

[jazzyisj (Jason)]

A group can only contains entities, not devices. My advice would be to exclude each device in your template as described in the readme.

[Mar1us]

ah ok - thank you!

[e-raser]

Someone ever ran into this issue (ignored entities group: maximum number of entities)?

Groups: maximum number of entities

Do groups have a maximum number of entities? I couldn’t find any at Group - Home Assistant Asking as I have this: group: problems_any_ignored: name: Problematische Entitäten (ignorierte) icon: mdi:debug-step-over entities: - ... - ... - ... with many entities. {{ expand('group.problems_any_ignored') | length }} gives: 128. …and that looks like a potential binary system limit to me. When adding more entities and reloading template sensors, the newly added on…

[jazzyisj (Jason)]

I have not run into that specific issue. Mobile, so I can’t test to confirm.

But as an immediate solution, are there common elements between some of your entities such as integration, naming, schema, etc. that you could use to filter them using other methods as described in the readme?

[e-raser]

Yes I could. But it seems like there’s another issue. The group had 131 elements before the last HA Core restart, then it went down to 128. So I need to investigate that.

[jazzyisj (Jason)]

Please update if you find anything out. Seems like something that should have been mentioned in the release notes if there was indeed a change.

[e-raser]

Solved by a simple restart, see Groups: maximum number of entities - #3 by e-raser

I did not update HA Core and I’m sure something like that would be mentioned in the breaking changes section. Sorry to bother you here.

Anyway I hope the topic will get an answer (in case IF there is a real limitation).

[itcsburnett (Steve)]

I’m struggling to get this to work and iterate only offline devices in the ZHA & ZWave_JS integrations. As those are mesh networks, they are really the only ones I am concerned about in my setup (I was troubleshooting a flakey ZHA network, and it turned out a tuya repeater had died).

Reading the docs, I came across that we can select just the entities from specific integrations. I can get it to work for one integration, but not both zha & zwave.

Here’s what works:

###################################################################################################
## Package - Unavailable Entities Sensor
## Count and list entities with a state of unknown or unavailable
## Home Assistant v2022.5 required. See README for customization options.
## https://github.com/jazzyisj/unavailable-entities-sensor/blob/main/README.md
###################################################################################################

# REQUIRED - This is the template sensor

template:
  - sensor:
      - name: "Unavailable Entities"
        unique_id: unavailable_entities
        icon: "{{ iif(states(this.entity_id)|int(-1) > 0,'mdi:alert-circle','mdi:check-circle') }}"
        state_class: measurement
        unit_of_measurement: entities
        state: >
          {% set entities = state_attr(this.entity_id,'entity_id') %}
          {{ entities|count if entities != none else none }}
        attributes:
          entity_id: >
            {% set ignore_seconds = 60 %}
            {% set ignored = state_attr('group.ignored_unavailable_entities','entity_id') %}
            {% set ignore_ts = (now().timestamp() - ignore_seconds)|as_datetime %}
            {% set entities = states
                |rejectattr('domain','in',['button','event','group','input_button','input_text','scene'])
                |rejectattr('last_changed','ge',ignore_ts) 
                |selectattr('entity_id','in',integration_entities('zwave_js'))
            {% set entities =  entities|rejectattr('entity_id','in',ignored) if ignored != none else entities %}
            {{ entities|map(attribute='entity_id')|reject('has_value')|list|sort }}


# OPTIONAL - Uncomment and add entities you want to ignore to this group.

group:
  ignored_unavailable_entities:
     entities:
      - sensor.1e0de372_f503aec8

I’ve tried adding zha in the same selectattr statement and get UNAVAILABLE:

        attributes:
          entity_id: >
            {% set ignore_seconds = 60 %}
            {% set ignored = state_attr('group.ignored_unavailable_entities','entity_id') %}
            {% set ignore_ts = (now().timestamp() - ignore_seconds)|as_datetime %}
            {% set entities = states
                |rejectattr('domain','in',['button','event','group','input_button','input_text','scene'])
                |rejectattr('last_changed','ge',ignore_ts) 
                |selectattr('entity_id','in',integration_entities('zwave_js','zha')) %}
            {% set entities =  entities|rejectattr('entity_id','in',ignored) if ignored != none else entities %}
            {{ entities|map(attribute='entity_id')|reject('has_value')|list|sort }}

then I tried just adding a separate selectattr line with zha:

        attributes:
          entity_id: >
            {% set ignore_seconds = 60 %}
            {% set ignored = state_attr('group.ignored_unavailable_entities','entity_id') %}
            {% set ignore_ts = (now().timestamp() - ignore_seconds)|as_datetime %}
            {% set entities = states
                |rejectattr('domain','in',['button','event','group','input_button','input_text','scene'])
                |rejectattr('last_changed','ge',ignore_ts) 
                |selectattr('entity_id','in',integration_entities('zwave_js'))
                |selectattr('entity_id','in',integration_entities('zha'))%}

Now I get 0 (zero) unavailable

Id also prefer to display the Device name rather than all the entities of the unavailable device if possible

[ktdt00]

I read through the thread and didn’t see it mentioned but if you’re using the code from the readme to display unavailable entities in the “More Info”, and you also want the default persistent notification to work too, you’ll also have to make the entity_id changes in the automation (or at least I did ) YMMMV

[mterry63 (Martin)]

@jazzyisj I also was interested if this template sensor could be modified to ignore entities of disabled devices after taking down my Patio Roku TV for the weather and disabling the device. After a bit of digging and testing, I can see that the core.entity_registry file marks entities as disabled by “config_entry” when the parent device is disabled.

Since this file is JSON, I worked out a command line sensor that uses jq to filter by entities that are marked as disabled by “config_entry”. Additionally, the jq filter has to flatten the resulting JSON to make the array of entities appear in the root level as command line doesn’t support specifying a path to pull data from.

Here is the resulting command line sensor configuration. I’m no JSON or jq expert, so the jq filter may easily be improved, but this sensor configuration has the count of disabled entities as the state, and an array of entity names as an attribute.

command_line:
  sensor:
      name: Disabled Device Entities
      unique_id: amn_disabled_device_entities
      json_attributes:
        - entities
      value_template: >
        {{ value_json.entities | length }}
      command: 'jq ''.data.entities |= map(select(.disabled_by? == "config_entry") | {name: .entity_id}) | del(.data.deleted_entities) | flatten | .[3]'' < .storage/core.entity_registry'

Adding this to configuration.yaml and reloading command line sensors results in this:

Each name is accessible via state_attr:

{{ state_attr('sensor.disabled_device_entities','entities')[0].name }}

Next steps are to incorporate checking these names in the template to ignore any matches.

This is kind of a hack, but I think it will work.

[bacco007]

@mterry63

command: 'jq ''.data.entities |= map(select(.disabled_by? == "config_entry") | {name: .entity_id}) | del(.data.deleted_entities) | flatten | .[3]'' < .storage/core.entity_registry'

I’ve extended this to add what integration the entity relates to, and to include all reasons for disabled_by - which might be assistance for someone else

'jq ''.data.entities |= map(select(.disabled_by? != null) | {name: .entity_id, platform: .platform, disabled_by: .disabled_by}) | del(.data.deleted_entities) | flatten | .[3]'' < .storage/core.entity_registry'

[mterry63 (Martin)]

Following up to completion, I modified the sensor in packages_unavailable_entities.yaml as follows:

under: 
{% set ignored = state_attr('group.ignored_unavailable_entities','entity_id') %}

I added:
{% set ignored2 = state_attr('sensor.disabled_device_entities', 'entities')|regex_replace(find='\[|\]|\{|\}|\'name\':', replace='') %}

and under :
{% set entities =  entities|rejectattr('entity_id','in',ignored)
    if ignored != none else entities %}

I added:
{% set entities =  entities|rejectattr('entity_id','in',ignored2)
    if ignored2 != none else entities %}

Checked the configuration, reloaded the template sensors and now my entities of disabled devices are ignored by the sensor.

I’m sure this could be improved but I’m pretty new at Jinja2 and Python.

[jazzyisj (Jason)]

            {% set entities = states
                |rejectattr('domain','in',['button','event','group','input_button','input_text','scene'])
                |rejectattr('last_changed','ge',ignore_ts) 
                |selectattr('entity_id','in',integration_entities('zwave_js'))
                |selectattr('entity_id','in',integration_entities('zha'))%}

Template filters are progressive so in the first selectattr filter you have told the template to select only entities in the zwave integration so the second selectattr is pointless becaue the zha integration entities were filtered out by the first selectattr statement.

Set the entities for each integration into list object variables and add them together to use in the filter.

{% set ignore_seconds = 60 %}
{% set ignored = state_attr('group.ignored_unavailable_entities', 'entity_id') %}
{% set ignore_ts = (now().timestamp() - ignore_seconds) | as_datetime %}
{% set zwave = states | selectattr('entity_id','in',integration_entities('zwave_js')) | list %}
{% set zha = states | selectattr('entity_id','in',integration_entities('zha')) | list %}
{% set entities = (zwave + zha) 
    |rejectattr('domain','in',['button','event','group','input_button','input_text','scene'])
    |rejectattr('last_changed','ge',ignore_ts) %}
{% set entities =  entities | rejectattr('entity_id', 'in', ignored) if ignored != none else entities %}
{{ entities | map(attribute='entity_id') | reject('has_value') | list | sort }}  

Make sure zha is the proper slug. I don’t use it so I can’t check.
Open the integration in Settings → Devices. It will be the last part of the url.

[jazzyisj (Jason)]

This is interesting. When I get a minute I’ll have a good look at this, I might add an example to the documentation.