Unavailable / Unknown Entity Monitoring - Template Sensor

Hello all,

Could someone help me figure out why I would be getting push notifications that have a title but body has no entities in it?

Essentially, what is happening is that I’ll get a notification of a device being an issue and it is correct. However, after I go fix the device which takes the unavailable entities sensor to 0, I get another notification that has no entities in it. (So a blank one)

Below is my automation. I haven’t been able to look at the persistent notification just yet so I’m not sure if it is doing the same thing or if it still looks fine. I just noticed this behavior on my phone recently.

Below is the automation I am using to call the sensor.—> I have verified that the sensor does have a value of 0.

alias: (ACTION- AUTOMATIC- SECURITY- NOTIFICATION) Unavailable Entities Notification
description: >-
  Create persistent notification if there are unavailable entities, dismiss if
  none.
trigger:
  - platform: state
    entity_id: sensor.unavailable_entities
    to: null
condition:
  - condition: template
    value_template: |
      {{ is_number(trigger.from_state.state)
          and is_number(trigger.to_state.state) }}
action:
  - if:
      - condition: numeric_state
        entity_id: sensor.unavailable_entities
        below: 1
    then:
      - service: persistent_notification.dismiss
        data:
          notification_id: unavailable_entities
    else:
      - service: persistent_notification.create
        data:
          title: Unavailable Devices
          message: |-
            - {{ expand(state_attr('sensor.unavailable_entities','entity_id'))
                  |map(attribute='name')|join('\n- ') }}
          notification_id: unavailable_entities
  - service: script.text_notify
    data:
      who: parents
      title: Unavailable Devices
      message: >-
        {{
        expand(state_attr('sensor.unavailable_entities','entity_id'))|map(attribute='name')|join('\n')
        }}
      group: null
      tag: null

Additionally, here is my package where I have only made changes to the individual entity area and select attr filters.

###################################################################################################
## Package - Unavailable Entities Sensor
## Count and list entities with a state of unavailable, unknown, or none (null)
## See README for customization options.
## https://github.com/jazzyisj/unavailable-entities-sensor/blob/main/README.md
###################################################################################################

# NOTE: Home Assistant v2021.12 required.  For older versions please see README
# REQUIRED - This is the template sensor
template:
  - sensor:
      - name: "Unavailable Entities"
        unique_id: unavailable_entities
        icon: "{{ 'mdi:alert-circle' if states('sensor.unavailable_entities')|int(0) > 0 else 'mdi:check-circle' }}"
        unit_of_measurement: entities
        state: >
          {% if state_attr('sensor.unavailable_entities','entity_id') != none %}
            {{ state_attr('sensor.unavailable_entities','entity_id')|count }}
          {% endif %}
        attributes:
          entity_id: >
            {% if state_attr('group.ignored_unavailable_entities','entity_id') != none %}
              {% set ignore_seconds = 3600 %}
              {% set ignore_ts = (now().timestamp() - ignore_seconds)|as_datetime %}
              {% set entities = states|rejectattr('domain','in',['group','button','automation','scene','script'])|selectattr('state','in',['unavailable','unknown','none'])|list %}
              {% set buttons = states.button|selectattr('state','eq','unavailable')|list %}
              {{ (entities + buttons)
                |rejectattr('entity_id','in',state_attr('group.ignored_unavailable_entities','entity_id'))
                |rejectattr('entity_id','search','keymaster')
                |rejectattr('entity_id','search','weatherbit_')
                |rejectattr('entity_id','search','media_player')
                |rejectattr('last_changed','ge',ignore_ts)
                |rejectattr('entity_id','search','weatheralerts_')
                |rejectattr('entity_id','search','octoprint_')
                |rejectattr('entity_id','search','westonspc_')
                |map(attribute='entity_id')|list }}
            {% endif %}

# REQUIRED - Add any entities you do not wish to monitor in this group.
# IMPORTANT - This group MUST exist even if empty for sensor template to render.
group:
  ignored_unavailable_entities:
    entities:
      - sensor.unavailable_entities # prevent template loop warnings?
      - binary_sensor.garage_door_access_control_barrier_sensor_low_battery_warning
      - binary_sensor.garage_door_access_control_barrier_sensor_not_detected_supervisory_error
      - binary_sensor.garage_door_access_control_barrier_unattended_operation_has_been_disabled_per_ul_requirements
      - binary_sensor.garage_door_home_security_tampering_product_cover_removed
      - binary_sensor.garage_door_low_battery_level
      - light.desk_lamp
      - sensor.gym_active_app
      - sensor.gym_active_app_id
      - sensor.living_room_active_app
      - sensor.living_room_active_app_id
      - sensor.master_bedroom_active_app
      - sensor.master_bedroom_active_app_id
      - select.living_room_channel
      - select.gym_channel
      - select.master_bedroom_channel
      - sensor.weatheralerts_alert_3_last_changed
      - sensor.weatheralerts_alert_4_last_changed
      - sensor.weatheralerts_alert_5_last_changed
      - sensor.weatheralerts_alert_3_most_recent_active_alert
      - sensor.weatheralerts_alert_4_most_recent_active_alert
      - sensor.weatheralerts_alert_5_most_recent_active_alert
      - sensor.washer_last_complete
      - sensor.pirateweather_precip
      - sensor.front_door_motion_away_count
      - sensor.lightning_near_home_lightning_azimuth
      - sensor.lightning_near_home_lightning_distance
      - sensor.holiday
# OPTIONAL - filter template loop warnings from the Home Assistant log.
logger:
  filters:
    homeassistant.components.template.template_entity:
      - "Template loop detected while processing event"
# OPTIONAL Example automation to demonstrate how you can utilize this sensor
# SEE Automation.yaml

If I have an entity or rejectattr (i.e. deleted entities and this file hasn’t been updated to match) that isn’t in my home assistant setup anymore, could that be causing this issue?

You need to indent the second service call code under the else block. Right now script.text_notify is not part if the if/else block and is running every time the automation runs even if there are no unavailable entities (so you get a blank list in your notifcation).

action:
  - if:
      - condition: numeric_state
        entity_id: sensor.unavailable_entities
        below: 1
    then:
      - service: persistent_notification.dismiss
        data:
          notification_id: unavailable_entities
    else:
      - service: persistent_notification.create
        data:
          title: Unavailable Devices
          message: |-
            - {{ expand(state_attr('sensor.unavailable_entities','entity_id'))
                  |map(attribute='name')|join('\n- ') }}
          notification_id: unavailable_entities

      - service: script.text_notify
        data:
          who: parents
          title: Unavailable Devices
          message: >-
            {{
            expand(state_attr('sensor.unavailable_entities','entity_id'))|map(attribute='name')|join('\n')
            }}
          group: null
          tag: null

If you add a tag to your push notification you could also add some code under the if block to automatically dismiss the push notifications when there are no unavailable entities similar to the persistent notification if you want.

1 Like

Geeze. Thank you Jason. I didn’t even realize my indention was wrong. Thank you so much for catching that.

Whaaaaatttttt??? Tell me more please, or possibly shhoowww me more if you happen to have a code sample around of this. That is a great idea. I had no idea you could do that. So you can make the push notification disappear off of your phone automatically when the device is corrected? Yeah, definitely would like to know more about this.

The more I use HA, the more I realize just how much better it is than other things out there. I must admit, you have to spend time in the trenches but once you get everything pretty well setup the way you want it, it’s really nice.

Thank you again Jason for alway being so helpful. Eventually I’ll get this thing figured out.

EDIT: For anyone stumbling upon this thread and looking for the concept (automatically clearing iOS notification) @jazzyisj was explaining above. Here is the completed automation.

The link to documentation for feature is: Notification Cleared | Home Assistant Companion Docs

alias: (ACTION- AUTOMATIC- SECURITY- NOTIFICATION) Unavailable Entities Notification
description: >-
  Create persistent notification if there are unavailable entities, dismiss if
  none.
trigger:
  - platform: state
    entity_id: sensor.unavailable_entities
    to: null
condition:
  - condition: template
    value_template: |
      {{ is_number(trigger.from_state.state)
          and is_number(trigger.to_state.state) }}
action:
  - if:
      - condition: numeric_state
        entity_id: sensor.unavailable_entities
        below: 1
    then:
      - service: persistent_notification.dismiss
        data:
          notification_id: unavailable_entities
      - service: script.text_notify
        data:
          who: parents
          title: Unavailable Devices
          message: clear_notification
          group: null
          tag: unavailable_entities
    else:
      - service: persistent_notification.create
        data:
          title: Unavailable Devices
          message: |-
            - {{ expand(state_attr('sensor.unavailable_entities','entity_id'))
                  |map(attribute='name')|join('\n- ') }}
          notification_id: unavailable_entities
      - service: script.text_notify
        data:
          who: parents
          title: Unavailable Devices
          message: >-
            {{
            expand(state_attr('sensor.unavailable_entities','entity_id'))|map(attribute='name')|join('\n')
            }}
          group: null
          tag: unavailable_entities
1 Like

Sorry I was away for a few days. Glad you got it figured out. Sounds like your config is really coming along :slight_smile:

No, your fine. Yeah, I’m very happy with it. HA makes all other stuff look so horrible. haha.

Def no master like yourself but I do have it doing everything I would want and it’s so customizable that you can literally make it “exactly” like you would want most of the time.

Thank you for all of the help you have given me Jason. You are very helpful.

@jazzyisj I’m sorry I keep coming back with more odd questions every little bit but the more I learn about things, things creep into my mind and I’ll be honest, you’re an expert on this stuff for sure. I’ve been reading about dead zwave nodes. If I had a “dead” zwave node would this essentially catch it by proxy? Or not at all?

My question stems from just general research about how HA works.

Thank you @jazzyisj if you or someone else can expand on this situation. Very appreciative again of everything you’ve taught me.

you’re an expert on this stuff for sure

Thank you, but I am far from that. I’ve just been at it a while.

If I had a “dead” zwave node would this essentially catch it by proxy? Or not at all?

If any of the entities for the zwave device have the state of unknown or unavailable after the node goes dead (they likely will) and you have not excluded them in the template or added them to your ignored entities group they will be reported by this sensor.

I forced a zwave node go dead and here the entities that get reported for this device.

@jazzyisj thank you for replying to this one Jason. Very helpful. I was hoping that was the case.

I’ve been using a version of this script since almost the time this thread was created. I recently decided to update to the newest version of the template sensor and noticed something different almost immediately. In the past when a sensor failed I would receive a notification (pushover) and at the same time when a sensor went back online, I would get a notification showing the updated list (minus any that were no longer unavailable).

With this new version, it doesn’t seem to send a notification when an entity is subtracted from the unavailable list. Is that the case, or am I doing something wrong?

To have your automation to trigger every time an entity becomes available/unavailable change your automation trigger to the entity_id attribute of the unavailable sensor, rather than the state.

      - platform: state
        entity_id: sensor.unavailable_entities
        attribute: entity_id
        to: ~
1 Like

I didn’t see this asked, but is there a way to add to the group a wildcard. like I don’t want to track anything that is a device_tracker. So instead of listing all of them, i would just add device_tracker.*. Also anything that is named a certain way, i.e. if there are a bunch of entities with the name master, I would like to add *master*

Is there a way to add this coding to the unavailble_entities yaml?

there a way to add to the group a wildcard

You build both of those right into the template, not in the ignore group. Group is for individual entities only. It’s thoroughly documented in the README on the git.

I got this, I meant to add it to the ignored_entities entities
like right now I have

ignored_entities:
entities:
- binary_sensor.updater
- binary_sensor.fios_router_wan_status
- binary_sensor.garage_back_door
- binary_sensor.garage_doors
- button.backup_reboot
- button.backup_shutdown
- button.tessy_flash_lights
- button.tessy_horn
- sensor.stevens_phone_charger_type
- sensor.refrigerator_temperature_measurement
- sensor.refrigerator_thermostat_cooling_setpoint
- switch.dryer_power_off
- switch.top_load_washer_power_off
- sensor.stevens_s22_ultra_charger_type
- update.home_assistant_supervisor_update
- binary_sensor.octoprint_printing

i was hoping to add another group like
ignored_entites_search
entities:
- \S+tessy_\S*
- ^(device_tracker)\S+

I understand. What I’m telling you is you can’t use templates in yami group configurations so you have to build them into the template itself. You can use regular expressions there.

that stinks, I was hoping to just add it to a group and not change the actual “sensor”

I did as was instructed but I now have an issue. my unavailable entities is below but I’m getting an “Invalid config for [group]: Entity ID ^(button|switch){1}(.octoprint_)\S+ is an invalid entity ID for dictionary value @ data[‘group’][‘ignored_entities_search’][‘entities’” and so on. What did I mess up?

sensor:
  - platform: template
    sensors:
      unavailable_entities:
        friendly_name: Unavailable Entities
        unit_of_measurement: entities
        icon_template: "{{ 'mdi:check-circle' if is_state('sensor.unavailable_entities','0') else 'mdi:alert-circle' }}"
        value_template: >
          {{ states|selectattr('state','in',['unavailable','unknown','none'])|rejectattr('domain','eq','group')
          |rejectattr('entity_id','in',state_attr('group.ignored_entities','entity_id'))
          |rejectattr('entity_id','search','^(button|switch){1}(\.octoprint_)\S+')
          |rejectattr('entity_id','search','\S+tessy_\S*')
          |rejectattr('entity_id','search','^(device_tracker)\S+')
          |rejectattr('entity_id','search','^(sensor)\S+(speaker_alarms|speaker_timers|display_alarms|display_timers)')
          |list|count }}
        attribute_templates:
          entities: >
            {{ states|selectattr('state','in',['unavailable','unknown','none'])|rejectattr('domain','eq','group')
            |rejectattr('entity_id','in',state_attr('group.ignored_entities','entity_id'))
            |rejectattr('entity_id','search','^(button|switch){1}(\.octoprint_)\S+')
            |rejectattr('entity_id','search','\S+tessy_\S*')
            |rejectattr('entity_id','search','^(device_tracker)\S+')
            |rejectattr('entity_id','search','^(sensor)\S+(speaker_alarms|speaker_timers|display_alarms|display_timers)')
            |map(attribute='entity_id')|list }}

Ok I am really lost, I updated my unavailable entitys to below and it will not filter out using the searches. Any help would be great.

template:
  - sensor:
      - name: "Unavailable Entities"
        unique_id: unavailable_entities
        icon: "{{ iif(states(this.entity_id)|int(0) > 0,'mdi:alert-circle','mdi:check-circle') }}"
        unit_of_measurement: entities
        state: >
          {% set entities = state_attr(this.entity_id,'entity_id') %}
          {% if entities != none %} {{ entities|count }} {% endif %}
        attributes:
          entity_id: >
            {% set ignore_seconds = 60 %}
            {% set ignored = state_attr('group.ignored_entities','entity_id') %}
            {% set ignore_ts = (now().timestamp() - ignore_seconds)|as_datetime %}
            {% set entities = states|rejectattr('domain','eq','group')
                |rejectattr('last_changed','ge',ignore_ts)
                |selectattr('state','in',['unavailable','unknown'])
                |rejectattr('entity_id','search','^(button|switch){1}(.octoprint_)\S+')
                |rejectattr('entity_id','search','\S+tessy_\S*')
                |rejectattr('entity_id','search','^(device_tracker)\S+')
                |rejectattr('entity_id','search','^(camera\.wyze_cam)\S+')
                |rejectattr('entity_id','search','^(sensor)\S+(speaker_alarms|speaker_timers|display_alarms|display_timers)')
                %}
            {% if ignored != none %}
              {% set entities =  entities|rejectattr('entity_id','in',ignored) %}
            {% endif %}
            {{ entities|map(attribute='entity_id')|list }}

Why not using only 1 rejectattr?


                |rejectattr('entity_id','search','device_tracker|master|wyze_cam')

Because some of the regedit is more complex. I just wanted to seperate them ench. But even with the rejectattr search it is still putting it in.

Set it back to the default template then add your filters one at a time, test them, them move on once you’ve got the current one working.

First thing I see is this |rejectattr('entity_id','search','^(device_tracker)\S+') Why use regex here?

You want to filter out all entities in the domain device tracker?
|rejectattr('domain','eq','device_tracker')

You want to filter out all entities that contain “camera.wyze_cam” in their entity_id?
|rejectattr('entity_id','search','camera.wyze_cam')

Etc. etc.