Problems with complex template sensor for monitoring offline devices

Configuration
1

My Zigbee and Bluetooth environment does not always run smoothly. I would therefore like to monitor them. How do I do this? I retrieve all entities from the corresponding integrations that have the value ‘unavailable’ and retrieve the device for the entity. I then count the devices. This is then list no. 1 If the total is greater than 0, I want to be notified on the mobile phone.

The challenge here is that I have lights that can be controlled via Zigbee - but don’t have to be. The lights also each have a normal switch. I therefore have to remove all the lights from the monitoring. This means that if a light is ‘unavailable’, it doesn’t matter. To do this, I create a list of all the lights. This is list 2.

I have created a template sensor that I want to use later in automations. This is the code I built for it. Basically, it checks whether there are any devices left after comparing list 1 with list 2. It then counts them and the result of the count is the state of the sensor.

- sensor:
    - name: "OfflineDevices"
      unique_id: "sensor.offline_devices"
      icon: mdi:CreditCardWirelessOff
      state: >
        {% set light_entities = states.light | list %}
        {% set device_names = namespace(my_lights=[]) %}
        {% for light in light_entities %}
          {% set device = light.attributes.friendly_name | string %}
          {% set device_names.my_lights = device_names.my_lights + [device] %}
        {% endfor %}
        {% set unique_device_lights = device_names.my_lights | unique | list %}
        {% set result=namespace(offline_devices=[]) %}
        {% set integrations=['mqtt', 'zwave_js', 'zha', 'homekit_controller', 'govee_ble', 'switchbot'] %}
        {% for integration in integrations %}
          {% for device in integration_entities(integration) | map('device_id') | sort | unique | list %}
            {% for entity in device_entities(device) %}
              {% if "unavailable" in expand(entity)|string %}
                {% set offline_name = iif(device_attr(device, "name_by_user"), device_attr(device, "name_by_user"), "ERROR", device_attr(device, "name")) %}
                {% if offline_name not in unique_device_lights %}
                  {% set result.offline_devices = result.offline_devices + [offline_name] %}
                {% endif %}
              {% endif %}
            {% endfor %}
          {% endfor %}
        {% endfor %}
        {{ result.offline_devices | unique | list | length }}
      attributes:
        list_of_offline_devices: >
          {% set light_entities = states.light | list %}
          {% set device_names = namespace(my_lights=[]) %}
          {% for light in light_entities %}
            {% set device = light.attributes.friendly_name | string %}
            {% set device_names.my_lights = device_names.my_lights + [device] %}
          {% endfor %}
          {% set unique_device_lights = device_names.my_lights | unique | list %}
          {% set result=namespace(offline_devices=[]) %}
          {% set integrations=['mqtt', 'zwave_js', 'zha', 'homekit_controller', 'govee_ble', 'switchbot'] %}
          {% for integration in integrations %}
            {% for device in integration_entities(integration) | map('device_id') | sort | unique | list %}
              {% for entity in device_entities(device) %}
                {% if "unavailable" in expand(entity)|string %}
                  {% set offline_name = iif(device_attr(device, "name_by_user"), device_attr(device, "name_by_user"), "ERROR", device_attr(device, "name")) %}
                  {% if offline_name not in unique_device_lights %}
                    {% set result.offline_devices = result.offline_devices + [offline_name] %}
                  {% endif %}
                {% endif %}
              {% endfor %}
            {% endfor %}
          {% endfor %}
          {{ result.offline_devices | sort | unique | join(', ') }}

I’m a beginner, so I don’t think the code is very good. But at the moment I can’t do any better. I have a few problems now.

1: I think HA has slowed down significantly since I enabled the sensor.
2: I keep getting the error

 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'homeassistant.util.read_only_dict.ReadOnlyDict object' has no attribute 'friendly_name' when rendering '{% set light_entities = states.light | list %} {% set device_names = namespace(my_lights=[]) %} {% for light in light_entities %}
  {% set device = light.attributes.friendly_name | string %}...

in the log and the log is full.

Is it possible to optimise the code or build it differently so that the queries don’t run all the time?

How do I get rid of the error? How can I find out which device does not have a ‘friendly_name’? The name is essential for the comparison between the two lists.

2

this is extremely slow because of (at least) 2 key things… because you don’t have a trigger defined and limited, it’s triggering pretty much off every entity changing from anything to anything.

and then in your code, you have triple nested loops. which is n*m*o. that’s crazy slow.

you’re using friendly names to try to match and eliminate entities… if you must match names, you should use the object_id, not friendly name.

but i think you should avoid all of the above if possible. your code is complex enough that i’m cautious not to try to rewrite it to fully working, however i think something like this would be key. this should eliminate a ton of your code:

{{ expand(integration_entities('zha')) | rejectattr('entity_id', 'match', 'light') | selectattr('state', 'eq', 'unavailable') | list }}

this expands all devices using the zha, it filters out all lights, then it selects just the resulting set that are ‘unavailable’… it then lists them out, but you can replace that with | count if you want the count of such objects.

this will do ton of the work you have in your code but in a single line and do it linearly in time complexity. and it doesn’t use friendly_name so all those errors should go away…

1 Like
3

You are shooting yourself in the foot by allowing your Zigbee lights to be powered off completely.

1 Like
4

FYI this does the same thing

{{ ['mqtt', 'zwave_js', 'zha', 'homekit_controller', 'govee_ble', 'switchbot']
   | map('integration_entities') 
   | sum(start=[])
   | select('is_state', 'unavailable')
   | map('device_attr','name') | unique | list }}
5

Thank you. This code works (And I’m just trying to learn why it works. Learning Python as a non-programmer is not that easy).

But how do I rule out lights? I tried to add

  | selectattr('entity_id', 'match', '^((?!light\.)[\w]+)\.')

before

  | select('is_state', 'unavailable')

but the I get the error

'str object' has no attribute 'entity_id'

which makes me thinking that entity_id is not available at that point.

I thought my goal is simple: list the names of the devices from the mentioned integrations that have ‘unavailable’ senors (and ignore details), ignore lights, write the rest into a list for display and additionally return the number of list items. But obviously I was completely wrong.

What can I do?

6 
reject('match', '^light.')
7

Just to note: this is not Python, this is the template language Jinja2. :slight_smile:

1 Like