Better Blueprint to report when any battery device (Zigbee/Zwave etc) or Zigbee/Zwave SmartSwtich has gone offline

I found it not obvious enough on how to get notified when battery Zigbee / Zwave devices, or smartplugs, have fallen off the network…

So, I made this blueprint that will report when any of your Zigbee/Zwave(Battery) devices falls off the network. Works with ZHA & Z2M. Works for any SmartPlugs / SmartSwitches also.

You can easily get a notification to your phone, or alerts spoken to your Alexa/Google speaker etc, when any device has gone offline.

This is an upgrade over my previous version - the main difference is this reports just the device that has fallen offline, whereas my previous version would report all sensors that were unreachable. This was unneccesarily verbose when one device had multiple sensors… Also cleaned up the code to make it more readable.

Open your Home Assistant instance and show the blueprint import dialog with a specific blueprint pre-filled.

Use {{offline_devices}} to get a list of all offline devices.
Below is an example on how you woud notify your phone with a list of offline devices.

service: notify.myphone
data:
  message: "The following devices are unavailable: {{ offline_devices }}"

Click here for direct link to yaml source on github.

Full source:

blueprint:
  name: Report offline zigbee/zwave/battery/smart plug devices
  description: Works with Smart Plugs, ZWave, Zigbee etc (Works with ZHA & Z2M)
  #By Tahutipai 2024-02-21
  #Originally Based on the work of Sybx @ https://community.home-assistant.io/t/low-battery-level-detection-notification-for-all-battery-sensors/258664
  #Note: This has been upgraded to report only the device that is offline, not multiple individual sensors within one device
  domain: automation
  input:
    time:
      name: Time to test on
      description: Test is run at configured time
      default: '10:00:00'
      selector:
        time: {}
    day:
      name: Weekday to test on
      description: 'Test is run at configured time either everyday (0) or on a given
        weekday (1: Monday ... 7: Sunday)'
      default: 0
      selector:
        number:
          min: 0.0
          max: 7.0
          mode: slider
          step: 1.0
    exclude:
      name: Excluded Sensors
      description: Battery sensors (e.g. smartphone) to exclude from detection. Only entities are supported, devices must be expanded!
      default: {entity_id: []}
      selector:
        target:
          entity:
            device_class: battery
    actions:
      name: Actions
      description: Call your notification here. {{offline_devices}} will replaced with the name of any offline devices
      selector:
        action: {}
  source_url: https://gist.github.com/Tahutipai/971bf0e07e50ce6190e0dacd73262e2e
variables:
  day: !input 'day'
  exclude: !input 'exclude'
  offline_devices: >-
    {% set result = namespace(offline_devices=[]) %}
    {% for sensor in states.sensor | selectattr('attributes.device_class', 'defined') | selectattr('attributes.device_class', '==', 'battery') %}
      {% if "unavailable" in sensor | string  and not sensor.entity_id in exclude.entity_id %}
        {% set result.offline_devices = result.offline_devices + [device_attr(device_id(sensor.entity_id), "name")] %}
      {% endif %}
    {% endfor %}
    {% for binary_sensor in states.binary_sensor | selectattr('attributes.device_class', 'defined') | selectattr('attributes.device_class', '==', 'battery') %}
      {% if "unavailable" in binary_sensor | string and not binary_sensor.entity_id in exclude.entity_id %}
        {% set result.offline_devices = result.offline_devices + [device_attr(device_id(binary_sensor.entity_id), "name")] %}
      {% endif %}
    {% endfor %}
    {% for switch in states.switch | selectattr('state','eq','unavailable') %}    
      {% if switch.entity_id not in exclude.entity_id %}
        {% set result.offline_devices = result.offline_devices + [device_attr(device_id(switch.entity_id), "name")] %}   
      {% endif %}
    {% endfor %}  
    {{result.offline_devices|join('\n')}}
trigger:
- platform: time
  at: !input 'time'
condition:
- '{{ offline_devices != '''' and (day | int == 0 or day | int == now().isoweekday()) }}'
action:
- choose: []
  default: !input 'actions'
mode: single
4 Likes

it seems that the blueprint will only report all battery operated devices (not just zwave or zigbee) or switches.

There is no option for other device types (e.g. lights)

And it doesn’t discriminate between zwave/zigbee and all other integrations.

Is that the correct intended behavior? if so the the title is a bit misleading. A better title would be “Better Blueprint to report when any Switch or Battery device has gone offline”

Example is I have a few zwave and zigbee bulbs that won’t report unavailable and a couple of non-zwave/zigbee ESPHome and various other switches that will report as unavailable. Obviously the extraneous sensors I don’t want to track can be fixed by adding them to the exclude list. But that doesn’t help with the missing light domain.

I don’t use blueprints but the title looked interesting to see how you filtered on zwave or zigbee and then saw that you really didn’t. At least as far as I can see.

1 Like

Ah, you are completely 100% right. Sorry about that, I’ve just upated the post to reflect your comments.

Yes: This blueprint works for all “Battery” and “Switch” devices. I’ve only ever had battery devices drop off the network myself (often from simply flat batteries). But after I posted my original blueprint for that, many people from the community wrote to me and asked me to extend it to also include their SmartPlugs…apparently that’s a problem… so I added that on request.

I’ve never personally had a single light bulb ever go offline in years - but that might be because I have Philips Hue, and have not tried other types yet. Hue is expensive but rock solid.
(And if a family members turns a physical light switch off, disconnecting a bulb’s power, I don’t want a notification about that - as we still use good 'ole fashioned light switches in some rooms :slight_smile: So that is “normal operation” in this house.

However…it would be easy to extend this blueprint to include lights…

I can still help you out with your original query though, I’ll give you some code you can use to filter by integration i.e exactly what you said you were looking for…

Do you use Z2M or ZHA?

This will show all offline Z2M devices (/all mqtt)

{{ states | selectattr('entity_id', 'in', integration_entities('mqtt')) | selectattr('state', 'in', ['unavailable', 'unknown', 'none']) | map(attribute='entity_id') | list }}

This will show all offline ZHA devices (combine with above to get all offline Zigbee devices, but I imagine very few people acutally use both Zigbee integrations…?)

{{ states | selectattr('entity_id', 'in', integration_entities('zha')) | selectattr('state', 'in', ['unavailable', 'unknown', 'none']) | map(attribute='entity_id') | list }}

Note that even the above wouldn’t work for my zigbee bulbs, as I run them through the Hue automation still, so if I wanted that, then I would filter for the Hue integration, instead of ZHA/Z2M…

So, if I truely wanted to build a “Zigbee Device Offline” for everyone, it would need to capture at least those three integrations, there are probably more…

I’m sure you can figure out ZWave from the above…:slight_smile:
(Let me know if you want code to progmatically generate a list of all Integrations, I find it handy to do that sometimes)

Thanks for the correction!

1 Like

I’ve already tried the integration_entities('integration') filter but it returns every entity associated with the integration even tho many are included in the same device (button, battery, sensor, switch, light, etc.).

So as in my case if a light bulb goes offline (which happens as you say when someone turns off the switch) then I would get a notification including all of the associated device entities (‘light.stairway_light’, ‘number.stairway_bulb_on_level’, ‘sensor.stairway_light_power’, ‘sensor.stairway_light_energy’) along with every other ‘button.’ entity associated with all the other zigbee and zwave devices that haven’t been ‘pressed’ so they are all ‘unknown’.

Right now I use a state trigger for a ‘representative’ entity_id for each device in question (battery_level sensors for battery powered stuff, light.x, switch.x, etc for non-battery powered stuff) goes to ‘unavailable’. But then the trigger entity list gets pretty long and has to be maintained when devices get added or removed.

It would be nice to just have a filter for integration_devices() so that you could use that without having to exclude all of the entities but one for each device. As that would be even harder to maintain by needing to exclude all of the massive number of extraneous entities associated with the devices.

I was just hoping that someone had figured out some ingenious way to filter on the integration and only get one entity for each device instead of all entities for each device.

If I’m correctly understanding (part of) the issue you describe (apologies, it’s late where I am), then ya that is basically the main “upgrade” between this blueprint and the one before it, more or less.

The problem I am referring to is “one device goes offline, thus all the entities of that device go offline”. In my case, I just grabed one (any/all) offline entities, and then worked backwards to get the Device.

So then I just report Offline Devices, not Offline Entities.

If that is what you are looking for, an example is on line 49 of the gist

The specific bit of code is:

device_attr(device_id(sensor.entity_id), "name")

i.e from the entity_id I find it’s device_id, and then using the attributes of the device, i grab the name field to report to the user.

(at the end I do a offline_devices|sort|unique|join, to get rid of any duplicate devices, not elegant but functional)

So, regardless of how many entities a device might have, we only report back the Single Device, once.

Is that what you are looking for? (I’m not entirely sure I understood your whole post tbh :sleeping_bed:)

1 Like

You’re right. That’s the bit I was missing. I forgot about the device_attr() method.

Thanks for that. I’ll need to play with the template to get it where I want it for my desired use.

yeah, that’s what I was looking for.

:+1:

1 Like

Actually I was just playing around with your template a bit but I think the issue is if the unavailable entity isn’t associated with a device (I have an MQTT switch that goes unavailable occasionally - like now) then the template for the switch portion will error because the ‘device_id()’ method won’t work if the entity has no associated device.

Here is how I solved that error:

.
.
{% for switch in states.switch | selectattr('state','eq','unavailable') %}
  {% if device_id(switch.entity_id) != None %}
    {% set result.offline_devices = result.offline_devices + [device_attr(device_id(switch.entity_id), "name")] %}
  {% endif %}
{% endfor %}
.
.

I removed the exclude entities for my testing but I think this should work if you add it back:

.
{% for switch in states.switch | selectattr('state','eq','unavailable') %}
  {% if switch.entity_id not in exclude.entity_id %}
    {% if device_id(switch.entity_id) != None %}
      {% set result.offline_devices = result.offline_devices + [device_attr(device_id(switch.entity_id), "name")] %}
    {% endif %}
  {% endif %}
{% endfor %}

then you could add the light domain as well:

.
{% for switch in states.switch | selectattr('state','eq','unavailable') %}
  {% if switch.entity_id not in exclude.entity_id %}
    {% if device_id(switch.entity_id) != None %}
      {% set result.offline_devices = result.offline_devices + [device_attr(device_id(switch.entity_id), "name")] %}
    {% endif %}
  {% endif %}
{% endfor %}
{% for light in states.light | selectattr('state','eq','unavailable') %}
  {% if light.entity_id not in exclude.entity_id %}
    {% if device_id(light.entity_id) != None %}
      {% set result.offline_devices = result.offline_devices + [device_attr(device_id(light.entity_id), "name")] %}
    {% endif %}
  {% endif %}
{% endfor %}

But again this will give you every device and not just those associated with the zwave or zigbee integration. I haven’t worked out a way to filter on just those yet.

----------EDIT----------------

I’ve done some more editing and I figured out a way to filter on only zwave & zigbee.

It’s not pretty and there may be a better way to do it to make it more efficient but as far as I know it should work.

Also in my playing around I noticed that adding entities by “name” might not always get the result thatr someone might want. I had a few of my devices that got a very generic name when I included them so getting a result like “Sengled G11-G13” isn’t going to help you get directly to the correct device. So you might want to investigate that part further. I started using “name_by_user” instead.

But here is the abomination template I came up with:

    {% set result = namespace(offline_devices=[]) %}
    {% for sensor in states.sensor | selectattr('attributes.device_class', 'defined') | selectattr('attributes.device_class', '==', 'battery') %}
      {% if "unavailable" in sensor | string  and not sensor.entity_id in exclude.entity_id %}
        {% if device_id(sensor.entity_id) != None %}
	      {% if (device_attr(device_id(sensor.entity_id), "identifiers") | list)[0] is defined %}
            {% if 'zha' in (device_attr(device_id(sensor.entity_id), "identifiers")| list)[0] or 'zwave_js' in (device_attr(device_id(sensor.entity_id), "identifiers")| list)[0] %}
              {% set result.offline_devices = result.offline_devices + [device_attr(device_id(sensor.entity_id), "name")] %}
          	{% endif %}
		  {% endif %}
	    {% endif %}    
      {% endif %}
    {% endfor %}
    {% for binary_sensor in states.binary_sensor | selectattr('attributes.device_class', 'defined') | selectattr('attributes.device_class', '==', 'battery') %}
      {% if "unavailable" in binary_sensor | string and not binary_sensor.entity_id in exclude.entity_id %}
        {% if device_id(binary_sensor.entity_id) != None %}
	      {% if (device_attr(device_id(binary_sensor.entity_id), "identifiers") | list)[0] is defined %}
            {% if 'zha' in (device_attr(device_id(binary_sensor.entity_id), "identifiers")| list)[0] or 'zwave_js' in (device_attr(device_id(binary_sensor.entity_id), "identifiers")| list)[0] %}
              {% set result.offline_devices = result.offline_devices + [device_attr(device_id(binary_sensor.entity_id), "name")] %}
            {% endif %}
		  {% endif %}
	    {% endif %}    
      {% endif %}
    {% endfor %}
    {% for switch in states.switch | selectattr('state','eq','unavailable') %}
	  {% if switch.entity_id not in exclude.entity_id %}
         {% if device_id(switch.entity_id) != None %}
	      {% if (device_attr(device_id(switch.entity_id), "identifiers") | list)[0] is defined %}
            {% if 'zha' in (device_attr(device_id(switch.entity_id), "identifiers")| list)[0] or 'zwave_js' in (device_attr(device_id(switch.entity_id), "identifiers")| list)[0] %}
		      {% set result.offline_devices = result.offline_devices + [device_attr(device_id(switch.entity_id), "name")] %}
	        {% endif %}
		  {% endif %}
	    {% endif %}
	  {% endif %}
    {% endfor %}
    {% for light in states.light | selectattr('state','eq','unavailable') %}
	  {% if light.entity_id not in exclude.entity_id %}
        {% if device_id(light.entity_id) != None %}
          {% if (device_attr(device_id(light.entity_id), "identifiers") | list)[0] is defined %}
            {% if 'zha' in (device_attr(device_id(light.entity_id), "identifiers")| list)[0] or 'zwave_js' in (device_attr(device_id(light.entity_id), "identifiers")| list)[0] %}
		      {% set result.offline_devices = result.offline_devices + [device_attr(device_id(light.entity_id), "name")] %}
	        {% endif %}
		  {% endif %}
	    {% endif %}
	  {% endif %}
    {% endfor %}
    {{result.offline_devices|sort|unique|join('\n')}}

it will only return zwave & zha devices that have a sensor or binary sensor battery device class or are a switch or light.

I hope it helps…even if you didn’t know you needed it… :laughing:

1 Like

Hey wow thanks for all of this :slight_smile:

Can you give me an example of this? Am curious, I don’t (yet) understand a switch that doesn’t have a device?

FYI I had a play around based on all the excellent work you did above. & I found a nice & very short way to get all offline devices, by integration.
The following code will report any offline Z2M/ZHA/ZWave devices,

(Note: I found when I switched to “name_by_user” that would fail for me on devices where it was “None”, so the below code includes error checking with fallback to just “name” when needed)


    {% set result=namespace(offline_devices=[]) %} 
    {% set integrations=['mqtt', 'zwave_js', 'zha'] %}
    {% for integration in integrations %}
      {% for device in integration_entities(integration) | map('device_id') | sort | unique | list %}
        {% for entity in device_entities(device) %}
          {% if entity.startswith(('switch','sensor','binary_sensor')) and "unavailable" in expand(entity)|string  %}
            {# add error checking, in case name_by_user==None #}
            {% set device_name=iif(device_attr(device,"name_by_user"), device_attr(device,"name_by_user"), "ERROR", device_attr(device,"name")) %}
            {% set result.offline_devices= result.offline_devices + [device_name] %}
          {% endif %}
        {% endfor %}
      {% endfor %}
    {% endfor %}
    {{result.offline_devices|sort|unique|list}}

So this code is conceptually much simpler than I had before :slight_smile:

  1. First interate through a list of integrations (Zwave, Zigbee etc)…
  2. Then for each integration, get a list of all its devices…
  3. The for each device, get a list of all it’s relevant entities [sensor, binary_sensor, switch]…
  4. Then just check if any of those entities are offline…

Hopefully this “just works” when adding other integrations, like Tasmota or whatever - I don’t know, I dont have any of those…By grabbing devices first, we avoid cedge ases where an entity doesn’t have a device. (which I dont understand yet, I don’t have anything like that?)

Note: the above doesn’t grab “all battery devices” or the light domain etc, like the previous code from you and I does. I guess the ultimate blueprint would add all of this together :slight_smile: Maybe with boolean checkboxes, so the user can choose what kind of devices they want scanned for…

I don’t personally have any battery devices that are not Zigbee or Zwave.

1 Like

Example Switch with no device I have is to turn my office fan on and off. It exists only as an entity related to mqtt.

I’m not sure if this is why, but I’m having this blueprint fail with ‘null’ as soon as triggered.

Also in terms of non-zigbee / zwave battery devices, phones or tablets for dashboards would be a common one.

Thanks for the further development. That’s way more compact than my first attempts. :laughing:

Back in the day we didn’t have the concept of devices so everything was just entities. Not everything has been switched to the device schema so there are still lots of things that don’t have associated devices. As in the example I used above.

It’s an MQTT switch that I use to turn on and off a computer monitor and it is configured manually via MQTT switch integration:

mqtt:
  switch:
    - name: "Digital Picture Frame Monitor"
      icon: mdi:panorama
      state_topic: "frame/monitor/status"
      command_topic: "frame/monitor/set"
      availability_topic: "frame/monitor/available"
      payload_on: "ON"
      payload_off: "OFF"
      state_on: "ON"
      state_off: "OFF"
      payload_available: "Online"
      payload_not_available: "Offline"
      qos: 0
      retain: true

that’s just one example but there are many others still around. I have several command line switches that also have no associated devices.

I think that may be due to the concept of the two definitions of “device”.

A switch is (almost?) always associated with a physical device of some kind.

But that physical device is separate from the concept of a device in HA which is generally just a collection of entities associated with a physical device.

For example one of my command line switches I use to turn on/off my camera night vision LED. The camera integration does not provide a switch entity in the device (HA version) to control that camera device (physical version) LED so I have to manually create one. And we don’t have the ability to create devices ourselves or add entities to existing devices either. So it becomes a switch without a (HA version) device.

I saw that as well but hadn’t had the time to try to work out the solution.

I guess my solution was going to be to go thru all my devices and make sure they all have “name_by_user” so I don’t get the above example generic “Sengled G11-G13” device name.

It’s ironic that I had just started working on this issue a couple of weeks ago when one of my zigbee plugs died and I didn’t figure it out for a few days. It runs a hot water heater recirc pump that isn’t used all the time so I didn’t notice right away that there was no immediate hot water at the tap. I got the notification working as I wanted it but as I mentioned it’s not easily maintainable.

Thanks again for your work (and collaboration) on this.

Tho I’ll still add the light domain as well to your template since I have some zwave and zigbee bulbs.

Thanks @kyemacdonald. The “null” failure is likely because that switch doesn’t belong to a Device.

Clarification: This is a physical switch right? That you want to get notified if it goes offline? Or is it some sort of virtual switch?

The reason I ask is there are two ways to handle this, we can either have the automation “ignore all switch entities that are not part of a Device” (thanks to advice from @finity) The other option is: if a Switch Entity is offline, and is not part of a Device, then I can just return the Entities name, instead of the “non-existant Device name”.

I don’t have anything like this, so I need advice from you two on how best to handle that kind of case…

In the meantime, FYI here is my earlier blueprint, basically the same except it returns Entities instead of Devices, so it should work for you now.

(I found this behaviour a bit annoying, if one Physical Device, with 5 sensors, fell off the network, then I would 5 notification lines on my phone instead of one - (or a long rambling list from spoken from Google Nest) - just saying the device name makes more sense, for me)

Thanks for all that info :slight_smile: I understand these switches now - they are still physical devices.

Question: when these kind of switches are offline, I can assume you would still want them reported? I.e would this logic make sense to you: “If one or more offline entities belong to a HA_Device, then just return that device name, but if the Entity doesn’t belong to any HA_Device, then just return that Entitys name instead”?

Ya it might be nice (in an ideal world) to extend the blueprint with some boolean checkboxes, so people can choose domains as suits their use case? i.e Checkboxes for domains: Zigbee, ZWave, AllBattery Devices, Lights etc

Would make the code in the blueprint a “fair bit more spaghetti” though. I gotta say I find this coding in HA/Yaml/Jinja really complex compared to e.g Python. It all seems so janky and fiddly to me

1 Like

I think that would be generally the case.

But there is already a thread for a generic “report any unavailable entity no matter the source” that’s been around for a while. And I used that for a bit but it was too generic so it became less useful since it reported too many things that I didn’t really care too much about.

Since I don’t use blueprints I can use the building blocks from the blueprints for the things I need a normal autoamtion to do. That way I can modify it for myself as needed.

My example isn’t a physical switch. It’s a command sent to a Sonoff RF Bridge that causes the bridge to broadcast that signal. The RF Bridge is a device in it’s own right, but commands sent to it are just entities I create and aren’t attached to any physical device (in HA terms).

From my perspective I don’t need to be notified if this mqtt switch is offline, as the only way it is offline is if HA is offline.

This was posted on the HA subreddit a few hours ago. I thought it could potentially be another approach - https://www.reddit.com/r/homeassistant/comments/1axg6mj/template_share_for_devs_how_to_get_all_offline/

Hi, is it possible to include smart lamps that went offline?
My lights are basically powered by mains switches installed in the house that normally switch a conventional lamp on/off and sometimes my wife or kids accidentally swithes off the mains switch that powers the smart lamp causing the automation cannot swith the smart lamp on.
Getting a alert on my phone I can recover the issue.

Whenever I try to run the automation, it never actually runs. In looking at the home-assistant.log I see this error:

2024-03-08 18:46:10.101 ERROR (MainThread) [homeassistant.components.automation.offline_sensors] Error rendering variables: TemplateError: Must provide a device or entity ID

I did not add any sensors to exclude and was just running the automation manually after creation. Any ideas? This is a bit over my head!

This blueprint (mistakingly) assumes that every Sensor belongs to a Device. (which is true for my home, but not true for everyone.

This this error is likely happening because you have some e.g a sensors or switch entity, that does not have a parent device.

I’m aware of the issue, and will release an updated blueprint that will fix the issue for you, hopefully next week. Thankyou for the feedback.

1 Like

Thank you so much for the explanation and working on a fix!

I’ve had to remove the battery device class filter out of the excludes block. For some reason the notification includes my WLED controller (which is currently offline) but since it’s not actually a battery device I’m unable to add it to the excludes filter. I’d prefer to just have a longer list of entities in the excludes section and have the ability to filter out whatever I’d like.

thanks for the BP, it works great.
But I have one note.

Via NabuCasa I have integrated my 8 Alexas to use them as Media Player.
Your BP reports all 8 Devices as offline and I can’t put them on the ignore List.

grafik

After some research I found, that each device has 2 switches, which are both unavailable. I think that’s why they are reported offline.

These switches get activated if there is music playing.

The same problem is with my Philips TV integration, which gets reported, because the switches are not available, if the TV is off.

So for now, I deactivated these switches, but I would be awesome If you could implement the possibility to exclude devices, like alexa/media Player.

tnxs, gz Stefan