Offline detection for Z2M devices with last_seen

[Kosy (Igor St)]

Unfortunetly I`m still stuck.
My config:

data_path: /share/zigbee2mqtt
socat:
  enabled: false
  master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777
  slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5
  options: '-d -d'
  log: false
mqtt:
  server: mqtt://192.168.0.111:1883
  user: mqtt
  password: '!secret mqttpwd'
serial:
  port: /dev/serial/by-path/pci-0000:00:07.0-usb-0:2.2:1.0-port0
external_converters: []
devices: devices.yaml
groups: groups.yaml
homeassistant: true
permit_join: false
advanced:
  last_seen: ISO_8601_local
  log_level: info
  pan_id: 6756
  channel: XX
  network_key:
    - XX 
  availability_blocklist: []
  availability_passlist: []
  ikea_ota_use_test_url: true
device_options:
  homeassistant:
    last_seen:
      enabled_by_default: true
blocklist: []
passlist: []
queue: {}
frontend:
  port: 8099
experimental: {}
availability: true
zigbee_herdsman_debug: false

This config beside entries from my previous post was running quite long.
I have no clue what I am doing wrong. I have compered everything with z2m docs, restarted HA , server few times and was manually putting offline value for homeassistant/status (according to docs). Weird thing is that there is no such message by itself in that topic.

[jarrah]

Ok, so I also see the same enabled_by_default message within MQTT explorer from home assistant:

Along with the hidden last_seen entity within HA:

The last_seen attribute is there as usual:

I rebooted HA and Z2M a few times again, as well as publishing manual offline/online message, but last_seen remains firmly hidden.

I see that Koenkk responded to my issue and suggested we use the availability feature for this instead: Device-Availability | Zigbee2MQTT

I guess this would just mean monitoring the offline / online state of a device?

[jarrah]

So in your HA states, you don’t see the last_seen attribute like this?

Or are you just finding that the last_seen entity remains hidden like mine?

[Kosy (Igor St)]

You are lucky I do not have attribute or hidden entity “last_seen”.
If You have that attribute this all that blueprint is about.
Avaiabilty is not just that good since this is presented in documentation.
My colegue also tested this settings and has last_seen in debug for devices that are out of network.

[jarrah]

It is strange that you don’t at least have the last_seen attribute because this is what the last_seen: ISO_8601_local gives you. Sorry for the obvious question, but are you running the latest Z2M (1.25.0) and HA (I’m on 2022.3.x - I’ll try 2022.4 just in case)?

My problem is that sensor.motion_bed4_last_seen entity remains hidden (along with all the other 124 Z2M devices), unless I manually enable it within HA. For me I’d rather have them all enabled automatically, which is what I hoped the `device_options’ method would do.

[Kosy (Igor St)]

I`m running Z2m :1.25.0-1 and HA core-2022.4.4.
You can use this if You have attribute.

[Kosy (Igor St)]

This weirdness comes from fact that Z2M config is moving from config to WebUI. See supervisor log for proof. Check Z2M WebUI > Settings > Generated Configuration (do not know how it is signed in your UI language) and then look for enabled_by_ default. Id it is not there edit ..\share\zigbee2mqtt\configuration.yaml
manually. That helped me with attributes.

[Scoff (Scott)]

I have set this up and am seeing the last_seen attribute on all my Z2M devices, but I’m not seeing what I expected. Perhaps I have misunderstood the functionality that this automation provides.
I have a device in Z2M which has been in an ‘off’ state for over 20 hours now (intentionally).
I’ve set the automation to run every day (option 0) at a set time. It is set to report on device that have a last_seen time of over 1 hour.
I’ve set a basic push notification to come through on thre HA app on my phone.
If I run the automation manually, I get the push notification but it doesn’t show any devices or information.
Is this because my device isn’t ‘sensor’?

Here is the device I expect to see reported:

[jarrah]

If you are using the latest version of the blueprint, it now looks for a last_seen entity rather than the last_seen attribute within an entity.

These need to be enabled within HA → Configuration → Devices
Select the _last_seen entity within Diagnostic:

Change to enabled and then restart HA:

Hope that makes sense?

[Scoff (Scott)]

Ah I understand now, thank you for explaining
I’ve enabled that previously disabled entity for the one device that has been offline, and added {{sensors}} to the start of the notification message, and ran the automation - works perfectly

[jarrah]

Ok, hopefully the following will help other users enable the _last_seen entity without having to manually enable them all.

After a discussion with Koenkk here, he said that setting the following in Z2M doesn’t by itself automatically make hidden _last_seen entities become enabled because “home assistant doesn’t update this after initial discovery”.

device_options:
  homeassistant:
    last_seen:
      enabled_by_default: true

The solution is to basically remove the MQTT integration and have HA discover all the Z2M entities again. He did however say that “this will reset all your entities in HA (icons, friendly names)”. This doesn’t luckily impact me, so I went ahead and did the following:

1. stop Z2M
2. remove MQTT integration in HA
3. restart HA
4. confirm devices/entities have been removed in HA
5. start Z2M
6. add and reconfigure MQTT integration in HA
7. restart HA

I now have all 125 _last_seen entities in HA!

btw, if anyone wants to test part of the blueprint in the Template Editor, I was using the following:

{% set result = namespace(sensors=[]) %}
{% for state in states.sensor | selectattr('attributes.device_class', '==', 'timestamp') %}
{% if 'last_seen' in state.entity_id and (states(state.entity_id) == 'unavailable' or ((as_timestamp(now()) - as_timestamp(states(state.entity_id))) > ((24 | int) * 60 * 60))) %}
{% set result.sensors = result.sensors + [state.name | regex_replace(find='_last_seen', replace='') ~ ' (' ~ relative_time(strptime(states(state.entity_id), '%Y-%m-%dT%H:%M:%S%z', 'unavailable')) ~ ')'] %}
{% endif %}
{% endfor %}
{{ result.sensors | join(', ') }}

@Mr_Groch it looks like you’re missing the underscores in the line below:

regex_replace(find=' last seen',

[Mr_Groch]

@jarrah

@Mr_Groch it looks like you’re missing the underscores in the line below:

regex_replace(find=' last seen',

Not really - this is for grabbing device name for notify - I’m using ‘last seen’ entity with name like ‘device last seen’, so I want to cut ’ last seen’ part from it (notice that regex_replace is done on state.name not state.entity_id)

[jarrah]

Interesting @Mr_Groch - it could be something to do with my testing via the template editor because the result without underscores looks like this:

If I then use find='_last_seen' it looks like this:

It’s also worth noting the lack of a timestamp when a device is unavailable due to having set availability: true in Z2M. Just mentioning it in case it helps anyone else who sees the same thing.

I’ve now disabled availability: true in HA so that timestamps are retained (plus this is the result using _last_seen):

[jarrah]

I’m intrigued as to what the difference is between our entities that would make your name have spaces because mine looks like this:

Maybe it’s because I don’t set a custom name so it uses the entity name by default?

[Mr_Groch]

Something is wrong with your setup, because your devices have name/friendly name the same as entity_id. Default behaviour in Z2M is to add ‘last seen’ (and other sufixes) with spaces in name/friendly name

[jarrah]

Z2M has so many options and legacy hangovers that it doesn’t surprise me that my setup is probably missing something. I’ll have a look through the docs to see if I can find something related.

[iz3man]

Tried to use this blueprint, but it doesn’t show any sensors when run. I already set the options and manually enabled LQI and last_seen

And when I run the code in template editor:

{% set result = namespace(sensors=[]) %}
{% for state in states.sensor | selectattr('attributes.device_class', '==', 'timestamp') %}
{% if 'last_seen' in state.entity_id and (states(state.entity_id) == 'unavailable' or ((as_timestamp(now()) - as_timestamp(states(state.entity_id))) > ((24 | int) * 60 * 60))) %}
{% set result.sensors = result.sensors + [state.name | regex_replace(find='_last_seen', replace='') ~ ' (' ~ relative_time(strptime(states(state.entity_id), '%Y-%m-%dT%H:%M:%S%z', 'unavailable')) ~ ')'] %}
{% endif %}
{% endfor %}
{{ result.sensors | join(', ') }}

it shows me this error:

UndefinedError: 'homeassistant.util.read_only_dict.ReadOnlyDict object' has no attribute 'device_class'

[jarrah]

Same error for me too. I suspect something changed within Zigbee2MQTT that no longer uses timestamp as a device class.

I’ve ended up using this alternative sensor to detect missing devices which has been working well for me:

template:
  - trigger:
      - platform: time_pattern
        hours: "/1"
        minutes: 0
    sensor:
      - unique_id: z2m_last_seen_entities
        name: "Z2M Last Seen Entities"
        state: >
          {% set lapsed_hours = 36 %}
          {% set ns = namespace(count=0) %}
          {% for state in states.sensor | selectattr('entity_id', 'search', '.*_last_seen$')  %}
            {% if states(state.entity_id) == 'unavailable' or ((as_timestamp(now()) - as_timestamp(states(state.entity_id),0)) > ((lapsed_hours | int) * 60 * 60)) %}
              {% set ns.count = ns.count + 1 %}
            {% endif %}
          {% endfor %}
          {{ ns.count }}     
        attributes:
          devices: >
            {% set lapsed_hours = 36 %}
            {% set result = namespace(sensors=[]) %}
            {% for state in states.sensor | selectattr('entity_id', 'search', '.*_last_seen$') %}
              {% if states(state.entity_id) == 'unavailable' or ((as_timestamp(now()) - as_timestamp(states(state.entity_id),0)) > ((lapsed_hours | int) * 60 * 60)) %}
                {% set result.sensors = result.sensors + [state.name | regex_replace(find='_last_seen', replace='') ~ ' (' ~ relative_time(strptime(states(state.entity_id), '%Y-%m-%dT%H:%M:%S%z', 'unavailable')) ~ ')'] %}
              {% endif %}
            {% endfor %}
            {{ result.sensors }}

Along with this automation to report it:

automation:
  - alias: Offline Zigbee Devices
    id: offline_zigbee_devices
    description: Sends notification for offline Z2m devices
    trigger:
      - platform: time
        at: '20:00'
    condition:
      - condition: template
        value_template: '{{states(''sensor.z2m_last_seen_entities'')|int > 0}}'
    action:
      - service: notify.signal_justme
        data:
          title: Missing Devices
          message: '{% set phrase = ''s are '' if states(''sensor.z2m_last_seen_entities'')|int > 1 else '' is '' %} 
                    The following sensor{{ phrase }}missing: {{ state_attr(''sensor.z2m_last_seen_entities'', ''devices'') | join(', ') }}'

[pannal]

Hey, thank you for this. I’m observing something strange, though: The z2m_last_seen_entities entity reports its value as “unknown”, even though I can use the template editor to get the state value of 0.

Does a state of 0 equal unknown?

Strike that. It’s working now. Took a couple of minutes.

Thank you!

[ChileHA]

Sorry for the dumb question. How can I use the template code if I have a sensors.yaml file?

Thanks!!