Dead sensor doesn’t show up as unavailable entity in HA

Hi all, I just added Zigbee in addition to z-wave and 433 devices (Ikea, price and flexibility reasons) and I used this config generated by the community to check if a device has come offline.

The script works great, whenever I unplug a device it becomes unavailable in HA, and I’m notified. With one exception: my Xiaomi Aqara water leak sensors.

They are battery powered and connected/routed via a Xiaomi smart plug. If I unplug the smart plug I get a notification, and the entity state for the smart plug is changed to “unavailable”. I then plug the smart plug back in, and make sure it’s online and functioning, and test the water leak sensor to make sure it’s connected.

If I then take out the batteries from the water leak sensor, nothing happens, the entity state doesnt change to “unavailable”. At the point of writing this the batteries have been out for 3 hours, and the time stamp in both Phoscon and deCONZ shows that it has been 3 hours since last communication.

I know it’s supposed to take longer for deconz to show an unavailable state for battery powered devices, but not this long? Does it have to do with the fact that the sensor is routed through a wall plug, it shouldn’t right, deconz knows how long since last communication occurred?

Anybody with any insights to share…? Thanks!

If you grabbed that gist a while back I think the version that is there now there was updated a bit.

If you use the developer tools to check the state of your water leak sensor (when the battery is out) what does it return? If you go through that thread @anon43302295 had a similar issue and I believe the solution was to add in a whitelist group for problematic entities. I thought the issue was resolved when Home Assistant started reporting “unavailable” states to the UI

I have to add some other text now because I replied in the wrong thread and the stupid computer won’t let me post it here without changes…

Thanks for the update! I used the code in your github package, and it works great for unavailable devices (I get the occasional false positive during startup, but nothing major). This issue is that the device never becomes unavailable.

According to this thread, the main problem in this case is that Phoscon which monitors the zigbee sensor and provides data to HA as an integration is not making a determination that the sensor is unavailable because it’s a battery powered device, so no change in status is being passed to HA.

To compound matters, HA only updates the attribute “last_triggered” only when the sensor has a change in state (e.g. whenever the water leak sensor toggles on/off), HA doesn’t take into account the timestamp of the last refresh. So the end result is that in HA the device always looks available.

I was thinking that there might be an attribute that gives me the last sensor refresh (as detailed here) and then looking at the time interval to make the determination if the refresh time is far enough in the past to conclude the sensor is unavailable (refresh time is always below 52 minutes). But I can’t figure out how to compare the timestamp provided by the attribute with the current time, and then check if the timestamp is more than 52 minutes in the past.

I think the attribute is provided by this {{ states.binary_sensor.ENTITY_NAME.last_changed }} (it looks right in the template editor in the Dev Tools), but I just can’t seem to get the syntax of the trigger and condition with an if statement right… Any help would be much appreciated! Maybe a template sensor of sorts?

EDIT: This is the code I’m adding in configuration.yaml to add a sensor for this check:

sensor:
  - platform: yr
    name: yr
    monitored_conditions:
      - temperature
  - platform: time_date
    display_options:
      - 'time'
      - 'date'
      - 'date_time'
  - platform: template
    sensors:
      ftx_water_sensor_availability:
        friendly_name: "FTX water sensor availability"
        value_template: >
        {% if as_timestamp(utcnow()) > (as_timestamp(states.binary_sensor.ftx_aqara_water_sensor.last_updated) + 3120) %} #Check if last update was more than 52 minutes ago
          0
        {% else %}
          1
        {% endif %}

However this gives me this error when I check my configuration:

                         Invalid config for [sensor.template]: expected dictionary for dictionary value @ data['sensors']. Got None
extra keys not allowed @ data['ftx_water_sensor_availability']. Got OrderedDict([('friendly_name', 'FTX water sensor availability'), ('value_template', '{% if as_timestamp(utcnow()) > (as_timestamp(states.binary_sensor.ftx_aqara_water_sensor.last_updated) + 3120) %} #Check if last update was more than 52 minutes ago\n  0\n{% else %}\n  1\n{% endif %}')]). (See ?, line ?). ```

It was an indentation fault… just needed some time away from the keyboard to see it :slight_smile:

This is the code with the correct indentation (in configuration.yaml) for anyone else wanting to use this solution. I also added “entity_id: sensor.time” which updates sensor every minute:

sensor:
  - platform: yr
    name: yr
    monitored_conditions:
      - temperature
  - platform: time_date
    display_options:
      - 'time'
      - 'date'
      - 'date_time'
  - platform: template
    sensors:
      ftx_water_sensor_availability:
        entity_id: sensor.time # updates sensor every minute
        friendly_name: "FTX water sensor availability"
        value_template: >
          {% if as_timestamp(utcnow()) > (as_timestamp(states.binary_sensor.ftx_aqara_water_sensor.last_updated ) + 3120) %}
          unavailable
          {% else %}
          available
          {% endif %}

But, now I’m wondering if I can perhaps set the state to ‘available/unavailable’ which will allow the automations provided in package from @jazzyisj to send an alert to avoid new automations for these sensors?

EDIT: Yes, setting the value to ‘available/unavailable’ makes it detectable to the automation by @jazzyisj so you only have to create a template sensor for battery powered zigbee devices, the notification is then already taken care of. Updated above code to reflect this.

1 Like

Glad you got it worked out. I’ll make a note of your technique in case it comes up again!

Well, looks like it was close, but no cigar. The sensor works, but the value provided in the attribute “last_updated” is not being updated properly.

However, when look at the device using the REST API provided by deCONZ, I get something more interesting in the response:

{
“config”: {
“battery”: 100,
“on”: true,
“pending”: [],
“reachable”: true,
“temperature”: 2300
},
“ep”: 1,
“etag”: “08955b25c056599828250f9219111267”,
“lastseen”: “2020-08-14T22:31:19.196”,
“manufacturername”: “LUMI”,
“modelid”: “lumi.sensor_wleak.aq1”,
“name”: “FTX Aqara Water Sensor”,
“state”: {
“lastupdated”: “2020-08-14T22:31:19.196”,
“lowbattery”: false,
“tampered”: false,
“water”: false
},
“swversion”: “20170721”,
“type”: “ZHAWater”,
“uniqueid”: “00:15:8d:00:02:4c:01:e4-01-0500”
}

The nested “lastupdated” with the value “2020-08-14T22:31:19.196” under “state” is the value I want!

I’ll have to go the REST route, but I have no clue how to set this up in HA. Any pointers setting up a rest sensor with nested attributes would be greatly appreciated :slight_smile:

Ok, this seems to work, I’ll mark it as a solution when I have tested it a bit more long term:

I updated configuration.yaml to include a REST sensor (sensor.json_ftx_water_sensor) that gets the nested ‘last_changed’ value directly from deCONZ and puts it as an attribute of the sensor. I then updated the availability check sensor to use this attribute instead of the one provided by HA.

This gives me the nested ‘last_changed’ value from the REST request in the attribute ‘sensor.json_ftx_water_sensor.last_updated’ which I can use. You can test this value in dev tools:

{{ states.sensor.json_ftx_water_sensor.last_updated }}

gives me:

2020-08-15 07:54:47.267667+00:00

My sensors section of configuration.yaml now looks like this:

sensor:
  - platform: yr
    name: yr
    monitored_conditions:
      - temperature
  - platform: time_date
    display_options:
      - 'time'
      - 'date'
      - 'date_time'
# Get the FTX water sensor 'state' values via REST towards deCONZ and put them in the sensor as attributes
  - platform: rest
    name: JSON FTX water sensor
    scan_interval: 300 # updates sensor every 5 minutes to not overload deCONZ
    resource: !secret ftx_aqara_water_sensor_resource_url
    json_attributes:
      - state
    value_template: '{{ json_ftx_water_sensor.state[0] }}'
    headers:
      Content-Type: application/json
      User-Agent: Home Assistant REST sensor
# The following sensor checks if FTX water sensor has refreshed its state within a certain timeframe (60 minutes/3600 seconds for Aqara water leak sensors), if not they are considered unavailable
  - platform: template
    sensors:
      rest_ftx_water_sensor:
        entity_id: sensor.time # updates sensor every minute
        friendly_name: "REST FTX water sensor"
        value_template: >
          {% if as_timestamp(utcnow()) < (as_timestamp(states.sensor.json_ftx_water_sensor.last_updated ) + 3600) %}
          available
          {% else %}
          unavailable
          {% endif %}

One small thing because I used @jazzyisj unavailability package is that the state of the rest sensor ‘sensor.json_ftx_water_sensor’ is always “unknown” which makes it trigger the notification automation. The solution for me was simply to put it in the package blacklist since I’m actually checking the sensor “sensor.rest_ftx_water_sensor”. If you’re not using this package you can disregard this paragraph.

When I add more battery powered zigbee devices, I simply need to add one REST sensor to get the right values from deCONZ, and one template sensor to make the determination if the device is still considered available.

For anyone struggling with the resource URL, it’s in the format:

http://YOUR_IP:YOUR_PORT/api/API_KEY/sensors/DEVICE_ID

You can change “sensors” to “lights” etc. More information and how to get started, generating the API key, etc, is available here. This also help with determining the DEVICE_ID.

3 Likes

Hi @goodyear77,

I’m struggling with resource url. I looked on the provided API documentation and I can’t get API key, I’m getting timeout. I’m using HA on Raspberry Pi with deCONZ plugin.

In Phoscon setup in help section I found API details but the structure looks different then your example:

https://YOUR_IP:YOUR_PORT/api/hassio_ingress/SOME_API_KEY/api/API_KEY_FOR_DECONZ/sensors/DEVICE_ID

and this works via the browser but for example not working using Podman (API client).
In the consequence the sensor is not updating.

Any idea what can be the reason? I did a try with Xiaomi zigbee motion sensor and button.

Regards,
Adam

Thanks, this helped me a lot. I also created automation that is sending a push notification to my phone if one of the sensors are not reachable.