Have HA core honor the ZHA status flags

Feature Requests
1

I would like for HA to stop recording invalid values as valid values.

The ZHA integration seems to handle invalid values from device as it should, but the handover to HA core isn’t working correctly, and I don’t know if the problem is with ZHA integration failing to send the correct invalid values or if HA core doesn’t read invalid values correctly.

For instance, I have create (for just one device) a filter device where I try to filter away the invalid values based on the ZHA status flag. This works, but I duplicate the data and my configuration is unmanageable if I create filter device for all my ZHA devices. Besides, I think my filter is wrong:

template:
  - sensor:
    - name: "Hovedmåler Summation delivered (filtreret)"
      unique_id: "hovedmaler_summation_delivered_filtered"
      unit_of_measurement: "kWh"
      device_class: "energy"
      state_class: "total_increasing"
      state: "{{ states('sensor.hovedmaler_summation_delivered') }}"
      availability: "{{ state_attr('sensor.hovedmaler_summation_delivered','status') != 'CHECK_METER' }}"
    - name: "Hovedmåler Instantaneous demand (filtreret)"
      unique_id: "hovedmaler_instantaneous_demand_filtered"
      unit_of_measurement: "W"
      device_class: "power"
      state_class: "measurement"
      state: "{{ states('sensor.hovedmaler_instantaneous_demand') }}"
      availability: "{{ state_attr('sensor.hovedmaler_instantaneous_demand','status') != 'CHECK_METER' }}"

Is it possible in HA somehow to make a custom filter driver that will work on all ZHA devices doing the above filtering, without duplicating all devices data?

2

The attribute value you are filtering on isn’t invalid to HA or ZHA though. There’s no way for either system to know that “CHECK_METER” is invalid. You might be able to do it in a custom quirk file, but I doubt it would work the way you want it to. With that said, what is the state of that sensor when that attribute reports “CHECK_METER”?

I’m guessing that if your case is that “CHECK_METER” is invalid, you’ll have to create a filter sensor for every device and use those instead of the actual sensors.

3

I am filtering on the “status” attribute that ZHA provides along with the value. If status is CHECK_METER, ZHA knows this value to be invalid. ZHA has parsed the received value from the device and noticed it is a magic value indicating the the value is invalid. ZHA passes there magic value on to HA core along with the status attribute indicating that HA core should not use the value as a reading. HA core ignores the status attribute and takes the value as a good value, which it isn’t, thereby ruining graphs and statistics.

If this won’t get fixed, can we then at least have an option to inject a filter driver between ZHA and HA core, allowing us to filter away invalid readings ourselves?

4

Looking at this from afar, meaning I have no context on the details, a clear determination needs to be made if it is something that should work and doesn’t, IE a bug, or something you would like to see work, IE an [FR].

I don’t know that either has been established. Once you have a good guess at that, you might want to clarify the top post here if an [FR] so you can solicit help changing it or put in an issue if the problem lies there. Reporting issues - Home Assistant.

5

This is why I write here, to determine if this is a feature request or a bug. My assumption was that inside values from Zigbee devices should not be interpreted as valid values.

I have tried to make posts about this previously, but they all end up failing because one group points at another group, and then no-one wanted to solve it. They also won’t say if this is a ZHA issue or HA core issue. This is an API issue that may not have been well defined, making both groups right.

So I basically hoped for a workaround. What I do is a poor solution at scale (too much data and to many devices). But if Homeless is meant to record invalid Zigbee values as valid values, this should be stated somewhere.

I would solve this myself, but I have no experience with python and home assistant/zha is a big project to start with. So I ask for some insight that could help reveal the true nature of the issue. I have already encircled it a bit by providing a workaround, but a proper fix should be made.

6

You’re making the (bold) assumption that ZHA “interpreted” anything.

You don’t say what device this is about, but I’d bet it’s the device itself that sends status == 'CHECK_METER'. In that case, neither HA or ZHA can do anything, and you’ll have to filter yourself.

2 Likes
7

The Zigbee protocol has no such feature. It does have invalid values. Using a proprietary tool, I have decided the message and it’s just like normal reasons, except the value is a magic value indicating inside value. The only difference really is the value. ZHA does check for these magic values. It is the only way it can decide that one value is normal and another is invalid and then set CHECK_METER.

Either that, or the response timer is different between the two readings and that triggers the CHECK_METER status. I will scan the ZHA source and see if I can find it.

8

I have bare a quick scan of zigpy and I didn’t find code relevant to invalid values. That doesn’t mean it isn’t there, but I will probably have to ask in the zigpy group for details. Thank you for the attention so far.

9

It evades me where you got the impression ZHA handles “invalid values”, tbh

10

By the fact that using the shown filter, my invalid values disappeared.

11

Is this a Develco (Frient) Energy meter?

12

The one in the OP? That’s a HA template filtering on a status field, nothing to do with ZHA :dizzy_face:

Well, good luck.

13

That is one of the devices, yes. I have another type that can rarely also give invalid values. This cannot reliably be handled at HA core level, as they magic value varies depending on the register type (24 bit, 48 bit, signed, unsigned, etc). Once the value reaches HA core, the type of lost.

14

Well, having that info up front (or when koying mentioned it at least) would have made this easier to help. But I was intrigued and kept searching.

I did find where zigbee2mqtt made a change a while back for CHECK_METER and another one. Something for the battery maybe. It has been a couple days since I found it and don’t remember.

I am not sure this should really be made on the software end. It sounds like the manufacturer is not following the standard. I think I’d push there.

Thinking about it, are they using CHECK_METER for something that you should not be skipping? Should something be checked in that case?

15

I am not fully aware of when the CHECK_METER flag is supposed to be used. But I know it isn’t restricted to battery devices, as I have seen that flag used on laid powered devices. I only have Develco metering devices. It is quite likely that Develco sets Check Meter flag when a reading is bad. If that is so, my filter works (and I know why). Still, it would be better if HA core didn’t record the invalid value at all.

Looking at the documentation from Zigbee alliance, I found this:

Check Meter: Set to true when a non fatal problem has been detected in the meter such as a measurement error, memory error, self check error.

The Develco devices do obey the standard in this regard.

I have an open issues on zigpy and they said that quirks cannot support invalid values. It needs to be supported internally in zigpy.