Invalid values not handled at all

If a Zigbee device returns an invalid value, Homeassistant takes that value as a normal reading and this is not the correct behavior.

The Zigbee protocol specifies a specific value for body data types that indicates that the value is invalid and shouldn’t be used as a normal reading.

Here is a list of values that indicate invalid values:

https://mmbnetworks.atlassian.net/wiki/spaces/SPRHA17/pages/110782747/ZCL+Specification+References

Currently, these values are treated as normal values and this makes it impossible to see normal values in the graphs. To fix this, values have to be manually corrected and statistics recalculated, something that is very difficult to do in Homeassistant.

I have created a filter for one of my devices, but I have numbers devices and they can also output invalid values.

I have noticed that the Zigbee backend seems to know that the value is invalid (STATUS indicates it), but the numeric value is still does in the database as a normal reading.

So I suggest to handle these values in a different way in Homeassistant.

The simplest way would be to look at the status flags and if it is a bad value, to store an string indicating this instead of the numeric value. That would be the expected behavior. The current behavior is lacking a simple filter.

Update:

I have to create copy-devices that are filtered, duplicating all data:

availability: "{{ state_attr('sensor.hovedmaler_summation_delivered','status') == 'NO_ALARMS' }}"

You did not mention if you are using the ZHA integration or not, however if you are using the ZHA integration then see replies below.

ZHA/zigpy developers do not frequent this community forum so instead suggest you post such feedback as a new discussion thread under the zigpy project on GitHub → zigpy/zigpy · Discussions · GitHub

Your device will likely need a ZHA Device Handler (a.k.a. quirk) for that device, so post service support request as an issue to that GitHub repository for discussion, and even if that is not the issue the ZHA community reading those issues can probably give you more information if you posted diagnostic information and debug logs there. Read and follow this → https://www.home-assistant.io/integrations/zha#how-to-add-support-for-new-and-unsupported-devices

If you do not get a respose then then post as an ZHA component issue to Home Assistant core → https://www.home-assistant.io/integrations/zha#reporting-issues

If my ‘googling’ is correct, looks like you have been looking for a solution to this for a while. Both on this forum and on GitHub. If I interpret correctly, your device is a 'metering device is based on a vendor chip ‘SPRHA17’ from MMB Network doing some ‘pulse counting’ to trying figure out usage, one of the harder IoT type sensing challenges from my experience. This chip seems to communicates on the Zigbee side using serial byte strings for the input/output it is doing with the physical components on the ‘other side’ of the SPRHA17 module.

It does look like from this GitHub post in 2022 on ZHA, that the communications protocol is a bit hard to work with. That said, again in this post a reference to Zigbee2MQTT device report, seems to indicate that the protocol has been successfully figured out for the device in the 2nd link below.

I do not have any first hand experience with this device below, or what I interpret to be your device, that said, my recommendation would be to ‘stand up’ a Zigbee2MQTT system with a TI26xx chip based coordinator and see if you can get your device operating via Zigbee2MQTT to Home Assistant. Or as the other poster indicated, write your own ZHA quirk module to handle the exceptions you note. Does seem like the core zigbee system should handle these edge cases, that said, there sure looks like a lot of ‘invalid’ values to deal with to translate to Home Assistant’s invalid state indicator.

Good hunting!

I am indeed using ZHA (not Z2M). But what I request is not a ZHA fault it seems, rather that the core HA send to record invalid numeric values in the database when it should record that it couldn’t retrieve a valid value (or not record anything at all). I say this because of I filter on the status field from ZHA, then all invalid numeric values can be removed and after instead recorded as proper invalid/missing values. This is possible because zha selling correctly identifies the the numeric value as a guilty value and sets the status flag accordingly.

This seems like a handover problem between zha and the core.

I have seen more than one device type with this problem, indicating this is not just a problem with one device, but a data problem. I have checked the byte sequence returned from the devices and the numeric values do indicate invalid values and zha apparently detected this and sets the status flag, but the core ignores the flag and assumes the value should be taken at face value.

As even invalid values can be dropped in the database (as strings indicating this so they won’t cause problems with graphs or statistics), some part of HA must be responsible for handling the transfer of invalid values, either zha per the core. I don’t know which, but I assume the core needs to bed able to handle the invalid values a bit better.

Invalid values are a normal part of the Zigbee protocol and every device returning values could (if data type allows for it) return invalid values. This is not a failure with the device and a quirk (multiple, as I would need quirks got all Zigbee devices) seems the wrong way to go when this appears to be a generic problem.

And yes, I have tried asking about this before but haven’t gained any interest. So I appreciateb the comments here.