How to reduce 'noise' when entity state changes?

Hi,

I use an integration for my Zehnder ventilation unit. It has some humidity and temperature entities.

When a temperature decreases from let say 14.8 to 14.7°C, it spits out quite some state changes in just one minute, probably because of rounding the values. See the charts below.

Mind the ‘noise’ at each drop/increase of the temperature:

Here is a zoomed-in part of the above one: it takes quite some fluctuation to go from a stable 14.80 to a stable 14.70 … This is what I want to reduce.

I don’t need that dense sample rate for a temp sensor: once a minute is more than enough.
So my question is: how can I debounce/limit the number of changes without adapting the integration?

Can I somehow hook up an automation in between to fix this? Other ways/ideas to fix this?

I also have some Sonoff switches that spit out temp/voltage changes too frequently, so a more generic way to reduce these kind of changes would be handy!

Thanks!
Gert

Could you create a sensor template that simply rounds the value provided by the integration’s entity and log/report on that sensor instead? Or even just to refine how often to change the value that you report on based on incremental changes?

Your suggested approach would function, but it will result in creating multiple entities containing redundant data.

I was hoping there might be a way to catch state updates before they get applied to the entities. Is there perhaps a hook that allows preprocessing state changes to eliminate the “noise values” early in the process?

Looking at this, there might be a way to further customize entities?

Does anybody know?

Many solutions on Home Assistant require creating somewhat redundant entities to manipulate results in the manner you desire, it comes with the territory. The only way to do what you want otherwise would be to manipulate the integration directly. I can’t even count how many such entities I have to do rounding or to do something that some integration I like doesn’t do natively.

I don’t think that will solve your issue of wanting to manipulate the results.

You probably are right about this.
But then I still have to figure out on how to reduce those entity changes …

Are you simply trying to round to a single decimal as your precision? Are you only wanting it to display a change if the change is in a range, say X more than last time? If you can elucidate then it’s just a matter of creating a value template that matches your requirements.

If you do want something like “only report every 2 minutes”, for example, then your sensor will need to have a timestamp attribute so you can calculate how much time has passed since your last calculation. For that matter you can use attributes to drive a lot of additional data if you desire.

Perhaps a sliding-window moving average can work. You can filter it to the closest 0.1 °C.

ESPHome - Deep Sleep with sliding_window_moving_average)

Looking at the chart above, none of these scenarios are good, because quite some data will be lost. I need to get rid of the spikes without losing precision.

This will be better, but will still removes details.

Yeah, but then every change of the orig entity will produce a change in the new entity. Although the value will be smoothed, it will still create unneeded state changes. It should drop the “noisy states” instead.

I might be asking too much though :thinking:

It sounds like you aren’t quite sure what you want to fix this to be. If every minute is what you want, you could simply create a number helper and then an automation that fires with a time pattern of /1 to run every minute and get the reading from the integration and write that to the number helper and then just use the number helper on your charts. The time pattern lets you dial in down to seconds how often to grab the value.

In your automation trigger:

platform: time_pattern
minutes: /1

I’m guessing the “noisy” states are the impact of systematic noise on top of an ADC threshold of the LSB, which appears to be 0.1 C. You wouldn’t be dropping the noisy states, you’d be low-pass filtering them and rounding the result. You’d still need to keep track of the time to make sure the influence of the runt pulses are properly weighted, but you wouldn’t lose resolution if properly implemented.

Ok, so by using the time pattern as CO_4X4 is suggesting, and using the low pass filter you are suggesting, I might have the perfect solution.

I will try that. Thanks both for helping me out! :ok_hand:

Integration and a low-pass filter are related. I think either one can work acceptably, but a sliding-window filter might provide a better time resolution at the cost of more complexity. Perhaps @CO_4X4 can offer an opinion on the two approaches; I don’t want to misrepresent his implementation.

You won’t, I was just suggesting a time pattern for automation.

1 Like