Help with filtering sensor values

Hello everyone,

I am trying to use an ultrasonic sensor via ESPhome to measure the level of water in a tank. Unfortunately, the sensor produces some false readings that make the gauge less than reliable when used for alerts or automations. Is there a way to “throw out” any sensor values that are a certain percentage below the previous (correct) values?

I want the sensor to remain fairly responsive - meaning I don’t want to wait to average the values over a long time to weed out the bad values because I would have to wait on the averaged value. I need to be alerted in a decent timeframe for level hikes/drops.

For example, I’d like to remove the negative spikes (red arrows) and have the chart keep the max values (red line). I hope this makes sense! Thanks!

Unfortunately ESPHome does not have an outlier filter, you will have to create this in home assistant:

The value doesn’t seem to change that much.
Perhaps you can use the
sliding_window_moving_average
which should average it out.

@Hellis81 thanks for the suggestion but when I try that the severe negative spikes bring the average down enough to make it look like the reservoir level is decreasing rather rapidly. I’m hoping removing the negative spikes instead of averaging them in will produce a better representation.

Also, the example chart I posted is somewhat less “noisy” than the sensor can get sometimes.

Thanks!

1 Like

Which is exactly what the outlier filter I linked to does.

How can it possibly give you negative spikes?

Because the spikes would be included in the average. Where as they would simply be discarded in the outlier filter.

Yes, I was just reading up on the Outlier filter. It looks like it should do the trick once I get my settings dialed in. Thanks!

I just opened a feature request for inclusion of this type of filter in ESPHome.

2 Likes

I meant how can the raw sensor data be negative?
I believe it’s either configured wrong or defective

I have the sensor value then calculated (height of reservoir 28cms - sensor reading = new value). I think condensation or perhaps something else sporadically causes reading of something over 28cms and therefore results in a negative value. I hope that makes sense.

It could be a double echo. Which sensor part number are you using?

I’m using an HC-SR04 with an ESP8266