Automatically generating avarages and detect outliers?

Configuration
1

Hi, I tried to search the forum, but had no success. I don’t even know if this is the right section to post. If not, accept my apologies and let me know the right place.

I am a somewhat advanced HA user doing many things with automation and templates. I am a professional developer (but with weak python skills).

I need to monitor water use in a “irrigation cistern system”. I need to protect against leakage and running out of water. It is somewhat complex and expensive to meter water use directly. However, I can use pump energy usage as a proxy for water usage (I’ve evaluated that before and it would work).

I have several daily irrigation time slots in the 10 to 20 minute range. Irrigation usually happens daily, so intervals usually re-occur daily (except in rainy periods). Each irrigation period matches one irrigation area.

Detecting empty cistern is simple: pump energy usage is zero to very-close-to-zero during the irrigation period.

Detecting leaks is more complex. During normal operations, the pump does not run continously (there is a small water buffer inside the system). If there is a (sufficiently large) leak, the pump either runs continously or much more often. In other words: the pump uses more energy in today’s irrigation interval than in “yesterdays”. To make things slightly more complex, water usage (and thus energy usage) can fluctuate somewhat due to external conditions. Side-note: that means that even I would monitor water usage directly, the base problem would persist.

The solution to the problem is to build the daily average usage of (pump energy/water) for each irrigation interval. Then, emit a warning if today’s interval usage is outside of some permited fluctuation.

Basically, that requires me to store daily usage per interval, build averages on that and compare the usage at the end of the current interval against that average. Note that I have only one pump and one sensor, so the intervals are not identified by the sensor but by the irrigation time frame.

I know I can do this by sending sensor data from HA to MQTT, reading and processing that data with an external script, and make this script notify HA on potential problems via MQTT/template sensor (and take up this warning via an automation).

I just wonder if there is a simpler way using only HA features directly.

Any help/hints would be deeply appreciated.

Many thanks,
Rainer

2

Just to be sure (since you don’t mention it): did you already check-out the Statistics integration?

1 Like
3

Actually, not before, but I just had a look. Thx for the suggestion. Unfortunately, statistics integration does not work for my case, because I have a single sensor, but multiple time frames. Well, one way around this may still be to create template sensors (one per timeframe representing logical sensors). It’s still complicated, because I would need to consistently increase these counter values while the actual sensor advances much more. I keep it in mind, but this sounds like it is easier to do via an external script communicating with MQTT.

That said, I realize that my question actually boils down to “Is there a capability in HA that permits me to run a custom (python?) script that has full access to databases as well as the HA objects?”. It turns out that I simulate this with my MQTT interface. I’ll also investigate in that direction. Any more hints are still welcome.