I’m facing a problem with processing sensor data in Home Assistant and need help to fix it. My power sensor takes readings at regular time intervals, but because of delays in wireless communication (from ZigBee bridge to WiFi router), Home Assistant ends up processing and storing the readings at uneven time intervals.
Right now, the times used are based on when Home Assistant receives the readings, not when the sensor actually takes them. Even though the sensor includes a timestamp in its report, Home Assistant isn’t using it. It’s making the calculated derivative unstable, which is not adequately fixed using a time window.
I exported the sensor’s data and HA’s derivative to verify this. I plotted them and then generated my own derivative by massaging the sensor data such that all samples were evenly spaced.
The following diagram shows a portion of my data with the issue I’m trying to remedy. (Please look past the orange/blue plots not being “in-phase”)
While trying to diagnose/fix this issue, I’ve considered that it might be an artefact of smoothing or the time window that HA’s derivative is set to use, but no matter what I set or simulate, I always get some events that don’t produce the derivative I’m expecting. I’ve only been able to get consistently correct results when the samples are spaced evenly.
Here’s a histogram showing the interval between samples, calculated from HA’s timestamps:
You can clearly see that the intervals between samples are normally quite stable, but plenty of cases aren’t. The samples that arrive at “exactly the same time” - essentially identical timestamps in the exported data - are particularly damaging to the derivative.
Is it possible for Home Assistant to handle sensor readings with their own timestamps? I’m guessing if yes, I’ll need to make code changes to the integration that’s providing the sensor data, which I’m comfortable doing.