Thoughts on Home Assistant not storing repeated values

Configuration
1

Firstly thanks to everyone that has been part of the HASS community… I have played with it for years but only jumped in seriously over the last couple of months and there is so much support it’s incredible.

I think I may be a bit crazy however… From what I can gather HASS stores sensor ‘values’ when they change (seems reasonable to minimise storage requirements) but I’m finding this is actually resulting in lost information. Here’s the logic:

  1. Assume a sensor updates every minute.
  2. For 10 minutes the sensor returns 0 (not unknown but a legit 0) and then 50 on the 10th minute
  3. HASS stores a 0 point @ t and a 50 @ t+10minutes
  4. You now plot this with any reasonable plotting scheme and it will draw a ramp (or fit a curve) between these points…

The issue being this plot is more wrong than it needs to be. From the data we know at t+9 the value was still zero and only then did it start to ramp, yet that information is lost and it is now indistinguishable from a situation where the update interval is 10m. Over the last month I read all sorts of suggestions and tried:

  1. Forcing values to be written through automations (homeassistant.update_entity)
  2. Template trigger sensors
  3. Adding states.sensor.time to templates
  4. Editing an attribute on update (eg. minute_counter: "{{ now().minute }}")

but came to the realisation that (I think) all these are about updating the sensor and do nothing to force the repeated values from actually being stored. With this I decided to add a tiny random offset ((range(0, 10) | random)/10000) when in a known stable state and all of a sudden I could see values every minute.

Anyway, long story short, surely introducing jitter to a sensor isn’t the only way to ensure values are stored? Please tell me I’m crazy and there is something blatantly obvious I’m missing (a min store interval?).

If you got to the end of this… thanks! :smiley:

1 Like
2

No. It will be a step. Those are the only points home assistant knows. Those are the only points plotted. No interpolation.

History graph:

Screenshot 2023-03-22 at 20-46-37 History – Home Assistant
Screenshot 2023-03-22 at 20-46-37 History – Home Assistant1766×445 6.83 KB

Even the new (and IMO awful) use of statistics graphs for more-info pop-ups uses such slight splines as to still be a step.

Untitled
Untitled1272×431 38 KB

2 Likes
3

Thanks for the response Tom. And you are correct that the built in History does render steps, and that (or technically a point graph) is the correct way. But for those that may want interpolation or smoothing (let’s call them heretics) and use other graphing tools has anyone found another way to get around this problem?
An idea is that by knowing the update interval (dt) you can insert a point with the old value @ t-dt (before the change) before plotting… but you’re still losing data for sensors with irregular update intervals (ie. was a repeated value actually returned and dropped or was there no actual update).

4

If there was no change it is a step. Why do you want to invent data that isn’t there?

5

This is a different question. Whether it stored those additional zeroes or not the step chart would be the same. If you are trying to assess the health of the device thats a separate topic.