So will it turn into normal after a period ? How am I going to track my real monthly duration I spent at home ?
It wont fix itself this month because you don’t have history that extends beyond today. If you want to track this months@home, set your datetime to start when you started collecting data. Otherwise it’s going to always add 11 days of your first state to the duration, if that’s the state you’re counting.
You need history for it to calculate, otherwise it makes assumptions. You don’t have the history, so temporarily set it to start on the 12th and then next month switch it to the 1st.
Do you know why the 12 missing days are added? I have purge set to 7 days, and when I tried it the purged days were not added. It made sense to me that if the data wasn’t there, that I also wasn’t home. Or was that because by chance I wasn’t home as the first state? It isn’t true this can be extrapolated backwards.
(I know I shouldn’t do this with a period longer than purge settings, but I an curious)
Yes, when you start up the history stats sensor looking for X state, but you the very first state in your database is X, it’s going to assume X was the state from the beginning of your configured time window.
v-- start v-- X (first in database) current time--v
| | |
\__________time added________/That does make sense for a new database or new sensor, but not so much for when a database is purged. In that case the first known value represents a change. For numerical is is not a bad assumption per se the value before was quite similar. But for binary sensors it is 100% sure wrong. In both cases unknown is more accurate. That is what also happens if Home Assistant is down for some time.
Unless you’re searching for off.
Keep in mind that history stats only knows about the state it’s looking for, nothing else. So there’s only so many ways it can count or not count that duration. And everyone wants something different.
I don’t get that. If you assume the first value you find was also there before, and the first value you find is off, then the value before was either on or unknown. Because off was stored as result of a change, and there’s nothing else it could have been. Afaik the database does not sture states unless there’s a change, so because of that the value before was not the value you found as the first.
The fact is that the data needed is not there. So the quick solution would be count it as unknown. The perfect solution would be for the history to either use the last value before the period, or unknown if that does not exist. But that comes at a performance hit, as you cannot optimise the search on the start if you do.
I do not believe every one wants something else, I think people want what is correct. There already is a feature request that amounts to the same that I am saying:
Sure, but history stats is old. I have no idea what was going through the dev’s mind when he first implemented it. I can guarantee there was a reason.
It does that. It doesn’t do that when nothing exists before hand, which is the situation we are in.
History_stats is not the same as statistics.
1 Like
Petro thank you very much for your explanation. Everything is clear now. For me I agree with Edwin that the data should be “unknown” or “0”. It should not give us a wrong data when there is no database to check. Interestingly the data comes up as it should with other newly created stats such as:
This is again monthly stats for being away from home. The code is selected from the 1.st day of the month and the template was also created yesterday. The result comes up as normal. It starts to count from day 1 and when there is no data until yesterday it shows as 0.
But my home status template behaves different. Anyway for now I solved it as pulling the date to yesterday(12th) as you proposed. Thank you very much.
You still aren’t understanding how it works. away is showing a small number because away is not the first state in your database. You’ve already proven that home was the first state in your database in your other post. That means, this is what’s happening:
Be aware that if you create a history stats sensor for “away”, it does not always mean “away from home”. If you create more zones, the state could also be “school” or whatever other zone you care to define and go to. This caution also applies to state triggers and conditions in automations. It is safer to use from: home than to use to: away.
If you look at this pretty normal use case in the link below, you can see the history_stats sensor is not doing what it should regarding states that initiated before the period at hand:
I think there’s still a bug in there that is pretty evident when using it on binary sensors.
Also, there’s something in your state transition graph above that is still not clear to me. The first found state in the window. You place it on the timeline in a spot where the state did not change. For a binary type sensor that is not happening. Why would the state be saved if it did not change at that point? The graph you drew supports your answer, but I do not think it supports a situation you would normally find in the history.
The only way to know the state at the start of the time period, is to know the last state that was recorded before that period. If there’s nothing in the history at all for that period, the only valid answer is unknown.
You keep focusing on there being a point before the period. My examples are with a fresh database, no points prior.
Have you looked at the post I referred to? There is data in the history there. It gets ignored. 10 hours worth. Until you expand the period to include it, then it gets counted.
And you also keep ignoring the fact that if data gets purged, there may also be nothing before in the database. And then too your example is not something that would happen in real life. You keep ignoring the fact that history stores changes. so the value before can be anything EXCEPT the value that was stored at that point. Unless we are talking about a whole new database. How long ago was that for you?
A new user would expect the values before the existence of the database not to be counted, If that user even uses a history stats sensor within 10 days of adopting HA. Never happens. It is a useless use case to begin with.
Listen, I know it wouldn’t happen in real life, I’m explaining how it works.
I hope we can agree that the post I refer to is not expected behaviour?
I didn’t look at the post, I’m on mobile at the moment. Regardless, no one needs my approval to write up an issue.
I get that, and I appreciate your contributions. I really do. I just wanted to point out there are problems with the current implementation.
1 Like
I seem to have started getting problems with my statistic sensors after upgrading to 2022.6.x versions.
I have statistics sensors set up to monitor the change of energy meters for the past 7d, 24h and 1h.
What I see is that when the 1h change is not present (no change in energy consumption for an hour) it will return “unknown” instead of 0.
As I understand this is due to no states within the window start/end.
There is something wrong with the stats, yes 
I have an issue in 2023.3.3 version with all history_stats. I have sensors counting occurances “events of ON” (from MQTT) since midnight, as well sensors counting time since midnight. The problem is that midnight does not reset the sensor.
This is my bug description: history_stats timeperiods gone haywire / 2023.3.3 · Issue #89696 · home-assistant/core · GitHub
By reading what you have had, then we likely have some similar data issue (HA not handling the data).
And yes, I as well did change some data retention (purge days) after 2023.3.3 update, from 20 to 15 days.
Thoughts?