I see this got more votes than some other things that made the newsletter. 
I’m sure it’s coming next month! What’s the point in asking for pain points otherwise eh!
Jokes aside I think the newsletter said things were to come so fingers crossed.
I mean, its all the public repo. I assume that if the change was submitted and accepted, someone would come by here and tell us about it. That’s my hope anyway. 
edit: And seems unlikely since @frenck seems fine with HA essentially providing incorrect device information.
1 Like
For every WTH task there are multiple things to consider:
- Number of votes
- Time it takes to make it
- Is it backwards compatible
- Is it a building block to something that was already desired by devs/community
- etc. (definitely others)
Votes are important but the things that are going to get done first are primarily going to be quick wins. These are backwards compatible things that either take very little time to make or we’re already partially complete.
This may have high votes but its not quick. It requires planning. And probably an adr since the only way to do it in a backwards compatible way is to add to the core state model.
I totally understand that it probably is a lot of work. I promise that I’m not (intentionally) bitching from the cheap seats. My point really was that if something was going to happen, it seems like us in the cheap seats probably would have heard about it by now. 
@frenck What are the potential drawbacks of making this assumption?
Isn’t HA already making assumptions all the time? If a sensor updates once a minute, isn’t HA technically assuming it’s state between updates? Even if a sensor is pushing state to HA, isn’t HA assuming state in between those pushes?
It seems more practical to approach it from a usability perspective. Being “technically corrent” here and setting the state to something at intervals and then immediately to unavailable/unknown wouldn’t be very usable. So we have these attributes (changed/updated_at) to let us know this information instead.
HA going down for a reboot and coming back up doesn’t seem any different to me than a gap in a sensor update that would happen naturally. To me it draws parallels with null. It’s not unknown, it’s not unavailable, it’s nothing (null), so nothing should change.
However, perhaps there is something I’m not considering in the reverse case. So to reiterate my original question - What are the potential usability drawbacks of having sensors not report as unavailable during downtime?
4 Likes
I mean posts from the original WTH section were still referenced in PRs occaisionally all the way up to when this one started. Just because nothing was started within the month of WTH does not mean it won’t be done.
There are 3000 contributors to HA, if one of them decides its super important to them and they have the time it could be done next week (very unlikely, I think this is harder then that but people can be surprising when they really put their mind to something). Or it may not be started for a year. Or it may never happen. There’s really no way to tell.
I can only say it won’t be ignored. People go back to the WTH posts a lot as support for PRs and changes and this is high up there in the votes.
1 Like
I’m pretty sure I recall reading about at least one PR to fix this issue being shot down. While its of course still possible, it seems unlikely. I’ll remain hopeful though!
I obviously have little knowledge of the underlying model so this might not make any sense at all.
A few suggestions point here to a new attribute, a proper “last-changed”. Alongside what is currently there. No need for backwards compatibility, just one that’s not reset on restart allowing users the flexibility.
This might be very difficult, I have no idea. But I’m not sure it needs to be quite backwards compatible though.
I was mainly addressing others concerns at Frenck’s wording and other indicators which potentially implied there was nothing wrong and this WTH was invalid.
3 Likes
I would like to add my 2c and react on the two parts of your statement @frenck. Thanks for your input here btw!
-
State of the object changes when HA is offline, so restoring the sate would be an assumption, not a reality:
- For most of the objects, HA should be the only source of truth, so they can’t change without HA telling them to change.
- For sensors it is normal state of affairs as HA is not aware of their change between two data points either. It just updates it’s own info about the state of the sensor when it receives it.
-
The state actually changes at startup, when connecting:
- This is actually a bit scary as there should not be a reason for such behavior and it is pretty dangerous thing to do. I will explain later in more detail why I say that, but in general the connect/disconnect action should not mess with the state of the object (i.e. switch, door contact).
As @CentralCommand correctly wrote:
You can’t just persist last_changed without also persisting the state of the entity.
So, as most of the people here agrees that changing the state during restart is in 100% cases wrong (my experience is definitely over 95%), I would propose to add an option to enable state persistence either globally or per entity.
Apart from other systems HA has DB where it keeps the current state of each object, so restoring the states from the DB should be fast and should not add any writes to disk (to not stress SD card). Still it can sometimes come handy to be able to control the persistency per entity.
To give a perspective to my viewpoint, I am not using HA as a main home automation system for my house because when I was constructing my house, HA was a little baby of Paulus not ready for the task of controlling critical home stuff like all lights, power sockets, heating, gates, etc., etc.
The commercial home automation system I use as a main system does support (per entity) opt-in of the state persistence over system restart and I have a few comments to this feature using it for 6+ years:
- I can’t imagine creating more complex automations without state persistency.
- My only issues with the state persisting over restart always came from the fact it is opt-in in my system, so whenever any automation broke it was always me forgetting to enable the state persistency for some entity
- Human beings naturally expects that object state in the system only changes when the real object changes, not when the system does some maintenance task (i.e. reboot). So when writing the automation they do write it the way that they rely on the fact the state will only change when the object changes. That’s why this WTH has so many upvotes I guess
So, I am loving HA and using it for almost 3 years now, but as I already have a main system which has state persistency I haven’t faced this restart issue much apart from few WTH moments when checking stuff in the UI which didn’t make any sense.
For my use-case HA servers mostly as a proxy, integrating stuff I either built myself or bought and which are incompatible with my main commercial system. For that I mostly don’t need persistency as the source of truth is not HA but my main system, which do have the persistency.
I seriously can’t imagine having to use HA as the only system as this lack of persistency would make a lot of things I do either impossible or much more complicated.
Also me being technical I can understand what happened and why, but other family members would be really confused by these state changes caused by restarts.
At last, I would like to get back to the dangerous factor of the possible state change caused by the HA reboot.
Imagine some powerful tool powered by or even controlled by HA. Someone would be cleaning it and suddenly it will turn on injuring him, just because HA rebooted and changed the state of it’s power from off to on.
This is not something imaginary. I have exactly that real issue with HA controlled smart socket powering my 3D printer.
Often when HA reboots that socket turns on powering up my printer. I agree 3D printer is not that dangerous, but still can go rogue and burn your house down…
I originally had the socket running Tasmota, configured to either remember the switch state or start with switch turned off. After two years of random issues I thought it must be Tasmota and a month ago flashed it to ESPHome, configured it with RESTORE_DEFAULT_OFF and hoped for the best. To my disappointment the issue is still present.
After reading the second part of your statement I now blame HA causing the socket to turn-on which is really unfortunate.
5 Likes
Thanks, and to show these entities in a Markdown card, presented with the most recent changes at the top, with relative time:
## Latest events:
{% set entities = state_attr('sensor.nominal_change_history', 'changes') | dictsort(false, 'value', reverse = True) %}
{% for entity in entities %}
* {{ states[entity[0]].name }}: {{ relative_time(state_attr('sensor.nominal_change_history', 'changes')[entity[0]]| as_datetime)}} ago
{% endfor %}
2 Likes
Since trigger-based binary sensors do not change their states during HA reboot - then why last-changed is reset on HA reboot?
Following a logic described in this thread - last-changed is reset since a state is changed - but here a state is NOT changed, right?
Asked about it here too.
So I think there is a case to be made that things which restore state (like trigger template entities) should also persist and restore last changed and last updated. It’s trickier with the many entities that do not restore state but for those that do I think the how is straightforward.
However there is an issue which is mentioned quite a lot in this thread. on and off may be the only states you care about but they aren’t the only possible states. There is also unknown and unavailable. And when the template integration is unloaded all it’s entities briefly become state unavailable. You can see this very clearly if you do “reload templates” from the UI while watching your template entity in developer tools.
Therefore that entity actually did have a state change. It briefly went to unavailable before returning to the state you’re interested in. Which is why what probably would need to happen here is for a new field to be created which tracks the last time an entity changed between two “nominal” states. “Nominal state” essentially being a state other then unknown, unavailable or any other more system states added in the future. This term is an unofficial one, it’s just what people settled on in the thread above.
1 Like
I do agree about the “templates reset” case.
Here what happens with 2 template binary sensors - a “normal” and a trigger-based (both depends on some “input_boolean”, the 2nd is triggered when this “input_boolean” changes):
The “input_boolean” on the top, the trigger-based sensor - on the bottom.
So, both sensors become of “undefined” state (even not specified here - may be “unavailable”, unclear for me), the “normal” sensor becomes “unknown” at HA reboot, but the trigger-based sensor does not change it’s state at HA reboot.
So, I even do agree that both sensors may have reset “last-changed” after “templates reset” - but I do not agree that the trigger-based sensor have “last-changed” reset on HA reboot.
I share my recipe for a lovelace card that works well with TheFes trick here:
I’ve been using the solution posted @TheFes for a few days successfully (thank you!), however I’ve noticed the following error message in Studio Code Server:
I believe this has only started showing since updating to 2023.5.3 and Studio Code Server 5.6.0
Just ignore it.
Maybe we should raise an all new WTH for a solution.
What about adding two attributes to the sensors;
- last_time_on
- last_time_off
which could be used for doors, alarms and everything with an on/off behaviour.
Does this line needs to be below the template sensor?
{{ state_attr(‘sensor.nominal_change_history’, ‘changes’)[‘input_boolean.test’] | as_datetime }}
Or does it have an own template sensor? Hm I’m not sure how to implement it.
Thanks!