What the heck is with the 'latest state change' not being kept after restart?

Every reboot I loose so much data about entities that I would like to keep on.
Last time guest visited my house?
Last time heating unit went on (to be checked in summer)?
Last time never-used light changed?

The answer is always the same: x hours/days ago, where x = date of restart.

I know I can use automation and store that data in input_date or whatever… But it’s not easily visible in UI, i.e. as a secondary-info.

image

That light was on last time 3 months ago, not 18 hours. :wink:

YES! +1

should be persistent.

2 Likes

Since last few releases HA database is so fast and optimized I hope there will be place for additional date to be kept and restored after restart of HA…

^^ The fact of knowing ‘aaah anyway the data is wrong…’ is what makes me sad the most

So a big problem with this is that currently all state changes are for states that just changed. So if we were to do this, it could only be done during startup (as no automations are active so writing non-current states would work).

But if we do it during startup, then what if you reload your input_booleans. You would expect it to still be the date in the past? Now that will break a lot of assumptions about the system, and it’s consumers of the API.

@balloob - the input_booleans were just example.
In my opinion there should be an option (if can’t be done for every HA instance), that before automations and all sensors are loaded, last-changed date is restored.

So, in example:
I turn on and off the light at 2020-08-18 18:30. The date is stored in ‘last_change’ attribute or in database (I’m not tech, just an example).
I do restart on 2020-08-18 19:00 and after that and before every other state is restored/automations ran, I’d like HA to take that ‘last_change’ and make it somehow visible in UI (for secondary info etc.).

I don’t require HA to re-assign everything including state-machine, automations etc.
I’d just like to have that data somewhere to be used (i.e. in templates) and shown (in UI secondary info) without need of use external storage entities and template sensors.

Just found an example screenshot of some domoticz’s theme.


Yellow-marked value is what I’d like to see in UI next to my never-used-light. Date it was last used, not since last restart.

3 Likes

@andriej @balloob

Also, the state of the sensor(or light/switch) can change during the restart. for example someone could manually turn light on/off from wall switch while HA was down. Or a zigbee motion sensor could get activated. So we have to compare the before and after states, and reinstate the old last_updated timestamp only when there is no change in the states.

On large systems I wonder how much time it would take to load, loop/compare and update?

It might be pretty fast after the state restore data is loaded, that might depend on the data structure.

Might be worth it depending on users requirements, might need a configuration option to enable.

Non-Startup state restore is interesting, maybe create a copy of the entities to be reloaded state information and use that data the same way the restore state data would be at startup?

Yes this annoys the crap out of me. I have a HA at my parents summer house and because it’s often unattended for the whole year I am restarting it every night just in case it hangs and can’t be rebootet. This helped me a couple of times over the years already to not need to fly from Sweden to Poland just to reboot the Raspberry Pi. But this makes the motion sensors a bit meaningless because they always show the reboot time which is in my case always < 24 hours so I need to go into the logs and look there.

Up-voted. This would be nice. There are kludgy ways around it (like most things). FWIW I use hass-variables (https://github.com/rogro82/hass-variables) and some automations and sensors for this.

Our front gate is an example.

variable.yaml:

front_gate_open:
  value: '0'
  restore: true

I have to create template switch for some of my entities that I want to track that triggers scripts to turn them on so I can set the variable (suppose I could also do this with an automation listening for the ‘to: on’ state change…

switches.yaml

- platform: template
  switches:
    front_drive_gate:
      friendly_name: "Front Driveway Gate"
      value_template: "{{ is_state('binary_sensor.front_gate_contact', 'on') }}"
      turn_on:
        service: script.front_gate_open
      turn_off:
        service: script.front_gate_close
      icon_template: "mdi:gate"

The key here is when the gate is OPENED the scripts sets the variable as now().timestamp()

scripts.yaml

front_gate_open:
  sequence:
  - condition: state
    entity_id: binary_sensor.front_gate_contact
    state: 'off'
  - service: switch.turn_on
    entity_id: switch.front_gate
  - service: variable.set_variable
    data:
      variable: front_gate_open
      value_template: "{{ now().timestamp() }}"

I then created a sensor to display amount of time since last open. Certainly many ways to display this time, it’s how I choose to do it.

sensors.yaml

    front_gate_last_open:
      value_template: >
        {% set elapsed = (as_timestamp(states('sensor.date_time').replace(',','')) - states('variable.front_gate_open') | int ) %}
        {% set days = (elapsed / 86400) | int %}
        {% set hours= ((elapsed  % 86400) / 3600) | int %}
        {% set mins = ((elapsed  % 3600) / 60) | int %}
        {% set secs = elapsed | int % 60 %}
        {% if days > 0 %} {{days}}d {%if hours < 10 %}0{%endif%}{{hours}}:{%if mins< 10 %}0{%endif%}{{mins}}
        {% elif hours > 0 %} {% if hours < 10 %}0{%endif%}{{hours}}:{%if mins< 10 %}0{%endif%}{{mins}}
        {% elif mins > 0 %} {{mins}}m
        {% else %} {{secs}}s {% endif %}

I then have a conditional card in my UI that shows how long the gate has been open, and survives restarts.

ui-lovelace.yaml

          - type: conditional
            conditions:
              - entity: binary_sensor.front_gate_contact
                state_not: "off"
            card:
              type: entities
              style: |
                ha-card #states {
                  margin-bottom: -5px !important;
                  padding-top: 5px !important;
                  padding-bottom: 0px;
                }
              entities:
                - entity: switch.front_drive_gate
                  type: custom:multiple-entity-row
                  name: Front Gate
                  state_color: true
                  toggle: false
                  show_state: false
                  entities:
                    - entity: switch.front_drive_gate
                      toggle: true
                      name: false
                      tap_action:
                        action: call-service
                        service: script.turn_on
                        service_data:
                          entity_id: script.front_gate_close
                    - entity: sensor.front_gate_last_open
                      name: Duration

Which displays this when it’s open:
image

Always a bunch of ways to skin a cat in HA, but this is certainly a round-about method to do it. (I understand I could create automation to remove some of these steps - but hey, I coded it a year ago and am too lazy to change [if it ain’t broke, don’t fix it]). :man_shrugging:

2 Likes

Markus,

thanks for the workaround, but as many can see there are so many things you must configure just to have the latest state change survive the restart (at it is stored in a separate entity).
Ideally, this should work in a similar way as @andriej posted.
Reload is recreating entities (right?), so this will cause the last state change to change, but for most use cases surviving a restart would be enough.
Knowing the limitations we can always restart HA after YAML changes to avoid loosing last state change (this won’t cause a reload right?).

Thinking further through it. I suppose I could create a variable.last_on and then use attributes to store the entity’s last_on/last_open times, along with automation listening for state to open/on to set these.

Could reduce to a single variable (with multiple attributes), a single automation (trigger.entity_id to set I’d think), and then would simply need multiple sensors to computer last_on time vs. now() and display each ‘nicely’.

Again, what I’m doing above works, just a few more steps to get there. And also again, it’s what I love about HA, typically > 1 way to get to where you wanna go.

It would be better to fix all the reasons why it needs restarting than to fix this.

1 Like

Think that’s currently the core of hass… (re)building the interface on the client

I think it won’t be necessary to have all your stuff restored on a reboot. Lots of things don’t matter.
An option to choose whether you want to restore it yes or no would be a really good idea.

1 Like

The system can crash or you can update HA to the latest version, then a restart is needed.
There always will be situations where a restart is needed.

+1
or change your config to add sensors and then a restart is needed…

That’s exactly the point.
System crash, power failure, even restoring a snapshot wouldn’t change a thing that person called “X” was visiting home last time 1 month ago; last time central heating unit was ON was at the end of winter; (examples list is growing).

I think you have to be careful with assumptions. If HA restarts, there’s really no way to know what happened when it wasn’t running. And that restart may not be a (relatively) quick restart. It could have not been running for a while. So, what happened to the “thing” that HA’s entity represents while HA was not running? Even checking that the state is the same as the last time HA was running isn’t enough because it could have changed to something else and then back again.

Basically there’s a gap in the data, and assuming nothing happened during that gap is problematic.

Now, it might be reasonable to add some new piece of data that is the last time a change was observed by HA. At least it would be clear that it is incomplete data - use at your own risk.

1 Like

Could be simple?

Before any restart copy states, then after restart

  1. Use copied states: make other color
  2. Check state per entity:
    a. If same a 1 (previous state): normal colour
    b. If different state: change it and use normal colour
    c. If unable to check state keep other colour

Sure but how do you know ha is about to restart? What about a crash, how do you plan that?