[DEPRECATED] How to make active timers survive a restart

northpower25 · June 6, 2020, 2:55pm

Thanks for your fast response. i do what you explain at first i start the timer and the entity input_text.active_timers has the value ‘gartenbewaesserung 1591451673’ After the restart i see no errors in the log and the input_text.active_timers the value ‘gartenbewaesserung 1591451673’ but the timer is idle

now i have additional tested 24:00:00 hours for the timer gartenbewaesserung and an additional timer named test with 168:00:00

input_text.active_timers shows the value gartenbewaesserung 1591452688,test 1591452910

after Restart both timers are idle

next test 00:10:00 minutes for the timer gartenbewaesserung and the additional timer named test with 168:00:00

input_text.active_timers shows the value gartenbewaesserung 1591455593,test 1591455002

after restart the timer gartenbewaesserung runs with the right time but the Timer test (with 168 Hours) is idle

My HA Version is 0.111.0b2 but i’m see this isue also in 0.110.2

it’s look like that the Script works with minutes correct but theare are Problems with Hours

123 · June 6, 2020, 3:23pm

Thank you for performing the suggested experiment and for the additional test data.

I agree with you that the problem appears to be in the python_script. It performs a simple calculation where it subtracts the current timestamp from the timer’s recorded timestamp. If the result exceeds 0 it represents the remaining duration for the timer (and the timer is restarted).

    duration = int(z[1]) - int(datetime.datetime.now().timestamp())
    if duration > 0:

If the result is zero, that means the timer has no remaining duration and is not restarted. Perhaps the calculation fails for timers with long durations (hours). I will test it later today.

northpower25 · June 6, 2020, 3:25pm

Now i make a last test i set the timer to 09:00:00 and that works so it look like the issue takes place with 10 hours and more.

123 · June 6, 2020, 5:36pm

I think I see the source of the problem and it’s not the python_script. The first automation assumes the duration attribute is always in this format HH:MM:SS but look at how the attribute is displayed when I set it to 30 hours like this: 30:00:00

Screenshot from 2020-06-06 13-29-38

It shows it as 1 day, 6:00:00 which is not in the expected HH:MM:SS format. That means the first automation calculates the wrong timestamp value. In fact, it’s so terribly wrong that when the python_script performs its calculation using that timestamp it results in a negative value. That falsely implies the timer has already expired so that’s why the python_script does not restart the timer.

I will need to revise the first automation and create a template that is able to handle durations in this format: 1 day, 6:00:00

northpower25 · June 6, 2020, 5:48pm

Make it sense to create an changerequest for the timer integration?

123 · June 7, 2020, 2:53pm

I have enhanced the first automation’s template so that it correctly handles durations expressed in days.

I have updated the first post in this thread. Simply replace the action portion of your existing automation.active_timers_save with what is shown in the first post or, for your convenience, what is shown below:

  action:
  - service: input_text.set_value
    data_template:
      entity_id: input_text.active_timers
      value: >
        {% set ns = namespace(timers = '') %}
        {% for t in states.timer | selectattr('state', 'eq', 'active') | list  %}
          {% set d = t.attributes.duration.split(':') %}
          {% if 'day' in d[0] %}
            {% set z = d[0].split(', ') %}
            {% set h = ((z[0].split()[0] | int) * 24 + (z[1].split(':')[0] | int)) * 3600 %}
          {% else %}
            {% set h = d[0] | int * 3600 %}
          {% endif %}
          {% set s = h + (d[1]|int*60) + (d[2]|int) %}
          {% set c = ',' if not loop.first else '' %}
          {% set ns.timers = ns.timers ~ '{}{} {}'.format(c, t.object_id, t.last_changed.timestamp()|int + s) %}
        {% endfor %}
        {{ ns.timers }}

What’s different is this section:

          {% if 'day' in d[0] %}
            {% set z = d[0].split(', ') %}
            {% set h = ((z[0].split()[0] | int) * 24 + (z[1].split(':')[0] | int)) * 3600 %}
          {% else %}
            {% set h = d[0] | int * 3600 %}
          {% endif %}
          {% set s = h + (d[1]|int*60) + (d[2]|int) %}

It checks if the duration’s hours portion contains the word day. If it does then it proceeds to extract the number of days, convert it to hours, combine it to any existing hours, then convert it to seconds.

northpower25 · June 7, 2020, 3:52pm

Great response rate from you! Now it works like a charm

Maximum Thanks !

Alfiegerner · June 30, 2020, 9:11pm

Thanks very much for this - I’ve just started using long running timers and also have a tendency to restart HA whilst tinkering so this has been very useful.

My use case also has paused timers, so I’ve made changes to include those:

template for setting input_text:

data_template:
  entity_id: input_text.active_timers
  value: >
    {% set ns = namespace(timers = '') %}  {% for t in states.timer |
    selectattr('state', 'in', ['active','paused']) | list  %}
      {% set d = t.attributes.duration.split(':') %}
      {% if 'day' in d[0] %}
        {% set z = d[0].split(', ') %}
        {% set h = ((z[0].split()[0] | int) * 24 + (z[1].split(':')[0] | int)) * 3600 %}
      {% else %}
        {% set h = d[0] | int * 3600 %}
      {% endif %}
      {% set s = h + (d[1]|int*60) + (d[2]|int) %}
      {% set c = ',' if not loop.first else '' %}
      {% set ns.timers = ns.timers ~ '{}{} {} {}'.format(c, t.object_id, t.last_changed.timestamp()|int + s, t.state) %}
    {% endfor %}  {{ ns.timers }}
service: input_text.set_value

and python script:

ts = hass.states.get('input_text.active_timers')
if ts is not None and ts.state != 'unknown' and ts.state != '':
  timers = ts.state.split(',')
  for t in timers:
    z = t.split()
    duration = int(z[1]) - int(datetime.datetime.now().timestamp())
    state = z[2]
    if duration > 0:
      service_data = {'entity_id': 'timer.{}'.format(z[0]), 'duration':'{}'.format(duration)}
      hass.services.call('timer', 'start', service_data, False)
      if state == "paused":
        hass.services.call('timer', 'pause', {'entity_id' : 'timer.{}'.format(z[0])}, False)

xenz · July 20, 2020, 8:22pm

Thank you very much for this, I know this will solve all my timer when I get it working, but I have been scratching my head for a few days. Added input_text, 2x automations and the pythonscript, after enabling a test timer for 30 minutes and restart HA via configuration > Server Controls > Server Management > restart, but the timer came back as idle. I would say out of 10 restart there maybe 1 time the timer will be resume okay, so that proved all scripts and yaml were added correctly.

Does anyone encounter such problem ?

Things I have tried and noticed:

after timer enabled and counting down, checked input_timer.active_timers state do indicate “test_timer 1595269963”. the state is registered, so the 1st automation work
with HA logger default set to “info”, reboot system and home-assistant.log show
2020-07-20 10:41:52 INFO (SyncWorker_0) [homeassistant.components.python_script] Executing active_timers_resume.py: {}
so the 2nd automation also work and called the pythonscript
I added couple logger.info line to the pythonscript to see if it read the value back correctly and here are some clue

When timer didn’t resume

2020-07-20 11:13:59 INFO (MainThread) [homeassistant.setup] Setting up python_script
2020-07-20 11:14:01 INFO (MainThread) [homeassistant.setup] Setup of domain python_script took 2.2 seconds.
2020-07-20 11:14:24 INFO (SyncWorker_2) [homeassistant.components.python_script] Executing active_timers_resume.py: {}
2020-07-20 11:14:24 INFO (SyncWorker_2) [homeassistant.components.python_script.active_timers_resume.py] active timers resume script ran
2020-07-20 11:14:24 INFO (SyncWorker_2) [homeassistant.components.python_script.active_timers_resume.py] ts= <state input_text.active_timers=; editable=False, min=0, max=255, pattern=None, mode=text, friendly_name=Active Timers @ 2020-07-20T11:14:02.986849-07:00>

Timer resumed correctly

2020-07-20 11:42:54 INFO (MainThread) [homeassistant.setup] Setting up python_script
2020-07-20 11:42:57 INFO (MainThread) [homeassistant.setup] Setup of domain python_script took 2.6 seconds.
2020-07-20 11:43:19 INFO (SyncWorker_0) [homeassistant.components.python_script] Executing active_timers_resume.py: {}
2020-07-20 11:43:19 INFO (SyncWorker_0) [homeassistant.components.python_script.active_timers_resume.py] active timers resume script ran
2020-07-20 11:43:19 INFO (SyncWorker_0) [homeassistant.components.python_script.active_timers_resume.py] ts= <state input_text.active_timers=test_timer 1595270700; editable=False, min=0, max=255, pattern=None, mode=text, friendly_name=Active Timers @ 2020-07-20T11:42:58.526629-07:00>
2020-07-20 11:43:19 INFO (SyncWorker_0) [homeassistant.components.python_script.active_timers_resume.py] timers= ['test_timer 1595270700']
2020-07-20 11:43:19 INFO (SyncWorker_0) [homeassistant.components.python_script.active_timers_resume.py] duration= 101

so seems like even the input_text.active_timers show the value in the developer tools > states, it is not saved properly. Thus, upon bootup, the script read an empty state, which caused the timer not to resume. And when the state were saved, it will read back properly and resume correctly.

Look into HA database noticed input_text.active_timers change and update instantantly whenever the state of the timer is changed
SELECT * FROM "states" WHERE "domain" LIKE 'input_text' ORDER BY "last_changed" DESC
Look into file system /.storage/core.restore_state and I believe this is where the pythonscript is getting the states from. And within the input_text.active_timers section, the state was never updated immediately. It update the state in random period.

Base on what I have found, if the input_text.active_timers state wasn’t updated correctly witin the .core.restore_state file, the timer may not resume properly.

Is it my HA configuration problem that causing the core.restore_state file not updating as often ?

123 · July 20, 2020, 9:16pm

Based on what you posted, the active timer was not resumed only when the the input_text was empty, otherwise it worked as expected.

Assuming we have an active timer, why would the input_text be empty?

For starters, the entity_id of an active timer is written to the input_text at the moment the timer starts. I see very little chance of it failing to do that.

As for the input_text itself, Home Assistant will periodically, and just before shutdown, store the input_text’s state to .storage/core.restore_state. On startup, it initializes the input_text’s state using the value recorded in core.restore_state. The database plays no part in the restoration process and the python_script (as does most everything else) gets the input_text’s value from the state machine (and not directly from core.restore_state).

Under what circumstances, during startup, would the input_text have no state value? This is just a guess but what if, during startup, the python_script executes before the input_text gets its old value restored?

One way to test this theory is to add a delay to the startup automation to (hopefully) prevent the python_script from executing before the input_text’s state is restored. If that fails to fix it, I’m not sure what’s grieving it nor how to mitigate it.

- alias: 'Active Timers Resume'
  trigger:
    platform: homeassistant
    event: start
  action:
    - delay: '00:00:05'
    - service: python_script.active_timers_resume

xenz · July 21, 2020, 4:54am

Yes, you’re correct, the active timer resume when input_text state value saves correctly, and it was not resumed only when the input_text state value was empty, and in my case, it wasn’t updated in the core.restore_state file for a very long period, not even before HA shutdown which is wierd.

I had my HA installed on Ubuntu, so I have two terminal open, one is monitoring the home-assistant.log file so I can tell when the python script is loading upon startup, another one is monitoring the change of the input_text.active_timer section within the core.restore_state file

When I start a timer, timer registered in the input_text.active_timer’ state value right away, but HA didn’t update the core.restore_state file until really really late, like 20min into a 30minutes timer late for example. so within the first 20min should I restart HA, state value isn’t in core.restore_state file, after reboot script reads an empty value lead to no timer resume.

activetimer1

when it work, the state value look like this :

"state": "test_timer 1595307325"

I don’t know why my HA doesn’t even update the core.restore_state file before shutdown either.

123 · July 21, 2020, 1:59pm

Good detective work!

I’ve never timed the interval used for storing state values but 20 minutes sounds about right.

That makes two of us. It’s supposed to do a final commit to core.restore_state during the shutdown process.

Let’s say that there’s no final commit operation. That means the only active timers that will be affected are the ones that started during the period between the last periodic commit and shutdown. In other words, worst case, during the last 20 minutes of operation before shutdown.

xenz · July 21, 2020, 3:55pm

Found an open issue that at HA github https://github.com/home-assistant/core/issues/36814#issuecomment-646804744. At least now I know I’m not the only one =) and at the same time Frederic look into the code confirm 15 min interval between updates for the core.restore_state file

123 · July 21, 2020, 3:59pm

Well, that confirms my theory about the lack of a final commit. That’s a bad bug. I’m surprised it hasn’t been prioritized for correction (now over one month old). No one has even been assigned to investigate it yet.

allanbell3d · August 18, 2020, 6:29am

First of all thanks Taras for the great work!!

I implemented this after someone suggested it in Discord and was very happy with the solution but I didn’t read this post and did not realize the limitation. I just found out from seeing some errors in the log file:

 [homeassistant.components.input_text] Invalid value: (length range 0 - 100)

My use case is 40 timers that need to run for 3 days each.

Any suggestions about how this could be implemented?

Thanks in advance!

allanbell3d · August 18, 2020, 12:40pm

Ok, so for anybody that might have the same problem, this is the work around I have implemented:

Create a bunch of input_text to store the different timers.
Using the include variant of the Active timers Save automation I repeat the service call as many times as needed referencing each time the different input_text we just created and its assigned timers in the include section.
In the python script I do the same, repeat the whole code for each input_text.

This seems to work, its not a very elegant solution but its the best I could come up with.

Update: Actually I have found a problem using the inclusion version of the automation, it was not updated to work with timers over 24 hrs, might want to add this to the first post:

- alias: 'Active Timers Save'
  trigger:
    platform: state
    entity_id:
    - timer.timer1
    - timer.timer2
    - timer.timer3
  action:
  - service: input_text.set_value
    data_template:
      entity_id: input_text.active_timers
      value: >
        {% set timers = [states.timer.timer1, states.timer.timer2, states.timer.timer3] 
                        | selectattr('state', 'eq', 'active')
                        | list %}
        {% set ns = namespace(timers = '') %}
        {% for t in timers %}
          {% set d = t.attributes.duration.split(':') %}
          {% if 'day' in d[0] %}
            {% set z = d[0].split(', ') %}
            {% set h = ((z[0].split()[0] | int) * 24 + (z[1].split(':')[0] | int)) * 3600 %}
          {% else %}
            {% set h = d[0] | int * 3600 %}
          {% endif %}
          {% set s = h + (d[1]|int*60) + (d[2]|int) %}
          {% set c = ',' if not loop.first else '' %}
          {% set ns.timers = ns.timers ~ '{}{} {}'.format(c, t.object_id, t.last_changed.timestamp()|int + s) %}
        {% endfor %}
        {{ ns.timers }}

123 · August 18, 2020, 1:40pm

Yes, I’m currently exploring the use of an input_select to store timer information. Each option in the input_select stores one timer’s information. An input_select has a set_options service but, for this application, an automation cannot be used and it will have to be a python_script.

The reason is because we have to dynamically build a list for set_options. A Jinja2 template cannot generate a list, it always generates a string. A python_script doesn’t have this limitation.

Unfortunately, there is no service call to retrieve an input_select’s options as a list UPDATE:
There’s no service call but all the options are indeed available as a list in the input_select’s attributes! That makes it very easy to iterate through all the options and requires none of the techniques I mentioned below.
(which would make it easy to iterate through its options). In addition, there’s no way to request the number of options. The best that it offers is a next_option service to step through the options. However, without knowing how many options exist, we can step through all of them over and over.

The simplest trick is to have the last option serve as a marker. In other words, the last option is never timer information but something like '###'. If one encounters it while stepping through the options, it indicates we have finished iterating through all available options.

I’m currently experimenting with this approach. I can easily create an input_select with 50 options. That means it is capable of storing the information for 50 timers (and up to 255 characters for each one). There may be an upper limit on the number of options supported by an input_select but I doubt anyone will ever encounter it.

123 · August 18, 2020, 4:21pm

Ha! That was a giant of waste of time!

I created the means to store time information in an input_select, and to retrieve it, only to discover something I should have known all along: an input_select’s options are initialized at startup.

No matter what options you set, on startup they revert to whatever is specified in the input_select’s configuration. Plus, you cannot configure an input_select without specifying options.

It appears the workaround you implemented (using multiple input_text entities) will have to suffice. I don’t intend to change the original code (you’re the first to encounter the 255 character limitation). Should anyone else be in the same position they have your workaround available.

Anyway, the exercise didn’t provide the result I wanted but it did serve to improve by knowledge of python.

Are you referring to what is listed in the first post under BugFix 1, 2020-0607?

allanbell3d · August 18, 2020, 6:25pm

Thanks for looking in to it Taras, apreciate your work!

Regarding your question: yes, I´m referring to that. I copied the code for the include version of the automation as its posted and that doesn’t have the bug fix implemented. Took some testing until I figured out why it was not working and included your modifications, now its working fine.

As a suggestion might be worth to add the 3 versions of the code with the bug fix implemented on the first post so people who use the exclude/include versions don´t need to figure out that the code for those needs to have the bug fixed added!

Anyway, thanks again, really useful script

123 · August 18, 2020, 6:42pm

Not following you. The correction reported in this post:

Is present in the topic’s first post.

[DEPRECATED] ~~How to make active timers survive a restart~~

[DEPRECATED] How to make active timers survive a restart