Automations not working - 1 month now

[123 (Taras)]

@klogg

by using that automation you lose the ability to have individual automations set to off at startup

True. That’s drawback #2.

As stated in the draft documentation, the suggested automation only mitigates drawback #1.

The automation forces all automations to on, just like adding initial_state: true to all automations, except it doesn’t cause the last_triggered attribute to be reset (drawback #1).

Would it be clearer if I reduced the second drawback to this:

You lose the ability to temporarily turn off an automation for a period longer than the next restart. At startup, the automation is always set to on regardless of its previous state.

[klogg]

I was thinking more of the drawback of using the automation being that you lose the ability of using initial_state: false for those automations you want to default to off at start up.

For example I have a couple of automations I turn on for testing purposes so I have initial_state: false but that means I could not use the automation to turn on all my automations at start up so I’d have to either trust the HA restore of list every one individually.

[DavidFW1960 (David)]

@123

Now your automations will be always be turned on at startup but won’t lose their last_triggered value.

I really like that. Great option.

[tom_l]

@anon43302295

All automations should be on always, and if you don’t want them to fire in certain circumstances you should use conditions.

An interesting philosophy but not one I subscribe to. e.g. seeing that my climate control automations have been disabled when I am away for an extended period is far more reassuring than leaving them on and relying on conditions. Using enable/disable allows for quick and easy feedback.

Also it allows for easy experimentation. Just last week I was trialling the use of my downstairs heat pump’s ability to heat the whole house. This heating source is automated but is disabled unless that area of the house is in use. Using your proposed system would have required altering a lot of conditions and reloading automations. Using my system all I had to do was toggle off the upstairs automations and enable the downstairs ones. It turned out that the heat pump was not up to the task so returning to the original setup was likewise just as easy.

[DavidFW1960 (David)]

@tom_l

I am away for an extended period is far more reassuring than leaving them on and relying on conditions

I have an input_boolean called home and away and as a condition in my automations, it checks if that is on or off and then does the action depending on that. Using an automation on/off is a poorer way of doing this and can have other consequences…

So Home means all my automations I want on when I am home will activate…
Similarly:

If I’m away they won’t run. Overtime means my wife is doing overtime and it adjusts the time some of my automations are activated via conditions as well and Holiday mode similarly adjusts times (so my coffee maker does not turn off during the day for example)

I find this approach superior to turning an automation on or off.

I also used to find that I have a lighting automation and if I’m away at night, I used to turn that off and then turn it back on that night but that would then trigger the automation and turn the lights on (and I would do that when I remembered to do it otherwise I would likely forget to turn it back on tomorrow and no lights) But the in-you-face input_booleans, colour coded… well that hasn’t happened. I can also turn those on/off via Google Assistant…

Anyway… my 2c

[tom_l]

@DavidFW1960

I have an input_boolean called home and away

I use an input select to set my automation ‘mode’ as there are more states than just home and away. I also have two guest modes and an ‘all off’ mode for HA-haywire-emergencies (which TBH I have never used and will probably remove). Automations are automatically enabled and disabled depending on the state of this input select. I could add conditions to each automation to check the state of this input select but I prefer having the list of which automations are enabled and disabled all in the one place (in the ‘mode’ automation).

@DavidFW1960

and can have other consequences

Like what?

I’m open to change just not yet convinced of the convenience.

[DavidFW1960 (David)]

I had some discussions with balloob when he was seeking feedback for removing the auto-generated groups like all automations and he said then to use an input_boolean condition was better so I switched at that time.

Other consequences like outlined in my post with unpredictable things happening when you re-enable the automation and then it triggers when you don’t expect it to.

In your case I would just use the input_select as a condition and expose that on a lovelace button but each to their own.

[anon43302295]

@tom_l

Using your proposed system would have required altering a lot of conditions and reloading automations…

Only because you didn’t write them to that system in the first place

[klogg]

I think the answer is no, but I just want to check…
Is it possible to use any kind of wild cards?

For example if you name all your lights automations beginning with the word ‘lights’ could you do something like this?

- alias: 'Start All Automations'
  initial_state: true
  trigger:
    - platform: homeassistant
      event: start
  action:
    - service: automation.turn_on
      entity_id: automation.lights*

[123 (Taras)]

@klogg

you lose the ability of using initial_state: false for those automations you want to default to off at start up

True but the suggested automation, as stated in the draft documentation, is designed to address one of the two drawbacks of using initial_state: true. It is not described as a solution for anything else.

It is beyond the documentations’s scope to offer an automation that handles both initial_state: true and false. That would involve a considerably more sophisticated automation (involving groups and/or templates). That seems too advanced for a documentation page that introduces users to automations.

[klogg]

@123

It is beyond the documentations’s scope to offer an automation that handles both initial_state: true and false .

I agree, I was just suggesting that to give some balance it could be pointed out as a drawback to using the startup automation.

But I’m easy, and for what it’s worth I’m happy to back your doc change as you have it.

[frenck (Franck Nijhof)]

Hi!

Thanks for all discussion. We are currently looking into this and discussing on how to resolve the general issues around this subject.

In short: the documentation is currently wrong, but there is a misconception on what initial_state does/is.
initial_state is the state the entity/automation has on startup of Home Assistant and overrides anything.
Without an initial_state, it will always try to restore the state from storage. If restoring of a state fails, the state for automation = ON/True/Enabled.

For other platforms/integrations, the state after restoring the state failed, depends on the implementation of that specific integration.

The confusion probably comes from the fact with people looking to set a default state in case restore has failed. It is not possible to change that specific behaviour and, in case of automations, will always be on.

Nevertheless, we are internally verifying everything and considering options to resolve the confusion around this.

[tom_l]

@frenck

The confusion probably comes from the fact with people looking to set a default state in case restore has failed. It is not possible to change that specific behaviour and, in case of automations, will always be on.

Except it’s not (unless I’ve misunderstood what you are saying). If you interrupt HA statrup and restore fails (and you dont have initial_state set) the default state for automations is off, not on. Hence - people recommending set initial_state to true.

[anon43302295]

^ this is exactly the problem.

[frenck (Franck Nijhof)]

Snippet from the code:

"""Allow to set up simple automation rules via the config file."""
DEFAULT_INITIAL_STATE = True


class AutomationEntity(ToggleEntity, RestoreEntity):
    """Entity to show status of entity."""

    async def async_added_to_hass(self) -> None:
        await super().async_added_to_hass()

        """If initial_state if configured, it will use that... """
        if self._initial_state is not None:
            enable_automation = self._initial_state

        """Initial state not set by user in config."""
        else:
            """Try to get previous state."""
            state = await self.async_get_last_state()

            """ If we have a previous state, use that."""
            if state:
                enable_automation = state.state == STATE_ON
                self._last_triggered = state.attributes.get('last_triggered')
                _LOGGER.debug("Automation %s initial state %s from recorder "
                              "last state %s", self.entity_id,
                              enable_automation, state)

            """If we do not have a previous state, use the default state, which is True."""
            else:
                enable_automation = DEFAULT_INITIAL_STATE
                _LOGGER.debug("Automation %s initial state %s from default "
                              "initial state", self.entity_id,
                              enable_automation)

Reference: https://github.com/home-assistant/home-assistant/blob/dev/homeassistant/components/automation/__init__.py#L221

edit: I do see something, but that needs some testing. Will dive into this tonight.

[tom_l]

@frenck

“”“If we do not have a previous state, use the default state, which is True.”“” else: enable_automation = DEFAULT_INITIAL_STATE

This is definitely not the observed behaviour.

[123 (Taras)]

@frenck

DEFAULT_INITIAL_STATE = True

The code appears to implement the correct fail-safe mode, namely when it’s unable to restore the automation’s previous state, it sets it to true. The automation is turned on. That’s definitely a better fail-safe mode than turning them all off.

HOWEVER, all automations turned off is the behavior people are reporting. The implication is that either the restoration feature has a faulty fail-safe mechanism or (long-shot) maybe the aborted startup process doesn’t even get to execute the restoration step.

What I find interesting is that before anyone pointed out the relevant fail-safe code, and how it defaults to on, it was generally accepted that startup failure would result in automations set to off not on.

In fact, that behavior was documented. The following excerpt comes from an older version of the documentation:

If you don’t set this then the previous state prior to restart is restored. However, if you shut down Home Assistant again before it finishes starting, any automation that doesn’t have the initial state set to true will be stored as being off, and those automations will be disabled at the next startup.

From an even older version:

If you don’t set this the previous state is restored. If you shut Home Assistant down before it finishes starting, the automation will be stored as being off, and your automations will be disabled at the next startup.

It was this observed behavior that led many users to wallpaper all their automations with initial_state: true to ensure they wouldn’t be disabled after a failed startup.

The current documentation, with its false and misleading statements, only makes a bad situation worse. Yet it’s just the tip of the iceberg. There’s no ‘automation’ in ‘home automation’ if a restart glitch disables all automations.

I applaud you, and all others involved, for starting an investigation into this issue. If you need our help to collect more data on this undesirable behavior, I’m ready and willing to assist.

---

EDIT
Here’s a data-point: I have never experienced all automations disabled after a restart. My test system is restarted several times daily (I use it to experiment with new automations in the course of assisting community members).

However, the startup process takes less than 10 seconds and it doesn’t rely on any external devices and services (zwave, zigbee, Google, Amazon, Nest, etc) other than MQTT which runs on a separate machine. In other words, there’s very little that can go wrong during my test system’s startup other than me making configuration errors. So my ‘surface area’ for startup failure is probably much smaller than for other people.

[firstof9]

@123

maybe the aborted startup process doesn’t even get to execute the restoration step.

This is what I assumed the case was and since at that point it sees the automations as all off, it records them as such, and thus the problem occuring.

[finity]

I do have one minor suggestion on you proposed doc change.

The following is only partially correct:

@123

The option can be set to one of two states:

• initial_state: true sets the automation to on at startup.
• initial_state: false sets the automation to off at startup.

However, the fully correct statement is:

The option can be set to one of two states:

 * initial_state: 'on' (or 'true') sets the automation to 'on' at startup.
 * initial_state: 'off' (or 'false') sets the automation to 'off' at startup.

And I believe I saw something recently that there is a plan to change the behavior to require ‘on/off’ instead of ‘true/false’ to make it more consistent with reality since what does an automation state being “true” mean? In the states page the state is either ‘on’ or ‘off’ not ‘true’ or ‘false’ so why not make the config match the desired state. I kind of agree.

Even if the required config language doesn’t change it still needs to be added that ‘on/off’ are valid settings. Those are what I use.

@123

Would it be clearer if I reduced the second drawback to this:

I think it would be clearer if it was specifically stated in the section on the start-up automation that all automations will be turned on no matter the state prior to restart.

Something like this:

@123

Now when Home Assistant starts, it will trigger automation.start_all_automations and set all automations to on no matter what the state was prior to start-up. Each automation’s last_triggered attribute will not be reset and its previous value (if any) will be preserved.

@123

Here’s a data-point: I have never experienced all automations disabled after a restart.

And to add another data point…

I have personally experienced the behavior where all automations were switched to ‘off’ during a failed system restart at least twice in addition to the expected initial restart after the restore storage mode config was initially implemented.

I think that’s why I thought that the default state of all new automations had been set to ‘off’. It made no sense to me that the default state was ‘on’ but also that the default state as happened in reality was that they “defaulted(?)” to ‘off’ on a restoration failure. That is where at least my confusion began.

[123 (Taras)]

I read the discussion in this PR titled Improve boolean validator, specifically this post by @frenck describing the synonyms for boolean true and false in YAML 1.1 and 1.2:

In YAML 1.1: y|Y|yes|Yes|YES|n|N|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF

In YAML 1.2: true|false

I’m nor sure what that portends for Home Assistant but I doubt it will ever discard YAML 1.1’s many synonyms for true and false. Therefore I do agree the documentation should show the common form used in Home Assistant, namely on and off.

Having said that, I think the correct format is to show on and off delimited by quotes but no delimiters for true and false.

The option can be set to one of two states:

• Use true or 'on' to always set the automation to on at startup.
• Use false or 'off' to always set the automation to off at startup.

Examples:
initial_state: 'on' sets the automation to on at startup.
initial_state: false sets the automation to off at startup.

If you’re OK with that, I’ll amend the draft documentation. I’ll also incorporate your suggestion to state “no matter what the state was prior to startup”.