Automations, from Zero to Hero

7. Advanced automation

Okay, now let’s take things to the next level:
Say, we have 3 cameras and 3 PIRs, the cameras are integrated with Home Assistant using Frigate or Blueiris. When they detect something, a motion sensor turns to on. They are all member of the group group.security.

We want to avoid false positives and trigger the big time alarm that will play a loud dog barking on your Sonos, the lights and all the circus, only if we’re fairly sure it’s not a false positives. Meaning not a spider bungee jumping in front of your camera, not a fat cat walking in front of a PIR.

What we want is a combination of at least 2 PIR or Camera being triggered independently within a timeframe of 90 seconds, which would help us identify it’s a burglar scouting your house. We want all that and more, in one automation. Yes, a full-featured security system, smarter than the one on the market, anti-false positive and all, just using automation…

We could trigger on a group change, but that would not tell us which sensor triggered and, if retriggered within a specific time, this is would be considered the same trigger, even if it’s a different member that triggered the group to on.

- id: "110020"
  alias: Alarm - Combined tiggers
  description: Combine different camera and PIR motion sensors triggers within 90s before setting off the alarm
  trigger:
    - platform: state
      entity_id:
        - binary_sensor.cam1_motion_trigger
        - binary_sensor.cam2_motion_trigger
        - binary_sensor.cam3_motion_trigger
        - binary_sensor.pir1_motion_detection
        - binary_sensor.pir2_motion_detection
        - binary_sensor.pir3_motion_detection
      to: "on"
  condition:
      - "{{ iif(expand('group.family_members') | map(attribute='last_changed') | select('gt', now() - timedelta(seconds=360)) | list | count > 0, true, false) }}"
      - "{{ expand('group.security') | map(attribute='last_changed') | select('gt', now() - timedelta(seconds=90)) | list | count > 1 }}"
      - "{{ states('group.innercircle_members') == 'not_home' or (today_at('23:00') <= now() <= today_at('23:59')) or (today_at('00:00') <= now() <= today_at('08:00')) }}"

Ok STOPPP. Slow-mo please:

  • The 1st condition says: if a family member just left the house, no need to ring the alarm, he has 6 minutes to leave in front of the CAM & PIR, and nothing will happen. We list how many members of the group family_member changed within the last 360s, count the number of items that changed and if >0, condition is true, proceed. If it’s less than 6 mins, the condition will return false.
  • The 2nd one says: if at least two members (strictly superior to 1 is 2) changed within 90 seconds, then we have an issue. Ok, be careful here that the auto-reset of your PIR and CAM are resetting (hence changing a status) after more than 90s. Otherwise, a PIR triggering and returning to normal would count 2. Except… The trigger only kicks if the entity is switching to on. So when they switch to off, the automation isn’t triggered, meaning it’s not an issue. This lesson was painful for me, but remember that no condition is evaluated if the trigger doesn’t fire in the first place, which is sometimes convenient.
  • The 3rd one says that the inner circle group (family members + friends device trackers) has to be away or that it’s after 23:00 and before 8am. If it’s day and we’re not here or night and we are home.

Ok let’s resume:

  action:
    - service: notify.emergency
      data_template:
        title: "ALARM!"
        message: >
          {{ "3 or more sensors triggered:\n" }}
          {% for sensor in expand('group.security') -%}
            {% if now() - sensor.last_changed < timedelta(seconds=100) -%}
              {{- " * " + state_attr(sensor.entity_id, 'friendly_name').partition(' ')[0] + "\n" -}}
            {% endif -%}
          {%- endfor %}
        data:
          priority: 1
          sound: bugle
    - service: media_player.volume_set
      data:
        entity_id: media_player.gaming_room
        volume_level: "0.7"
    - service: media_player.play_media
      data:
        entity_id: media_player.gaming_room
        media_content_type: music
        media_content_id: "http://192.168.1.1:8123/local/dog.wav"
    - if: "{{ (as_timestamp(now()) - as_timestamp(state_attr('automation.combined_3_tiggers','last_triggered'), 0) | int > 600) }}"
      then:
        - service: camera.snapshot
          entity_id: camera.blueiris_cam1
          data:
            filename: "/config/www/tmp/snapshot_cam1.jpg"
        - service: notify.info
          data_template:
            title: "Cam1"
            message: 'Alert - {{now().strftime("%H:%M %d-%m-%Y")}}'
            data:
              attachment: "/config/www/tmp/snapshot_cam1.jpg"
              priority: 0
              sound: intermission
  • We want a notification and send the names of all sensors that went on for the last 100 seconds.
  • We want to set the Sonos level in the gaming room to 70%… Loud.
  • Let’s play the big bad dog sound
  • And finally, if this very automation was last run more than 10 minutes ago, we allow ourselves to snapshot one of the camera.
    and then call whatever the notification service (mine is pushover) to send yourself the picture of the cam.

8. Very advanced automation

[WARNING]: This section isn’t for beginners. You may discouraged if you have not practiced HA a lot and try to understand the next part. It may not be necessary to discourage yourself thinking “I don’t get it”. The learning curve is sometimes steep, that is normal, no need to break your head on too many new concepts at once. Bookmark it, and return to it later when you’ve bagged some experience points in HA automation.

Ready? This one will combine what we’ve learned previously with advanced templating and loops.
We’ll dissect one advanced automation, step by step, to understand the full power of HA automations & templates. This one is in charge of the Air Conditioning system. (HVAC automation)

A bit of background here. Daikin provides great hardware, below-average software & API and probably the worse remotes I have ever had. On top of that … Remotes? in 2023? Really? Naaaaa

So, I skip on the obvious: 2 other automations that calculate the proper HVAC mode and the proper HVAC temperature, per room, depending on outside temp, inside each room temp, hour of the day, day of the week. They calculate the proper value and set accordingly input_text.XXX_hvac_mode and input_number.ac_temp_XXX per room, every 15 minutes.

Also, all A/C units are listed in a group named: group.ac_splits

ac_splits:
  name: AC splits
  entities:
    - climate.ac_kitchen
    - climate.ac_library
    - climate.ac_parent_bedroom
    - climate.living_room
    - climate.office

Now to the big bad wolf:

- id: "230000"
  alias: A/C - Pilot all A/C mode & temp
  description: Set A/C mode & temp, per room, every half an hour
  trigger:
    - platform: time_pattern
      minutes: "/30"
  action:
    - repeat:
        for_each: >
          {% set data = namespace(splits=[]) -%}
            {%- for climate in expand('group.ac_splits') -%}
              {% set data.splits = data.splits + [climate.entity_id] %}
            {%- endfor -%}
          {{ data.splits }}
        sequence:
          - variables:
              mapper: >
                { 'climate.ac_kitchen':        ["{{ states('input_number.ac_temp_kitchen_final') | float }}","{{ states('input_text.kitchen_hvac_mode')        }}", "{{ states('binary_sensor.kitchen_door_sensor_access_control_kitchen')                    }}", "{{ states("input_boolean.ac_kitchen_override") }}" ],
                  'climate.ac_library':        ["{{ states('input_number.ac_temp_library_final') | float }}","{{ states('input_text.library_hvac_mode')        }}", "{{ states('binary_sensor.library_door_sensor_window_door_is_open')                       }}", "{{ states("input_boolean.ac_library_override") }}" ],
                  'climate.living_room':       ["{{ states('input_number.ac_temp_living_final')  | float }}","{{ states('input_text.living_hvac_mode')         }}", "off", "{{ states("input_boolean.ac_living_override")  }}" ],
                  'climate.office':            ["{{ states('input_number.ac_temp_office_final')  | float }}","{{ states('input_text.office_hvac_mode')         }}", "off", "{{ states("input_boolean.ac_office_override")  }}" ]}
          - if: "{{ now().hour not in (1,2,3,4,5,6,7,8) }}"
            then:
              - service: climate.set_temperature
                data:
                  temperature: "{{ mapper[repeat.item][0] }}"
                  hvac_mode: >
                    {{ iif((mapper[repeat.item][2] == 'on' or mapper[repeat.item][3] == 'on'), 'off', mapper[repeat.item][1]) }}
                target:
                  entity_id: "{{ repeat.item }}"
              - service: system_log.write
                data:
                  message: >
                    {{ repeat.item + ' temp: ' + mapper[repeat.item][0] + ', mode: ' +  mapper[repeat.item][1] }}
                  level: warning
              - delay: "00:00:02"
    - service: script.notify_and_log
      data:
        title: "A/C settings update:"
        message: >
          {{"-----------------------------------\n
          | Room    | Temp | Mode, Stat, Ovrd\n
          | Kitchen | " + states('input_number.ac_temp_kitchen_final') + ' | ' + states('input_text.kitchen_hvac_mode') + ', ' + states('binary_sensor.kitchen_door_sensor_access_control_kitchen') + ', ' + states("input_boolean.ac_kitchen_override")
          + "\n| Library | " + states('input_number.ac_temp_library_final') + ' | ' + states('input_text.library_hvac_mode') + ', ' + states('binary_sensor.library_door_sensor_window_door_is_open') + ', ' + states("input_boolean.ac_library_override")
          + "\n| Parents | " + states('input_number.ac_temp_parents_final') + ' | ' + states('input_text.parent_bedroom_hvac_mode') + ', ' + states('binary_sensor.parent_bedroom_door_sensor_access_control_window_door_is_open') + ', ' + states("input_boolean.ac_parents_override")
          + "\n| Living  | " + states('input_number.ac_temp_living_final') + ' | ' + states('input_text.living_hvac_mode') + ', off, ' + states('input_boolean.ac_living_override')
          + "\n| Office  | " + states('input_number.ac_temp_office_final') + ' | ' + states('input_text.office_hvac_mode') + ', off, ' + states('input_boolean.ac_office_override')
          + "\n------------------------------------"}}

Let’s break the interesting parts of this big baby down:

- repeat:
    for_each: >
      {% set data = namespace(splits=[]) -%}
        {%- for climate in expand('group.ac_splits') -%}
          {% set data.splits = data.splits + [climate.entity_id] %}
        {%- endfor -%}
      {{ data.splits }}

Ok this basically list all AC splits (the units blowing hot/cold) and create a convenient list with it.
Then, we’ll iterate on this list and apply the below actions.

The repeat.item is passed to the actions below in the automation. So basically, at every loop, the repeat.item is advancing in the list data.splits and you replace the supposed entity_id or whatever target you’re working on, using repeat.item, so here, it’ll take the value, climate.ac_kitchen then climate.ac_library then climate.ac_parent_bedroom, climate.living_room, climate.office, etc.

sequence:
  - variables:
      mapper: >
        { 'climate.ac_kitchen_2': ["{{ states('input_number.ac_temp_kitchen_final') | float }}","{{ states('input_text.kitchen_hvac_mode') }}", 
        [...]
  - if: "{{ now().hour not in (1,2,3,4,5,6,7,8) }}"
    then:
      [...]
- service: script.notify_and_log
 [...]

Here, the sequence will play at each loop, so here, for each AC unit in the group we iterate upon.
We test if it’s nighttime and then do stuff. Outside of this loop, so at the end of the full cycle, is a script action that notifies me and log in HA system logs, if a input_boolean.debug is set to on.

Now at the beginning of the sequence of action, you see a variable, a giant table named mapper (I invented the name. It’s not required to be named like this), with a specific structure. The ones familiar with Python will get what it is. Let’s use a simplified version:

{ 
    'climate.kitchen': ["{{ states('input_number.kitchen_temp') }}","{{ states('input_text.kitchen_mode') }}", "{{ states('binary_sensor.kitchen_door')}}", "{{ states("input_boolean.kitchen_override") }}" ],
    'climate.living': ["{{ states('input_number.living_temp') }}","{{ states('input_text.living_mode') }}", "{{ states('binary_sensor.living_door')}}", "{{ states("input_boolean.living_override") }}" ],
    [...]
}

So in this table, I store the air conditioner entity name as repeat.item, then mapper[repeat.item][0] is the temperature, [1] is the mode (cool/hot), [2] is the state of the kitchen door (if it’s opened, no need to warm or cool), and the last one is an override. Another automation controls it. This other automation detects if the mode and temp differ from the ones the system calculated. If so, it means someone took over manually, using the remote. Like they turned off the unit of changed the temperature directly. You don’t want to mess with their preference, so I suspend sending orders to this A/C unit until 8Am if it happened at night or for 2h otherwise.

The target temperature and hvac modes are caculated by other automations and finally, the zwave door position detector knows if the door of the kitchen is opened or closed.

- if: "{{ now().hour not in (1,2,3,4,5,6,7,8) }}"
  then:
    - service: climate.set_temperature
      data:
        temperature: "{{ mapper[repeat.item][0] }}"
        hvac_mode: >
          {{ iif((mapper[repeat.item][2] == 'on' or mapper[repeat.item][3] == 'on'), 'off', mapper[repeat.item][1]) }}
      target:
        entity_id: "{{ repeat.item }}"
    - service: system_log.write
      data:
        message: >
          {{ repeat.item + ' temp: ' + mapper[repeat.item][0] + ', mode: ' +  mapper[repeat.item][1] }}
        level: warning
    - delay: "00:00:02"
  • If the hour is not during night (between 1:00 and 8:00), you set the proper temp & mode, otherwise, you do not. You set the temperature using the proper service by reading the mapper[repeat.item][0] from the mapper table and the mode by reading mapper[repeat.item][1]. If a door is opened or there was a manual bypass with the remote, turn the unit off.
  • Log and write what happened.
  • Then we wait for 2s, not to burst too much on the fragile Daikin API, before repeating for the next AC unit.
    - service: script.notify_and_log
      data:
        title: "A/C settings update:"
        message: >
          {{"-----------------------------------\n
          | Room    | Temp | Mode, Stat, Ovrd\n
          | Kitchen | " + states('input_number.ac_temp_kitchen_final') + ' | ' + states('input_text.kitchen_hvac_mode') + ', ' + states('binary_sensor.kitchen_door_sensor_access_control_kitchen') + ', ' + states("input_boolean.ac_kitchen_override")
          + "\n| Library | " + states('input_number.ac_temp_library_final') + ' | ' + states('input_text.library_hvac_mode') + ', ' + states('binary_sensor.library_door_sensor_window_door_is_open') + ', ' + states("input_boolean.ac_library_override")
          + "\n| Parents | " + states('input_number.ac_temp_parents_final') + ' | ' + states('input_text.parent_bedroom_hvac_mode') + ', ' + states('binary_sensor.parent_bedroom_door_sensor_access_control_window_door_is_open') + ', ' + states("input_boolean.ac_parents_override")
          + "\n| Office  | " + states('input_number.ac_temp_office_final') + ' | ' + states('input_text.office_hvac_mode') + ', off, ' + states('input_boolean.ac_office_override')
          + "\n| Living  | " + states('input_number.ac_temp_living_final') + ' | ' + states('input_text.living_hvac_mode') + ', off, ' + states('input_boolean.ac_living_override')
          + "\n------------------------------------"}}

A script call notify_and_log now spits the whole status of the system in the logs and a notification.
This script only acts if the input_text.debug is on, so I don’t receive any debugging statement if it’s not.
The debug is turned on and off through a simple button press on the lovelace interface.

A better way to iterate in an array was proposed to me by @123 tara:

  action:
    - repeat:
        for_each:
          - hvac: climate.ac_kitchen
            temp: input_number.ac_temp_kitchen_final
            mode: input_text.kitchen_hvac_mode
          - hvac: climate.ac_library
            temp: input_number.ac_temp_library_final
            mode: input_text.library_hvac_mode
            [... YOUR LIST...]
        sequence:
          - if: "{{ states(repeat.item.mode) != 'off' }}"
            then:
              - service: climate.set_temperature
                data:
                  temperature: "{{ states(repeat.item.temp) }}"
                target:
                  entity_id: "{{ repeat.item.hvac }}"

9. Debugging tools

You’ll typo, create failed logic, forget the impact of one automation on another, etc.
Beyond keeping it clean (files structures, naming convention, variables, etc. ), the need of debugging will kick in. Sadly, this is not where HA excels yet. I’m sure this will progress, but in the meantime, cryptic messages will pop in the logs, explaining you… something pretty unuseful and very unclear. How to get out of this spiral?

Log your automations output
obvious but sending logs and then reviewing them with the command line using ha core logs will help.
Now you also often get more detailed output using this one than the settings->system->log.
Log early in the actions if your automation fails after, otherwise, you’ll get no traces.

Send yourself notifications
Logs are good, but getting your output on a notification can be very convenient.
Specifically, pushover is very good for that use.

Traces
In settings->automations, select the automation you want to debug and look for “traces” in the three-dot menu on the right. There is a wealth of information to be found and exploited there. Most of my successes in debugging complex automations leveraged it at some point.

Use the developer tab

  • Templates: if you write a template, it should evaluate properly in the developer->template tab. Otherwise, it just won’t in HA

  • Services: it’s so easy to just use the GUI and add parameters, look then in YAML mode to cut/paste the content. But also, you can check why a service isn’t firing by manually testing the parameters sent to it in this tab.

  • States: obvious but still useful, which entity is in what state.

10. Tips & tricks

  • Enforcing your variable types, aka “casting” and variables defaults. You’ll eventually get some errors saying you can’t compare STR and INT or something similar. To avoid this, think about the default values and typecasting. It’s essentially true for the custom templated sensors you’ll create, but has a strong repercussion on automation hygiene:
{{ states('whatever.variable') }}: classical form, but could be error-prone
{{ states('whatever.variable') | int }}: this one is set to be evaluated as an integer
{{ states('whatever.variable') | in(0) }}: this one is set to be evaluated as an integer and if in an unknown state, defaults to 0
{{ states('whatever.variable') | string | default("Nothing", true) }}: this one is set to be evaluated as a string and is set to "Nothing" if in an unknown state.
  • After a reload, head to the logs and filter through Automation: easy way of seeing if all goes according to plan. Optionally, you can set a growing counter that will increase only when an automation warning or error is detected in the logs, and graph it. If the graph is growing fast, you have an issue.

  • How {{ }} works: Those curly brackets will trigger the interpretation of whatever is inside. so if you mix strings and variables:
    “{{ state_attr(‘sensor.imeon’,‘Battery_current’) | float(0) }}”
    “{{ now().hour not in (1,2,3,4,5,6,7,8) }}”
    “{{ repeat.item + ’ temp: ’ + mapper[repeat.item][0] + ', mode: ’ + mapper[repeat.item][1] }}”

Now be careful if you improperly mix ’ and ", namely if you use only " before and after {} and inside them, it will not be interpreted and generate an error.
Rule of thumb, keep double quote exterior and simple quote inside like in the 3rd example.

  • How direct template or to the line template work
- service: climate.set_temperature
  data:
     hvac_mode: >
          {{ iif(states('A') == 'on' or states('B') == 'cool', 'off', 'on') }}

and:

 - service: climate.set_temperature
   data:
       hvac_mode: "{{ iif(states('A') == 'on' or states('B') == 'cool', 'off', 'on') }}"

are the same. notice the " used when inline with the parameter hvac_mode in the second example, which are not need in the 1st example because of the above >

Closing word

Here we are. I probably missed points, typoed a lot and will make tons of edit, but hopefully, you’ll walk the path that took me months in weeks.

I’m sure the big shots here will find innovative or elegant ways to optimize my templates, and I’m eagerly looking for it.

Constructive comments are welcomed, don’t hesitate to add you own advanced tricks.
Good sharing, everyone.

14 Likes

Very nice writeup. I use the Docker install so some of the screens or ideas are not relevant or the same but it is still very useful and I suspect most new users are not going the docker route as they get their feet wet.

1 Like

Thanks @pcwii
You mean the fact you use HA in a docker environment limits the capacity to create/run automation?
For my education, what are the main differences or limitations you are facing using HA in a docker env?

There are differences for sure, I am not sure about limitations I guess that depends on your perspective.
Things I like about docker are…

  • everything I need for HA is in the container, I never have dependency issues since all the dependencies are present regardless of the host OS.
  • every “addon” I have needed I have found an additional docker container for that I run to give me that function.
  • I run watchtower which automatically updates my HA with the latest builds when they are available. Some people may be intimidated by this as I need to be aware of breaking changes and make changes as soon as I can. Since I am always running the latest build I find this more manageable than running an old build and having to deal with many surprises when I try to run the latest build.

Docker has it’s own learning curve, and I use it quite a bit with work so I am not intimidated by this. For what it’s worth I also use Portainer as I find it helpful for managing my docker compose installs. One day I will create a post about my installs.
Here are most of the compose files “stacks” I run.

2 Likes

oh I definitely have to look into this indeed. I use HAOS myself, on a PI dedicated to this, but indeed, if the hardware fries, I can’t relocate, where the containers approach allows you to do it at the snap of a finger.

1 Like

no, not at all.

there are only a few limitations to a non-supervised install type but none of them can’t be fairly easily overcome.

Probably the biggest one for non-techie users is the lack of add-ons. But if you can install and run HA in a Docker container you can likely figure out how to run your own equivalent containers for those add-ons anyway.

the other one is system backups but if you know Docker or a use venv then you can manage those things yourself fairly trivially too.

TBH, I can’t really think of any other important differences.

the upside of not using a supervised install type is that you have total access and control of the host OS. Which means you can use the machine to do other things along side HA and HA won’t complain about it or totally handicap your HA install because the system is “unsupported”.

3 Likes

Well nah. HA has specific copy and paste instructions for installing docker and then a containerised HA. You don’t really have to know anything about the innards of docker, or what to do other than copying and pasting.

OTOH getting stuff like mosquitto appdaemon esphome z2m etc going in docker is not quite so cut & paste. An addon is easier. Also Ingress is great in the addons.

I didn’t mean to imply it was just cut & paste but if you know docker well enough to get HA Container running then configuring the equivalent docker containers for the add-ons shouldn’t be too much harder.

the only container I ever really struggled with and eventually gave up on was NUT.

I ended up just installing it on the host instead (the big benefit of HA Container) and it’s been fine.

Well it is. Running ha in docker with all the addons aka containers isn’t that difficult. I really don’t understand why people think that this is something special, hard, or I don’t know what. Setting up docker security is difficult.

2 Likes

Is there a way to execute an action in a condition section that can affect the state values (e.g force a polling of a device for its status before evaluating that status in the condition statement)?

When you do something like:

condition:

  • “{{ states(‘whaterver.entity’) == ‘on’ }}”

The state is evaluated at the precise moment the condition is evaluated.

But if you need to trigger an action of the device, a script to refresh, etc. I don’t think it’s doable in a condition. You could use another automation to refresh at a certain rate this status, store it in a variable and instead use this variable in your automation.

Like automation A is polling, and updating a variable every 5 minutes.
Automation B is using this variable instead of the original device state in its conditions.

I was afraid of that – what I settled on was using other sensors to trigger the query so that the device status is correct when the automation runs. It was be best I could do given that the devices (two Schlage Z-Wave plus locks) don’t report when they are manually locked/unlocked.

or since you can use a condition in an action, what you could refresh the state before evaluating it before launching the action if the evaluation match your criteria.

- service: script.poll_zwavedevice_state 
- if: "{{ whatever.variable == "whatever.state" }}"
  then:
    - service: whatever.service

and the script is just updating whatever.variable when called

It turned out the easy way to do this was to have two automations.

The first automation has the trigger then polls as an action, waits a few sec, then invokes a second automation that has the conditions to check in it and the notification action.

What was the size limitation issue mentioned at the end of the first post? It appears to prevent the guide from functioning as a wiki, which is one of the points of the category.

sorry, I meant the size of the post.
It didn’t allow me to make it in just one post, I had to fragment it in 2.
There is probably a better place or way of doing it, but I’m not familiar with it.

Thanks - just that I was thinking of doing a longish guide myself and wondered if there was a max number of characters or something.

If it were possible to add to your excellent write up, I might insert a line warning against using device ids in triggers and conditions - a lot of people seem to be doing it nowadays and it’s hardly ever necessary. Just stores up trouble for the future.

2 Likes

absolutely right, send me a PM with your write-up of the pb/solution and I’ll add it under your name.
Anyone with good advices alike, I’ll make an appendix with your own tricks!

Sorry, wasn’t making myself clear. I am doing a write-up of something else - just wondering what the character limit is.

I would guesstimate 16000 chars @Stiltjack.

1 Like