High frequency Automation/Script {{ templates }} performance tuning?

[ferventgeek (Patrick Hubbard)]

I’m hoping a core event processing guru can help me understand how to design Automations that remain efficient under high-frequency event load (>1000/min).

Background

I’m a former developer and have been doing home automation and multi-platform integration for 20+ years, but I’m new to Home Assistant. I’m coming from a fairly complex Universal Devices IoX environment, with Insteon/Z-Wave plus multiple PG3 Node Servers, including a couple I hacked together to close integration gaps.

Home Assistant has been a huge step change for me. Its modern architecture, breadth of integrations, consistently strong documentation, and community are all outstanding. I’ve closed integration gaps that had been open for years in a matter of minutes. What used to be “I’ll code around that someday” is now: find the integration, read the docs, test, done. I’m genuinely grateful for the work that’s gone into this project.

Context

I built an Automation that can trigger up to ~1000 times per minute when downstream elements are active. As a concrete example, a dimming hold event can generate 20–40 events/sec while active.

As you can imagine, I’d like to minimize processing overhead as much as possible for Automations like this one:

I’ve done a fair amount of reading, including threads like this one:

Order of variables in automation?

I’m declaring some variables at the top of my automation to simplify some of the logic later. For example: variables: is_daytime: >- {{ (now().hour * 60 + now().minute > 5 * 60 + 00) and (now().hour * 60 + now().minute < 23 * 60 + 00) }} is_outside_camera: "{{ source_camera != 'familyroom' and source_camera != 'familyroom_pool' }}" should_describe: "{{ (not is_daytime) or is_outside_camera }}" I occasionally get errors in the logs that is_daytime is undefined, leading me to beli…

I’m trying to build a mental model for how Automations are executed internally, and I have a few questions based on that.

Questions

1. When defining a block of convenience variables in an Automation, are those templates executed eagerly when the Automation run starts, or are they evaluated lazily as needed?

   For example, if I have an early “gross filter” condition that yeets ~95% of events before they reach more complex logic, will the templates for variables defined later already have been evaluated, or only for the events that pass the filter?

2. Is there any performance advantage to moving the bulk of action logic into a script, and using a smaller “router” Automation that filters the event stream and only calls the script for qualifying events?

   In other words, can this reduce load on the event processing path, or does the cost of the Automation run still dominate either way?

3. For conditional branches that rely on templates, are those templates evaluated on demand in execution order, or are they all resolved up front when the Automation run begins?

   This is slightly different from #1. I’m trying to understand execution behavior for conditional evaluation, not just variable assignment.

Working assumption

My current assumption is that:

• events enter the core processing pipeline
• Automations with matching triggers are evaluated
• for each matching Automation, templates are all rendered before the run
• then the action sequence executes

Note: I assume that enqueued Automatons don’t evaluate templates as they’re queued, only when they actually execute.

If that’s roughly correct, then it seems like pushing larger blocks of templating and branching into scripts might help reduce Jinja2 overhead in high-frequency scenarios, assuming that scripts are not evaluated unless called from an Automation.

I’d appreciate any guidance or corrections to the model in my head.

Thanks again. I’m already feeling very at home in this community.

[mirekmal (Mirek Malinowski)]

@ferventgeek

I built an Automation that can trigger up to ~1000 times per minute when downstream elements are active. As a concrete example, a dimming hold event can generate 20–40 events/sec while active.

Well, while I can’t comment on your assumptions, I think that philosophy of HA is somehow different from what you might expect. It is build for home automation, when delays of of 1 second do not matter much. For this reason also dimming handling is different in HA. For normal consumer devices HA receives just 2 events: button pressed (but not released) and button released. Button pressed event should trigger action (start dimming light with speed of 5% per second) that should continue to run until button release event is observed (stop dimming). Please note that this is device HW that internally handles appropriate command, not HA, so no need to handle tenths of events per second. Obviously you can find in forum some examples of scripts that are doing this programatically, but this is usually for rather old devices.

[arturpragacz]

Generally speaking anything more frequent than 1/s should be throttled before it even reaches Home Assistant.

But to answer your question: everything is evaluated according to its place in the code, nothing is pre-evaluated.

[ferventgeek (Patrick Hubbard)]

Thanks @mirekmal. I’m wasn’t worried about latency- rather the raw number of times my Automation is triggered and trying to minimize the impact on the HA event queue.

It’s a raw event trigger:

triggers:
  - trigger: event
    event_type: isy994_control

I have a hybrid system with a lot of Insteon and z-wave and for whatever reason the IoX (isy99 now esiy/PG3) device sometimes floods the above trigger rule. Using Insteon and z-wave mini remotes to dim Hue and change Sonos volume is a natural instinct, but has been fraught for years.

However… I just found jsonmx’s handy Philips Hue Smooth Dimmer, which eliminates funky timer dimming scripts and calls Hue’s native start/stop native dimming instead. In HA you can wire the Insteon’s press+hold/release, (fade up/down/stop) events 1:1 directly to hue_dimmer:

hue_dimmer.raise
hue_dimmer.lower
hue_dimmer.stop

Philips Hue Smooth Dimmer then calls Hue’s internal raise/lower/stop transition behavior. That eliminates the need for fractional-second looping shenanigans.

So with that I nuked the dimming programs on the IoX system, uninstalled the eisy’s Hue integration, and moved everything over to HA’s native Hue integration. Now the above HA Automation fires only a few dozen times /min.

So it’s more or less moot and much more responsive.

[ferventgeek (Patrick Hubbard)]

Thanks @arturpragacz. I’m glad to know it’s procedurally evaluated. I’d expect that for the yaml-driven python underneath, but didn’t know if the jinja2 was the same. Thank you for the detail.