I am a computer science student working on my Bachelor’s thesis focused on Home Assistant automation reliability. Specifically, I am looking into cases where an automation is syntactically valid (passes YAML configuration check or is built via the UI editor without errors), yet fails or behaves unexpectedly in real-world operation (e.g., due to restarts/reloads, single vs restart modes, unavailable states, lost delay/for timers, or missing feedback from devices, etc.).
To base my work on real community scenarios rather than theory, I would love to hear about your recent experiences.
Could you share the last automation that behaved differently than you expected?
What was it supposed to do vs. what did it actually do?
How did you notice the issue, and what was the actual root cause? (e.g., trace analysis, logs, trial-and-error)
How did you fix it?
(If you are willing to share a snippet of the original/fixed YAML, please make sure to remove any sensitive data/tokens first!)
As Many people these days uses AI, and copy/paste, it’s questionable whether you could even call it “real community scenarios”
Im sure you could ask Your AI, to create such an “automation/scenario” , and paste it into your Bachelor Thesis ( And still make your Thesis look like, You don’t use AI )
Also for you to be sure It actually is “reliable” examples you have to verify this on your own Install as well as the “Posters” Install ( As they for sure in no way are identical Installs ) … If your Thesis should be “Reliable”
What it was supposed to do: Turn the heating on in the morning according to a schedule, but only if the user was out of bed. Getting up to go to the loo didn’t count. If the user had a lie-in, heating didn’t come on until they got up.
What actually happened: Worked fine at first, but after a few days the heating came on according to the schedule, whether the user was up or not.
The automation: Two triggers - schedule and “user awake” flag. It ran when either turned on, but the heating only started when both were on.
The real problem: Turning the “user awake” flag off. It was turned on by room changes (Bermuda integration, not motion sensors, allowing the user to move around without their phone undetected), but it turned off at a set time. If the user stayed up past that time, it never turned off.
The issue: Bad design.
I would suggest that bad design is by far the most common cause of failure. The issues you mention can cause problems, but in my experience they are so rare they hardly figure in “reliability”.
The real problem with Home Assistant is it’s users.
I had an automation that warned about low battery levels in Bluetooth thermometers, and such. It included a list of devices and a single threshold for all of them.
Recently decided to expand it to include other types of devices (phones, tablets, etc.), while allowing for different thresholds to be set for each. So I added a dictionary:
And a variable to search for devices with a low battery:
variables:
lowBatt: >-
{% macro getLow_macro(item, returns) -%}
{% set n = item.name if item.name is string else item.entity_id %}
{% set batt = item.state|float(-1)|round %}
{% set wrn = override.get(item.entity_id, defaultTh) %}
{% if batt < wrn %}
{% do returns( (n, batt) ) %}
{% endif %}
{% endmacro -%} {% set getLow = getLow_macro | as_function -%}
{{ entities|expand|map('apply',getLow)|reject('none')|list }}
I was very happy with this achievement, until a tablet ran out of charge without any warning I wrote it in such a way that items added only to the dictionary were not taken into account at all. Fix for the lowBatt variable, lines at the very bottom:
{% set fullList = entities+override.items()|map(attribute=0)|list %}
{{ fullList|expand|map('apply',getLow)|reject('none')|list }}
Two from my own setup. A state trigger with to: on and for: 5 minutes never fired: the sensor turned on right around the time I reloaded automations, stayed on for about 10 minutes, and nothing happened until I added a time_pattern trigger that checks the same condition every minute. The other one ran 42 times in October and turned on a SmartThings scene meant to switch my AC off through an IR blaster, but I’d moved that IR blaster to Zigbee2MQTT at the end of August, so as far as I can tell it hadn’t done anything for two months.
"Thanks everyone for the shared automations and incidents! As part of my bachelor thesis research on automation reliability, I’d also love to understand your practical experience:
How did you actually discover that the automation failed? (Did you check traces/logs immediately, or did you only notice when something physically went wrong in your home?)
Which types of automations do you consider truly critical where failure is unacceptable (HVAC, EV charging, security vs. ambient lighting)?
If a diagnostic tool/linter existed for HA, what kind of help would you expect (just warnings, explanation of WHY it failed, or automated fix suggestions)?
How tolerant would you be to false positive warnings?"
For my two, on your first question: the for: 5 minutes one I noticed physically, because I was sitting in the living room on Sep 14 and the light didn’t come on. The SmartThings scene one I only found when I removed the SmartThings hub on Oct 24 and went through what still depended on it. The recorder shows that automation ran 42 times in October.
I wrote it in such a way that items added only to the dictionary were not taken into account at all. Fix for the