Why the heck do automations require Jinja templates instead of real Python?

Archive Month of "What the heck?!" (2020)
#3

I’m not sure where you got that information but you don’t have to write it like that.

this should work fine:

value_template: >
  {%- macro GetDroppedZigbee() -%}
    {% for state in states.sensor -%}
      {%- if state.attributes.linkquality %}
        {%- if "Ikea" not in state.name and "linkquality" in state.name and (state.attributes.linkquality < 15 or (as_timestamp(now()) - as_timestamp(state.last_updated) > (12 * 60 * 60) )) -%}
          X 
        {%- endif -%}
      {%- endif -%}
    {%- endfor %}
  {%- endmacro -%}
  {%- if GetDroppedZigbee()[0] =="X" -%}
    true
  {%- else -%}
    false
  {%- endif -%}

Of course that’s assuming your templates are correct in the first place. I didn’t check that.

3 Likes
#4

Agreed, but it could be written much more cleanly in pure Python (e.g. no “{%” required), which is undoubtedly more familiar to more people.

#5

The templates are python
It’s a limited subset but still quite extensive as some of our leading lights prove on a daily basis.
The python is wrapped in jinja2 which is just print formatting
This allows the template to be printed the the python sandbox, this allows some of the more dangerous commands to be either scope limited or just forbidden (ie safe)
What you have seen from Tediore and finity are examples of how it should be done.
Neat, consise and legible.
Finity translated yours.
From a quick look at it, it doesn’t make much sense to me.
A template just resolves one instance and yours seems to be aiming at two.
Like Tediore I haven’t spent much time trying to absorb it’s content.
But the original is a bit of a train wreck, where did you get that from ?

1 Like
#6

The only thing I really miss from the jinja templates are some kind of “global” functions or macros.
Since they are so tightly scoped, quite often you need to repeat a lot of code for different scripts or automations, or even inside an automation or a script because you need to do some data manipulation both in the trigger, the condition and one or more of the actions.

#7

There is always AppDaemon for python lovers :slight_smile:

8 Likes
#8

+1 for AppDaemon and there’s also PyScript now.

5 Likes
#9

Sometimes I wish there was a vote against button. Jinja and templating is perfect the way it is. Not only would this be a huge breaking change, it would be for little to no benefit. People already have issues trying to return values properly with Jinja. Imagine forcing them to use return. Not to mention the casting is way worse. It’s not a simple filter that has a default. It’s a try catch. And that’s just the tip of the iceberg. BLEH no thank you.

11 Likes
#10

Sure they’re based on Python, but something like this is far earlier to understand:

def GetDroppedZigbee():
    for state in states.sensor:
        if state.attributes.linkquality:
            if "Ikea" not in state.name and "linkquality" in state.name and (state.attributes.linkquality < 15 or (as_timestamp(now()) - as_timestamp(state.last_updated) > (12 * 60 * 60) )):
            # do something

if GetDroppedZigbee()[0] == "X":
    value = True
else:
    value = False
4 Likes
#11

Jinja is not user friendly when you want to do anything more substantial than a one-liner. Given the importance of templates, it’s a shame they can’t be made more intuitive.

But why can’t there be two modes of evaluating a template? One that uses Jinja and another that uses a restricted version of Python?

#12

That’s easier for you and other people familiar with programming but not for the average joe. And you have the ability to do this with AppDaemon and Pyscript.

4 Likes
#13

While having to ways does make it more accessible to each users requirement it does add to the complexity of maintaining the system as a whole

#14

I’m suggesting (mostly) a subset of Jinja (i.e. the Python part), so it should be easier for the average Joe.

#15

Have you considered that complex templates are probably an edge case and not the majority of templates? Yes we see lots of complex templates on the forum here and in our own home automation I’m sure most of us spend a good chunk of our time wading through complex templates in the template editor. But I think if you added up the total number of templates used and compared the number that are ‘complex’ to the number that look like this:

{{ trigger.to_entity.attributes.friendly_name }} is home!

or maybe throw in a simple control construct once in a while like this:

{% if is_state('person.a', 'home') %}
  do stuff for home
{% else %}
   do stuff for not home
{% endif %}

I think you’d probably find most templates stay below that level of complexity. I am absolutely speculating here although I can confirm that my own usage of templates in my own HA matches this. Devs hopefully have more insight into this then I do if one wants to weigh in.

So I think if you want to make your case you should think about the impact of your idea in use cases like this. Templating seems to work pretty well in these simple use cases and I feel like introducing python here would definitely make these more complex. Is it worth adding complexity in the simple use case in order to slightly reduce complexity in some of the more complicated ones? I’m doubtful personally but you’re welcome to make the case.

For me personally, I really just wish there was an easy way to create custom filters for use in my own HA instance. If I could do that I think that would be more then enough for me. Then I could simplify my complex ones by defining filters for common behaviors and leave the simple ones as is.

2 Likes
#16

TBH how did you measure that? IMO jinja is as hard as programming languages to average joe. Or maybe even harder because of language construct and multiplicity of appearance of particular characters which makes the code even more difficult to read.

IMO the problem is, that when basic usage of templates requires less effort than programming, very quickly it turned into scripting method (see all feature request). It made the templating insanely unfriendly readability-wise

1 Like
#17

I agree that example is difficult to read but, frankly, it’s like someone was purposely trying to win an obfuscation competition.

Most of that mess appears to be due to how the Automation Editor stores YAML. Beyond that, there are some design choices the author made that seem to unnecessarily complicate the template. For example, why make a macro if the template only calls it once (and so case-specific that it’s not likely to be re-usable elsewhere)?

I believe this does the exact same thing and is considerably more legible:

value_template: >
  {% set ns = namespace(exists = false) %}
  {% for s in states.sensor | selectattr('attributes.linkquality', 'defined') %}
    {% if "Ikea" not in s.name and "linkquality" in s.name and
         (s.attributes.linkquality < 15 or
           now().timestamp() - s.last_updated.timestamp() > 12*60*60) -%}
      {% set ns.exists = true %}
    {% endif %}
  {% endfor %}
  {{ ns.exists }}

Personally, I would have preferred if automations employed python from the very start but with simplifications. To understand what I mean, compare how you call a service using python_script to the recently introduced pyscript.

python_script:

hass.services.call('light', 'turn_on', {'entity_id': 'light.kitchen', 'color_name': 'blue', 'transition': 2, 'brightness': 125}, False)

pyscript

light.turn_on(entity_id='light.kitchen', color_name='blue', transition=2, brightness=125)

However, I believe it’s far too late to re-engineer how existing automations work. There are alternatives available: appdaemon, python_script, and pyscript.

3 Likes
#18

I didn’t measure anything, but I helped like hundreds of people here on the forum with SIMPLE template errors like missing a comma, incorrect use of brackets, wrong type of quotes used etc. and then I should expect these people to write it in Python and explain them what a method is, how to call a method, not mix tabs with spaces and maybe 100’s of other nuances? No, thanks…
IMHO now it’s too late to introduce Python as the native automation platform as people are used to Jinja for years now. Also as mentioned multiple times, for advanced users with lots of complex templates there’s always the possibility to use AppDaemon, python scripts or pyscript to get the python experience.

4 Likes
#19

Yep, I’ve been here ~4 years and this hits the nail on the head. In fact, I still see the same people making those same mistakes. Then out pops a python error and it’s deer in headlights. Just imagine this python errors everywhere…

2 Likes
#20

Isn’t that more the case for Jinja than pure Python (e.g. the brackets and %s)? Not saying Python is the best solution, but it seems a step up from Jinja.

Also there are lots of resources for Python, so people wouldn’t have to come here if it was a purely syntactical issue.

#21

I have also noticed that a lot of people love to make stuff overly complicated. Trying to cram as many conditions and corner cases into a single script or automation, instead of just spltting stuff up in smaller, more readable and more easily maintainable pieces.

2 Likes
#22

Or create a template sensor for intermediate storage instead of overly complex computations in jinja.