Variables and Templates, bug or expected anomaly?

Why do these two scripts give different outputs?

sequence:
  - variables:
      tab: >
        {{ '\n' }}{{ '{:>5}'.format('- ') }}

  - service: notify.my_file
    data:
      message: >
        Text Line 1
        {{ tab }} Text Line 2

Writes (ignoring the newline and formatting):

Text Line 1 -Text Line 2

Whereas, without using the variable and hard coding the newline and formatting

sequence:
  - service: notify.my_file
    data:
      message: >
        Text Line 1
        {{ '\n' }}{{ '{:>5}'.format('- ') }} Text Line 2

Writes:

Text Line 1
     - Text Line 2

For message text I use two newlines like this:

  - service: notify.my_file
    data:
      message: >
        Text Line 1

        {{ tab }} Text Line 2

You could also do:

  - service: notify.my_file
    data:
      message: |
        Text Line 1
        {{ tab }} Text Line 2

It’s about YAML whitespace control. Check out the demos here:

1 Like

Thanks, that link is very useful!

(I’ve been working under the misapprehension for about 6 or 7 years that > and | were equivalent and interchangeable :person_facepalming:: :person_shrugging: )

Ok, I’ve played around with this but I still can’t get the service call to respect the variables defined in the script.

The new line issue seems to have clouded things (because of the | vs. >).
Just using the tab formatting will not work.

sequence:
  - variables:
      tab: >
        {{ '{:>5}'.format('- ') }}

  - service: notify.my_file
    data:
      message: |
        Text Line 1
        {{ tab }} Text Line 2

Writes this:

Text Line 1
- Text Line 2

Not this which is what I am after

Text Line 1
     - Text Line 2

As I said before if I include the tab formatting in the message itself rather than as a variable it works.

Try this

sequence:
  - variables:
      tab: >
        {{ '\t-' }}
  - service: notify.my_file
    data:
      message: |
        Text Line 1
        {{ tab }} Text Line 2

Nope, same thing. The variables don’t ‘carry across’ properly.

Here is a skeleton script (generic apart from the notify service name)

script:
  
  test_file_write:
    sequence:

      - variables:
          event_time: >
            {{ as_timestamp(now()) | timestamp_custom('%X') }}
          default_message: >
            {{ event_time }} - THIS IS A TEST MESSAGE
          tab1: >
            {{ '\t-' }}
          tab2: >
            {{ '{:>18}'.format('- ') }}
          newline: >
            {{ '\n' }}

      - service: notify.log_system_event
        data:
          message: |
            {{ default_message }}
            {{ newline }} Preceded with newline variable (There should be a blank line above this one)
            {{ '\n' }} Prefixed with hard coded \n (There should be a blank line above this one)
            {{ tab1 }} Preceded with tab1 variable
            {{ '\t-' }} Preceded with hard coded \t-
            {{ tab2 }} Preceded with tab2 variable (using format)
            {{ '{:>18}'.format('- ') }} Preceded with hard coded tab using format

And here is the output

11:26:44 - THIS IS A TEST MESSAGE
 Preceded with newline variable (There should be a blank line above this one)

 Prefixed with hard coded \n (There should be a blank line above this one)
- Preceded with tab1 variable
	- Preceded with hard coded \t-
- Preceded with tab2 variable (using format)
                -  Preceded with hard coded tab using format

It appears that variables will not work :person_shrugging:

This is a variable, after the variable is executed, it’s resolved in HA. This leads it to have all leading and trailing whitespace removed, and now the value can be accessed as a variable in the remainder of the automation.

This is a message, the resolution is not performed here because all outputs will be text.


To get around this, make the output of tab complex (a list or dictionary). The resolver does not recursively traverse complex objects, so the whitespace won’t be stripped.

sequence:
  - variables:
      tab: >
        {{ {'text': '\n{:>5}'.format('- ') } }}
  - service: notify.my_file
    data:
      message: >
        Text Line 1
        {{ tab.text }} Text Line 2
1 Like

No, there should be a carriage return. Not a blank line. A blank line would be 2 carriage returns. The variable is stripping your whitespace.

Correct, because the message is retaining the carriage return inside the template.

Same as above


script:
  
  test_file_write:
    sequence:

      - variables:
          event_time: >
            {{ as_timestamp(now()) | timestamp_custom('%X') }}
          default_message: >
            {{ event_time }} - THIS IS A TEST MESSAGE
          tests: >
            {{ {
              'tab1': '\t-',
              'tab2': '{:>18}'.format('- '),
              'newline': '\n'
            } }}

      - service: notify.log_system_event
        data:
          message: |
            {{ default_message }}
            {{ test.newline }} Preceded with newline variable (There should be a blank line above this one)
            {{ '\n' }} Prefixed with hard coded \n (There should be a blank line above this one)
            {{ test.tab1 }} Preceded with tab1 variable
            {{ '\t-' }} Preceded with hard coded \t-
            {{ test.tab2 }} Preceded with tab2 variable (using format)
            {{ '{:>18}'.format('- ') }} Preceded with hard coded tab using format

Thanks for all of that. very helpful as usual.
And I think I have it all working now.
But more importantly I understand why!

A quick side question:
Is there any certainty that variables will be defined and assigned in the order they are declared?
Simply put, can one variable reliably reference another one defined higher up in the same variable block?

There is no certainty… however it’s an ordered dictionary and I personally treat it like it has 100% certainty. I’d suggest you do the same. Ordered dicts are sorted based on the order in which an item was added. If HA changes that base class (unlikely), then you’d run into issues.

FYI you can look at my configuration on github, you’ll notice I heavily rely on this assumption in automations and scripts.

1 Like

Ok…

So is it possible to have an input_text containing a message line that is then used in various scripts?
The catch is that I would like that message line to include variable names which get resolved in the script.

For example:

input_text:
  message_line:

message_line contains a string, for example,

"The time is {{ now() }}"

Is there any way to use this in a script and have it resolve the contents of the input_text?

For example as an illustration, this, which doesn’t work:

script:
  persistent_notify:
    sequence:
      - variables:
          message_as_list: >
            {% set message_line = states('input_text.message_line') %}
            {{
              {
                'message': message_line 
              }
            }}


      - service: notify.persistent_notification
        data:
          message: >
            {{ message_as_list.message }}

Which would (ideally) produce a persistent notification with a message of:

The time is 2024-02-08 09:10:25.364207+00:00

That will work. What’s “Not working”? Are you nesting the variable in a if step?

Ok in an attempt to simplify I seem to have simplified to the point of it working.

What doesn’t work is using any kind of formatting in the input_text.
So if I assign the string \n to the input_text it doesn’t give a newline.

Is that possible?

I hope that makes sense?

(I also tried the string '\n')

What are you trying to do with the input_text? Why do you need it? Pretty sure they strip whitespace via the service call. You may be able to retain it doing…

      - service: notify.persistent_notification
        data: "{{ message_as_list }}"

or

      - variables:
          message_as_list: >
            {% set message_line = states('input_text.message_line') %}
            {{
              {
                'value': message_line 
              }
            }}
      - service: input_text.set_value
        data: "{{ message_as_list }}"

No that doesn’t do what I’d like, it’s pretty much what I tried.

Since you asked, I have a system of logging ‘events’. Every one of my ‘subsystems’ e.g. heating, lights, irrigation (optionally) write to their own log. Every log is formatted in the same way so rather than repeating the variables section in ever script that handles logging I thought I could use input texts.

It has the added benefit of being able to change the formatting easily.

So,

For the sake of getting to bottom of whether what I want to do is actually possible, here is a complete simple example.

input_text:
  test_file_line:
    max: 255

  test_file_new_line:
    max: 255

  test_file_tab:
    max: 255

script:
  
  test_file_write:
    sequence:

      - variables:
          message_as_list: >
            {% set message = states('input_text.test_file_line') %}
            {% set newline = states('input_text.test_file_new_line') %}
            {% set tab = states('input_text.test_file_tab') %}
            {{
              {
                'message': message,
                'newline': newline,
                'tab'    : tab
              }
            }}

      - service: notify.log_system_event
        data:
          message: >
                    {{ defaults.message }}
                    {{- defaults.newline }}{{ defaults.tab }}Line 2
                    {{- defaults.newline }}{{ defaults.tab }}Line 3

If I set the input texts as follows:
        test_file_line: {{ now() }} - Somthing here - and something here
test_file_new_line: \n
          test_file_tab: '{:>27}'.format('- ')

I get the following one line written to the file:

2024-02-09 14:28:28.554068+00:00 - Something here - and something here\n'{:>27}'.format('- ')Line 2\n'{:>27}'.format('- ')Line 3

Whereas I would like 3 separate lines:

2024-02-09 14:28:28.554068+00:00 - Something here - and something here
                                                                  - Line 2
                                                                  - Line 3


(The input text for newline and tab arose simply to stop having to remember how to format those things. For example I have trouble remembering where it is \n and where it is <br> and the syntax for format is not intuitive to me).

Ok, in my opinion you’re going about this wrong. But if you want to use the input_texts, you can create new lines by just repeating the service call.

script:
  
  test_file_write:
    variables:
      items:
      - "{{ states('input_text.test_file_line') }}"
      - "{{ states('input_text.test_file_new_line') }}"
      - "{{ states('input_text.test_file_tab') }}"
    sequence:
    - repeat:
        for_each: "{{ items }}"
        sequence:
        - service: notify.log_system_event
          data:
            message: "{{ repeat.item }}"

That gives me almost the same as I was getting before. The only difference being that each line is indeed a new line.

All of the formatting is still being treated literally:

2024-02-09 18:41:18.983237+00:00 - Something here - something here - and something here
\n
'{:>27}'.format('- ')

But in any case I can’t let this go withouit reponse…

I can honestly think of no situation you would say that and I wouldn’t be interested in how you think I should be doing it.

If you have the time and inclination I’d be interested to hear.


To recap, I want to be able to write the same lines which contain formatting and variable names to many different files from many different scripts.

The reason to use input texts was simply to avoid re-defining the lines in every script and to be able to change the formatting and variables in one place and without having to reload all the scripts

Here is an actual real world example output that I have been using in my testing:

23:57:53.457035 LOW    - DOOR_OR_WINDOW_EVENT                (Initiated By automation.access_monitoring_door_or_window_event - Triggered By binary_sensor.front_door)
                         Front Door CLOSED
                         --------------------

Which is created every time a door or window is opened or closed by this service call:

      #=== Write To Log
      - service: script.turn_on
        entity_id: script.access_monitoring_write_to_log
        data:
          variables:
            log_event:  DOOR_OR_WINDOW_EVENT
            event_time: >
              {{ trigger.to_state.last_changed }}
            mode: >
              {{ trigger.to_state.state }}
            triggered_by: >
              {{ trigger.entity_id }}
            initiated_by: >
              {{ this.entity_id }}

Followed by:

script:
  
  #===================================
  #=== Access Monitoring Write To Log
  #===================================
  access_monitoring_write_to_log:
    alias: Access Monitoring Write To Log
    sequence:

      - variables:
          #=== Event Time
          event_time: >
            {{ as_timestamp(event_time | default(now())) | timestamp_custom('%X.%f') }}

          #=== Defaults
          defaults: >
            {{
              {
                'message': event_time ~ '{:<7}'.format(' ' ~ this_log_message_severity | upper ) ~ ' - ' ~ '{:<35}'.format(log_event) ~ ' (Initiated By ' ~ initiated_by ~ ' - Triggered By ' ~ triggered_by ~ ')',
                'newline': '\n',
                'tab'    : '{:>25}'.format(' ')
              }
            }}

          #=== Extra Message Lines
          extra_message_lines: >
            {% if log_event == 'DOOR_OR_WINDOW_EVENT' %}
              {{ defaults.newline }}{{ defaults.tab }}{{ triggered_by.split('.')[1].replace('_', ' ') | title }} {{ {'on': 'open', 'off': 'closed'}[mode] | upper }}
            {% endif %}

      #=== Write To The Log
      - service: notify.log_access_monitoring
        data:
          message: >
            {{ defaults.message }}
            {%- if extra_message_lines is defined %}
              {{- defaults.newline }}{{ defaults.tab }}{{ extra_message_lines }}
            {%- endif %}
            {{- defaults.newline }}{{ defaults.tab }}--------------------

I’ll be a bit before I can read & reply fyi. Going on a trip, but I’m not ghosting you.

Edit: this doesn’t seem to use input texts, so it seems you are doing what I’d do.

Edit: had a chance to look. You’re being limited by the notification service. You may need to swap to a command line command that writes to a file instead.

Thanks, no problem.

No, the example I posted in that reply was just to show how I have it working. I’d like to move to input texts.

For some reason I hadn’t thought of a command line.
Yes, I’ll look into that

(Have a good trip!)

I’ve tried a command line which I use successfully elsewhere to delete the first line or all lines in a file

shell_command:
  #=== Delete all lines from a log file
  logging_log_file_delete_all_lines: >
    truncate -s 0 /config/my_log_{{ log_name }}.log

  #=== Delete the first line of a log file
  logging_log_file_delete_first_line: >
    sed -i '1d' /config/my_log_{{ log_name }}.log

But I can’t the command to work for adding a line to the end of a file. None of these seem to work

  #=== Write a line to a log file using printf
  logging_log_file_write_a_line_printf: >
    printf "{{ line }} printf" >> /config/my_log_{{ log_name }}.log

  #=== Write a line to a log file using echo
  logging_log_file_write_a_line_echo: >
    echo {{ line }} echo >> /config/my_log_{{ log_name }}.log


  #=== BASIC Write a line to a log file using printf
  logging_log_file_write_a_line_printf_basic: >
    printf "testline printf" >> /config/my_log_system_event.log

  #=== BASIC Write a line to a log file using echo
  logging_log_file_write_a_line_echo_basic: >
    echo testline echo >> /config/my_log_system_event.log

Here are the responses to the two ‘basic’ commands:
image

image