Template rebuilds after 117.x needed

[Mariusthvdb (Marius)]

OK, my first and only error in the log so far after updating to beta 117.0b1 caused by the new templating options for native types
please have a look with me how to solve this using the CC variable. I know its not core, but this shouldn’t have ti do with that.

error:

While executing automation automation.update_last_motion
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/automation/__init__.py", line 426, in async_trigger
    await self.action_script.async_run(
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 1010, in async_run
    await asyncio.shield(run.async_run())
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 245, in async_run
    await self._async_step(log_exceptions=False)
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 253, in _async_step
    await getattr(
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 460, in _async_call_service_step
    await service_task
  File "/usr/src/homeassistant/homeassistant/core.py", line 1402, in async_call
    processed_data = handler.schema(service_data)
  File "/usr/local/lib/python3.8/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.8/site-packages/voluptuous/schema_builder.py", line 594, in validate_dict
    return base_validate(path, iteritems(data), out)
  File "/usr/local/lib/python3.8/site-packages/voluptuous/schema_builder.py", line 432, in validate_mapping
    raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: template value should be a string for dictionary value @ data['attributes_template']

and relevant part of the automation:

    action:
      - delay:
          seconds: 2
      - service: variable.set_variable
        data:
          variable: last_motion
          attributes_template: >
            {
             "history_1":"{{states('variable.last_motion')}}",
             "history_2":"{{state_attr('variable.last_motion','history_1')}}",
             "history_3":"{{state_attr('variable.last_motion','history_2')}}",
             "history_4":"{{state_attr('variable.last_motion','history_3')}}",
             "history_5":"{{state_attr('variable.last_motion','history_4')}}",
             "history_6":"{{state_attr('variable.last_motion','history_5')}}",
             "history_7":"{{state_attr('variable.last_motion','history_6')}}",
             "history_8":"{{state_attr('variable.last_motion','history_7')}}",
             "history_9":"{{state_attr('variable.last_motion','history_8')}}",
             "history_10":"{{state_attr('variable.last_motion','history_9')}}"
            }
          value: >
            {{trigger.to_state.name|replace(' sensor motion','')}}:
            {{as_timestamp(trigger.from_state.last_changed)|timestamp_custom('%X')}}

[123 (Taras)]

@Mariusthvdb

I know its not core, but this shouldn’t have ti do with that.

Are sure about that? This custom component is expecting to receive a string for attributes_template and in 0.117 your template is now supplying a dictionary.

Try setting the new legacy_templates option to true and see if that eliminates the error.

[Mariusthvdb (Marius)]

well, not sure, and I have reported it as an issue.

but, have a look at this:

    action:
      service: mqtt.publish
      data:
        topic: location/marijn
        retain: true
        payload: >-
          {
            "latitude": "{{state_attr('zone.home','latitude')}}",
            "longitude": "{{state_attr('zone.home','longitude')}}",
            "battery_level": {{states('sensor.calltheboss_battery_level')|int}},
            "gps_accuracy": 0
          }

which now reports this:

Last logged: 11:17:03 PM

Marijn Presence Update from Router or Bluetooth: Error executing script. Unexpected error for call_service at pos 1: payload must be a string, bytearray, int, float or None.
While executing automation automation.marijn_presence_update_from_router_or_bluetooth

.....
TypeError: payload must be a string, bytearray, int, float or None.

[Mariusthvdb (Marius)]

@123

Try setting the new legacy_templates option to true and see if that eliminates the error.

yes it does and the automation works as expected

so I guess it must be doable then?

[123 (Taras)]

Try this with legacy_templates set to true first and then to false (its default value):

{{'{{"latitude": "{}", "longitude": "{}", "battery_level": {}, "gps_accuracy": 0 }}'.format(state_attr('zone.home','latitude'), state_attr('zone.home','longitude'), states('sensor.calltheboss_battery_level')|int)}}

The set of inner double-braces is not a typo.

[petro]

remove your > in both cases… Not sure how that worked in the first place

[Burningstone]

There’s an issue with json payloads and the new templates see here.

[Mariusthvdb (Marius)]

thanks, Ive added my bits to that issue, if they’re related, I hope this will aid in fixing it.

[Mariusthvdb (Marius)]

I now notice the error is still here:

voluptuous.error.MultipleInvalid: template value should be a string for dictionary value @ data['attributes_template']

but the automation works as before.

not seeing any errors on the MQTT automation in legacy_mode, and behaving normally.

[Mariusthvdb (Marius)]

@123

{{'{{"latitude": "{}", "longitude": "{}", "battery_level": {}, "gps_accuracy": 0 }}'.format(state_attr('zone.home','latitude'), state_attr('zone.home','longitude'), states('sensor.calltheboss_battery_level')|int)}}

unfortunately:

TypeError: payload must be a string, bytearray, int, float or None.

Hope the GitHub issue is causing this, and this is a bug…to be solved anytime soon. meanwhile on 117.0b2 and still happening.

@petro

remove your > in both cases… Not sure how that worked in the first place

yeah, for years now. why would you doubt that? I have more than a few automations with the exact code and they all update the topics correctly.
tried to take the > out, but that doesn’t fix it either unfortunately

or scripts like this:

  set_hue_threshold:
    alias: Set Hue Indoors threshold
    mode: restart
    icon: mdi:cog
    sequence:
      service: rest_command.set_hue_command
      data:
        command: config
        type: sensors
        data: >
          {"tholddark": {{states('input_number.slide_hue_threshold')|int}} }
        id: >
          {% set mapper =
            {'Laundry':'25',
             'Dining table':'53',
             'Auditorium':'45',
             'Frontdoor':'195',
             'Library':'21',
             'Dorm':'57',
             'Front room':'9',
             'Guest room':'35',
             'Attic':'13'} %}
          {% set state = states('input_select.select_hue_motion_sensor') %}
          {% set id = mapper[state] if state in mapper %}
          {{id}}

all working perfectly.

And now see this error:

TypeError: encoding without a string argument

dont think I like this update…
dont really get when we PR a small thing like adding a device_class or so, this gets blown away for being a breaking change, while this HUGE breaking change gets merged without any proper guide how to change the templates we have all been using for ages, and now stop doing what they supposed to do, hence breaking not only a single script or automation, but the reliability of one’s complete homeautomation.

grrr

[Mariusthvdb (Marius)]

now noticed this merged PR https://github.com/home-assistant/frontend/pull/7439

taking care of

Now that not every template result comes back as a string, we print the type of the result and stringify objects.

not really are what that does, because if I enter the mqtt template, it simply returns that with the filled out template values, and no formatting or coding changes to it at all.
It does say:

Result type: string

which is exactly the first type the error says the result should be… it says so for all above reported templates.

should this help us rewrite the templates for the new engine? If so, how?

or, does this proof the issue to be a bug, (because it is returning the correct types).

[petro]

The difference is you’re payloads are formatted dictionaries and when you use the > it turns them into strings…

Those other templates are not formatted dictionaries and have nothing to do with it.

To clarify…

removing the arrow in your last template

        data: {"tholddark": {{states('input_number.slide_hue_threshold')|int}} }

or

        data:
          tholddark: >
            {{states('input_number.slide_hue_threshold')|int}}

or

        data:
          tholddark: "{{states('input_number.slide_hue_threshold')|int}}"

[Mariusthvdb (Marius)]

@petro

        data:
          tholddark: >
            {{states('input_number.slide_hue_threshold')|int}}

yes, thanks!

and this (current) template:

      - service: rest_command.set_hue_rule
        data:
          type: rules
          id: 1
          data: >
            {% set schedule = states('sensor.master_bedroom_schedule') %}
            {% set alarm = states('sensor.alarmclock_wd_time') %}
            {"conditions":[{"address":"/config/localtime","operator":"in","value":"T{{alarm}}:00/{{schedule.split('/')[1]}}"},
                           {"address":"/sensors/48/state/presence","operator":"eq","value":"true"},
                           {"address":"/sensors/48/state/presence","operator":"dx"},
                           {"address":"/sensors/83/state/status","operator":"lt","value":"1"},
                           {"address":"/sensors/49/state/dark","operator":"eq","value":"true"}] }

would then translate to:

      - service: rest_command.set_hue_rule
        data:
          type: rules
          id: 1
          data:
            conditions: >
              {% set schedule = states('sensor.master_bedroom_schedule') %}
              {% set alarm = states('sensor.alarmclock_wd_time') %}
              [{"address":"/config/localtime","operator":"in","value":"T{{alarm}}:00/{{schedule.split('/')[1]}}"},
                           {"address":"/sensors/48/state/presence","operator":"eq","value":"true"},
                           {"address":"/sensors/48/state/presence","operator":"dx"},
                           {"address":"/sensors/83/state/status","operator":"lt","value":"1"},
                           {"address":"/sensors/49/state/dark","operator":"eq","value":"true"}]

? Struggling a bit with the variables that are set in the beginning of the template.

this doesnt work:

    sequence:
      service: rest_command.set_hue_command
      data:
        command: config
        type: sensors
        data:
          sensitivity: >
            {% set mapper =
              {'Low':'0',
               'Medium':'1',
               'High':'2'} %}
            {% set state = states('input_select.select_hue_sensitivity') %}
            {% set sensitivity = mapper[state] if state in mapper %}
            {{sensitivity}}

coming from:

    sequence:
      service: rest_command.set_hue_command
      data:
        command: config
        type: sensors
        data: >
          {% set mapper =
            {'Low':'0',
             'Medium':'1',
             'High':'2'} %}
          {% set state = states('input_select.select_hue_sensitivity') %}
          {% set sensitivity = mapper[state] if state in mapper %}
          {"sensitivity": {{sensitivity}} }

thing is, I need the value to be the full line including the embracing braces and double quotes, just like it is posted here:

[petro]

Did you try it? the command will be sent with the data structure that comes from yaml.

this in yaml:

data:
  sentitivity: 2

translates to this in code

{'data': {'sensitivity': 2}}

So data already has the shape you want.

[Mariusthvdb (Marius)]

of course I tried it…and yes, I was surprised too.

tried all of the variations. There was no error btw, but, also no effect. Note I do need the double quotes, because at one time I had a slip of the keyboard and changed all my double quotes to singles, and these commands stopped working. (as described in the link above the Clip api debugger)

maybe I should add this data is passed into a script:

  set_hue_command: #not named sensitivy because the command can be used to set other entities/attributes also
    url: !secret url_set_hue_command
    method: put
    payload: >
      {{data}}

in the end filling in everything in:

url_set_hue_command: http://192.168.1.212/api/dredactedq/{{ type }}/{{ id }}/{{ command }}

could that be the issue here?

[petro]

then wait for the bug fix to arrive

[Mariusthvdb (Marius)]

so here I am on 117.0b5 meanwhile, with all ‘fixes’ in place, and none of the posted templates above work in the new templating engine. either the original or the suggestions people have made.

what’s even more astonishing is that my ‘old’ rgb template in this thread still works (in the new template engine), while the modernized version doesn’t…

because of this I am forced to use legacy mode unfortunately, because I need all of these to function properly.

please if you have 117.0b5 installed, could you have another look? thanks!

rest_command and MQTT payload are still not working, below is a separate post in the now fixes CC Variable

[Mariusthvdb (Marius)]

separate post for this:
Frenck suggested an solved the issue for the CC Variable here and use it like this.

for reference on this community forum I’ll repost the conclusion:

• using Ha pre 117: don’t change the CC and use attributes_template: in the service-call use legacy_templates: true

• using HA 117.+: change the CC as Frenck suggests here and change to use attributes: in the service-call

      - service: variable.set_variable
        data:
          variable: last_motion
          attributes:
            history_1: "{{ states('variable.last_motion') }}"
            history_2: "{{ state_attr('variable.last_motion','history_1') }}"
            history_9: "{{state_attr('variable.last_motion','history_8')}}"
            history_10: "{{state_attr('variable.last_motion','history_9')}}"
          value: >
            {{trigger.to_state.name|replace(' sensor motion','')}}:
            {{as_timestamp(trigger.from_state.last_changed)|timestamp_custom('%X')}}

[123 (Taras)]

I have to admit that, at the moment, I am unclear as to how this new enhancement works. My preconceived notions of its behavior were completely wrong, primarily that you cannot influence the result’s type through casting (for example, a final string filter doesn’t guarantee the result will be considered to be a string).

I look forward to experimenting with tomorrow’s release version to gain a better understanding of how it works.

[Mariusthvdb (Marius)]

on the rest_command posted above, Ive edited the second data: field to message_body: to prevent confusion, and to have it relate to the CLIP managers language exactly:

rest_command:

  set_hue_command:
    url: !secret url_set_hue_command
    method: put
    payload: >
      {{message_body}}

and script:

script:

  set_hue_sensitivity:
    alias: Set Hue Indoors sensitivity
    mode: restart
    icon: mdi:cog
    sequence:
      service: rest_command.set_hue_command
      data:
        command: config
        type: sensors
        message_body: >
          {% set mapper =
            {'Low':'0',
             'Medium':'1',
             'High':'2'} %}
          {% set state = states('input_select.select_hue_sensitivity') %}
          {% set sensitivity = mapper[state] if state in mapper %}
          {"sensitivity": {{sensitivity}} }
        id: >
          {% set mapper =
            {'Laundry':'22',
             'Dining table':'52',
             'Auditorium':'44',
             'Frontdoor':'194',
             ......
             'Guest room':'31',
             'Attic':'12'} %}
          {% set state = states('input_select.select_hue_motion_sensor') %}
          {% set id = mapper[state] if state in mapper %}
          {{id}}

just to be sure there’s no confusion on the field names and their respective functionality. Above is legacy-style, working up to 117. Need to transform this but am out of ideas…