Is there any way to prevent template warnings / errors on startup for yet non-existing state attributes?

[derandiunddasbo (Andreas Börner)]

Some integrations always tend to be a bit slow with setting up during startup leading to templates being evaluated before entities used in that templates provide all their attributes.

In my case it’s mostly the knx integration that isn’t able to gather all states on startup before HA wants to evaluate some templates using cover positions for template binary sensors. Depending on how the sensors are defined, I get errors or warnings on startup.

For example:

binary_sensor:
 - platform: template
    sensors:
      eg_entertainment_kinomodus:
        friendly_name: EG Entertainment Kinomodus
        value_template: "{{ not is_state('sensor.beamer_status','standby') and state_attr('cover.eg_wohnen','current_position') < 100 }}"

leads to an error like this on most startups:

2021-11-16 16:50:01 ERROR (MainThread) [homeassistant.helpers.event] Error while processing template: Template("{{ not is_state('sensor.beamer_status','standby') and state_attr('cover.eg_wohnen','current_position')|default(100) < 100 }}")
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 398, in async_render
    render_result = _render_with_context(self.template, compiled, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 1698, in _render_with_context
    return template.render(**kwargs)
  File "/usr/local/lib/python3.9/site-packages/jinja2/environment.py", line 1304, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.9/site-packages/jinja2/environment.py", line 925, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
TypeError: '<' not supported between instances of 'NoneType' and 'int'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 514, in async_render_to_info
    render_info._result = self.async_render(variables, strict=strict, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 400, in async_render
    raise TemplateError(err) from err
homeassistant.exceptions.TemplateError: TypeError: '<' not supported between instances of 'NoneType' and 'int'
2021-11-16 16:50:01 ERROR (MainThread) [homeassistant.components.template.template_entity] TemplateError('TypeError: '<' not supported between instances of 'NoneType' and 'int'') while processing template 'Template("{{ not is_state('sensor.beamer_status','standby') and state_attr('cover.eg_wohnen','current_position')|default(100) < 100 }}")' for attribute '_state' in entity 'binary_sensor.eg_entertainment_kinomodus'

I also tried using a default filter for state_attr(), but that doesn’t prevent any errors:

binary_sensor:
 - platform: template
    sensors:
      eg_entertainment_kinomodus:
        friendly_name: EG Entertainment Kinomodus
        value_template: "{{ not is_state('sensor.beamer_status','standby') and state_attr('cover.eg_wohnen','current_position')|default(100) < 100 }}"

Defining the binary sensor as a trigger based template instead doesn’t help much either:

template:
  - trigger:
      - platform: state
        entity_id: sensor.beamer_status
      - platform: numeric_state
        entity_id: cover.eg_wohnen
        attribute: current_position
        below: 100
    binary_sensor:
      - name: EG Entertainment Kinomodus
        state: "{{ not is_state('sensor.beamer_status','standby') and state_attr('cover.eg_wohnen','current_position')|default(100) < 100 }}"

This just “replaces” the error with a (not so scary looking… ) warning:

2021-11-16 17:11:41 WARNING (MainThread) [homeassistant.components.homeassistant.triggers.numeric_state] Error initializing 'Trigger Update Coordinator' trigger: In 'numeric_state' condition: attribute 'current_position' (of entity cover.eg_wohnen) does not exist

I really can’t blame the author of the knx integration for this behavior, as the integration is gathering states of all knx entities as fast as possible, i.e. flooding the knx bus with read requests on startup. But for the amount of knx entities in my house it’s unavoidable that this takes some time.

Is there any way to prevent these warnings?

[Mikefila]

Updating Templates with the new default values in 2021.10.x

I keep seeing the same question about the new default argument for the templating. This post will cover all possible ways to define default. All the methods that were changed: acos as_timestamp asin atan atan2 cos float log round sin sqrt strptime tan timestamp_custom timestamp_local timestamp_utc Each of them have the exact same default functionality but a different number of arguments. There are multiple ways to use these functions, and this is where all the confusion lies. Understandi…

[farmio (Matthias Alphart)]

We are thinking about not adding / setting the entities unavailable before the first run of the state updater. But don’t expect this any time soon, could be a quite big change.

In the meantime you can try to just check if the attribute exists / is None in your template before comparing it with <, but I don’t know how to do this in jinja.

[123 (Taras)]

@derandiunddasbo

any way to prevent these warnings?

Add the availability option in your Template sensors. If the template within availability evaluates to false, the sensor’s state template is not evaluated.

However that won’t prevent the issue with the Numeric State Trigger and cover.eg_wohnen. Which integration is used for that cover entity?

[farmio (Matthias Alphart)]

Do you have any

[xknx.log] Error: KNX bus did not respond in time (2.0 secs) to GroupValueRead request for:

messages in your logs?

[derandiunddasbo (Andreas Börner)]

Yes, I have one of these warnings in my log, but it’s always the same device and the same group address. That’s an issue with this specific device, which seemingly didn’t take a power outage about a year ago very well and is acting a bit weird since then. One of the issues is, that this device refuses to reply to read requests for a specific GA (the current brightness value for one of it’s four dimming actuators). But I suspect, that this doesn’t affect setting up all other entities:

2021-11-17 11:59:00 WARNING (MainThread) [xknx.log] Error: KNX bus did not respond in time (2.0 secs) to GroupValueRead request for: 3/3/113

[derandiunddasbo (Andreas Börner)]

@123

Which integration is used for that cover entity?

This is a knx cover entity. But as said, I don’t think, there currently isn’t much that can be done from the integration’s side, apart from the long term goal to completely change the way, knx entites are set up during startup as @farmio wrote yesterday.

[derandiunddasbo (Andreas Börner)]

@123

Add the availability option in your Template sensors. If the template within availability evaluates to false, the sensor’s state template is not evaluated.

- platform: template
    sensors:
      eg_entertainment_kinomodus:
        friendly_name: EG Entertainment Kinomodus
        value_template: "{{ not is_state('sensor.beamer_status','standby') and state_attr('cover.eg_wohnen','current_position')|default(100) < 100 }}"
        availability_template: "{{ not state_attr('cover.eg_wohnen','current_position') == None }}"

I added an availability template to the sensor and at least on the last restart there were no errors in the log. But as they don’t appear on every restart, I can’t tell for sure, if they are gone for good.

Edit: Just restarted HA a second time and unfortunately the error re-appeared, even with the availability template set:

2021-11-17 14:09:00 ERROR (MainThread) [homeassistant.helpers.event] Error while processing template: Template("{{ not is_state('sensor.beamer_status','standby') and state_attr('cover.eg_wohnen','current_position')|default(100) < 100 }}")
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 398, in async_render
    render_result = _render_with_context(self.template, compiled, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 1698, in _render_with_context
    return template.render(**kwargs)
  File "/usr/local/lib/python3.9/site-packages/jinja2/environment.py", line 1304, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.9/site-packages/jinja2/environment.py", line 925, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
TypeError: '<' not supported between instances of 'NoneType' and 'int'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 514, in async_render_to_info
    render_info._result = self.async_render(variables, strict=strict, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 400, in async_render
    raise TemplateError(err) from err
homeassistant.exceptions.TemplateError: TypeError: '<' not supported between instances of 'NoneType' and 'int'

Either the availability_template isn’t working as expected, or it doesn’t prevent the value_template from being evaluated anyways.

[farmio (Matthias Alphart)]

@derandiunddasbo

But I suspect, that this doesn’t affect setting up all other entities

It doesn’t if it’s just one. If it were multiple GAs it would slow down initialization of others because only 2 parallel reads are sent - so others would have to wait 2 seconds.

[123 (Taras)]

Try this version:

- platform: template
    sensors:
      eg_entertainment_kinomodus:
        friendly_name: EG Entertainment Kinomodus
        value_template: "{{ states('sensor.beamer_status') != 'standby' and state_attr('cover.eg_wohnen','current_position')|int(100) < 100 }}"
        availability_template: "{{ states('cover.eg_wohnen') not in ['unavailable', 'unknown'] }}"

[derandiunddasbo (Andreas Börner)]

@123

Try this version:

No, that’s raising the same error as before. It’s because of the way, the integration currently adds entities during startup. F.e. a cover entity is initialized without the current_position attribute at first. The attribute and it’s value is only added, after the actuator replied the read request from the state updater. Thus, the entity is available immediately on startup, but the current_position attribute is added some seconds later. And before that, HA already tries to evaluate the template, failing with an error because of the unknown attribute.

Actually, I just tried an overly complicated but virtually bullet proof availability template, but the error is still raised. Which leads me to the conclusion, that using an availability_template doesn’t prevent the value_template from being evaluated at all:

  - platform: template
    sensors:
      eg_entertainment_kinomodus:
        friendly_name: EG Entertainment Kinomodus
        value_template: "{{ not is_state('sensor.beamer_status','standby') and state_attr('cover.eg_wohnen','current_position')|default(100) < 100 }}"
        availability_template: "{{ states.cover.eg_wohnen.attributes|select('match', 'current_position') |list|length == 1 }}"

[123 (Taras)]

@derandiunddasbo

the entity is available immediately on startup, but the current_position attribute is added some seconds later.

Wow, that’s inconvenient.

[123 (Taras)]

@derandiunddasbo

leads me to the conclusion, that using an availability_template doesn’t prevent the value_template from being evaluated at all

FWIW, I copy-pasted your Template Sensor configuration into my test system (version 2021.11.1).

  - platform: template
    sensors:
      eg_entertainment_kinomodus:
        friendly_name: EG Entertainment Kinomodus
        value_template: "{{ not is_state('sensor.beamer_status','standby') and state_attr('cover.eg_wohnen','current_position')|default(100) < 100 }}"
        availability_template: "{{ states.cover.eg_wohnen.attributes|select('match', 'current_position') |list|length == 1 }}"

My test system doesn’t have any of the entities referenced in the template. After restarting Home Assistant, the sensor’s state value is simply unavailable and there are no error or warning messages in the log.

I get the same result after executing Reload Template Entities (no error messages).

[derandiunddasbo (Andreas Börner)]

That’s interesting. I suspect, the error only occurs, if the entities in the value template are existing but the attribute isn’t. Maybe you could verify this by replacing sensor.beamer_status and cover.eg_wohnen with two existing entities (of course the replacement for cover.eg_wohnen mustn’t have a current_position attribute ).

[123 (Taras)]

I have to admit that I made a mistake in my previous post.

I had said I used your version of the Template Sensor:

      eg_entertainment_kinomodus:
        friendly_name: EG Entertainment Kinomodus
        value_template: "{{ not is_state('sensor.beamer_status','standby') and state_attr('cover.eg_wohnen','current_position')|default(100) < 100 }}"
        availability_template: "{{ states.cover.eg_wohnen.attributes|select('match', 'current_position') |list|length == 1 }}"

However, I mistakenly used my version of the Template Sensor:

      eg_entertainment_kinomodus:
        friendly_name: EG Entertainment Kinomodus
        value_template: "{{ states('sensor.beamer_status') != 'standby' and state_attr('cover.eg_wohnen','current_position')|int(100) < 100 }}"
        availability_template: "{{ states('cover.eg_wohnen') not in ['unavailable', 'unknown'] }}"

The result was as described: it produced no error or warning messages and its state value is unavailable.

I just finished testing your version (double-checked; definitely your version) and its state value is unavailable but it does produce error messages. However, the errors are due to a known issue.

Your availability template uses direct referencing to get the entity’s attribute. This method is guaranteed to fail, with an error, if either the entity or the attribute doesn’t exist (in my test case neither the entity or the attribute exist):

states.cover.eg_wohnen.attributes

It’s always best to use the states() and state_attr() functions. For more information, refer to Templating - States.

I suggest you try this version.

      eg_entertainment_kinomodus:
        friendly_name: EG Entertainment Kinomodus
        value_template: "{{ states('sensor.beamer_status') != 'standby' and state_attr('cover.eg_wohnen','current_position')|int(100) < 100 }}"
        availability_template: "{{ state_attr('cover.eg_wohnen','current_position') != none and states('sensor.beamer_status') not in ['unavailable', 'unknown'] }}"

The availability template references the current_position attribute using the state_attr() function. If the attribute does not exist, the function will return none (null object). As long as the function returns something other than none then it means the attribute exists and the sensor is considered to be available. For good measure, I also included a check for the state of sensor.beamer_status.

[123 (Taras)]

BTW, this might be of interest to you.

• Prior to 2021.10.X, if the availability_template evaluated to false, the sensor’s value_template was not evaluated.

• In 2021.10.X, even if the availability_template evaluated to false, the sensor’s value_template is still evaluated … mostly because it was checking for functions that lacked default values and would be reported as warnings (not errors).

The new behavior was reported in this Issue:

Allow specifying a super template for async_track_template_result

In response to that Issue, this PR reverted the behavior to pre-2021.10.X where if the availability_template evaluated to false, the sensor’s value_template should not be evaluated.

Allow specifying a super template for async_track_template_result

PR 58477 was included in 2021.11.X.

There’s a followup PR (to improve startup performance) that will (probably) appear in the next release:

Improve startup of unavailable template entities

[derandiunddasbo (Andreas Börner)]

@123

Your availability template uses direct referencing to get the entity’s attribute. This method is guaranteed to fail, with an error

I’m aware of that problem, but actually deliberately used direct referencing for the last version, because as far as I understand, it would only fail, if the entity doesn’t exist at all. But in this case, another distinct error is raised, directly referencing the availability template:

2021-11-18 16:49:06 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'None' has no attribute 'attributes' when rendering '{{ states.cover.eg_wohnenxxx.attributes|select('match', 'current_position') |list|length == 1 }}'

I think the availability template worked fine, despite using direct references, because the errors always referenced the value template, not the availability template.

@123

In 2021.10.X, even if the availability_template evaluated to false, the sensor’s value_template is still evaluated … mostly because it was checking for functions that lacked default values and would be reported as warnings (not errors).

I think, this was the actual reason for the raised errors. I already suspected that earlier, but thank you for confirming, that there’s an open issue for that.

@123

      eg_entertainment_kinomodus:
        friendly_name: EG Entertainment Kinomodus
        value_template: "{{ states('sensor.beamer_status') != 'standby' and state_attr('cover.eg_wohnen','current_position')|int(100) < 100 }}"
        availability_template: "{{ state_attr('cover.eg_wohnen','current_position') != none and states('sensor.beamer_status') not in ['unavailable', 'unknown'] }}"

This last version of yours indeed didn’t raise any errors on the last three startups. But I’m not sure why. The additional check for sensor.beamer_status doesn’t seem to be relevant, it’s even working without.

But when omitting the second check, the remaining template should technically be equivalent to the one I tried yesterday:

        availability_template: "{{ not state_attr('cover.eg_wohnen','current_position') == none }}"

(Yes, it was an upper case “None” in my yesterday’s post, but I also tried this version with a lower case “none”.)

I can only guess, that the used availability template doesn’t make any difference, because in the current HA version the value template is evaluated regardless of the availability template result and that your change from

state_attr('cover.eg_wohnen','current_position')|default(100) < 100

to

state_attr('cover.eg_wohnen','current_position')|int(100) < 100

is the reason, the errors don’t appear anymore. But I really don’t know, why the int() filter would behave different from the default() filter.

[123 (Taras)]

@derandiunddasbo

deliberately used direct referencing for the last version, because as far as I understand, it would only fail, if the entity doesn’t exist at all

That’s true but, as a test for an availability template, it’s not particularly robust because it makes an assumption the entity is defined. If it’s not defined, the entire template fails with an error (which is not a true/false result but an outright error). Anyway, if you feel comfortable with that risk, the template can be reduced to:

{{ states.cover.eg_wohnen.attributes.current_position is defined }}

@derandiunddasbo

there’s an open issue for that

Actually the Issue is closed. It was addressed by the PR listed above.

@derandiunddasbo

why the int() filter would behave different from the default() filter

Because they serve different purposes.

• The default filter is used to assign a default value if the supplied value is undefined.

• The int filter converts the supplied value to an integer but if cannot it reports a default value.

• int(100) cannot convert none to an integer so it reports its default value (100).

• default(100) was supplied with a value (none) so it had no need to report its default value (100).