Template changes (I don't understand...)

[tescophil (Phil)]

Just upgraded to the latest HA core-2021.10.0 and get hundreds of template warnings like this :

2021-10-06 22:16:45 WARNING (MainThread) [homeassistant.helpers.template] Template warning: 'timestamp_custom' got invalid input '00:00' when compiling template 

'{% if (states.sensor.time.state >= ('00:00' | timestamp_custom('%H:%M', False))) and (states.sensor.time.state < ('00:30' | timestamp_custom('%H:%M', False))) %}

 <Do some things...>

{% endif %}' 

but no default was specified. Currently 'timestamp_custom' will return '00:00', however this template will fail to render in Home Assistant core 2021.12

So, I have no idea what I’m supposed to do here as clearley I don’t understand what’s going on. Why is ‘00:00’ (or any time specified in this format) suddenly an invalid imput to the custom_timestamp filter…?

[tescophil (Phil)]

I’ll answer my own question here…

Don’t know, Don’t care…, handling time in HA has always been a over complex PITA…, lots of confising and contradictory stuff on the boards about this…, so I reverted back to the most basic tests, which actually work:

{% if states.sensor.time.state >= '00:00' and states.sensor.time.state < '00:30' %}0000_to_0030
{% elif states.sensor.time.state >= '00:30' and states.sensor.time.state < '01:00' %}0030_to_0100
{% elif states.sensor.time.state >= '01:00' and states.sensor.time.state < '01:30' %}0100_to_0130
{% elif states.sensor.time.state >= '01:30' and states.sensor.time.state < '02:00' %}0130_to_0200
{% elif states.sensor.time.state >= '02:00' and states.sensor.time.state < '02:30' %}0200_to_0230
{% elif states.sensor.time.state >= '02:30' and states.sensor.time.state < '03:00' %}0230_to_0300
{% elif states.sensor.time.state >= '03:00' and states.sensor.time.state < '03:30' %}0300_to_0330
{% elif states.sensor.time.state >= '03:30' and states.sensor.time.state < '04:00' %}0330_to_0400
{% elif states.sensor.time.state >= '04:00' and states.sensor.time.state < '04:30' %}0400_to_0430
{% elif states.sensor.time.state >= '04:30' and states.sensor.time.state < '05:00' %}0430_to_0500
{% elif states.sensor.time.state >= '05:00' and states.sensor.time.state < '05:30' %}0500_to_0530
{% elif states.sensor.time.state >= '05:30' and states.sensor.time.state < '06:00' %}0530_to_0600
{% elif states.sensor.time.state >= '06:00' and states.sensor.time.state < '06:30' %}0600_to_0630
{% elif states.sensor.time.state >= '06:30' and states.sensor.time.state < '07:00' %}0630_to_0700
{% elif states.sensor.time.state >= '07:00' and states.sensor.time.state < '07:30' %}0700_to_0730
{% elif states.sensor.time.state >= '07:30' and states.sensor.time.state < '08:00' %}0730_to_0800
{% elif states.sensor.time.state >= '08:00' and states.sensor.time.state < '08:30' %}0800_to_0830
{% elif states.sensor.time.state >= '08:30' and states.sensor.time.state < '09:00' %}0830_to_0900
{% elif states.sensor.time.state >= '09:00' and states.sensor.time.state < '09:30' %}0900_to_0930
{% elif states.sensor.time.state >= '09:30' and states.sensor.time.state < '10:00' %}0930_to_1000
{% elif states.sensor.time.state >= '10:00' and states.sensor.time.state < '10:30' %}1000_to_1030
{% elif states.sensor.time.state >= '10:30' and states.sensor.time.state < '11:00' %}1030_to_1100
{% elif states.sensor.time.state >= '11:00' and states.sensor.time.state < '11:30' %}1100_to_1130
{% elif states.sensor.time.state >= '11:30' and states.sensor.time.state < '12:00' %}1130_to_1200
{% elif states.sensor.time.state >= '12:00' and states.sensor.time.state < '12:30' %}1200_to_1230
{% elif states.sensor.time.state >= '12:30' and states.sensor.time.state < '13:00' %}1230_to_1300
{% elif states.sensor.time.state >= '13:00' and states.sensor.time.state < '13:30' %}1300_to_1330
{% elif states.sensor.time.state >= '13:30' and states.sensor.time.state < '14:00' %}1330_to_1400
{% elif states.sensor.time.state >= '14:00' and states.sensor.time.state < '14:30' %}1400_to_1430
{% elif states.sensor.time.state >= '14:30' and states.sensor.time.state < '15:00' %}1430_to_1500
{% elif states.sensor.time.state >= '15:00' and states.sensor.time.state < '15:30' %}1500_to_1530
{% elif states.sensor.time.state >= '15:30' and states.sensor.time.state < '16:00' %}1530_to_1600
{% elif states.sensor.time.state >= '16:00' and states.sensor.time.state < '16:30' %}1600_to_1630
{% elif states.sensor.time.state >= '16:30' and states.sensor.time.state < '17:00' %}1630_to_1700
{% elif states.sensor.time.state >= '17:00' and states.sensor.time.state < '17:30' %}1700_to_1730
{% elif states.sensor.time.state >= '17:30' and states.sensor.time.state < '18:00' %}1730_to_1800
{% elif states.sensor.time.state >= '18:00' and states.sensor.time.state < '18:30' %}1800_to_1830
{% elif states.sensor.time.state >= '18:30' and states.sensor.time.state < '19:00' %}1830_to_1900
{% elif states.sensor.time.state >= '19:00' and states.sensor.time.state < '19:30' %}1900_to_1930
{% elif states.sensor.time.state >= '19:30' and states.sensor.time.state < '20:00' %}1930_to_2000
{% elif states.sensor.time.state >= '20:00' and states.sensor.time.state < '20:30' %}2000_to_2030
{% elif states.sensor.time.state >= '20:30' and states.sensor.time.state < '21:00' %}2030_to_2100
{% elif states.sensor.time.state >= '21:00' and states.sensor.time.state < '21:30' %}2100_to_2130
{% elif states.sensor.time.state >= '21:30' and states.sensor.time.state < '22:00' %}2130_to_2200
{% elif states.sensor.time.state >= '22:00' and states.sensor.time.state < '22:30' %}2200_to_2230
{% elif states.sensor.time.state >= '22:30' and states.sensor.time.state < '23:00' %}2230_to_2300
{% elif states.sensor.time.state >= '23:00' and states.sensor.time.state < '23:30' %}2300_to_2330
{% elif states.sensor.time.state >= '23:30' and states.sensor.time.state <= '23:59' %}2330_to_0000
{% endif %}

This is used to set the correct 30min tariff on a utility_meter. Any better way of doing this ?

[123 (Taras)]

@tescophil

I don’t understand what’s going on

21.10.0 now warns you when you misuse many functions that, in previous versions, just quietly let you make mistakes. Templating changes in 2021.10.0 - Breaking Change

For example, timestamp_custom expects to receive a timestamp but you supplied it with a string:

'00:00' | timestamp_custom('%H:%M', False)

00:00 is not a timestamp, it’s a string and in previous versions this mistake would be ignored and the result of timestamp_custom would simple be the string itself (i.e. it would ignore whatever pattern you supplied and simply pass the string as-is). Whatever you think you were trying to achieve with the timestamp_custom function, it wasn’t actually doing it.

2021.10.0 now flags mistakes like that (with a warning message). 2021.12.0 will cause an error message.

@tescophil

Any better way of doing this ?

Yes. This computes the time-range that the current time falls into without requiring sensor.time.

{% set h = now().hour %}
{% set m = now().minute %}
{{'{:02d}{:02d}_to_{:02d}{:02d}'.format(h, 0 if m < 30 else 30, h if m < 30 else h+1 if h < 23 else 0, 30 if m < 30 else 0) }}

[mobile.andrew.jones (Andrew Jones)]

@123

{% set h = now().hour %}
{% set m = now().minute %}
{{'{:02d}{:02d}_to_{:02d}{:02d}'.format(h, 0 if m < 30 else 30, h if m < 30 else h+1 if h < 23 else 0, 30 if m < 30 else 0) }}

Ooooo that’s elegant.
I need to start saving snippets like these to Google Keep so I can easily find them again!

[mobile.andrew.jones (Andrew Jones)]

@123

'{:02d}{:02d}_to_{:02d}{:02d}'

Just a quick question if you don’t mind. I’ve been on the Jinja docs, and the Python docs, and even stackoverflow. But I can’t find the answer. Are the colons the same as % in Python formatting?

EDIT: Nevermind - I managed to find it. string — Common string operations — Python 3.12.2 documentation

[123 (Taras)]

The colon in {:02d} is part of what’s known as ‘new-style’ of string formatting.

Quick comparison: Python String Formatting Best Practices – Real Python

Tutorial: Advanced formatting with string format

Python version 3.6 introduced f-strings (yet another way to format a string) but that’s not supported by Home Assistant’s Jinja2 interpreter.

[tescophil (Phil)]

@123 Jinha2 Ninja !

Awsome, thanks… Been trying to squash various warnings in my (hacky) templates…, but this is still bugging me…

TemplateError('UndefinedError: 'mappingproxy object' has no attribute 'results'') while processing template 'Template("

{% if states.sensor.octopus_agile_daily_tariff is defined %}
  {% if states.sensor.octopus_agile_daily_tariff.attributes is defined %} 
    {% if states.sensor.octopus_agile_daily_tariff.attributes.results is defined %} 
       <Some other Stuff>

So, asking if an attribute is defined throws an error because its not defined ???

[123 (Taras)]

I don’t know the answer to that one because I can’t replicate it. When I try the same thing with a non-existent attribute, it works as expected.

[tescophil (Phil)]

I think it might be related to another sensor in the template, its just not logging the error correctly.

I have several command line sensors that get electricity tariffs from a public API. When these fail (as they sometimes do), I can’t work out how to catch the error eg.

- platform: command_line
  name: Octopus Agile Daily Tariff
  command: "curl 'https://api.octopus.energy/v1/products/AGILE-18-02-21/electricity-tariffs/E-1R-AGILE-18-02-21-E/standard-unit-rates/?period_from={% set ts_now = as_timestamp(now()) %}{{ (ts_now) | timestamp_custom ('%Y-%m-%d') }}T00:00:00Z&period_to={{ (ts_now) | timestamp_custom ('%Y-%m-%d') }}T23:59:00Z'"
  value_template: "Updated {{ as_timestamp(now()) | timestamp_custom('%H:%M') }}"
  scan_interval: 3600
  json_attributes:
    - "results"

Sometimes this returns a section of HTML with ‘Bad Gateway’, and then generates an ‘Unable to parse JSON’ error message in the logs. Any way to catch this ?

[mobile.andrew.jones (Andrew Jones)]

Well the preferred way to access is attributes is state_attr(sensor,attr) so depending on what results would normally contain, then I would be looking at something like testing

state_attr('sensor.octopus_agile_daily_tariff','results')[0]['valid_from']

[123 (Taras)]

That won’t work as well as you might think it will. If state_attr fails to get the value of results, the value it does return isn’t a list but the template will attempt to get the zeroth item from it (and cause an error).

[123 (Taras)]

@tescophil

returns a section of HTML with ‘Bad Gateway’, and then generates an ‘Unable to parse JSON’ error message in the logs. Any way to catch this ?

I don’t know of anything in the RESTful Sensor that allows you to perform error handling (i.e. validate the received data and handle errors gracefully). However the Command Line Sensor lets you invoke any external command so, instead of using curl, you could call a python script to perform the task. Python provides you with all the flexibility needed to process the data and handle errors. However, if you don’t know how to program in python then that might be a significant hurdle.

Here’s an example of what I mean:

RESTful sensor, parsing json

I modified the code and this works for me. I now know on which days you can put out garden waste, refuse, bulk items, etc. #!/usr/bin/python3 import requests import json from datetime import datetime import sys if len(sys.argv) == 2: days = {'Mandag':'Monday', 'Tirsdag':'Tuesday', 'Onsdag':'Wednesday', 'Torsdag':'Thursday', 'Fredag':'Friday'} types = {'Dagrenovation':0, 'MadamSkrald':2, 'Storskrald':3, 'Haveaffald':5 } body = {'adrid': 24750,'common': False} headers = {'…

[FutureTense]

I’m having a similar problem. first, let me try to explain the situation. I’m trying to determine if the current time is “in range” of variables for each day of the week. To try and simplify things for discussion (it took me hours to create this post ) let’s assume we are looking at the entities only for Sunday. The input_datetime entities the user sets has no date component. e.g.

  sun_start_date_frontdoor_1:
    name: "Start"
    has_time: true
    has_date: false

The code below is trying to set the start and end input_datetime entities to “midnight” with no date, as has_date is set to false.

- service: input_datetime.set_datetime
  data_template:
	entity_id: "input_datetime.{{ days[repeat.index - 1] }}_start_date_frontdoor_{{ code_slot | string }}"
	time: "{{ '00:00' | timestamp_custom('%H:%M') }}"

- service: input_datetime.set_datetime
  data_template:
	entity_id: "input_datetime.{{ days[repeat.index - 1] }}_end_date_frontdoor_{{ code_slot | string }}"
	time: "{{ '00:00' | timestamp_custom('%H:%M') }}"

And this is what the log is showing.

Template warning: 'timestamp_custom' got invalid input '00:00' when compiling template '{% set days = ["sun", "mon", "tue", "wed", "thu", "fri", "sat"] %}
   time: "{{ '00:00' | timestamp_custom('%H:%M') }}"' but no default was specified. 

Currently 'timestamp_custom' will return '00:00', however this template will fail to render in Home Assistant core 2021.12
   time: "{{ '00:00' | timestamp_custom('%H:%M') }}"'

Any idea what’s the problem?

[Didgeridrew]

You don’t need to use a template to set a static time value.

- service: input_datetime.set_datetime
  data_template:
	entity_id: "input_datetime.{{ days[repeat.index - 1] }}_start_date_frontdoor_{{ code_slot | string }}"
	time: "00:00"