Have I found a flaw in template sensors?

Configuration
1

I am having a problem with errors in my log caused by a template sensor when the value_template has no value. I know the normal way to avoid this but I might have found a flaw in HA where this is not possible to do (or more likely of course, I don’t know how to do it!)

Please read on if this is your area of expertise and don’t be put off by the length of this post, most of it is just for illustrative purposes :wink:

I use a sensor platform (UK Transport but don’t stop reading if you don’t use it) that returns nested JSON about future trains.

  - platform: uk_transport
    app_id: !secret uk_transport_app_id
    app_key: !secret uk_transport_api
    queries:
      - mode: train
        origin: LIV
        destination: WAT      
    scan_interval: 86400

And here is the JSON

{
  "station_code": "LIV",
  "calling_at": "WAT",
  "next_trains": [
    {
      "origin_name": "Liverpool Street",
      "destination_name": "Watford",
      "status": "ON TIME",
      "scheduled": "09:26",
      "estimated": "09:26",
      "platform": "4",
      "operator_name": "TfL Rail"
    },
    {
      "origin_name": "Liverpool Street",
      "destination_name": "Watford",
      "status": "ON TIME",
      "scheduled": "11:12",
      "estimated": "11:12",
      "platform": "4",
      "operator_name": "TfL Rail"
    }
  ],
  "unit_of_measurement": "min",
  "friendly_name": "Next train to WAT",
  "icon": "mdi:train"
}

The docs suggest templating like this:

value_template: '{{ states.sensor.next_train_to_wat.attributes.next_trains[x].scheduled }}'

which works (obviously) but as you would expect it throws an error if there is no next_trains[x].

I experimented with various alternatives:

value_template: '{{ value_json.next_trains[x].scheduled }}

which also works but still throws an error if next_trains[x] doesn’t exist. (also it’s a bugger to test as I can’t see a way to do it in the template tool!)

Is there a way to template this sensor without it throwing an error if the next_trains[x] doesn’t exist. For example, I tried:

value_template: '{{ state_attr('sensor.next_train_to_wat', 'next_trains[x].scheduled' }}'

which doesn’t work, and nor does

value_template: '{{ state_attr('sensor.next_train_to_wat', 'next_trains[x]', 'scheduled' }}'`
2

Not to be a total PITA but you can test it in the template tool. Setup a test JSON and then you can test JSON to your hearts content… The default example when you go into the template tool shows you how to do that.

You haven’t shown the sensor definition either which would make it easier to troubleshoot…

3

Both fair points!
I have edited the OP to include the sensor.

And yes I can simulate the JSON but only to test using value_json, not states, state_attr or any variation on that.

4

And that is the name of the sensor it creates? And the JSON you posted is what you see (an example anyway) as attributes when you click on that sensor in the dev tool?

5

I think this would work (see http://jinja.pocoo.org/docs/dev/templates/#list-of-builtin-tests for reference)

{% if value_json.next_trains[x] is defined %}
 get the actual value
{% else %}
  display the error message
{% endif %}
2 Likes
6

@DavidFW1960 yes and yes

7

I’ll try it.
And Thank you!!!

EDIT: tried it in the template tool and it looks like it does work. I can’t test it for real until tonight though when the trains stop but thanks again!

8

Ok, so for completeness for anyone coming here with the exact same problem, that didn’t quite work but this does (so exactly the right principle from @gpbenton):

      {% if states.sensor.next_train_to_wat.attributes.next_trains[3] is defined %}
        {{ states.sensor.next_train_to_wat.attributes.next_trains[3].scheduled }}
      {% else %}
        Not scheduled
      {% endif %}

at least it did in the template tool and it does when trains exist. I’ll see tonight what happens when they stop running.

3 Likes
9

@gpbenton
I’m afraid that doesn’t work…
(But looking at the errors I wonder if I have the value_template using value_json crrect?)

is defined only seems to check for the existence of an element within the array, not the array itself, If the array itself doesn’t exist then HA still throws an error.

When there were no next_trains neither of these

`{% if value_json.next_trains[0] is defined %}` caused this,

2018-11-23 23:53:57
ERROR (MainThread) [homeassistant.components.sensor.template] Could not render template Next train scheduled: UndefinedError: 'value_json' is undefined

and

`{% if states.sensor.next_train_to_wat.attributes.next_trains[0] is defined %}` caused this,

2018-11-23 23:53:57
ERROR (MainThread) [homeassistant.components.sensor.template] Could not render template Next train status: UndefinedError: 'mappingproxy object' has no attribute 'next_trains'

To recap this is what I am trying

  - platform: template
    sensors:
      next_trains_scheduled:
        friendly_name: 'Next train scheduled'
        entity_id: sensor.next_train_to_wat
        value_template: >
          {% if value_json.next_trains[0] is defined %}
            {{ states.sensor.next_train_to_wat.attributes.next_trains[0].scheduled }}
          {% else %}
            Not scheduled
          {% endif %}

      next_train_status:
        friendly_name: 'Next train status'
        entity_id: sensor.next_train_to_wat
        value_template: >
          {% if states.sensor.next_train_to_wat.attributes.next_trains[0] is defined %}
            {{ states.sensor.next_train_to_wat.attributes.next_trains[0].status }}
          {% else %}
            ' '
          {% endif %}
10

@gpbenton
After I posted the above it occurred to me that maybe I could do this instead, i.e check the array is defined rather than a single element. I have tested it in the template tool and it works but next_trains exists so I can’t test it properly until late tonight.

      {% if states.sensor.next_train_to_wat.attributes.next_trains is defined %}
        {{ states.sensor.next_train_to_wat.attributes.next_trains[0].status }}
      {% else %}
        ' '
      {% endif %}

Any thoughts?

11

That would seem to me that you need to check all the elements in the tree individually like

{% if (value_json is defined) and (value_json.next_trains[0]) is defined %}
12

Good point…
I’ll give that a try.

Update: As before this doesn’t work:

{% if (value_json is defined) and (value_json.next_trains[0]) is defined %}

Bu this does

{% if (states.sensor.next_train_to_wat.attributes.next_trains is defined) and
            (states.sensor.next_train_to_wat.attributes.next_trains[0] is defined) %}

At least, as before, it did in the template tool and it does when trains exist. I’ll see what happens again when they stop running.

13

If you are trying to get the first item in an array, do this as a check:

{% if states.sensor.next_train_to_wat.attributes.next_trains is defined and 
     states.sensor.next_train_to_wat.attributes.next_trains | length >= 1 %}

This checks to see if items are in the list and the list is defined. You can use any operation for the > sign. I’d use greater than though because if the list is larger than 1, you’ll still want this to work.

14

Please bear with me because I’m still learning how the Jinja2 template engine works.

Assumption: The template engine evaluates all tests in the if to determine if the overall result is true or false. So in the following example it first determines if the list next_trains exists then proceeds to determine if the list’s length is non-zero.

{% if states.sensor.next_train_to_wat.attributes.next_trains is defined and 
     states.sensor.next_train_to_wat.attributes.next_trains | length >= 1 %}
  • So if the list is undefined, the first test gracefully returns false.
  • Then it proceeds to determine the length of an undefined variable. Wouldn’t this result in an error?

Or does the template engine stop its evaluation at the first false it finds? In other words, it recognizes there are two tests connected by and, if the first test results in false it doesn’t bother evaluating the second test.

15

No, if the first value is false, it won’t even resolve the second because the condition has not been met. Pretty much all languages work this way with AND or OR.

With OR, if the first value is true, it pops out and moves on.

16

That’s been my experience but I needed to confirm this works the same way. Thank you for clearing that up!

17

Why is this better than what I (@gpbenton ) had? Isn’t the way I am doing it, checking the list is defined and then checking if the specific item in the list I am interested in exists?

Doesn’t your method only work for the first item in the list? I am not always interested in the first item in the list as indicated in the original post with an index of ‘x’ being used. (Although subsequent posts have indeed always referred to next_trains[0] which may have affected your view.)

I’m not being critical, I just want to understand!

18

it’s not necessarily better. It’s just how index checking is normally done.

Yes

No, checking against the length of the list allows you do do things like this:

{% if list | length >= 5 %}
  {{ list[0] }} {{ list[1] }} {{ list[2] }} {{ list[3] }} {{ list[4] }}
{% endif %}

In this case, you can safely access all items up to the 5th item.

The list[0] is defined only checks the first item. You have your thought process backwards on that.

1 Like
19

Perfect explanation, thanks.

20

hey @klogg

I am facing the same issue as you. How did you manage to resolve the issue mate?

thanks