Template sensor calculations when values don't exist

Configuration
1

I’m pulling some data from a restAPI from my Neurio, but occasionally the connection is reset or fails and the restAPI call gets nothing. I’ve got sensors set for real time consumption and generation and those sensors do a good job of dealing with the fact there’s a failure. No blips in my graphs, they seem to just ignore the fact there was no data for a moment. This is desired.

        unit_of_measurement: Ws
        value_template: "{{ state_attr('sensor.neurio_raw', 'channels')[3]['eImp_Ws']  }}"

image

Then I have a second set of sensors that do some math based on those real time numbers. Those sensors DON’T handle the fact that the restAPI call failed and that the sensors these calculations are based on don’t exist for a moment and as such, I get a graph like this below for the following sensor:

      calculated_generation:
        friendly_name: 'Daily Generation'
        unit_of_measurement: 'kWh'
        value_template: "{{ ((states('sensor.generation_imported') | int - states('input_number.generation_at_midnight') | int) / 3600000) | round(2) }}"

image

What can I do to account for this erroneous data and bring my graph back to normal/accurate?

2

This should keep the same value if the sensor state used for the calculation is unknown.

calculated_generation:
  friendly_name: 'Daily Generation'
    unit_of_measurement: 'kWh'
    value_template: >
      {% if (states('sensor.generation_imported') in ['unavailable', 'unknown', 'none'] %}
        {{ (states('sensor.calculated_generation') }}
      {% else %}
        {{ ((states('sensor.generation_imported') | int - states('input_number.generation_at_midnight') | int) / 3600000) | round(2) }}
      {% endif %}
1 Like
3

Template Sensor

Check out the availability_template option, you can do a check like:

availability_template: >-
  {%- if states('sensor.generation_imported') -%}
     True
  {%- else -%}
     False
  {%- endif -%}

So if the state is a False value, including None, which would equate to Unavailable, this should then ignore data when the availability_template evaluates to False. If the sensor can have a value of Zero, then you have be more explicit about your availability_template test.

Example:
if states('sensor.generation_imported') == None

4

I believe the availability_template can be reduced to this:

        availability_template: "{{ states('sensor.generation_imported') != None }}"
5

What tom_l suggested seems to have worked, but this looks like it would be cleaner, though I tried adding it to my sensor and it had no effect. Do you have an example of how this would be added to the current sensor setup, or another example of how it’s used?

6

Regarding checking a state of sensor.

I have seen in many examples (including here - https://www.home-assistant.io/cookbook/#example-configurationyaml, https://github.com/Petro31/home-assistant-config/blob/62fddff4247b7c3c81d33b00a3b5048174a70c6e/sensor.yaml) something like:

value_template: "{% if states('sensor.source_sensor') -%}
                 {{ states('sensor.source_sensor') }}
                 {%- else -%}
                 {{ states('sensor.destination_sensor') }}
                 {%- endif %}"

The point was to use a source sensor’s value only if this sensor is meaningful (ready, actually updated etc).
As I understood the 1st condition is true if a value of “states(‘sensor.source_sensor’)” is NOT zero - at least it is in C/C++ and I proposed that same logic is in Perl, Jinjia2 etc. Later I found that I was wrong.

Asked myself - what if the source sensor’s value MAY BE zero (i.e. it is a normal behaviour).
Then I started looking for more precise methods.

Also, in many other places here in forum I’ve seen smth like:

value_template: "{% if not states('sensor.source_sensor') in ['unavailable','unknown','none'] -%}
                 {{ states('sensor.source_sensor') }}
                 {%- else -%}
                 {{ states('sensor.destination_sensor') }}
                 {%- endif %}"

Note: initially by mistake in this post it was specified unavialable instead of unavailable, my thanks to this - Template sensor calculations when values don't exist.

To check if it works used a “Template” feature in HA.
Specified this:

{% if states('sensor.test_zero_value') in ['unavailable','unknown','none'] -%}
first = {{ states('sensor.test_zero_value') }}
{%- else -%}
second = {{ states('sensor.test_zero_value') }}
{%- endif %}
%%%%%%%%%%%%%
{% if not states('sensor.test_zero_value') -%}
first = {{ states('sensor.test_zero_value') }}
{%- else -%}
second = {{ states('sensor.test_zero_value') }}
{%- endif %}

Then changed a value of sensor.test_zero_sensor to 0, unavailable, unknown, none.
Here are the output results:

second = 0
%%%%%%%%%%%%%
second = 0

first = unavailable
%%%%%%%%%%%%%
second = unavailable

first = unknown
%%%%%%%%%%%%%
second = unknown

first = none
%%%%%%%%%%%%%
second = none

So, the check in ['unavailable','unknown','none'] works fine.

But the check for “zero” failed.
Found here (https://jinja.palletsprojects.com/en/master/templates/#if) a possible answer:
you can use it to test if a variable is defined, not empty and not false.
So it is not about “= 0”.
Does that mean that these "is defined, not empty and not false" are covered by "if states(..)" ?
According to my test the answer is NO - look at the output results, they are always “second = …” for the 2nd check.
So why people used this "if states(..)" method?

Moreover, why " unavailable','unknown','none" ?
Can anyone give me a link to a document/code explaining possible states and their meaning?
What I think is (from my experience):

  1. Unavailable - the sensor is created but its state is not determined.
    Example: the state of thermometer device which cannot be polled (the device is switched off, connection lost, …).
  2. Unknown - the sensor does not exist.
    Example: the sensor with RAM usage of remote PC retrieved by Netdata integration - since that PC was not available during HA startup, so the sensor was not created at HA startup.
  3. None - do not know any possible reasons.
7

The value is called unavailable not unavialable. It means the sensor’s value isn’t currently available (typically due to loss of communication). A state value of unknown is usually due to the sensor not receiving any value (so it’s current state is unknown).

8

First, thank you very much for the reply!

Thank you, that was a typo; funny moment that to check the conditions I used the same “unavialable” (copy-paste) as a state)).
Updated my 1st post, replaced unavialable with unavailable.

Right, like I wrote above.

Well, it may happen at least in two cases:

  1. My scenario mentioned above - “the sensor does not exist”.
  2. If the sensor depends on another sensor which is “unavailable” or does not exist.

What about “None”, do you know anything about it?

Also, I am confused about using “{% if state(XXXX) %}” as a condition since it returns “true” in case of “unavailable”.

9

The states() function always returns a value. As you have already discovered, it will report unknown if the entity doesn’t have a value due to not receiving one or, less often, due to a non-existent entity.

Personally, I have never used {% states('sensor.whatever') %} as a test to determine if the entity’s value exists. The use of states() ensures a value will be returned even though it might be unavailable or unknown. As far as I know, states() will never return none which represents the complete absence of any value (i.e. null).

10

That’s interesting about ‘none’ never being returned by states(). I should probably stop testing for it.

11

Perhaps the only time it would return none is if the entity’s state value is none? :man_shrugging: I’ll leave these edge-cases for someone else to explore.

12

Look at this:

The file is full of this statements:

          {% if states.zwave.garage_door.state %}
            {{ states.zwave.garage_door.attributes.battery_level }}
          {% else %}
            0
          {% endif %}

I believe that the person is quite well-informed about HA ))

13

I am very glad that you joined this discussion because I’ve learned about this " in [‘unavailable’,‘unknown’,‘none’]" from you )))

So, finally shall we use something like this:

value_template: "{% if not states('sensor.source_sensor') in ['unavailable','unknown'] -%}
                 {{ states('sensor.source_sensor') }}
                 {%- else -%}
                 {{ states('sensor.destination_sensor') }}
                 {%- endif %}"

???

14

Are you aware of the difference between accessing an entity’s state value directly versus using the states() function?

Use the Template Editor to get the value of a non-existent entity using both techniques. It will quickly demonstrate the difference.

EDIT

It’s at least the second time you have referred to Petro’s sensors as proof of … something. Of what exactly? As the definitive way to determine if a value exists? Instead of guessing, simply ask him.

15

No, not looking for a proof, I am just asking questions )).

Actually, it is not about a particular expert, I have already told that I have seen this (or similar) approach in many places in Cookbook or here in the Conference, that is why I am asking, may be I am missing something, I am quite a new person here, still learning and ready to listen to people’s opinion.

I know about “states.sensor.XXX.state vs states(‘sensor.XXX’)” only what is specified here - Templating - Home Assistant.

A small test using “Templates”.
There are 2 sensors here:

  • sensor.test_zero_value - a real sensor (currently manually set to ‘unavailable’);
  • sensor.test_zero - non-existing sensor.
{{ states.sensor.test_zero.state }}
{{ states('sensor.test_zero') }}
$$$$$$$$$$$$$$$$$$$$$$$$
{{ states.sensor.test_zero_value.state }}
{{ states('sensor.test_zero_value') }}

Output:


unknown
$$$$$$$$$$$$$$$$$$$$$$$$
unavailable
unavailable

As we can see:

  1. Both methods show ‘unavailable’ if the sensor is ‘unavailable’.
  2. First method returns empty value, if the sensor does not exist.

My assumption - it is properly to use:

if states('sensor.XXXX') != 'unavailable' and states('sensor.XXXX') != 'unknown'

or

if not states('sensor.source_sensor') in ['unavailable','unknown']

if it is equal to it,
than:

if states('sensor.XXX')

or

if states.sensor.XXX.state
16

If it was this question:

Can anyone give me a link to a document/code explaining possible states and their meaning?

The states were explained.

You’re making an assumption about the examples you’ve selected. You’re assuming the authors are attempting to avoid using any and all possible invalid values. Perhaps the authors of those examples weren’t concerned with all of them just one or two cases.

The first three responses of this thread explain how to go about it (either use a more comprehensive test and/or employ the availability_template).

17

OK, never mind.

That was about states() in ['unavailable', 'unknown', 'none'], now it seems to be corrected to in ['unavailable', 'unknown'] as you may notice. Mission accomplished.

18

how about asking a property which should give a value if the device is there like

{{ state_attr('sensor.ble_temperature','battery_level')|float > 0}}

which returns false if the device is not there.