Jinja loop scoping – Average temperature sensor

ok, so maybe I hsould describe what i want to do…

I just want to get a reading of the total power in use in the house… so I can identify peaks.

maybe a better way is to find all devices which have the unit of measurement W and kW (* 1000) and add them up?

which I suppose is what you are implying with this:

{% set x = states.sensor
   | selectattr('attributes.device_class', 'eq', 'power')
   | map(attribute='state') | map('float') | list %}
{{ (x | sum) / (x | count) }}

device_class and unit_of_measurement are separate attributes and have different purposes.

device_class helps the Lovelace UI select the appropriate icon to display for the sensor. It can also be used by templates to quickly and efficiently select all sensors of the same device_class.

unit_of_measurement indicates what the sensor’s state value represents. If some of your power sensors report in Watts and other in KiloWatts then you have to take that into consideration when calculating total or averages.

So we would need to select all sensors using W separately from the sensors using kW. The ones using kW would need to be multiplied by 1000 so that they have parity with the ones using W.

{% set x = states.sensor
   | selectattr('attributes.device_class', 'eq', 'power')
   | selectattr('attributes.unit_of_measurement', 'eq', 'W')
   | map(attribute='state') | map('float') | list %}
{% set y = states.sensor
   | selectattr('attributes.device_class', 'eq', 'power')
   | selectattr('attributes.unit_of_measurement', 'eq', 'kW')
   | map(attribute='state') | map('float') | map('multiply', 1000) | list %}
{{ (x + y) | sum / (x + y) | count }}

EDIT

Simplified the template’s last line.

2 Likes

yeah, nice. That is fantastic.

Thank you for all of your help, this is what I have ended up with, I have had to ensure it does not count its self (sensor.average_power) and when HASS is starting check if there are any values so there isn’t a ‘ZeroDivisionError: division by zero’ in the logs:

        {% set x = states.sensor
           | selectattr('attributes.device_class', 'eq', 'power')
           | selectattr('attributes.unit_of_measurement', 'eq', 'W')
           | rejectattr('entity_id', 'eq', 'sensor.average_power')
           | map(attribute='state') | map('float') | list %}
        {% set y = states.sensor
           | selectattr('attributes.device_class', 'eq', 'power')
           | selectattr('attributes.unit_of_measurement', 'eq', 'kW')
           | rejectattr('entity_id', 'eq', 'sensor.average_power')
           | map(attribute='state') | map('float') | map('multiply', 1000) | list %}
        {% if ((x | count) + (y | count)) > 0 %}
          {{ ((x | sum) + (y | sum)) / ((x | count) + (y | count)) }}
        {% else %}
          0
        {% endif %}

anyway, we will see where this experiment takes me

You can reduce the last five lines to two.

        {% set x = states.sensor
           | selectattr('attributes.device_class', 'eq', 'power')
           | selectattr('attributes.unit_of_measurement', 'eq', 'W')
           | rejectattr('entity_id', 'eq', 'sensor.average_power')
           | map(attribute='state') | map('float') | list %}
        {% set y = states.sensor
           | selectattr('attributes.device_class', 'eq', 'power')
           | selectattr('attributes.unit_of_measurement', 'eq', 'kW')
           | rejectattr('entity_id', 'eq', 'sensor.average_power')
           | map(attribute='state') | map('float') | map('multiply', 1000) | list %}
        {% set qty = (x + y) | count %}
        {{ (x + y) | sum / qty if qty > 0 else 0 }}

ok, so in the end this is where I ended up:

    average_power:
      device_class: power
      friendly_name: Average Power
      unit_of_measurement: W
      value_template: >-
        {% set x = states.sensor
           | selectattr('attributes.device_class', 'eq', 'power')
           | selectattr('attributes.unit_of_measurement', 'eq', 'W')
           | rejectattr('entity_id', 'eq', 'sensor.average_power')
           | rejectattr('entity_id', 'eq', 'sensor.total_power')
           | map(attribute='state') | map('float') | list %}
        {% set y = states.sensor
           | selectattr('attributes.device_class', 'eq', 'power')
           | selectattr('attributes.unit_of_measurement', 'eq', 'kW')
           | rejectattr('entity_id', 'eq', 'sensor.average_power')
           | rejectattr('entity_id', 'eq', 'sensor.total_power')
           | map(attribute='state') | map('float') | map('multiply', 1000) | list %}
        {% if ((x | count) + (y | count)) > 0 %}
          {{ ((x | sum) + (y | sum)) / ((x | count) + (y | count)) }}
        {% else %}
          0
        {% endif %}

    total_power:
      device_class: power
      friendly_name: Total Power
      unit_of_measurement: W
      value_template: >-
        {% set x = states.sensor
           | selectattr('attributes.device_class', 'eq', 'power')
           | selectattr('attributes.unit_of_measurement', 'eq', 'W')
           | rejectattr('entity_id', 'eq', 'sensor.average_power')
           | rejectattr('entity_id', 'eq', 'sensor.total_power')
           | map(attribute='state') | map('float') | list %}
        {% set y = states.sensor
           | selectattr('attributes.device_class', 'eq', 'power')
           | selectattr('attributes.unit_of_measurement', 'eq', 'kW')
           | rejectattr('entity_id', 'eq', 'sensor.average_power')
           | rejectattr('entity_id', 'eq', 'sensor.total_power')
           | map(attribute='state') | map('float') | map('multiply', 1000) | list %}
        {{ ((x | sum) + (y | sum)) }}

They seem to work fine, but now I get this error in the logs:

2020-10-12 09:45:45 WARNING (MainThread) [homeassistant.components.template.template_entity] Template loop detected while processing event: <Event state_changed[L]: entity_id=sensor.total_power, old_state=<state sensor.total_power=513.9200000000001; unit_of_measurement=W, friendly_name=Total Power, device_class=power @ 2020-10-12T09:43:45.185308+11:00>, new_state=<state sensor.total_power=499.35; unit_of_measurement=W, friendly_name=Total Power, device_class=power @ 2020-10-12T09:44:45.145237+11:00>>, skipping template render for Template[{% set x = states.sensor
   | selectattr('attributes.device_class', 'eq', 'power')
   | selectattr('attributes.unit_of_measurement', 'eq', 'W')
   | rejectattr('entity_id', 'eq', 'sensor.average_power')
   | rejectattr('entity_id', 'eq', 'sensor.total_power')
   | map(attribute='state') | map('float') | list %}
{% set y = states.sensor
   | selectattr('attributes.device_class', 'eq', 'power')
   | selectattr('attributes.unit_of_measurement', 'eq', 'kW')
   | rejectattr('entity_id', 'eq', 'sensor.average_power')
   | rejectattr('entity_id', 'eq', 'sensor.total_power')
   | map(attribute='state') | map('float') | map('multiply', 1000) | list %}
{{ ((x | sum) + (y | sum)) }}]

any thoughts on how to quieten the loop detection here or is there an actually problem (I can no see it)?

Thanks

so for anyone else interested, I got a reply on discord for this:

Barbados SlimYesterday at 4:18 PM
@jurgenweber there was a new issue with .115 where template entities that refer to other template entities that appear lower in the yaml config would throw a false template loop error (among other errors). It was fixed in that the entities now will eventually update state once the dependency is loaded, but it still shows the error in the logs. Since both of those sensors refer to the other one (and therefore you won’t be able to get them in the correct order), you’ll always get that error. You could open an issue about it, or just ignore it. Or maybe somehow reconfigure the templates so that they don’t refer to one another.
this was the original issue, just for reference: 'Reload Template Entities' breaks any template entities that refer to other template entities. · Issue #40611 · home-assistant/core · GitHub
also it kind of is a template loop. since they both refer to themselves. not necessarily a problem after initial loading but it’s not surprising that it is being flagged as a loop since it is indeed self-referential. so even if you fix the issue I described before, you might still get an error. Not sure.

Barbados SlimYesterday at 4:32 PM

maybe you could set an attribute in the two template sensors and use your reject filter to reject them based on that attribute instead of referencing the entity_id
not sure if that would work or not, ive never done anything quite like this. just an idea

I have now tried:

    average_power:
      device_class: power
      friendly_name: Average Power
      unit_of_measurement: W
      attribute_templates:
        template_sensor: >-
          true
      value_template: >-
        {% set x = states.sensor
           | selectattr('attributes.device_class', 'eq', 'power')
           | selectattr('attributes.unit_of_measurement', 'eq', 'W')
           | rejectattr('attributes.template_sensor', 'eq', 'true')
           | map(attribute='state') | map('float') | list %}
        {% set y = states.sensor
           | selectattr('attributes.device_class', 'eq', 'power')
           | selectattr('attributes.unit_of_measurement', 'eq', 'kW')
           | rejectattr('attributes.template_sensor', 'eq', 'true')
           | map(attribute='state') | map('float') | map('multiply', 1000) | list %}
        {% if ((x | count) + (y | count)) > 0 %}
          {{ ((x | sum) + (y | sum)) / ((x | count) + (y | count)) }}
        {% else %}
          0
        {% endif %}

without success:

2020-10-14 15:48:54 WARNING (MainThread) [homeassistant.components.template.template_entity] Template loop detected while processing event: <Event state_changed[L]: entity_id=sensor.average_power, old_state=<state sensor.average_power=14.893658536585368; template_sensor=true, unit_of_measurement=W, friendly_name=Average Power, device_class=power @ 2020-10-14T15:46:54.519707+11:00>, new_state=<state sensor.average_power=15.721951219512196; template_sensor=true, unit_of_measurement=W, friendly_name=Average Power, device_class=power @ 2020-10-14T15:47:54.512962+11:00>>, skipping template render for Template[{% set x = states.sensor
   | selectattr('attributes.device_class', 'eq', 'power')
   | selectattr('attributes.unit_of_measurement', 'eq', 'W')
   | rejectattr('attributes.template_sensor', 'eq', 'true')
   | map(attribute='state') | map('float') | list %}
{% set y = states.sensor
   | selectattr('attributes.device_class', 'eq', 'power')
   | selectattr('attributes.unit_of_measurement', 'eq', 'kW')
   | rejectattr('attributes.template_sensor', 'eq', 'true')
   | map(attribute='state') | map('float') | map('multiply', 1000) | list %}
{% if ((x | count) + (y | count)) > 0 %}
  {{ ((x | sum) + (y | sum)) / ((x | count) + (y | count)) }}
{% else %}
  0
{% endif %}]

I suggest you report this new behavior as an Issue in the GitHub core repository.

The error message appears to be related to the new way that templates are rendered (introduced in version 0.115). Your template used to work without error (prior to 0.115) but now it causes a problem.

finally remembered to do this; https://github.com/home-assistant/core/issues/42003#issuecomment-714030131

Tbh, this could well be an unexpected user error, since the template simply tries to find all sensors with the device_class: power, so its doing its job correctly.

Even if this would be seen as a bug, as an intermediary solution, wouldn’t this be fixed by taking out the device_class on this sensor? Or simply reject the entity_id?

|rejectattr('entity_id','eq','sensor.average_power')

Always have to be careful with these generators, that’s why we create blacklist groups for these situations :wink:

          {{states|selectattr('state','in',['unavailable','unknown','none'])
                  |rejectattr('entity_id','in',ignore_list)
                  |rejectattr('entity_id','in',state_attr('group.entity_blacklist','entity_id'))
                  |rejectattr('domain','in',['group','media_player'])
                  |list|length}}
1 Like

It’s not a bug, your template is referencing it self when you search the sensor domain and you’re inside a template sensor. Ignore the warning or make sure it doesn’t happen. A bug would be a continuous loop, this message is letting you know that’s not happening.

If you don’t want the sensor referencing itself, then take @Mariusthvdb’s advice and use rejectattr

EDIT: Users may want to revert to the previous state of the current sensor. That means they want to have a circular reference. Automatically filtering the entity out would break all these cases.

In your case, you should be filtering it out because you don’t want to add it into the averages. This warning is letting you know that you’re doing that. Essentially doing it’s job.

And, if you don’t want to see the warning but want the circular reference, use logger and set the logger level to error or critical. Warnings will no longer appear.

1 Like

I’m trying to do something similar with the weather integration. I’m trying to average the daily temperature using the hourly forecast from the NWS integration. This should be simple, but I’m clearly missing something. The sensor template I wrote is (where state_attr('weather.kmsn_hourly','forecast') is a dictionary of hourly forecasts with the key temperature for each forecast):

        {% set t_avg = 100|float %}
        {% for f in state_attr('weather.kmsn_hourly','forecast') %}
          {% set t_avg = t_avg + f['temperature']|float %}
        {% endfor %}
        {{t_avg}}    

The value comes out to be 100. I had originally set it to zero and the value came out to be zero. I plugged this into the Developer Tools Template and, when I print out the values for t_avg it always prints to 100. The values for f['temperature']|float are correct, but for some reason the value for t_avg never changes. What am I doing wrong? I haven’t even gotten to the averaging part… Thanks for any help.

You would need to use a namespace to get the information out of the loop…

But you don’t need to do any of that:

{{ state_attr('weather.kmsn_hourly','forecast')
| map(attribute='temperature') 
| list | average }}
1 Like

Wow - that was incredibly easy! Thanks, Drew!

1 Like

I just realized that the forecast dictionary goes out 3 days. Is there a way to limit/filter to the next 24 hours only?

Use selectattr() to keep only forecasts with today’s date in their datetime:

{{ state_attr('weather.kmsn_hourly','forecast')
| selectattr('datetime', 'search', now().date()|string )
| map(attribute='temperature') 
| list | average }}
2 Likes