Template loop detected while processing event

[Tomaae]

Hey,
I have been having this issue for a while, but cant seem to find a solution. Sensor itself works well, but error message appears quite often in core log:

2020-11-07 16:26:00 WARNING (MainThread) [homeassistant.components.template.template_entity] Template loop detected while processing event: <Event state_changed[L]: entity_id=sensor.current_consumption, old_state=<state sensor.current_consumption=236.0; unit_of_measurement=W, friendly_name=Current Power, icon=mdi:transmission-tower @ 2020-11-07T16:25:58.361623+01:00>, new_state=<state sensor.current_consumption=237.0; unit_of_measurement=W, friendly_name=Current Power, icon=mdi:transmission-tower @ 2020-11-07T16:25:59.364586+01:00>>, skipping template render for Template[{% set ns = namespace(states=[]) %} {% for s in states.sensor %}
  {% if s.entity_id.startswith('sensor.') and s.object_id.endswith('_power') and s.state not in ['unavailable', 'unknown' ] %}
    {% set ns.states = ns.states + [ s.state | float ] %}
  {% endif %}
{% endfor %} {{ ns.states | sum | round(2) }}]

Full sensor configuration:

- platform: template
  sensors:
    current_consumption:
        friendly_name: Current Power
        unit_of_measurement: W
        icon_template: mdi:transmission-tower
        value_template: >
          {% set ns = namespace(states=[]) %}
          {% for s in states.sensor %}
            {% if s.entity_id.startswith('sensor.') and s.object_id.endswith('_power') and s.state not in ['unavailable', 'unknown' ] %}
              {% set ns.states = ns.states + [ s.state | float ] %}
            {% endif %}
          {% endfor %}
          {{ ns.states | sum | round(2) }}

It is based on configuration I found in several places here.
Interesting is, that I have kWh sensor which is almost identical, but that one never shows any errors.
I dont see how it can loop, since it is not self-referencing for sure.

Any idea what could be causing this message?

[MT0321]

Did you ever solve this? I’m encountering the same behavior.

[Tomaae]

No, I didnt. I have already tried bunch of things, but this issue just wont go away.

[123 (Taras)]

@Tomaae

it is not self-referencing for sure

You’ve defined a Template Sensor called sensor.current_consumption.

This line collects all sensors including sensor.current_consumption.

{% for s in states.sensor %}

There are other ways to do this. You can create a group containing all power-related sensors except sensor.current_consumption. The template will simply need to use expand() and you won’t need a for-loop. Alternately, if all your power-related sensors have a device_class set to power, your template can simply use selectattr('attributes.device_class', 'eq', 'power) to get them.

If you many sensors with device_class set to power and you don’t want the sum of all of them, you can use custom attributes. Create a custom attribute for each sensor (could be something simple like type: power) and then use a template like selectattr('attributes.type', 'eq', 'power).

[Tomaae]

@123

many sensors with device_class set to power and you don’t want the sum of all of them, you can use custom attributes. Create a custom attribute for each sensor (could be somet

It does not collect all sensors, It only cycles through them all. See line right under your quote:

{% if s.entity_id.startswith('sensor.') and s.object_id.endswith('_power') and s.state not in ['unavailable', 'unknown' ] %

Entity name must end with “_power”, otherwise it will not be counted in.

They are all ESP devices, so I should be able to add attribute I think. Never tried that.
But not sure it would help, since I still need to cycle through all sensors to find all power readings. I have many and add/remove more often, so I dont want static method to do this.

[123 (Taras)]

@Tomaae

It does not collect all sensors, It only cycles through them all.

Paste this into the Template Editor:

          {% for s in states.sensor %}
              {{ s.name }}
          {% endfor %}

It’ll report the names of all sensors in your system, including sensor.current_consumption. That’s the collection the for-loop cycles through. The first line within your for-loop is the test it applies to each member of the collection.

Paste this into the Template Editor

          {% for s in states.sensor if s.entity_id.startswith('sensor.') and s.object_id.endswith('_power') and s.state not in ['unavailable', 'unknown' ] %}
              {{ s.name }}
          {% endfor %}

It should print only the names of the sensors that match the criteria. You may want to try that construct in your Template Sensor. It may fix the issue (without any major changes to the template).

[Tomaae]

thanks, going to give it a try

[123 (Taras)]

Given that the template is using states.sensor you could probably remove the initial startswith('sensor.') because only sensors are included in the collection.

NOTE

Depending on how the new template analysis works, it may still cause the error message if the analysis takes into consideration states.sensor without evaluating the balance of the template. In other words, it may creates listeners for all sensors just because it sees states.sensors in the template. I’m hoping the new template analysis technique (introduced a few versions ago) is more sophisticated than that. If not, every time sensor.current-consumption is updated, it will (because it’s part of states.sensor cause the template to be re-evaluated (which is a loop, of course).

[Tomaae]

Ah I see, now I understand the issue. Thanks for an explanation.
So far no error, but I didnt used to get it every time. I give it some time and report here.

[Tomaae]

No, that didnt helped. I’m still getting the same error:

2020-12-16 10:26:22 WARNING (MainThread) [homeassistant.components.template.template_entity] Template loop detected while processing event: <Event state_changed[L]: entity_id=sensor.current_consumption, old_state=<state sensor.current_consumption=207.0; unit_of_measurement=W, friendly_name=Current Power, icon=mdi:transmission-tower @ 2020-12-16T10:26:20.723463+01:00>, new_state=<state sensor.current_consumption=191.0; unit_of_measurement=W, friendly_name=Current Power, icon=mdi:transmission-tower @ 2020-12-16T10:26:21.725318+01:00>>, skipping template render for Template[{% set ns = namespace(states=[]) %} {% for s in states.sensor if s.entity_id.startswith('sensor.') and s.object_id.endswith('_power') and s.state not in ['unavailable', 'unknown' ] %}
  {% set ns.states = ns.states + [ s.state | float ] %}
{% endfor %} {{ (ns.states | sum) | round(2) }}]

I have checked and it is not possible to add attributes for esphome sensors.

[petro]

If you loop through all the sensors, regardless of filters applied, that message has a chance to appear.

@Tomaae

WARNING

That means it’s not a error. You can safely ignore it, it’s just letting you know that you have a template that might reference itself. If you want it out of your logs, remove it by configuring logger to ignore warnings for the template sensor integration.

[123 (Taras)]

@Tomaae

not possible to add attributes for esphome sensors

It’s possible to add custom attributes to Home Assistant’s entities by defining them in customize.yaml or via the menu: Configuration > Customizations. You can add whatever extra information you want to each entity.

Customizing Entities

However, the warning message may be due to the template’s reliance on states.sensor (which will automatically include sensor.current_consumption). My suggestion to use custom attributes still relies on using states.sensor so I now doubt it would prevent the warning.

The following suggestion avoids the use states.sensor. However, it requires that you create a group containing all of the power-related sensors. The Template Sensor is reduced to this:

- platform: template
  sensors:
    current_consumption:
      friendly_name: Current Power
      unit_of_measurement: W
      icon_template: mdi:transmission-tower
      value_template: >
        {{ expand('group.my_sensors') | rejectattr('state', 'in', ['unavailable', 'unknown'])
            | map(attribute='state') | map('float') | sum | round(2) }}

[Tomaae]

@123

rejectattr

Thanks, thats probably only option I have.
Could you point me to the documentation for rejectattr and other such commands used with expand? I’m not able to find it.
I would like to learn additional options. Also I would like to know if it is possible to add additional filter similar to endswith using this method.

Removing warnings globally for all template sensors is not an option. I need them in case I mess up anything in the future.
I’m using HA for over a year now, but I still keep discovering new things and new ways to do things almost weekly, so I tinker with it a lot.
It sort of became my hobby, from configuration to creating integrations

[petro]

@Tomaae

Could you point me to the documentation for rejectattr and other such commands used with expand?

This is Jinja, it’s not something that is specific to Home Assistant. I suggest google Jinja, here’s a reference to built in jinja filters.

@Tomaae

Also I would like to know if it is possible to add additional filter similar to endswith using this method.

It is not possible. EDIT: Seeing that you know how to code, an easier solution would be to add a new filter thats similar to rejectattr/selectattr that accepts startswith/endswith. Adding a test to it would require you to override the whole method.

[Tomaae]

Ah, I see. I have much to learn

Here is what I’m testing now:
Group:

energy:
    name: Energy
    entities:
        - sensor.dac_daily_energy
        - sensor.dac_power
        - sensor.dell_monitor_daily_energy
        - sensor.dell_monitor_power
        - sensor.eizo_monitor_daily_energy
        - sensor.eizo_monitor_power
        - sensor.grow_light_5_daily_energy
        - sensor.grow_light_5_power
        - sensor.ha_daily_energy
        - sensor.ha_power
        - sensor.headphones_daily_energy
        - sensor.headphones_power
        - sensor.hp_monitor_daily_energy
        - sensor.hp_monitor_power
        - sensor.marantz_daily_energy
        - sensor.marantz_power
        - sensor.nas_daily_energy
        - sensor.nas_power
        - sensor.nc_gw1_daily_energy
        - sensor.nc_gw1_power
        - sensor.pc_daily_energy
        - sensor.pc_power
        - sensor.prusa_daily_energy
        - sensor.prusa_power
        - sensor.usb_daily_energy
        - sensor.usb_power

Sensors:

sensor:
- platform: template
  sensors:
    energy_total:
      friendly_name: Total Energy
      unit_of_measurement: kWh
      icon_template: mdi:calendar-today
      value_template: >
        {{ expand('group.energy') 
        | selectattr('attributes.unit_of_measurement', 'equalto', 'kWh')
        | rejectattr('state', 'in', ['unavailable', 'unknown'])
        | map(attribute='state') | map('float') | sum | round(2) }}
          
- platform: template
  sensors:
    current_consumption:
      friendly_name: Current Power
      unit_of_measurement: W
      icon_template: mdi:transmission-tower
      value_template: >
        {{ expand('group.energy') 
        | selectattr('attributes.unit_of_measurement', 'equalto', 'W')
        | rejectattr('state', 'in', ['unavailable', 'unknown'])
        | map(attribute='state') | map('float') | sum | round(2) }}

utility_meter:
  daily_energy:
    source: sensor.energy_total
    cycle: daily

  monthly_energy:
    source: sensor.energy_total
    cycle: monthly

I just didnt wanted to create 2 groups if it was not necessary. It will be easier to maintain this way.
It is unfortunate that I need to use groups to avoid that warning spam. I would prefer for it to be fully automatic, as I keep adding and removing those esp devices a lot.

[petro]

if you want to be super lazy with your group. Create an automation that fires at startup, iterates your sensors and creates the group with all entities that end in ‘energy’. You could even have it execute on template reload.

[Tomaae]

I can have an automation to create and update a group? that would actually solve the issue, if I could have group update like every hour.
I have to research this.
Having dynamic groups could be really powerful tool.

[123 (Taras)]

@Tomaae

automation to create and update a group

So what do you envision you will use in the automation’s Template Trigger? Obviously something that monitors all sensors to check for new ones, like states.sensor … (EDIT: Just joking. No need to monitor all sensors. It’s highly unlikely that new sensors are being created so frequently.)

The very end of the first post in this thread demonstrates how to use a python_script to dynamically generate a group.

PSA: Turn on/off all lights in Home Assistant 0.104+ (group.all_* changes)

Seeing that group.all_* was removed from states, here are some useful canned items that will get you the same result as before. Using Services to perform an action on all entities Using all inside the entity_id field will allow you to use turn everything on or off inside a domain. You just need to pair it with the correct command. This should work for pretty much every service that has entity_id. Replaces # Do not use, this is out of date. - service: homeassistant.turn_on entity_id: grou…

[petro]

no reason to have it update every hour. Just have it update when things reload or you restart. Every hour is overkill.

[123 (Taras)]

@Tomaae

I keep adding and removing those esp devices a lot.

What is the use-case that causes such frequent additions/removals of sensor entities?