Template sensor value to Dev Tools inconsistent

[lwsrbrts (Lewis Roberts)]

I have a template sensor which I use to obtain a list of select entities that match a specific state.

template:
  - sensor:
      - name: "SHP Circuits Manual Mode"
        unique_id: shp_circuits_manual_mode
        icon: "mdi:home-automation"
        state: >-
          {{ states.select
          | select('search', 'shp_circuit_\d+_mode')
          | map(attribute='entity_id')
          | select('is_state', 'Battery')
          | list
          }}

As you can see, this searches for all select entities with a name matching a regular expression. There are 10 circuits in total, the last one “10” is usually in an Off state. The available states of these selects are: Auto, Grid, Battery, Off.

In creating the template, I am only interested in obtaining the entities with the state of Battery.

The problem I’m having is that when I systematically go through the selects and change them to Battery, the sensor value (state) updates as normal…until I change the last circuit to Battery. The last select changed has no effect on the sensor. If I change the select back to Auto for example, it shouldn’t be included as per the sensor’s template, but if I change it to Battery, it should be included, because its state is Battery.

Why doesn’t the sensor update to include this circuit’s entity ID?

Obviously I have taken the template itself over to Developer Tools and performed the same tests there…and there, it works just as expected: I change the select to Auto, it’s removed from the list, I change it to Battery, it’s included in the list. Why is the sensor giving a different result?

I’ve tinkered with it endlessly, Dev Tools shows the correct results when I change the state of the select but the sensor omits the last select changed to Battery

I’ve tried:

• Trigger based sensor with the template
• Simplifying the template to use only select - same result.
• Using rejectattr instead - same result.
• Writing/rewriting the template in the sensor any number of ways - same result.

Dev Tools with the template works fine - when I change the state of the select, it’s represented in the Dev Tools output instantly. The sensor just doesn’t add the entity name of last changed select.

Dev Tools:

Actual states:

The sensor (circuit 7 is missing as that was the last one that I changed to Battery:

[petro]

How fast are you making these changes? Templates are throttled to at most 1 update per second when using states.<domain>. If you pump 10 changes in under a second, you’ll only see 1 update until the next second.

EDIT: If you want to avoid the 1 second throttle, this ridiculously stupid template would do that.

{{ range(1,11) | map('string') | map('slice', 1, '_mode') | map('sum', start=[]) | map('join') | map('reverse') | map('slice', 1, '_tiucric_phs.tceles') | map('sum', start=[]) | map('join') | map('reverse') | list }}

or you can use namespace to do the same thing

{% set ns = namespace(items=[]) %}
{% for i in range(1,11) %}
  {% set ns.items = ns.items + [ 'select.sph_circuit_' ~ i ~ '_mode' ] %}
{% endfor %}
{{ ns.items }}

but I believe 123 found the real problem which is that you’re hitting the character limit on the state.

[123 (Taras)]

The state value of any entity is limited to storing a maximum of 255 characters.

Attempting to store nine instances of ‘select.shp_circuit_X_mode’ exceeds that limit.

I suggest you store the list of select entities in the Template Sensor’s attributes. An attribute can store up to 16Kb and the value’s type isn’t limited to just string.

Let me know if you need help configuring the Template Sensor’s attributes option.

[walberjunior]

What if you delete part of the name?

 | replace("select.","") 

[lwsrbrts (Lewis Roberts)]

Thank you. I was genuinely hoping that the solution would be something very simple but somewhat non-obvious! I can’t say I’d realised there was a character limit on the state.

Thanks for the offer, I’m using the attributes already to generate some “friendly names” so I’ll re-work the template to be a binary sensor and use an attribute to store the entity names.

The hours I’ve spent on this

[lwsrbrts (Lewis Roberts)]

Thanks for replying to help me through this - as you mention, 123 seems to have hit the nail on the head.

To answer your question though, in terms of rate of change, I was only doing one every 5 or so seconds…usually how long it would take for me to navigate between dashboards and click to select the state I wanted, so it wasn’t quick by any means.

Thanks again and for the example template which I’m sure others may find handy!

[123 (Taras)]

Sure, that will serve to reduce the overall length of the stored state value. However if more select entities are ever added then the problem will reappear. Plus it assumes that the application can make do without a complete entity_id.

Generally speaking, it’s not recommended to store a string in an entity’s state if there’s a risk it can exceed the storage limit.

The other advantage of storing it as an attribute is that the value’s type, which is a list in this case, remains as a list. That makes it easier to access individual items. In contrast, state converts everything to a string, so the value may have the appearance of a list but it’s no longer a list.

[lwsrbrts (Lewis Roberts)]

This is what I’ve put together - frankly, the triggers probably aren’t needed since the select domain is already in scope but meh…it’s there.

template:
  - trigger:
      - platform: time_pattern
        minutes: "/5"
      - platform: state
        entity_id:
          - select.shp_circuit_1_mode
          - select.shp_circuit_2_mode
          - select.shp_circuit_3_mode
          - select.shp_circuit_4_mode
          - select.shp_circuit_5_mode
          - select.shp_circuit_6_mode
          - select.shp_circuit_7_mode
          - select.shp_circuit_8_mode
          - select.shp_circuit_9_mode
          - select.shp_circuit_10_mode
        not_to:
          - unavailable
          - unknown
    binary_sensor:
      - name: "SHP Circuits Manual Mode"
        unique_id: shp_circuits_manual_mode
        icon: "mdi:home-automation"
        state: >-
          {% set circuits = states.select
          | select('search', 'select.shp_circuit_\d+_mode')
          | map(attribute='entity_id')
          | reject('is_state', ['Auto', 'Off'])
          | list | count
          %}
          {{ circuits > 0 }}
        attributes:
          circuit_list: >-
            {{ states.select
            | select('search', 'select.shp_circuit_\d+_mode')
            | map(attribute='entity_id')
            | reject('is_state', ['Auto', 'Off'])
            | list
            }}
          friendly_names: >-
            {{ states.select
            | selectattr('entity_id', 'search', 'shp_circuit_\d+_mode')
            | rejectattr('state', 'in', ['Auto', 'Off'])
            | map(attribute='entity_id')
            | expand 
            | map(attribute='attributes.friendly_name')
            | list
            | sort
            | regex_replace(find='SHP ', replace='')
            | regex_replace(find=' Mode', replace='')
            }}

[petro]

you definitely want to optimize that last template.

            {{ states.select
            | selectattr('entity_id', 'search', 'shp_circuit_\d+_mode')
            | rejectattr('state', 'in', ['Auto', 'Off'])
            | map(attribute='attributes.friendly_name')
            | sort
            | regex_replace(find='SHP ', replace='')
            | regex_replace(find=' Mode', replace='')
            }}

You’re adding extra operations and you’re converting to alist in the middle as well as using expand when you already have the state objects. You want to keep it all generators if possible, which this template above does.

[lwsrbrts (Lewis Roberts)]

I knew there’d be a benefit to posting what I’d finalised on

Thank you for the suggestion - now changed in my template.