Template sensor: how to reduce a number of calculations? Tried to use a namespace as a state

[Ildar_Gabdullin]

There is a set of sensors:

• sensor.test_1
• sensor.test_2
• sensor.test_3

Each sensor has a set of attributes like:

- name: sensor.test_1:
  state: ...
  attributes:
    attr_1: ...
    attr_2: ...
    attr_3: ...

Now I need to have some composite “sensor.composite_123”:

- name: sensor.composite_123:
  state: ...
  attributes:
    comp_attr_1: ...
    comp_attr_2: ...

whose state & attributes depend on states & attributes of those 3 sensors.

Assume that here is an algorithm of calculating the composite state & composite attributes:

step 1 - some calculation
comp_attr_1: ...
step 2 - some calculation
state: ...
step 3 - some calculation
comp_attr_2: ...

Here is how I can split this algorithm when defining a template sensor:

- name: sensor.composite_123:
  state: >-
    step 1 - some calculation
    comp_attr_1: ...
    step 2 - some calculation
    {{ return result for the state }}

  attributes:
    comp_attr_1: >-
      step 1 - some calculation
      comp_attr_1: ...
      {{ return result for the comp_attr_1 }}

    comp_attr_2: >-
      step 1 - some calculation
      comp_attr_1: ...
      step 2 - some calculation
      state: ...
      step 3 - some calculation
      comp_attr_2: ...
      {{ return result for the comp_attr_2 }}

It seems cumbersome since there is a repeating code.

Tried another option - keeping results of the whole algorithm in some namespace:

- name: sensor.composite_123:
  state: >-
    step 1 - some calculation
    comp_attr_1: ...
    step 2 - some calculation
    state: ...
    step 3 - some calculation
    comp_attr_2: ...
    
    set some_ns = namespace(state,attr_1,attr_2)
    some_ns.state = calculated state
    some_ns.attr_1 = calculated comp_attr_1
    some_ns.attr_2 = calculated comp_attr_2

    {{ some_ns }}

  attributes:
    attr_state: >-
      {{ states("sensor.composite_123").state }}
    comp_attr_1: >-
      {{ states("sensor.composite_123").attr_1 }}
    comp_attr_2: >-
      {{ states("sensor.composite_123").attr_2 }}

The idea is to perform calculations once, then access results from a “composite” state.

So I created a sensor with this state:

<Namespace {'p_s': 'asuswrt', 'place': None, 'p_d': 2.0087780952453613, 'p_a': 'off', 'b_s': 'life360', 'b': '52', 'b_d': 610.0081288814545, 'b_a': 'off', 'bc_src': 'life360', 'bc': 'off', 'bc_d': 610.0081720352173, 'bc_a': 'off', 'p': 'home'}>

But - I cannot access its attributes:

I can convert this “namespace string” to a “dictionary-looking” string:

{% set ns = "<Namespace {'p_s': 'asuswrt', 'place': None, 'p_d': 2.0087780952453613, 'p_a': 'off', 'b_s': 'life360', 'b': '52', 'b_d': 610.0081288814545, 'b_a': 'off', 'bc_src': 'life360', 'bc': 'off', 'bc_d': 610.0081720352173, 'bc_a': 'off', 'p': 'home'}>"%}
{{ns}}
{% set dict = (ns|string).split("<Namespace ")[1].split(">")[0] %}
{{dict}}

But still this “dictionary-looking” string does not work as a dictionary - I cannot do any of these:

{{ dict['p'] }}
{{ dict[p] }}
{{ dict.p }}

Probably I should not use a namespace? May be I should create some kind of dictionary and return it?
Or may be there is some 3rd way?

Update:
Managed to extract an argument from that “dictionary-looking” string:

{% set dict_2 = dict.replace("'","\"") %}
dict_2 = {{dict_2}}

{{(dict_2|from_json).place}}

But these tricks seem like a hack; probably there is a better solution.

[Ildar_Gabdullin]

Anyway, there is a “255 chars maximum length” restriction for states.
Means - I cannot insert all required info into the result JSON string.
Of course I may use some “long_state” attribute which does not have this “255 chars” limit instead of a state.
But may be there is a better workaround?

[Didgeridrew]

It seems like you are trying to force a square peg into a round hole… Try something like:

- name: sensor.composite_123:
  state: >-
    {{ (this.attributes.state_attr if this is defined) | default('calculating state') }}
  attributes:
    namespace: >-
      step 1 - some calculation
      comp_attr_1: ...
      step 2 - some calculation
      state: ...
      step 3 - some calculation
      comp_attr_2: ...
    
      set some_ns = namespace(state,attr_1,attr_2)
      some_ns.state = calculated state
      some_ns.attr_1 = calculated comp_attr_1
      some_ns.attr_2 = calculated comp_attr_2
      {{ some_ns }}
    state_attr: >-
      {{ this.attributes.namespace.state }}
    comp_attr_1: >-
      {{ this.attributes.namespace.attr_1 }}
    comp_attr_2: >-
      {{ this.attributes.namespace.attr_2 }}

[Ildar_Gabdullin]

@Didgeridrew

It seems like you are trying to force a square peg into a round hole… Try something like:

Have tried it by yourself?
It does not work.

---

In short:

Sensors:

template:
  - sensor:
      - name: test_long_state_ns_0
        state: &ref_state_ns >-
          {% set ns = namespace(state=[],elevation=[]) -%}
          {%- set ns.state = states("sun.sun") -%}
          {%- set ns.elevation = state_attr("sun.sun","elevation") -%}
          {{ ns }}

      - name: test_long_state_ns_1
        state: *ref_state_ns
        attributes:
          attr_elevation: >-
            {% set LONG_STATE = states("sensor.test_long_state_ns_1") -%}
            {{ LONG_STATE.elevation }}

      - name: test_long_attr_ns_0
        state: "123"
        attributes:
          long_attr: &ref_attr_ns >-
            {% set ns = namespace(state=[],elevation=[]) -%}
            {%- set ns.state = states("sun.sun") -%}
            {%- set ns.elevation = state_attr("sun.sun","elevation") -%}
            {{ ns }}

      - name: test_long_attr_ns_1
        state: "123"
        attributes:
          long_attr: *ref_attr_ns
          attr_elevation: >-
            {% set LONG_ATTR = state_attr("sensor.test_long_attr_ns_1","long_attr") -%}
            {{ LONG_ATTR.elevation }}

Log:

2022-07-16 23:48:46 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'str object' has no attribute 'elevation' when rendering '{% set LONG_STATE = states("sensor.test_long_state_ns_1") -%} {{ LONG_STATE.elevation }}'
2022-07-16 23:48:46 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'str object' has no attribute 'elevation' when rendering '{% set LONG_ATTR = state_attr("sensor.test_long_attr_ns_1","long_attr") -%} {{ LONG_ATTR.elevation }}'

Created sensors:
accessing namespace in attributes - failed.

Dev tools with sensors w/o accessing a namespace:

Dev tools with attempt to access a namespace in attributes:

I have already said - accessing a namespace DOES NOT work - see my 1st post:

The only way which worked for a “long value as a state” case - is converting a namespace into json with some tricks (replacing ' with ", using from_json):

Check these similar sensors, the only difference is where this “namespace → json” conversion takes place:

      - name: test_long_state_ns_5
        state: >-
          {% set ns = namespace(state=[],elevation=[]) -%}
          {%- set ns.state = states("sun.sun") -%}
          {%- set ns.elevation = state_attr("sun.sun","elevation") -%}
          {{ ns }}
        attributes:
          attr_elevation: >-
            {% set LONG_STATE = states("sensor.test_long_state_ns_5") -%}
            {%- set dict = (LONG_STATE|string).split("<Namespace ")[1].split(">")[0] -%}
            {%- set dict = dict.replace("\u0027","\u0022") -%}
            {{ (dict|from_json).elevation }}

      - name: test_long_state_ns_6
        state: >-
          {% set ns = namespace(state=[],elevation=[]) -%}
          {%- set ns.state = states("sun.sun") -%}
          {%- set ns.elevation = state_attr("sun.sun","elevation") -%}
          {%- set dict = (ns|string).split("<Namespace ")[1].split(">")[0] -%}
          {%- set dict = dict.replace("\u0027","\u0022") -%}
          {{ dict }}
        attributes:
          attr_elevation: >-
            {% set LONG_STATE = states("sensor.test_long_state_ns_6") -%}
            {{ (LONG_STATE|from_json).elevation }}

But if the “long value” is stored as an attribute - there is a difference: no need to replace quotation marks and using from_json if that “namespace → json” conversion takes place in a “source attribute”:

      - name: test_long_attr_ns_5
        state: "123"
        attributes:
          long_attr: >-
            {% set ns = namespace(state=[],elevation=[]) -%}
            {%- set ns.state = states("sun.sun") -%}
            {%- set ns.elevation = state_attr("sun.sun","elevation") -%}
            {{ ns }}
          attr_elevation: >-
            {% set LONG_ATTR = state_attr("sensor.test_long_attr_ns_5","long_attr") -%}
            {%- set dict = (LONG_ATTR|string).split("<Namespace ")[1].split(">")[0] -%}
            {%- set dict = dict.replace("\u0027","\u0022") -%}
            {{ (dict|from_json).elevation }}

      - name: test_long_attr_ns_6
        state: "123"
        attributes:
          long_attr: >-
            {% set ns = namespace(state=[],elevation=[]) -%}
            {%- set ns.state = states("sun.sun") -%}
            {%- set ns.elevation = state_attr("sun.sun","elevation") -%}
            {%- set dict = (ns|string).split("<Namespace ")[1].split(">")[0] -%}
            {{ dict }}
          attr_elevation: >-
            {% set LONG_ATTR = state_attr("sensor.test_long_attr_ns_6","long_attr") -%}
            {{ LONG_ATTR.elevation }}

Oddly enough that the “source attribute” is presented as a dictionary (which does not happen when the “long value” is stored as a state) - see the 2nd sensor:

[Didgeridrew]

@Ildar_Gabdullin

Oddly enough that the “source attribute” is presented as a dictionary (which does not happen when the “long value” is stored as a state)

That is the normal behavior… States are always strings, attributes can be other data types.

[Ildar_Gabdullin]

@Didgeridrew

States are always strings, attributes can be other data types.

Thank you for your clarification! It really makes sense.
Also, it explains (I guess so) why in case of using “long value as an attribute” there is no need to use “from_json”.

But I am more interested about these issues:

1. In your 1st post you suggested to address namespaces directly w/o any conversion to dicts.
   This does not work.
   Am I doing something wrong or that was just your hypothetical proposal?

2. OK, in case of large repeating calculations we may perform them ONCE, create a namespace with results and then provide “shortcuts” to these results via attributes.
   Is it the only way?

[petro]

@Ildar_Gabdullin

In your 1st post you suggested to address namespaces directly w/o any conversion to dicts.
This does not work.
Am I doing something wrong or that was just your hypothetical proposal?

Why are you using namespace? Why arn’t you using a dict in the first place? These test sensors don’t make sense as a dict would solve them in 1 line. What’s the ACTUAL template that you’re using in your environment?

e.g. you’re doing this:

          long_attr: >-
            {% set ns = namespace(state=[],elevation=[]) -%}
            {%- set ns.state = states("sun.sun") -%}
            {%- set ns.elevation = state_attr("sun.sun","elevation") -%}
            {{ ns }}

when this is the same without needing to convert

          long_attr: >-
            {{ {'state': states("sun.sun"), 'elevation': state_attr("sun.sun","elevation") } }}

[Ildar_Gabdullin]

@petro

These test sensors don’t make sense as a dict would solve them in 1 line.

The initial problem is described in the 1st post.
All output values are calculated in some algorithm involving “for” loops - so, I needed namespaces to define values in loops.
The output of an algorithm is a set of values in a namespace.
Finally, I could place all values from the resulting namespace into output dictionary - and yes, this may be done in one line.
But to define values I needed namespaces…

[petro]

can you post the template that makes this?

@Ildar_Gabdullin

So I created a sensor with this state:

<Namespace {'p_s': 'asuswrt', 'place': None, 'p_d': 2.0087780952453613, 'p_a': 'off', 'b_s': 'life360', 'b': '52', 'b_d': 610.0081288814545, 'b_a': 'off', 'bc_src': 'life360', 'bc': 'off', 'bc_d': 610.0081720352173, 'bc_a': 'off', 'p': 'home'}>

I’ll change the code to make it output a dict while dynamically adding to it. Then you can play around in the future

[Ildar_Gabdullin]

@petro

Then you can play around in the future

Thank you very much, I will try to create a simplified example repeating my case in general.

[petro]

no need, just post the whole thing so I can get a general idea

I only need to see how you’re using namespace and how you add things to it

[Ildar_Gabdullin]

The code is too large…
I prepared this simplified example.

Assume there is an object (there are many of them, actually).
Info about this object is gathered from five sources (“source_1” … “source_5”).
Every source gives a set of parameters (“parameter_1” … “parameter_x”); some of them are received explicitly, some of them need to be calculated.

I need to:

• collect these parameters from each source;
• process them (+ using data from other sensors);
• calculate some output parameters (“calculated_parameter_1” … “calculated_parameter_x”);
• make these output parameters available via attributes.

template:
  - sensor:
      - name: ....
        ...
        attributes:
          long_state: >-
            ### this structure will keep results of the algorithm
            {% set nsLATEST = namespace(calculated_parameter_1,
                                        ...
                                        calculated_parameter_x) -%}

            ### this structure data will keep data received from all sources
            {%- set nsOBJECT_DATA = namespace(object_data=[]) -%}   

            ## sources  
            {%- set SOURCES = ['source_1', 'source_2', 'source_3', 'source_4', 'source_5'] -%}

            ### defining parameters for each source
            {%- for source in SOURCES -%}

              ### here we will collect values for the current source
              {%- set nsSOURCE_DATA = namespace(source=source,
                                                parameter_1="not_defined",
                                                parameter_2="not_defined",
                                                ...
                                                parameter_x="not_defined") -%}
          
              ## acquiring data from the current source
              {%- set SOURCE_SENSOR = "sensor." + OBJECT + "_" + source|lower -%}
              {%- if not states(SOURCE_SENSOR) in ["unavailable","unknown"] -%}
  
                ##### defining "parameter_1" for the current source
                ... calculations involving loops ...
                {%- set nsSOURCE_DATA.parameter_1 = ... -%}

                ....

                ##### defining "parameter_x" for the current source
                ... calculations involving loops ...
                {%- set nsSOURCE_DATA.parameter_1 = .... -%}
              {%- endif -%}
  
              ## add calculated values to the common stack
              {%- set nsOBJECT_DATA.object_data = nsOBJECT_DATA.object_data + [nsSOURCE_DATA] -%}
            {%- endfor -%}
  
            ### process gathered data collected in nsOBJECT_DATA
            .....
            ### put processed data into nsLATEST
            {%- set nsLATEST.calculated_parameter_1 = ... -%}
            ...
            {%- set nsLATEST.calculated_parameter_x = ... -%}

            ## convert namespace into dictionary
            {%- set dict = (nsLATEST|string).split("<Namespace ")[1].split(">")[0] -%}
            {{dict}}

          calculated_parameter_1: >-
            {% set STATE = states(this.entity_id) -%}
            {%- if not STATE in ["unavailable","unknown"] -%}
              {%- set LONG_STATE = state_attr(this.entity_id,"long_state") -%}
              {%- if LONG_STATE is not none -%}
                {{ LONG_STATE.calculated_parameter_1 }}
              {%- endif -%}
            {%- endif %}

          .....

          calculated_parameter_x: >-
            {% set STATE = states(this.entity_id) -%}
            {%- if not STATE in ["unavailable","unknown"] -%}
              {%- set LONG_STATE = state_attr(this.entity_id,"long_state") -%}
              {%- if LONG_STATE is not none -%}
                {{ LONG_STATE.calculated_parameter_x }}
              {%- endif -%}
            {%- endif %}

[petro]

here’s an example

   {%- set add_param1 = True %}
   {%- set add_param2 = False %}
   {%- set add_param3 = True %}
   {%- set OBJECT = 'a' %}
   {%- set SOURCES = ['source_1', 'source_2', 'source_3', 'source_4', 'source_5'] -%}
   {%- set ns = namespace(latest=[], data=[]) %}
   {% for source in SOURCES %}
     {% set SOURCE_SENSOR = "sensor." + OBJECT + "_" + source|lower -%}
     {% set ns.data = [('source', SOURCE_SENSOR)] %}
     {% if add_param1 %}
       {% set ns.data = ns.data + [('param1', 'somevalue1')] %}
     {% endif %}
     {% if add_param2 %}
       {% set ns.data = ns.data + [('param2', 'somevalue2')] %}
     {% endif %}
     {% if add_param3 %}
       {% set ns.data = ns.data + [('param3', 'somevalue3')] %}
     {% endif %}
     {% set ns.latest = ns.latest + [ dict.from_keys(ns.data) ] %}
   {% endfor %}
   {{ ns.latest }}

i used add_param to simulate the paramter looping.

The main points here are to use a single namespace, there’s no need for more that one… ever. At the begining of each loop, reset your ‘loop data’ by setting it equal to an empty list. Then, when adding things, make sure you add them as a key value pair. I.e. (‘key’, ‘value’) where ‘key’ is always a string and the value is any python type.

When adding the data to your “total object”, just add a dictionary.

[Ildar_Gabdullin]

Well, I need some time to learn it…
Thank you!
I’ll come back with a feedback.