New template function: natsort -> natural sorting of sensor values

Development
1

If you sort sensor values or attributes with the “sort” filter, they get sorted by their ASCII value. But if you have numbers, the result ist not what you want:

{{ ["5","1","31","23","9","22","11"]|sort }}

gives you [‘1’, ‘11’, ‘22’, ‘23’, ‘31’, ‘5’, ‘9’]. So if you sort for example battery sensors, the list is not in the order you want it. So I developed a “natsort” filter. “natsort” splits text and numbers and sorts the numbers separate from the text.

So

{{ ["5","1","31","23","9","22","11"]|natsort }}

gives you [‘1’, ‘5’, ‘9’, ‘11’, ‘22’, ‘23’, ‘31’].

Another example for testing: battery values:

{{ states.sensor| selectattr('attributes.device_class', 'defined')|selectattr('attributes.device_class', 'eq','battery')|natsort(attribute="state")|list|join("\n\n") }}

I would like to make a change request to helpers/template.py if this function is desired.
For this, it is also necessary that HA installs the natsort module (dependency). How is this achieved?

1 Like
2

That is not a list of numbers (integers). Those are strings.

This is simply fixed as follows:

{% set l = [55, 4, 33, 2, 11] %}
{{ l | sort }}
{% set l = ["55", "4", "33", "2" , "11"] %}
{{ l | sort }}
{{ l | map("int") | list | sort }}
3

Wow, amazing!
Now do the same magic with

{% set l = ["55%", "4%", "33%", "2%", "11%", "100%"] %}

(this is what you get if you fetch the list of battery sensors)

1 Like
4

Haha, nice challenge!

I think your suggestion is a good idea for this case. I’m simply showing this as a somewhat contrived alternative.

I had to jump through a few hoops for this one…

{% set l = ["55%", "4%", "33%", "2%", "11%", "100%"] %}
{{ l }}
{% set l_as_sorted_ints = l | map('replace', '%', '') | map("int") | list | sort %}
{{ l_as_sorted_ints }}
{% set l_in_json_obj = '{"l": ["' + (l_as_sorted_ints | join('%","')) + '%"]}' %}
{{ l_in_json_obj }}
{{ (l_in_json_obj | from_json)['l'] }}
{{ (l_in_json_obj | from_json)['l'][3] }}

Output:

['55%', '4%', '33%', '2%', '11%', '100%']
[2, 4, 11, 33, 55, 100]
{"l": ["2%","4%","11%","33%","55%","100%"]}
['2%', '4%', '11%', '33%', '55%', '100%']
33%

Neither Jinja2 nor HA’s templating engine has a way to split a string, hence the from_json trick.

5

Which integrations (that you’re using) include a percent symbol in their battery value?

The reason why I am asking is because none of the integrations I use do that.

6 
{{ ["55%", "4%", "33%", "2%", "11%", "100%"]
  | map('replace', '%', '')
  | map('int', 0)
  | sort
  | map('string')
  | map('regex_replace', '(\d+)', '\\1%')
  | list
}}
1 Like
7

Well done!
My Version:

{% set l = ["55%", "4%", "33%", "2%", "11%", "100%"] %}
{{ l|natsort }}

And now this list of battery levels without “natsort”:

{% for i in states.sensor|natsort(attribute="state")|map(attribute='entity_id') %}
{%- if state_attr(i,"device_class")=="battery"  and states(i)!='unavailable' and states(i)!='unknown' -%}
{{ states(i)|int(0) }}%  {{ state_attr(i,"friendly_name") }}
{% endif -%}
{% endfor %}
8

That master has spoken. Nice one!

1 Like
9

Which integrations append % to the battery level?

I just double-checked and confirmed every integration I have that reports battery level does so without appending a % symbol.

FWIW, I use the Auto Entities custom card to display a sorted list of names and levels of all entities whose device_class is battery. No template required.

10

i still haven’t figured out why my idea of natsort is so fiercely opposed. all strings that combine numbers and characters are extremely difficult to sort. e.g. “50 mph”.
Also, sort(attribute=“abc”) does not work correctly if the attribute contains numbers.

11

Welcome back! Do you have time now to answer the question that was posed two months ago?

1 Like
12

Sensor values are always strings, it does not matter if there is a “%” sign or not. The natsort algorithm is not specifically for the percent-sign and not specifically for battery values. If you sort battery values, “100” comes first, then “20”.

13

An entity’s state value is always a string, even when it’s numeric. In contrast, an entity’s attributes can contain a number, string, list, boolean, etc.

The example in your August 18th post shows a list of string values where each numeric string value includes a percent character. That would be unusual for Home Assistant because integrations typically do not include symbols, like percent, in a numeric string value.

BTW, I am not sure why you believe there’s fierce opposition to your proposal. Personally, I am indifferent to it, largely because I have rarely encountered the problem that natsort sets our to solve.

I encourage you to submit it as a PR in the Core repository. The development team is the final arbiter of what gets included in Home Assistant.

1 Like
14

This is an example where natsort can be used and there is no possibility to do the same without natsort:

states.sensor | natsort(attribute=“state”)

15

Why would one ever do this? This would be a mix of units and domains. If it’s just illustrative, it would be better to pick a better example.

I reread the topic up to that point, but I don’t get the same impression.

If it wasn’t clear: I’m not opposed. It’s just that the use case must be understood well — and it needs to be logged as an FR.

And this isn’t a good example either, since this isn’t normal. Integrations should split their state and unit of measurement. This is a specific problem that needs to be addressed with the owner of that integration, regardless of your proposal.

It does, because the solution I gave to this before is the accepted way of doing things: map to int or float, which again means you need to provide sufficient incentive for this to be implemented through your use cases.

1 Like
16

Ok, example again:

{% for i in states.sensor|natsort(attribute="state")|map(attribute='entity_id') %}
{%- if state_attr(i,"device_class")=="battery"  and states(i)!='unavailable' and states(i)!='unknown' -%}
{{ '100' if states(i)=="full" else states(i)|int(0) }} {{ state_attr(i,"friendly_name") }}
{% endif -%}
{% endfor %}

This gives you a list of battery levels from your sensors.
it looks like this:

22 _TZE200_hhrtiq0x TS0601 Battery
45 IKEA of Sweden FYRTUR block-out roller blind Battery
52 LUMI lumi.weather Battery
56 LUMI lumi.sensor_ht Battery
59 Lumi Weather 1 Battery
66 Aqara Wassersensor 3 Battery
70 Doorcontact Battery
70 SOS Button 1 Bed Battery
100 Tint_remote Battery

For this example, I remove the precent sign from the code to avoid endless discussions about the percent sign.