Trying to clean json list

duceduc · November 21, 2020, 3:24am

I am trying to unique, sort this raw json. I almost got it but it is deleting my , separators at the end.

template editor:

{% set value =
{"id":"e79341ccf8ea","address":"e7:93:41:cc:f8:ea","rssi":-51,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":87}},{"id":"f4f839cd40fd","address":"f4:f8:39:cd:40:fd","rssi":-43,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":97}},{"id":"e79341ccf8ea","address":"e7:93:41:cc:f8:ea","rssi":-50,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":87}},{"id":"c289d47408fd","address":"c2:89:d4:74:08:fd","rssi":-79,"serviceData":{"model":"T","modelName":"WoSensorTH","temperature":{"c":20.9,"f":69.6},"fahrenheit":false,"humidity":47,"battery":100}},{"id":"f4f839cd40fd","address":"f4:f8:39:cd:40:fd","rssi":-43,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":97}},{"id":"f4f839cd40fd","address":"f4:f8:39:cd:40:fd","rssi":-44,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":97}},{"id":"f4f839cd40fd","address":"f4:f8:39:cd:40:fd","rssi":-42,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":97}},{"id":"e79341ccf8ea","address":"e7:93:41:cc:f8:ea","rssi":-51,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":87}},{"id":"f4f839cd40fd","address":"f4:f8:39:cd:40:fd","rssi":-45,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":97}},{"id":"e79341ccf8ea","address":"e7:93:41:cc:f8:ea","rssi":-50,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":87}},{"id":"e79341ccf8ea","address":"e7:93:41:cc:f8:ea","rssi":-51,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":87}},
%}

{% for devices in value | unique(attribute='id') | sort(attribute='id') %}
{{ devices }}
{%- endfor %}

results:

{'id': 'c289d47408fd', 'address': 'c2:89:d4:74:08:fd', 'rssi': -79, 'serviceData': {'model': 'T', 'modelName': 'WoSensorTH', 'temperature': {'c': 20.9, 'f': 69.6}, 'fahrenheit': False, 'humidity': 47, 'battery': 100}}
{'id': 'e79341ccf8ea', 'address': 'e7:93:41:cc:f8:ea', 'rssi': -51, 'serviceData': {'model': 'H', 'modelName': 'WoHand', 'mode': False, 'state': False, 'battery': 87}}
{'id': 'f4f839cd40fd', 'address': 'f4:f8:39:cd:40:fd', 'rssi': -43, 'serviceData': {'model': 'H', 'modelName': 'WoHand', 'mode': False, 'state': False, 'battery': 97}}

123 · November 21, 2020, 5:12am

It’s not “deleting” them, it’s doing exactly what you asked it to do. The only thing that gets printed is each item containing the id attribute.

If given a, b, c, d, e and you ask to ‘print the letters’ it won’t print the commas.

Are you planning to use this in a Template Sensor or is this just something that you’re experimenting with in the Template Editor?

duceduc · November 21, 2020, 5:14am

I see. Yes, I am planning to use the Rest platform following this sample.

123 · November 21, 2020, 5:30am

If you’re using version 0.117.X or lower, this will work:

{{ value | unique(attribute='id') | sort(attribute='id') | join(', ')  }}

If you are using 0.118, it will produce something a bit different. In 0.118 templates can produce values whose type is not only string but can be list, int, float, dict, etc. Because of that, the same template using in 0.118 will generate a list. That means you will see all three results within [ ] because it’s considered to be a list.

I have no idea how to prevent it other than to avoid using a comma (that’s the symbol normally used to separate items in a list) and use something like a semi-colon (which is never used to separate items in a list).

{{ value | unique(attribute='id') | sort(attribute='id') | join('; ')  }}

duceduc · November 21, 2020, 5:56am

{{ value | unique(attribute=‘id’) | sort(attribute=‘id’) | join(', ') }}

Yes, I have tried that and it produced a long vertical list with spaces in between each characters. I’ve reformatted it to have it shown in horizontal view. Something like this.

{ ' i d ' : ' c 2 8 9 d 4 7 4 0 8 f d ' , ' a d d r e s s ' : ' c 2 : 8 9 : d 4 : 7 4 : 0 8 : f d ' , ' r s s i ' : - 8 7 , ' s e r v i c e D a t a ' : { ' m o d e l ' : ' T ' , ' m o d e l N a m e ' : ' W o S e n s o r T H ' , ' t e m p e r a t u r e ' : { ' c ' : 2 0 . 8 , ' f ' : 6 9 . 4 } , ' f a h r e n h e i t ' : F a l s e , ' h u m i d i t y ' : 4 9 , ' b a t t e r y ' : 1 0 0 } } , { ' i d ' : ' e 7 9 3 4 1 c c f 8 e a ' , ' a d d r e s s ' : ' e 7 : 9 3 : 4 1 : c c : f 8 : e a ' , ' r s s i ' : - 7 0 , ' s e r v i c e D a t a ' : { ' m o d e l ' : ' H ' , ' m o d e l N a m e ' : ' W o H a n d ' , ' m o d e ' : F a l s e , ' s t a t e ' : F a l s e , ' b a t t e r y ' : 8 7 } } , { ' i d ' : ' f 4 f 8 3 9 c d 4 0 f d ' , ' a d d r e s s ' : ' f 4 : f 8 : 3 9 : c d : 4 0 : f d ' , ' r s s i ' : - 4 2 , ' s e r v i c e D a t a ' : { ' m o d e l ' : ' H ' , ' m o d e l N a m e ' : ' W o H a n d ' , ' m o d e ' : F a l s e , ' s t a t e ' : F a l s e , ' b a t t e r y ' : 9 7 } }

I’ve tried using it to grab data from it, but getting this error.

TemplateSyntaxError: expected token ‘,’, got ‘integer’

I am on V 117.0

Maybe it will work if I just use as is. I was copying and pasting to the template editor and maybe it is adding the spaces unintentionally?

123 · November 21, 2020, 6:10am

I’m not seeing that behavior in 0.117.5.

duceduc · November 21, 2020, 6:15am

Ah. I was using the for loop.

This works.

{{ value | unique(attribute=‘id’) | sort(attribute=‘id’) | join(', ') }}

123 · November 21, 2020, 2:00pm

Don’t forget that it will work differently in 0.118 (for the reason I mentioned above).

Here’s the same template’s output in 0.118. Take note of the leading [ indicating the result is now a list whereas in 0.117 it’s a string.

EDIT

I just learned that this behavior in 0.118’s Template Editor may be due to a bug.

github.com/home-assistant/core

Templates: Round is no longer honored on whole numbers. e.g. 1.0, 2.0, etc.

opened 04:32PM - 19 Nov 20 UTC

closed 12:21AM - 10 Jun 21 UTC

Petro31

templates

## The problem  Rounding is no longer honored as a result of the new template typing. All whole numbers will be truncated to an int. ex: ``` {{ '1.1' | round(1) }} # returns 1.1 {{ '1.0' | round(1) }} # returns 1 {{ '1.1' | float | round(1) }} # returns 1.1 {{ '1.0' | float | round(1) }} # returns 1 ``` After follwing the code in the forgiving_round method, it appears as if the bug is beyond that. I would assume that it's coming from the new template changes because the values that leave forgiving_round contain the floating point decimals. ## Environment  - Home Assistant Core release with the issue: 0.118 - Last working Home Assistant Core release (if known): 0.116.4 - Operating environment (OS/Container/Supervised/Core): NA - Integration causing this issue: template - Link to integration documentation on our website: NA

If I add one more line of text after the template, the Template Editor reports the same result as it did in 0.117 (result is a string, not a list).

duceduc · November 28, 2020, 9:21am

Now that I got it sorted and omit duplicates base on it’s id. The next step is to create a sensor and grab details from it.
I have tried creating a rest sensor, but HA does not register the sensor entity. I guess the code isn’t correctly set. The template sensor entities could be seen though.

{'id': 'c289d47408fd', 'address': 'c2:89:d4:74:08:fd', 'rssi': -76, 'serviceData': {'model': 'T', 'modelName': 'WoSensorTH', 'temperature': {'c': 17.4, 'f': 63.3}, 'fahrenheit': False, 'humidity': 62, 'battery': 100}}, {'id': 'c5c6df9448bc', 'address': 'c5:c6:df:94:48:bc', 'rssi': -58, 'serviceData': {'model': 'c', 'modelName': 'WoCurtain', 'calibration': True, 'battery': 100, 'position': 1, 'lightLevel': 1}}, {'id': 'e79341ccf8ea', 'address': 'e7:93:41:cc:f8:ea', 'rssi': -61, 'serviceData': {'model': 'H', 'modelName': 'WoHand', 'mode': False, 'state': False, 'battery': 97}}, {'id': 'f4f839cd40fd', 'address': 'f4:f8:39:cd:40:fd', 'rssi': -43, 'serviceData': {'model': 'H', 'modelName': 'WoHand', 'mode': False, 'state': False, 'battery': 95}}

Sensor:

sensor: 
  - platform: rest
    name: Switchbot Devices
    resource: http://192.168.1.30:5003
    scan_interval: 300
    value_template: >
      {% set devices = value_json | unique(attribute='id') | sort(attribute='id') | join(', ') %}
      {{ devices }}

  - platform: template
    sensors:
      hall_temperature:
        value_template: '{{ states.sensor.switchbot_devices[0].serviceData.temperature.c }}'
        device_class: temperature
        unit_of_measurement: '°C'
        friendly_name: 'Hall Temperature'
      hall_battery:
        value_template: '{{ states.sensor.switchbot_devices[0].serviceData.battery }}'
        device_class: battery
        unit_of_measurement: '%'
        friendly_name: 'Hall Battery'
      curtain_cr_left_battery:
        value_template: '{{ states.sensor.switchbot_devices[1].serviceData.battery }}'
        device_class: battery
        unit_of_measurement: '%'
        friendly_name: 'Curtain CR Left Battery'

123 · November 28, 2020, 3:48pm

The state value of all entities is always a string. It doesn’t matter if it’s appearance is numeric, list, or dict, it is handled as a string.

The state value of sensor.switchbot_devices is a string but your Template Sensors are attempting to use it like a dict:

states.sensor.switchbot_devices [0].serviceData.temperature.c