Trying to clean json list

I am trying to unique, sort this raw json. I almost got it but it is deleting my , separators at the end.

template editor:

{% set value =
{"id":"e79341ccf8ea","address":"e7:93:41:cc:f8:ea","rssi":-51,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":87}},{"id":"f4f839cd40fd","address":"f4:f8:39:cd:40:fd","rssi":-43,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":97}},{"id":"e79341ccf8ea","address":"e7:93:41:cc:f8:ea","rssi":-50,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":87}},{"id":"c289d47408fd","address":"c2:89:d4:74:08:fd","rssi":-79,"serviceData":{"model":"T","modelName":"WoSensorTH","temperature":{"c":20.9,"f":69.6},"fahrenheit":false,"humidity":47,"battery":100}},{"id":"f4f839cd40fd","address":"f4:f8:39:cd:40:fd","rssi":-43,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":97}},{"id":"f4f839cd40fd","address":"f4:f8:39:cd:40:fd","rssi":-44,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":97}},{"id":"f4f839cd40fd","address":"f4:f8:39:cd:40:fd","rssi":-42,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":97}},{"id":"e79341ccf8ea","address":"e7:93:41:cc:f8:ea","rssi":-51,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":87}},{"id":"f4f839cd40fd","address":"f4:f8:39:cd:40:fd","rssi":-45,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":97}},{"id":"e79341ccf8ea","address":"e7:93:41:cc:f8:ea","rssi":-50,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":87}},{"id":"e79341ccf8ea","address":"e7:93:41:cc:f8:ea","rssi":-51,"serviceData":{"model":"H","modelName":"WoHand","mode":false,"state":false,"battery":87}},
%}

{% for devices in value | unique(attribute='id') | sort(attribute='id') %}
{{ devices }}
{%- endfor %}

results:

{'id': 'c289d47408fd', 'address': 'c2:89:d4:74:08:fd', 'rssi': -79, 'serviceData': {'model': 'T', 'modelName': 'WoSensorTH', 'temperature': {'c': 20.9, 'f': 69.6}, 'fahrenheit': False, 'humidity': 47, 'battery': 100}}
{'id': 'e79341ccf8ea', 'address': 'e7:93:41:cc:f8:ea', 'rssi': -51, 'serviceData': {'model': 'H', 'modelName': 'WoHand', 'mode': False, 'state': False, 'battery': 87}}
{'id': 'f4f839cd40fd', 'address': 'f4:f8:39:cd:40:fd', 'rssi': -43, 'serviceData': {'model': 'H', 'modelName': 'WoHand', 'mode': False, 'state': False, 'battery': 97}}

It’s not “deleting” them, it’s doing exactly what you asked it to do. The only thing that gets printed is each item containing the id attribute.

If given a, b, c, d, e and you ask to ‘print the letters’ it won’t print the commas.

Are you planning to use this in a Template Sensor or is this just something that you’re experimenting with in the Template Editor?

I see. Yes, I am planning to use the Rest platform following this sample.

If you’re using version 0.117.X or lower, this will work:

{{ value | unique(attribute='id') | sort(attribute='id') | join(', ')  }}

If you are using 0.118, it will produce something a bit different. In 0.118 templates can produce values whose type is not only string but can be list, int, float, dict, etc. Because of that, the same template using in 0.118 will generate a list. That means you will see all three results within [ ] because it’s considered to be a list.

I have no idea how to prevent it other than to avoid using a comma (that’s the symbol normally used to separate items in a list) and use something like a semi-colon (which is never used to separate items in a list). :man_shrugging:

{{ value | unique(attribute='id') | sort(attribute='id') | join('; ')  }}

{{ value | unique(attribute=‘id’) | sort(attribute=‘id’) | join(’, ') }}

Yes, I have tried that and it produced a long vertical list with spaces in between each characters. I’ve reformatted it to have it shown in horizontal view. Something like this.

{ ' i d ' : ' c 2 8 9 d 4 7 4 0 8 f d ' , ' a d d r e s s ' : ' c 2 : 8 9 : d 4 : 7 4 : 0 8 : f d ' , ' r s s i ' : - 8 7 , ' s e r v i c e D a t a ' : { ' m o d e l ' : ' T ' , ' m o d e l N a m e ' : ' W o S e n s o r T H ' , ' t e m p e r a t u r e ' : { ' c ' : 2 0 . 8 , ' f ' : 6 9 . 4 } , ' f a h r e n h e i t ' : F a l s e , ' h u m i d i t y ' : 4 9 , ' b a t t e r y ' : 1 0 0 } } , { ' i d ' : ' e 7 9 3 4 1 c c f 8 e a ' , ' a d d r e s s ' : ' e 7 : 9 3 : 4 1 : c c : f 8 : e a ' , ' r s s i ' : - 7 0 , ' s e r v i c e D a t a ' : { ' m o d e l ' : ' H ' , ' m o d e l N a m e ' : ' W o H a n d ' , ' m o d e ' : F a l s e , ' s t a t e ' : F a l s e , ' b a t t e r y ' : 8 7 } } , { ' i d ' : ' f 4 f 8 3 9 c d 4 0 f d ' , ' a d d r e s s ' : ' f 4 : f 8 : 3 9 : c d : 4 0 : f d ' , ' r s s i ' : - 4 2 , ' s e r v i c e D a t a ' : { ' m o d e l ' : ' H ' , ' m o d e l N a m e ' : ' W o H a n d ' , ' m o d e ' : F a l s e , ' s t a t e ' : F a l s e , ' b a t t e r y ' : 9 7 } }

I’ve tried using it to grab data from it, but getting this error.

TemplateSyntaxError: expected token ‘,’, got ‘integer’

I am on V 117.0

Maybe it will work if I just use as is. I was copying and pasting to the template editor and maybe it is adding the spaces unintentionally?

I’m not seeing that behavior in 0.117.5.

Ah. I was using the for loop.

This works.

{{ value | unique(attribute=‘id’) | sort(attribute=‘id’) | join(’, ') }}

Don’t forget that it will work differently in 0.118 (for the reason I mentioned above).

Here’s the same template’s output in 0.118. Take note of the leading [ indicating the result is now a list whereas in 0.117 it’s a string.


EDIT

I just learned that this behavior in 0.118’s Template Editor may be due to a bug.

If I add one more line of text after the template, the Template Editor reports the same result as it did in 0.117 (result is a string, not a list).

1 Like

Now that I got it sorted and omit duplicates base on it’s id. The next step is to create a sensor and grab details from it.
I have tried creating a rest sensor, but HA does not register the sensor entity. I guess the code isn’t correctly set. The template sensor entities could be seen though.

{'id': 'c289d47408fd', 'address': 'c2:89:d4:74:08:fd', 'rssi': -76, 'serviceData': {'model': 'T', 'modelName': 'WoSensorTH', 'temperature': {'c': 17.4, 'f': 63.3}, 'fahrenheit': False, 'humidity': 62, 'battery': 100}}, {'id': 'c5c6df9448bc', 'address': 'c5:c6:df:94:48:bc', 'rssi': -58, 'serviceData': {'model': 'c', 'modelName': 'WoCurtain', 'calibration': True, 'battery': 100, 'position': 1, 'lightLevel': 1}}, {'id': 'e79341ccf8ea', 'address': 'e7:93:41:cc:f8:ea', 'rssi': -61, 'serviceData': {'model': 'H', 'modelName': 'WoHand', 'mode': False, 'state': False, 'battery': 97}}, {'id': 'f4f839cd40fd', 'address': 'f4:f8:39:cd:40:fd', 'rssi': -43, 'serviceData': {'model': 'H', 'modelName': 'WoHand', 'mode': False, 'state': False, 'battery': 95}}

Sensor:

sensor: 
  - platform: rest
    name: Switchbot Devices
    resource: http://192.168.1.30:5003
    scan_interval: 300
    value_template: >
      {% set devices = value_json | unique(attribute='id') | sort(attribute='id') | join(', ') %}
      {{ devices }}

  - platform: template
    sensors:
      hall_temperature:
        value_template: '{{ states.sensor.switchbot_devices[0].serviceData.temperature.c }}'
        device_class: temperature
        unit_of_measurement: '°C'
        friendly_name: 'Hall Temperature'
      hall_battery:
        value_template: '{{ states.sensor.switchbot_devices[0].serviceData.battery }}'
        device_class: battery
        unit_of_measurement: '%'
        friendly_name: 'Hall Battery'
      curtain_cr_left_battery:
        value_template: '{{ states.sensor.switchbot_devices[1].serviceData.battery }}'
        device_class: battery
        unit_of_measurement: '%'
        friendly_name: 'Curtain CR Left Battery'   
      

The state value of all entities is always a string. It doesn’t matter if it’s appearance is numeric, list, or dict, it is handled as a string.

The state value of sensor.switchbot_devices is a string but your Template Sensors are attempting to use it like a dict:

states.sensor.switchbot_devices [0].serviceData.temperature.c