Parse an malformed JSON string

Hi!

I’m trying to create a template sensor based on a MQTT server topic.

It contains a malformed JSON string, like the one below:

{"SerialReceived":"{"circ":[0,0,0,-9,-9,-9,-9,-9,-9],"circdm":[-9,-9,-9],"ventdm":[-9],"vent":[0],"DHT":[25,86.5,0,0,26.16333]}"}

With this sensor I can extract the valid JSON part:

sensor:
  - platform: mqtt
    name: "result_serial"
    state_topic: "tele/sonoff/RESULT"
    value_template: '{{ value|regex_replace(find="{\"SerialReceived\":\"", replace="",ignorecase=False)|regex_replace(find="\"}", replace="",ignorecase=False) }}'

The problem is that sensor.result_serial is string, not a JSON, for HASS. How can I convert this new string to a JSON? I’ve tried {{ value_json|… in place of {{ value|… but it seems that the malformed string should be fixed before converting it.

PD: the whole part is an Arduino Nano that prints on serial the “good” JSON part. A ESP-01 with Sonoff Tasmota firmware acts like a serial MQTT bridge, receiving the JSON on serial and sending the string on a MQTT topic, adding “SerialReceived” on the beginning, but breaking the JSON.

Thanks in advance!

Why not just use an appropriate regex pattern to extract the value you want from the payload?

For example, this will extract -9 from ventdm

{{ value | regex_findall_index('\"ventdm\":\[(-?\d+)\]') }}

Thanks a lot! This solve the problem, because AFAIK a string can’t be converted to dictionary inside value_template.

I’ve managed to extract 26.16333 of DHT tag with:

value | 
    regex_findall_index('\"DHT\":(\[.*?\])') | 
    regex_findall_index('\[(.*)\]') | 
    regex_findall_index('(-?\d+\.?\d*),?', 4)

I didn’t figure how to nest those regex on a single regex_findall_index call.

Cheers!

This regex pattern will match a positive or negative number, integer or floating point, with up to 5 decimal places.

-?\d+\.?\d{0,5}

To ensure it only extracts the last number in the DHT list, we use the following pattern:

DHT\":\[.+,(-?\d+\.?\d{0,5})\]

Here’s the resulting Jinja2 template:

{{ value | regex_findall_index('DHT\":\[.+,(-?\d+\.?\d{0,5})\]') }}

If you know the number can potentially have more than 5 decimal places, adjust the 5 in {0,5} to the appropriate value.

For future reference, this is a very handy tool for experimenting with regex patterns: