Extract MQTT Json payload values that're in square brackets

Vish2024 · January 19, 2025, 7:50pm

Hello,

I have this mqtt sensor with a payload that looks like this :

{
    "deduplicationId": "abcde",
    "time": "2025-01-17 23:22:42",
    "deviceInfo": {
        "tenantId": "afbfcfde",
        "tenantName": "xxxxabcde",
        "tags": {k}
    },
    "confirmed": false,
    "data": "01234abcde",
    "object": {
        "RO2_status": "OFF",
        "RO1_status": "ON"
    },
    "rxInfo": [
        {
            "uplinkId": 0123401234,
            "rssi": -65,
            "snr": 24,
            "channel": 2,
            "rfChain": 1,
            "metadata": {
                "region_config_id": "01234US",
                "region_common_name": "1234US"
            },
            "crcStatus": "CRC_OK"
        }
    ],
    "txInfo": {
        "frequency": 0123400,
        "modulation": {
            "lora": {
                "bandwidth": 125000,
                "spreadingFactor": 8,
                "codeRate": "CR_1_2"
            }
        }
    }
}

I’m looking to extract “RO_1 status” as state value. “time”, “rssi” and “snr” I want to extract as attributes.

Here’s my attempt :

mqtt
  sensor:
          - name: "ABXY"
            unique_id: 4
            state_topic: "event/sample"
            value_template: "{{ value_json.object.RO1_status}}"
            json_attributes_topic: "event/sample"
            json_attributes_template: >
               { "Time": "{{value_json.time}}",
                 "RSSI": {{value_json.rxInfo.rssi}},
                 "SNR": {{value_json.rxInfo.snr}} }

This unfortunately isn’t giving me anything. The sensor shows as unavailable. If I get rid of attributes’ template, I’m able to see the state value.

Any idea what I’m doing wrong? I’m guessing maybe the square brackets (the “rxInfo” section of the payload) might have something to do with this - not sure.

Thank you.

123 · January 20, 2025, 3:43am

rssi and snr are keys in a dictionary. The dictionary itself is the zeroth item of a list in the rxInfo key. So you need to indicate you want the zeroth item of rxInfo

mqtt:
  sensor:
    - name: "ABXY"
      unique_id: 4
      state_topic: "event/sample"
      value_template: "{{ value_json.object.RO1_status}}"
      json_attributes_topic: "event/sample"
      json_attributes_template: >
         { "Time": "{{value_json.time}}",
           "RSSI": {{value_json.rxInfo[0].rssi}},
           "SNR": {{value_json.rxInfo[0].snr}} }

Vish2024 · January 20, 2025, 6:15am

Thanks for the reply.

This didn’t work unfortunately. I can see the state value this time but not any of the attributes.

What’s interesting is when I do this :

mqtt:
  sensor:
    - name: "ABXY"
      unique_id: 4
      state_topic: "event/sample"
      value_template: "{{ value_json.object.RO1_status}}"
      json_attributes_topic: "event/sample"
      json_attributes_template: >
         { "RSSI": {{value_json.rxInfo[0].rssi}} }

ie. I’m calling upon just the one attribute here (rssi and not time and snr)
I see the state value and rssi as an attribute. When I add the comma and include other attributes, it doesn’t show.

Weird.

Troon · January 20, 2025, 8:17am

EDIT: removed due to Taras’ work below.

123 · January 20, 2025, 2:30pm

What’s also weird is when I tested the payload you posted in a JSON validator, it failed to pass. In particular,
the following is invalid:

Error: Parse error on line 7: ...", "tags": {k} }, "confi
                       ----------------------^ Expecting 'STRING', '}', got 'undefined'

It appears to be an incomplete attempt to define a dictionary. Do all payloads contain this error or just some of them?

In addition, zero-padded numbers like 012345 aren’t understood to be valid integers.

I don’t understand how the MQTT Sensor can report anything correctly when it receives invalid JSON.

Have you modified the posted payload (just for hiding some details from the public) or is that an actual unmodified payload?

123 · January 20, 2025, 2:47pm

What I suggested is in the same format as what I have suggested to other users (and tested myself) and it has been confirmed to work.

I agree it’s weird that it works when specifying just one attribute but not for two attributes. It shouldn’t make a difference and the technique has been successfully tested with far more than just two attributes.

As I mentioned earlier, the main problem was that your original template didn’t refer to the zeroth item in the list. Now all that remains is to determine why your MQTT Sensor misbehaves when supplied with a valid template.

BTW, you need to confirm the received payload is always valid JSON otherwise we’re chasing ghosts here.

EDIT

In addition, check the log for warnings and errors related to your MQTT Sensor.

Troon · January 20, 2025, 2:55pm

Without outer {{ }} brackets? Effectively building a string that looks like a dictionary rather than a Jinja template that returns one?

I’m surprised by that which is why I made my suggestion: I ass-u-med it might have been an omission on your part, recognising that is rare.

123 · January 20, 2025, 3:35pm

I just finished testing my example and it works.

I created two MQTT Sensors:

sensor.abxy1 uses the configuration I posted above.
sensor.abxy2 uses your configuration.

I fixed three JSON validation errors in the posted payload and then published it to event/sample.

Valid JSON payload example

{
    "deduplicationId": "abcde",
    "time": "2025-01-17 23:22:42",
    "deviceInfo": {
        "tenantId": "afbfcfde",
        "tenantName": "xxxxabcde",
        "tags": "k"
    },
    "confirmed": false,
    "data": "01234abcde",
    "object": {
        "RO2_status": "OFF",
        "RO1_status": "ON"
    },
    "rxInfo": [
        {
            "uplinkId": 123401234,
            "rssi": -65,
            "snr": 24,
            "channel": 2,
            "rfChain": 1,
            "metadata": {
                "region_config_id": "01234US",
                "region_common_name": "1234US"
            },
            "crcStatus": "CRC_OK"
        }
    ],
    "txInfo": {
        "frequency": 123400,
        "modulation": {
            "lora": {
                "bandwidth": 125000,
                "spreadingFactor": 8,
                "codeRate": "CR_1_2"
            }
        }
    }
}

Here are the results:

The first sensor works correctly and confirms what I suggested is valid; it reports state and three attributes.
The second sensor reports state but none of the attributes. It also reports the following warning in the Log:

tl;dr

When supplied with a valid JSON payload, the MQTT Sensor configuration I posted above works correctly. The resulting sensor displays state and three attributes (Time, RSSI, and SNR).

Troon · January 20, 2025, 5:44pm

Huh, that’s interesting. I wonder if the “erroneous JSON” is related to the Jinja template outputting single-quoted keys despite the doubles in the template?

Feels like a bug that a template that outputs a dict fails, whereas a string with embedded templates that looks like a dict passes…

I suspect my version might be able to be cajoled into working with to_json — might have a go later. That feels even more wrong though.

123 · January 20, 2025, 6:04pm

FWIW, this works (note the inclusion of to_json filter).

{{ { "Time": value_json['time'],
     "RSSI": value_json['rxInfo'][0]['rssi'],
     "SNR": value_json['rxInfo'][0]['snr'] } | to_json }}

So whether this way or the way I originally suggested, the resulting MQTT Sensor is able to have three attributes. It’s unclear why the OP fails to get positive results with a proven solution.

EDIT

To be clear, it doesn’t work with the payload the OP posted because it’s invalid JSON (contains three errors).

Troon · January 20, 2025, 6:08pm

Thanks for trying it — that means json_attributes_template needs to be a JSON string rather than a Jinja template, although it supports templates in its construction.

It’s an unusual feature, only (I think) found on MQTT. I seem to recall it being proposed and rejected on REST, where it would be useful for processing responses that aren’t dicts at their top level.

123 · January 20, 2025, 6:15pm

When json_attributes_template was first released (it was preceded by a far more limited way to get attributes; like in REST) I recall having to do a fair bit of experimentation to be able to extract values and rename keys. The example in the docs is kind of skimpy.

Anyways, at this point it’s up to the OP to explain why the posted payload is invalid and what are the related errors/warnings (if any) in the Log.

Vish2024 · January 20, 2025, 8:35pm

Hey Taras,

Yes I did change the mqtt payload in the example above to hide details. Apologies, I did not think it would cause mischief.

The specific error you mentioned :

Error: Parse error on line 7: ...", "tags": {k} }, "confi
                       ----------------------^ Expecting 'STRING', '}', got 'undefined'

in actuality, has no “k” in curly brackets. ie. it’s just :

"tags": {}

I tried what you suggested initially once again:

json_attributes_template: >
         { "Time": "{{value_json.time}}",
           "RSSI": {{value_json.rxInfo[0].rssi}},
           "SNR": {{value_json.rxInfo[0].snr}} }

And it’s working fine. It’s showing me the state value + all 3 attributes. I really have no idea why it didn’t work the first time.

This isn’t the first sensor that I’m seeing experience this issue.

I currently have another sensor (with a much smaller payload and without any square brackets) that’s giving me the exact same problem even though the json attributes template I’m using has the exact same format (only different variable names).

I’m thinking it’s probably something else and not to do with the syntax I’m using.

Anyways, thanks a lot for suggesting rxInfo[0] - that solves this for me.

Cheers.