How to template MQTT sensor JSON as entity + attributes

Configuration
#1

Hello guys,
I know that there are many topics on templating subjects and believe me I went through a lot of them!
I have in my configuration.yaml an MQTT sensor:

mqtt:
  sensor:
# TXsoil sensors
    - name: "sms_txsoil1"
      state_topic: "sms_wf1_out_tx-o-txsoil1"
      value_template: '{{ value_json.Humidity }}'
      unit_of_measurement: '%'

The string sent by this sensor is:

{"B":94,"W":1,"Humidity":8,"Date":20220909,"Time":1900,"Name":TXsoil1,"Device":TXsoil,"H":"CE6408"}%

My goal would be to extract “Humidity” as entity and the rest of the values as attributes of this entity.

At the moment the only thing that I got with the code in the configuration.yaml is:
"unkwnown"
Since I was not even able to extract one value from the json string I stop there start searching for solutions and as I said couldn’t find anythng meaningful.

Can anybody help?

thanks

Luca

MQTT is Listening (I think) but not parsing as a sensor
#2

Luca,
No answer for you but offering moral support as I am stumped with a similar query and result.
Dan

#3

Thanks Dan! waiting for somebody more proficient!

#4

Just as a check, an MQTT topic is usually of the form xxx/yyy/zzz. It doesn’t have to be, but just wanted to double-check first to make sure that your topic was not the issue.

#5

Is this definitely what is posted on the MQTT topic, and you haven’t made any mistakes? Because the % at the end of this string of text, makes this invalid JSON which Home Assistant won’t be able to parse.

#6

So in addition to what @mobile.andrew.jones is saying, I noticed you had some invalid json inside the object that will cause errors.

Specifically

Those 2 fields, the results are unquoted strings which is not valid. That means value_json will not work for you.

You’ll have to use regex

      value_template: >
        {{ value | regex_findall('"Humidity":([0-9.]+),')[0] }}

EDIT: Also you have odd quoting that you have to overcome, so you’ll have to use multiline notation.

1 Like
#7

feedbacks clear… this is exactly the content of the message that I receive from the MQTT sensor. From my understanding there is a “%” simbol that is not in line with the JSON format as well as the unquoted strings. Thanks for your feedbacks, I’ll come back when I will have the JSON formt corrected.

#8

folks,
I got the MQTT payload corrected with all quotes (thanks Petro!)

{"B":92,"W":1,"Humidity":7,"Date":20220915,"Time":2023,"Name":"TXsoil1","Device":"TXsoil","H":"CE6407"}

And now the command below actually extracts the desired value (7) :

mqtt:
  sensor:
# TXsoil sensors
    - name: "sms_txsoil1"
      state_topic: "sms_wf1_out_tx-o-txsoil1"
      value_template: '{{ value_json.Humidity }}'
      unit_of_measurement: '%'

Now I am back to my original questions: how to extract from the MQTT payload “Humidity” as entity and the rest of the values as attributes of this entity.

thanks

#9 
mqtt:
  sensor:
# TXsoil sensors
    - name: "sms_txsoil1"
      state_topic: "sms_wf1_out_tx-o-txsoil1"
      value_template: '{{ value_json.Humidity }}'
      unit_of_measurement: '%'
      json_attributes_template: "{{ value_json | tojson }}"

Reference: JSON attributes template configuration

#10

This should identical. I.e. if it can be converted from json into value_json, then it’s already a json string, so there’s no reason to double convert

      json_attributes_template: "{{ value }}"
#11

Interestingly, the example in the documentation uses to_json even when the sample payload contains an obvious JSON dictionary. Its use in this situation seems redundant to me as well but I deferred to the example.

#12

huh, odd. :man_shrugging:

#13

ok, made it work:

mqtt:
  sensor:
# TXsoil sensors
    - name: "sms_txsoil1"
      state_topic: "sms_wf1_out_tx-o-txsoil1"
      value_template: '{{ value_json.Humidity }}'
      unit_of_measurement: '%'
      json_attributes_topic: "sms_wf1_out_tx-o-txsoil1"
      json_attributes_template: '{{ value_json | tojson }}'

actually suggestion from 123 was correct but you have to add this line:

      json_attributes_topic: "sms_wf1_out_tx-o-txsoil1"

Final result in the States tab is:

image
image1032×267 15.2 KB

So exactly what I was looking for.

Thanks to all and hope this topic will be useful

2 Likes
#14

Observations originally reported in this post cannot be duplicated so I have collapsed it (content retained for the curious).

Original post

Actually, you should not have to include json_attributes_topic. Or at least you did not have to include it in the past. If it was left out, the sensor would simply default to using the state_topic as the source for attributes.

However, its new behavior differs from the past and may due to a bug. Here’s what I mean:

I can confirm your observation that when the MQTT Sensor is first created, if you leave out json_attributes_topic it fails to create attributes from the payload received via the state_topic. Afterwards, if you add json_attributes_topic it will then create attributes.

With json_attributes_topic

image

image
image786×191 14.4 KB

However, I then removed json_attributes_topic (reloaded ‘Manually created MQTT entities’, as always, to load the new configuration) and the sensor continued to create attributes and report their correct values.

Without json_attributes_topic

image

Attributes are still created and updated with the new published values.

image
image781×195 14.7 KB

Just to confirm I wasn’t imagining things, I restarted Home Assistant, published a new payload and the sensor reported attributes … all without a json_attributes_topic. That’s the correct behavior it always had … until now when it fails to do this when the sensor is initially created.

I repeated the test to confirm my observations.

  1. I removed the MQTT Sensor’s configuration, reloaded, and it disappeared from the system.
  2. I added the MQTT Sensor’s configuration (without json_attributes_topic), reloaded, and it appeared (with no value or attributes).
  3. I published a payload and the sensor reported a value but no attributes.
  4. I added json_attributes_topic to the sensor’s configuration and reloaded.
  5. I published a new payload and the sensor reported a value and attributes.
  6. I removed json_attributes_topic from the sensor’s configuration and reloaded.
  7. I published a new payload and the sensor reported a value and attributes (i.e. without having a json_attributes_topic).

This new behavior suggests it’s a bug that occurs exclusively when the MQTT Sensor is initially created.

NOTE

The test was performed using 2022.8.7.

EDIT

Fixed typos.

#15
Original post

If you have a moment to spare, see if you can duplicate my results.

  • Confirm your existing sensor is currently reporting attribute values.
  • Remove the following line and then reload ‘Manually created MQTT entities’
      json_attributes_topic: "sms_wf1_out_tx-o-txsoil1"
  • Publish a new payload to sms_wf1_out_tx-o-txsoil1.
  • Confirm the sensor continues to report attribute values.

Let me know your results. If they are identical to mine, I will report this as a bug in the GitHub Core repository. In addition, let me know which version of Home Assistant you are using.

UPDATE

Well, oddly enough, I can no longer replicate my own results. :thinking: :man_shrugging: :man_facepalming:

I’ll have to chalk it up to a mistake in my testing procedure (where exactly, I don’t know) because I can no longer get it to work without including json_attributes_topic. Very strange because I swear it used to work without it in previous versions. Oh well.

#16

very strange…