Help with MQTT sensor and attributes

MassiPi · June 18, 2023, 10:10am

Hello,
sadly i need some more help with MQTT sensors
This is an example of my mqtt data (as you can see, i have a json inside a json field)

{"timestamp": 1686941544.2616673, 
"data": 
	"{\"1\": 
		{\"roomId\": 1, \"roomName\": \"Sala\", \"actualTemp\": 27.6, \"setTemp\": 20.0, \"valveAperture\": 0, \"status\": true}, 
	\"2\": 
		{\"roomId\": 2, \"roomName\": \"Cucina\", \"actualTemp\": 28.0, \"setTemp\": 19.0, \"valveAperture\": 0, \"status\": false}, 
	\"3\": 
		{\"roomId\": 3, \"roomName\": \"Bagnetto\", \"actualTemp\": 27.1, \"setTemp\": 19.0, \"valveAperture\": 0, \"status\": false},
	\"4\":
		{\"roomId\": 4, \"roomName\": \"Bagno\", \"actualTemp\": 27.2, \"setTemp\": 19.0, \"valveAperture\": 0, \"status\": false}, 
	\"5\": 
		{\"roomId\": 5, \"roomName\": \"Camera\", \"actualTemp\": 27.4, \"setTemp\": 19.5, \"valveAperture\": 0, \"status\": false},
	\"6\": 
		{\"roomId\": 6, \"roomName\": \"Studio\", \"actualTemp\": 27.6, \"setTemp\": 19.0, \"valveAperture\": 0, \"status\": false}}
	"}

Now i’d like to create 6 sensors, one for each room, where the value is the actual temp and as attributes i’d have valve aperture, target temp and the status.

I tried with something like that:

#temperatures
    - name: "Temperature Room 1"
      state_topic: massi/sensors/roomTemp
      value_template: "{{ value_json.data[1].actualTemp }}"
      suggested_display_precision: 1
      unit_of_measurement: "°C"
      json_attributes_topic: massi/sensors/roomTemp
      json_attributes_template: >
        { "targetTemp": {{ value_json.data[1].setTemp }},
          "valveAperture": {{ value_json.data[1].valveAperture }},
          "statusError": {{ value_json.data[1].status }}
        }

obviously nothing works The point - i think - is that the nested json is not parsed. Indeed i get this error:

Template variable warning: 'str object' has no attribute 'actualTemp' when rendering '{{ value_json.data[1].actualTemp }}'

but i’ve not understood how to get this parsed.
I’d also like to add:

how can i set the unit of measurement of attributes?
are also attributes logged and graph-ed?
do you see anything very very wrong in what i’m trying to do?

Thanks

koying · June 18, 2023, 10:30am

As far as HA is concerned, everything under “data” is one big plain string, not json, so you cannot use json parsing here.
Maybe some clever jinja substring will do, but it will be tricky.

petro · June 18, 2023, 10:34am

Odd that your device would do that. But this will work:

    - name: "Temperature Room 1"
      state_topic: massi/sensors/roomTemp
      value_template: "{{ value_json.data[1].actualTemp }}"
      suggested_display_precision: 1
      unit_of_measurement: "°C"
      json_attributes_topic: massi/sensors/roomTemp
      json_attributes_template: >
        {% set info = value_json.data[1] | from_json %}
        { "targetTemp": {{ info.setTemp }},
          "valveAperture": {{ info.valveAperture }},
          "statusError": {{ info.status }}
        }

koying · June 18, 2023, 10:41am

Nice.

Likely

    - name: "Temperature Room 1"
      state_topic: massi/sensors/roomTemp
      value_template: >
        {% set info = value_json.data | from_json %}
        {{ info["1"].actualTemp }}
      suggested_display_precision: 1
      unit_of_measurement: "°C"
      json_attributes_topic: massi/sensors/roomTemp
      json_attributes_template: >
        {% set info = value_json.data | from_json %}
        { "targetTemp": {{ info["1"].setTemp }},
          "valveAperture": {{ info["1"].valveAperture }},
          "statusError": {{ info["1"].status }}
        }

or similar, though.
I actually have no clue on the scope of a jinja variable. Could “info” be reused in json_attributes_template as well?

petro · June 18, 2023, 10:44am

You can only reuse info inside each field. We are both wrong btw… after looking at his data (was focused on the json), this is the correct template:

      value_template: >
        {% set info = value_json.data | from_json %}
        {{ info.actualTemp }}
      json_attributes_template: >
        {% set info = value_json.data | from_json %}
        { "targetTemp": {{ info.setTemp }},
          "valveAperture": {{ info.valveAperture }},
          "statusError": {{ info.status }}
        }

koying · June 18, 2023, 10:49am

Mmm… Not sure, after from_json, “info” is

{
  "1": {
    "roomId": 1,
    "roomName": "Sala",
    "actualTemp": 27.6,
    "setTemp": 20.0,
    "valveAperture": 0,
    "status": true
  },
  "2": {
    "roomId": 2,
    "roomName": "Cucina",
    "actualTemp": 28.0,
    "setTemp": 19.0,
    "valveAperture": 0,
    "status": false
  },
  "3": {
    "roomId": 3,
    "roomName": "Bagnetto",
    "actualTemp": 27.1,
    "setTemp": 19.0,
    "valveAperture": 0,
    "status": false
  },
  "4": {
    "roomId": 4,
    "roomName": "Bagno",
    "actualTemp": 27.2,
    "setTemp": 19.0,
    "valveAperture": 0,
    "status": false
  },
  "5": {
    "roomId": 5,
    "roomName": "Camera",
    "actualTemp": 27.4,
    "setTemp": 19.5,
    "valveAperture": 0,
    "status": false
  },
  "6": {
    "roomId": 6,
    "roomName": "Studio",
    "actualTemp": 27.6,
    "setTemp": 19.0,
    "valveAperture": 0,
    "status": false
  }
}

petro · June 18, 2023, 10:50am

Ugh, need my morning wakeup juice

koying · June 18, 2023, 10:54am

OP, keep in mind the quotes in info["1"] are relevant.
info[1] would assume info is an array, which it isn’t.

MassiPi · June 18, 2023, 11:34am

koying:

    - name: "Temperature Room 1"
      state_topic: massi/sensors/roomTemp
      value_template: >
        {% set info = value_json.data | from_json %}
        {{ info["1"].actualTemp }}
      suggested_display_precision: 1
      unit_of_measurement: "°C"
      json_attributes_topic: massi/sensors/roomTemp
      json_attributes_template: >
        {% set info = value_json.data | from_json %}
        { "targetTemp": {{ info["1"].setTemp }},
          "valveAperture": {{ info["1"].valveAperture }},
          "statusError": {{ info["1"].status }}
        }

guys you are amazing!
I can now get the temperature
However, attributes are not working since i get this error:

Erroneous JSON: { "targetTemp": 19.5, "valveAperture": 0, "statusError": True }

this is not the source json (it’s using attributes names as keys), i can’t understand this

Do you also know if i can set also a unit of measurement for each attribute?
Thanks!!

petro · June 18, 2023, 11:38am

whatever is outputting that is not formatting in JSON, so when attempts to translate it to json, it fails. It’s because json doesn’t understand True, only true. IMO you should fix whatever is populating that information to ensure it outputs proper json.

MassiPi · June 18, 2023, 11:45am

this is VERY strange, since the json comes from a python script generating it with

output['data'] = json.dumps(data)

afaik this manages correct capital letters, isn’t it?

petro · June 18, 2023, 11:47am

yes but you already have other oddities inside your data. Your whole setup is creating oddness. If that’s python code, post the whole thing.

MassiPi · June 18, 2023, 11:52am

you were right, the issue is about the boolean capital letter, it’s now working with a rough | lower

However, if i can’t graph attributes i loose one of my targets, so i fear i’ll have to setup different sensors.

Thanks for your help

For info, this is my python code generating oddness

                #first part
                output = {}
                data = {}
                output['timestamp'] = time.time()
                for id, room in rooms.items():
                        if id != 0:
                                data[id] = {}
                                data[id]['roomId'] = id
                                data[id]['roomName'] = room['Name']
                                data[id]['actualTemp'] = room['Heating']['TempNow']
                                data[id]['setTemp'] = room['Heating']['TempSet']
                                data[id]['valveAperture'] = room['Heating']['Aperture']
                                data[id]['status'] = False
                                for devices in room['Devices'].values():
                                        for device in devices.values():
                                                data[id]['status'] = data[id]['status']

                output['data'] = json.dumps(data)

                publish.single(....

petro · June 18, 2023, 12:12pm

well, that is odd code. That device for loop is doing… nothing. Can you post the room devices dictionary?

I’ll add it to this and you can use this instead:

                output = {'data':[], 'timestamp': time.time()}
                for id, room in rooms.items():
                    if id != 0:
                        output['data'].append({
                            'roomId': id,
                            'roomName': room['Name'],
                            'actualTemp': room['Heating']['TempNow']
                            'setTemp': room['Heating']['TempSet']
                            'valveAperture': room['Heating']['Aperture']
                            'status': False
                        })

                publish.single(json.dumps(output))

FYI after we make these changes, you’ll need to update your code in HA.

MassiPi · June 18, 2023, 12:35pm

no, that’s my fault, i copied the code from a putty session, so just took a piece of row, the code is checking a list of device boolean fields to set the status to true if something is wrong

for devices in room['Devices'].values():
					for device in devices.values():
						data[id]['status'] = data[id]['status'] or not device['Info']['InfoValidity'] or device['Info']['Error'] or device['Info']['BatteryLow'] or not device['Info']['LinkOk']

understood your code, makes sense to avoid the nested json

petro · June 18, 2023, 12:37pm

Ok, so… Personally, I’d just keep room zero, you don’t need to use it.

EDIT: I see wha tyou’re trying to do with status, this should do the same thing…

                output = {'data':[], 'timestamp': time.time()}
                for id, room in rooms.items():
                    statuses = []
                    for devices in room['Devices'].values():
                        for device in devices.values():
                            statuses.append(
                                not device['Info']['InfoValidity'] or device['Info']['Error'] or device['Info']['BatteryLow'] or not device['Info']['LinkOk']
                            )
                    output['data'].append({
                        'roomId': id,
                        'roomName': room['Name'],
                        'actualTemp': room['Heating']['TempNow'],
                        'setTemp': room['Heating']['TempSet'],
                        'valveAperture': room['Heating']['Aperture'],
                        'status': any(statuses),
                    })

                publish.single(json.dumps(output))

So now your mqtt should just be what you originally had.

    - name: "Temperature Room 1"
      state_topic: massi/sensors/roomTemp
      value_template: "{{ value_json.data[1].actualTemp }}"
      suggested_display_precision: 1
      unit_of_measurement: "°C"
      json_attributes_topic: massi/sensors/roomTemp
      json_attributes_template: >
        { "targetTemp": {{ value_json.data[1].setTemp }},
          "valveAperture": {{ value_json.data[1].valveAperture }},
          "statusError": {{ value_json.data[1].status }}
        }

petro · June 18, 2023, 12:51pm

Your other option is to break when it’s true…

                output = {'data':[], 'timestamp': time.time()}
                for id, room in rooms.items():
                    status = False
                    for devices in room['Devices'].values():
                        for device in devices.values():
                            if not device['Info']['InfoValidity'] or device['Info']['Error'] or device['Info']['BatteryLow'] or not device['Info']['LinkOk']:
                                status = True
                                break
                    output['data'].append({
                        'roomId': id,
                        'roomName': room['Name'],
                        'actualTemp': room['Heating']['TempNow'],
                        'setTemp': room['Heating']['TempSet'],
                        'valveAperture': room['Heating']['Aperture'],
                        'status': status,
                    })

                publish.single(json.dumps(output))