Mqtt sensor - how to load attributes

Hello
I have a multiple sensors for shelly devices, each one representing state, battery, tilt etc.
I would like to have all those values in single sensors. Secondary values like battery or tilt could be places as attributes.

I’m struggling to achieve that.
how can I create such sensor, which automatically reads values from topic into attributes?

Lets’s be the code below a starting point:

  - platform: mqtt
    name: "PCroom Window"
    state_topic: "shellies/shellydw-pc/sensor/state"
    payload_on: "open"
    payload_off: "close"
    device_class: "window"
    json_attributes_topic: "shellies/shellydw-pc/sensor/"

I’ve added json_attributes_topic believing that it’s enough. but it’s not.

here is how the message structure looks like:

obrazek

Any help appreciated.

json_attributes_topic allows you to specify one MQTT topic that receives a payload containing a dictionary in JSON format. For example:

{"tilt": -1, "vibration": 0, "lux": 1838, "battery": 93}

It can readily convert that into attributes. If the JSON structure is more complicated, one can specify a template to extract the names and values using json_attributes_template.

In your case, there is no single topic containing all the desired attributes; each has its own topic.

1 Like

I though shellydw-pc/sensor is the topic containing all attributes (state, tilt, vibration, lux, battery). But obviously I was wrong.
How can I use json_attributes_template then? All documentation I found works with single attribute. I haven’t succeed so far applying it to my case.

thank you

The option you have is to create mqtt sensors for all the topics you want.

Is there the only option? As I wrote in OP I don’t need all those values to be tracked by sensors.
Actually I have them already as sensors but I found it overengeenered. instead of 15 sensors I have 45. it’s insufficient.
I’d like to have them as attributes, without need of creating sensors.

Here’s an example of using json_attributes_template.

If given this topic:

sensor/cpu

and this payload:

{
	"used_space": 25,
	"sys_clock_speed": 1500,
	"data": {
		"cpu_temp": 43.0,
		"voltage": 0.8500,
		"cpu_load": 1.25
	},
	"memory": "False",
	"swap": "False"
}

This MQTT Sensor extracts values from the payload and assigns them to attributes using custom names.

  - platform: mqtt
    name: monitor
    state_topic: sensor/cpu
    value_template: "{{ value_json.used_space }}"
    json_attributes_topic: sensor/cpu
    json_attributes_template: >
      { "speed": {{value_json.sys_clock_speed}},
        "temperature": {{value_json.data.cpu_temp}},
        "voltage": {{value_json.data.voltage}},
        "load": {{value_json.data.cpu_load}},
        "memory": "{{value_json.memory}}",
        "swap": "{{value_json.swap}}" }

If the our data you wish to use for attributes is not published to a single topic then json_attributes_topic and json_attributes_template probably can’t help you.

I say “probably” because there is one thing I have not tried. You can try setting json_attributes_topic to:

shellies/shellydw-pc/sensor/#

That’s a ‘wild card’ character and means to subscribe to all topics below shellies/shellydw-pc/sensor. If that is even allowed in this context, the next challenge is composing a template that understands what was just received and to which attribute it belongs to. The only thing it will receive (if it works) will be the raw data and won’t know if it belongs to tilt or lux or whatever.

6 Likes

I carried out an experiment and the results don’t bode well for your application.

I created this MQTT sensor:

  - platform: mqtt
    name: "test sensor1"
    state_topic: "test/sensor1"
    json_attributes_topic: "test/sensor1/+"
    json_attributes_template: >
      {"other_data":"{{value}}"}

Note the plus symbol at the end of json_attributes_topic. That means to subscribe to all sub-topics of test/sensor1 but not test/sensor1 itself.

  1. When I publish 22 to test/sensor1 it becomes the sensor’s state value.
  2. When I publish 150 to test/sensor1/whatever it becomes the value of an attribute named other_data.

Screenshot from 2020-07-18 16-28-44

The limitation is that if now publish hello to test/sensor1/something it will replace the existing value of other_data. The template has no means of determining which sub-topic received the payload. Without knowing the payload’s source, it can’t assign it to a different attribute. For what you want to do, that’s a showstopper.


NOTE

The only possible workaround I can think of is if the values for tilt, vibration, lux, and battery are distinctly different. For example, if the value of vibration can only be a negative number then any received negative numbers can be presumed to belong to vibration. However, I doubt that’s the case here. the value of lux can easily overlap with the permissible range of values for battery so there’s no uniqueness that can be used to differentiate the source of the data.

1 Like

Thank-you so much, this great write up coupled with using MQTTExplorer and I can finally use attributes.

I notice the double quotes in the second line, and then I am confused

It depends on whether you want the JSON value to be interpreted as a string or a number (or another type like boolean, list, etc).

Look at the example posted in the Syntax section of this Wikipedia entry for JSON. Notice how number values are not wrapped in quotes?

1 Like

I just went through a similar issue so documenting my results in this thread. MQTT sensors are only updated in home assistant when their associated topic is updated. In other words:

  • The sensor’s state is only updated when the state_topic is updated on the MQTT server
  • The sensor’s attributes are only updated when the json_attributes_topic is updated on the MQTT server
  • The sensor’s availability is only updated when the availability_topic is updated on the MQTT server
  • etc…

If you need multiple attributes, the best case scenario is that you have them all fed into a single MQTT topic in JSON format. You can’t collect information from different MQTT topics beyond what was mentioned above by @123 . However you can have templates reference whatever home assistant sensor or attribute you want. So if you have one home assistant sensor created for each MQTT topic, and you want to add a new sensor which contains all those other sensor states as attributes in your new “master” sensor, you could use something like the following:

  - platform: mqtt
    name: "master sensor"
    state_topic: "test/sensor1"
    json_attributes_topic: "test/sensor1/randomattribute"
    json_attributes_template: >
      { 
        "attribute1":"{{ states(/"sensor.mymqttsensor1/") }}",
        "attribute2":"{{ states(/"sensor.mymqttsensor2/") }}"
      }

This is all fine and dandy, except for what I wrote at the beginning of this reply, which is that the attributes won’t update unless there’s an update to the MQTT topic that the particular portion of the sensor is driven by. So state changes of the two sensors used in the template above won’t be reflected in the master sensor attributes until an update is posted to the MQTT topic defined by json_attributes_topic which is test/sensor1/randomattribute. This applies even though the value from that randomattribute topic isn’t used anywhere in the template.

A workaround could be devised by creating an automation which uses service: mqtt.publish to post an update to that specific MQTT topic to force the attributes to update. Or you could instead use an automation to create a JSON payload to the json_attributes_topic topic, which really just moves the complicated template out of your MQTT sensor config and into your automation. But I think the latter is the cleaner way to go.

2 Likes

I’ve been recently trying to solve a similar problem, and I ended up coding a trigger based template sensor. It’s close to the automation idea above but you can have all the code in one place.
I’ll explain with a “living” example (part of my sensor). There are 2 MQTT topics that I want to combine into 1 sensor with attributes:

1) ebusd/bai/SetMode
{
     "hcmode": {"value": "auto"},
     "flowtempdesired": {"value": 0.0},
     "hwctempdesired": {"value": 46.0},
     "hwcflowtempdesired": {"value": null},
     "disablehc": {"value": 1},
     "disablehwctapping": {"value": 0},
     "disablehwcload": {"value": 0},
     "remoteControlHcPump": {"value": 0},
     "releaseBackup": {"value": 0},
     "releaseCooling": {"value": 0}
}
2) ebusd/bai/Status01
{
     "0": {"name": "temp1", "value": 55.5},
     "1": {"name": "temp1", "value": 53.0},
     "2": {"name": "temp2", "value": 11.875},
     "3": {"name": "temp1", "value": null},
     "4": {"name": "temp1", "value": null},
     "5": {"name": "pumpstate", "value": "off"}
}

From the 1st topic, I need the value of ‘disablehc’, which informs whether the heating mode is enabled (value 0) or disabled (1).
From the 2nd topic, I need the value of ‘pumpstate’, which can be one of: “overrun”,“on”, or “off”.
Combination of the 2 values encodes the current operation mode of my heating device, which I want to show in my sensor’s state. Individual values ​​from both topics go to attributes. I also want to know what time last heating cycle started and ended (another 2 attributes)

Sensor code
- trigger:
  - trigger: homeassistant
    event: start
    id: 'ha_start'
  - trigger: event
    event_type: event_template_reloaded
    id: 'template_reload'
  - trigger: mqtt
    topic: ebusd/bai/SetMode
    value_template: "{{ value_json }}"
    id: 'setmode'
  - trigger: mqtt
    topic: ebusd/bai/Status01
    value_template: "{{ value_json }}"
    id: 'status01'
  sensor: 
      # State meaning:
      # 'unknown' - no data
      # 'error' - unexpected value in MQTT data
      # 'idle' - idle
      # 'hwc' - hot water
      # 'hc' - central heating
      # 'hc_p' - central heating, paused
      # 'hc_hwc' - central heating and hot water
    - name: Heating Device
      unique_id: heating_device_id
      state: >-
        {% if trigger.id == 'ha_start' %}
          unknown
        {% elif trigger.id == 'setmode' %}
          {% set p = this.attributes.get('pump_state') %}
          {% if p == 'unknown' %}
            unknown
          {% else %}
            {% set j = trigger.payload|from_json %}
            {% set hc = j['disablehc']['value'] == 0 %}
          {% endif %}
        {% elif trigger.id == 'status01' %}
          {% set hc = this.attributes.get('hc_active') %}
          {% if hc == 'unknown' %}
            unknown
          {% else %}
            {% set j = trigger.payload|from_json %}
            {% set p = j['5']['value'] %}
          {% endif %}
        {% else %}
          {{ states(this.entity_id) }}
        {% endif %}
        {% if hc is defined and p is defined %}
          {% if hc %}
            {{ 'hc_hwc' if p == 'overrun' else 'hc' if p == 'on' else 'hc_p' if p == 'off' else 'error' }}
          {% else %}
            {{ 'hwc' if p == 'overrun' else 'idle' if p in ['on','off'] else 'error' }}
          {% endif %}
        {% endif %}
      attributes:
        # is heating mode active
        hc_active: >-
          {% if trigger.id == 'ha_start' %}
            unknown
          {% elif trigger.id == 'setmode' %}
            {% set j = trigger.payload|from_json %}
            {{ j['disablehc']['value'] == 0 }}
          {% else %}
            {{ this.attributes.get('hc_active') }}
          {% endif %}
        pump_state: >-
          {% if trigger.id == 'ha_start' %}
            unknown
          {% elif trigger.id == 'status01' %}
            {% set j = trigger.payload|from_json %}
            {{ j['5']['value'] }}
          {% else %}
            {{ this.attributes.get('pump_state') }}
          {% endif %}
        # start time of last heating cycle; it will survive HA restart 
        hc_start: >-
          {% set prev = this.attributes.get('hc_start') %}
          {% if trigger.id == 'setmode' %}
            {% set j = trigger.payload|from_json %}
            {% set prevHcActive = this.attributes.get('hc_active') %}
            {% set prevHcActive = true if prevHcActive == 'unknown' else prevHcActive %}
            {{ now() if j['disablehc']['value'] == 0 and not prevHcActive else prev }}
          {% else %}
            {{ prev }}
          {% endif %}
        # end time of last heating cycle; it will survive HA restart
        hc_end: >-
          {% set prev = this.attributes.get('hc_end') %}
          {% if trigger.id == 'setmode' %}
            {% set j = trigger.payload|from_json %}
            {% set prevHcActive = this.attributes.get('hc_active') %}
            {% set prevHcActive = false if prevHcActive == 'unknown' else prevHcActive %}
            {{ now() if j['disablehc']['value'] == 1 and prevHcActive else prev }}
          {% else %}
            {{ prev }}
          {% endif %}

So the attributes basically act as the original MQTT sensors and are only updated when the corresponding MQTT topics have changed, otherwise - they just keep returning their previous value.

If you need to combine values from 2 MQTT topics (as in the state of my sensor), you get the new value from topic’s respective event trigger, and the value of other topic from it’s attribute (as that didn’t change).

Bonus: contrary to “normal” MQTT sensors, you can decide what should happen if templates get reloaded or Home Assistant gets restarted (that only matters, if your MQTT messages don’t have RETAIN flag set). So you can decide per-attribute; e.g. I chose to restore my sensor completely in case of template reload but render it ‘unknown’ after HA restart - except for attributes ‘hc_start’ and ‘hc_end’, which restore their values even then.

1 Like