Different variables from a single MQTT topic

KPL · February 16, 2024, 2:23pm

Hi guru’s!

A newbie here; using HA for a few weeks, in Docker.
I am still trying to get myself comfortable with HA, browsing&playing around, but still unable to get some basic thing to work.
There are some topics with similar things in subject, but those seem to be about a different problem.

I have multiple Tasmota-based devices that are running tasmota rules to perform specific tasks, like thermostat in this case.
Tasmota devices themselves were successfully discovered by HA automatically, but I want to get the information that is published when rules execute.
Normally, Tasmota’s STATE topics are used to obtain this info, but I had to suppress them to once-per-minute to save the bandwidth.

So, when rules execute, they publish many multiple things under RESULT topic:

$ grep pow1 /var/log/mosquitto/traffic.log | grep RESULT 
stat/tasmo/tasmopow1/RESULT {"Var2":"-8"}
stat/tasmo/tasmopow1/RESULT {"Add2":"-7.000"}
stat/tasmo/tasmopow1/RESULT {"Mem1":"16"}
stat/tasmo/tasmopow1/RESULT {"Var1":"4.1"}
stat/tasmo/tasmopow1/RESULT {"Mem1":"16"}
stat/tasmo/tasmopow1/RESULT {"Event":"Done"}
stat/tasmo/tasmopow1/RESULT {"POWER1":"1"}
stat/tasmo/tasmopow1/RESULT {"POWER1":"0"}
stat/tasmo/tasmopow1/RESULT {"Var2":"16"}
stat/tasmo/tasmopow1/RESULT {"Add2":"17.000"}
stat/tasmo/tasmopow1/RESULT {"Var1":"4.2"}
stat/tasmo/tasmopow1/RESULT {"Mem1":"16"}
stat/tasmo/tasmopow1/RESULT {"Event":"Done"}
stat/tasmo/tasmopow1/RESULT {"POWER1":"1"}
stat/tasmo/tasmopow1/RESULT {"Var2":"16"}
stat/tasmo/tasmopow1/RESULT {"Add2":"17.000"}
stat/tasmo/tasmopow1/RESULT {"Var1":"4.1"}
stat/tasmo/tasmopow1/RESULT {"Mem1":"16"}
stat/tasmo/tasmopow1/RESULT {"Event":"Done"}
stat/tasmo/tasmopow1/RESULT {"POWER1":"1"}
stat/tasmo/tasmopow1/RESULT {"Var2":"16"}
stat/tasmo/tasmopow1/RESULT {"Add2":"17.000"}
stat/tasmo/tasmopow1/RESULT {"Var1":"4.2"}
stat/tasmo/tasmopow1/RESULT {"Mem1":"16"}
stat/tasmo/tasmopow1/RESULT {"Event":"Done"}
stat/tasmo/tasmopow1/RESULT {"POWER1":"1"}

I want to capture specific things:

stat/tasmo/tasmopow1/RESULT {"Mem1":"16"}
stat/tasmo/tasmopow1/RESULT {"POWER1":"1"}

mqtt:
  - sensor:
    - name: test tasmopow1 power1
      unique_id: tasmopow1_power1
      state_topic: stat/tasmo/tasmopow1/RESULT
      # value_template: "{{ value_json['POWER1'] }}"
      value_template: "{{ value_json.POWER1 }}"

  - sensor:
    - name: test tasmopow1 mem1
      unique_id: tasmopow1_mem1
      state_topic: stat/tasmo/tasmopow1/RESULT
      # value_template: "{{ value_json['POWER1'] }}"
      value_template: "{{ value_json.Mem1 }}"

Normal mqtt sensors do capture them, but since multiple kinds of variables are reported under the same topic, captured state is lost when next RESULT contains different data in it, and a warning is logged.

2024-02-16 16:00:32.204 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'POWER1' when rendering '{{ value_json.POWER1 }}'
2024-02-16 16:00:32.205 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'Mem1' when rendering '{{ value_json.Mem1 }}'

I think I noticed once a recommendation how to deal with this, using template trigger and sensor, ~~but I can not find that anymore~~ found it: Hold/retain sensor value between mqtt messages -- SOLVED! THX.
I tried that approach, but did not see any change to behaviour, probably failed to adapt it to my situation:

template:
    - trigger:
        - platform: mqtt
          topic: stat/tasmo/tasmopow1/RESULT
          payload: "True"
          value_template: "{{ value_json.Mem1 }}"
      sensor:
        - unique_id: test_tmpl_mem1
          name: "test tmpl Mem1"
          state: >-
            {{ trigger.payload_json.Mem1 }}

It’s not quite clear how was this supposed to work, or how was i supposed to see it’s results.

Is there a document somewhere, that would describe things like value_json, trigger.payload_json? What other options are there? I see value_json is giving me dictionary’s value, is there a way to obtain it’s key? How can I check if a dictionary contains particular key, without causing a warning?

Obviously I do not have enough understanding of basic concepts; I would be happy if somebody helped me out, directing me to docs/examples/whatever on this.

KPL

123 · February 16, 2024, 9:39pm

Others have encountered the same situation where a single MQTT topic is used to report different parameters. In effect the topic’s data is “multiplexed”. What I have suggested to others is to use an automation to “demultiplex” the data. It simply re-publishes the received data to unique topics and your sensors subscribe to those topics.

Create the following automation.

alias: 'tasmopow1 demultiplexer'
id: tasmopow1_demultiplexer
trigger:
  - platform: mqtt
    topic: stat/tasmo/tasmopow1/RESULT
condition: []
action:
  - variables:
      key_name: "{{ trigger.payload_json.keys() | list | first }}"
  - service: mqtt.publish
    data:
      topic: "tasmopow1/{{ key_name }}"
      payload: "{{ trigger.payload_json[key_name] }}"
      retain: true

Then configure you MQTT Sensors like this:

mqtt:
  - sensor:
    - name: tasmopow1 power1
      unique_id: tasmopow1_power1
      state_topic: tasmopow1/POWER1

  - sensor:
    - name: tasmopow1 mem1
      unique_id: tasmopow1_mem1
      state_topic: tasmopow1/Mem1

The initial state of each sensor will be unknown and will change to a nominal value when one is received. The sensor’s value will be restored after a restart because it’s published to the broker as a retained message.

KPL · February 18, 2024, 3:35pm

Thanks, Taras! After some strange struggle with file permissions this seems to work.

Now I want to understand: HOW
There are json dictionary-related keywords like payload_json, key_name, keys() - where these come from? I have not seen a manual that would list and describe them.
Those do not appear like part of yaml or part of jinja; what are they?
Tutorials just show them being used, but never refer to documentation.

123 · February 18, 2024, 6:31pm

Home Assistant’s scripting language is a combination of:

YAML
Jinja
python ("pythonic syntax")

So it can be challenging to find a single resource that covers all three of them.

YAML is basically means of structuring data.
Jinja is a templating language with python-like syntax so some of the things you see in the template are borrowed from python such as keys().

KPL · February 18, 2024, 6:56pm

So basically one has to sit and read examples, shared by others.
I was sure there must be a way to access “key” part of a json dictionary, but had never seen one in use.
Thanks!

123 · February 18, 2024, 7:31pm

If you haven’t already done so, click the link in my previous post. It leads to a tutorial, on W3schools.com, for keys(). The same site explains many other aspects of python that are also employed by Jinja.

In addition, the documentation section on Templating is a must-read. Home Assistant supports most but not all of Jinja’s features plus it adds a few things of its own.

KPL · February 18, 2024, 8:18pm

I know python a bit and use dictionaries a lot, but keys() was the only here that looked familiar in this context.

Now I have found things like first, last, list in jinja’s docs, but that does not help to understand which parts of jinja, python, etc can be used together and in which way.

This makes a sense when reading it, but it needs a good understanding to be able to write this, as there are elements of different languages linked:

trigger.payload_json.keys() | list | first

Not clear yet, where things like value_json comes from.

Anyway, lot of reading ahead.

123 · February 18, 2024, 8:33pm

It depends on the type of trigger. In this case, it’s an MQTT Trigger so the documentation for the resulting trigger variable shows what’s available (payload_json if the received payload is expected to be in JSON format).

KPL · February 19, 2024, 12:10pm

So I was able to modify that automation so it parses similar mqtt data for all my tasmota devices, getting device name from mqtt topic, so I do not have to write a separate automation for each of them.
Probably that means I actually understood something