How to remove errors from MQTT sensor with JSON data

Configuration
1

I have setup MQTT sensor, it is working but i get lots of errors.
Error look like this:

2024-08-22 15:12:51.756 WARNING (MainThread) [homeassistant.components.mqtt.mixins] Erroneous JSON: 
2024-08-22 15:12:52.053 WARNING (MainThread) [homeassistant.components.mqtt.mixins] Erroneous JSON: 
2024-08-22 15:12:52.054 ERROR (MainThread) [homeassistant.helpers.template] Error parsing value: 'AA' is undefined (value: {"bus":"0","type":"rx","ts":14552,"frame":[{"id":2029,"dlc":8,"rtr":false,"extd":false,"data":[4,98,116,72,4,170,170,170]}]}, template: {% if value_json.frame[0].id == 1999 %}
  {% set PID = value_json.frame[0].data[0:5] %}
  {% if PID == [16,9,98,29,218] %}
    {% set AA = value_json.frame[0].data[6] %}
{% endif %} {% endif %} {% if AA | round(0) == 20 %}
  in
{% else %}
  out
{% endif %})

Setup for sensors are like this

    - name: e-Up_charging_status
      unique_id: eup_charging_status
      icon: mdi:ev-plug-type2
      state_topic: "wican/e-up/can/rx"
      payload_on: "C"
      payload_off: "B"
      value_template: >-
        {% if value_json.frame[0].id == 2029 %}
          {% set PID = value_json.frame[0].data[0:4] %}
          {% if PID == [4,98,116,72] %}
          {% set AA = value_json.frame[0].data[4] %}
        {% endif %}
        {% endif %}
        {% if AA | round(0) == 0 %}
          B
        {% elif AA | round(0) == 4 %}
          C
        {% endif %}

      
    - name: "e-Up!_odo1"
      unique_id: "eup_odo1"
      state_topic: "wican/e-up/can/rx"
      unit_of_measurement: "km"
      state_class: "measurement"
      value_template: >-
        {% if value_json.frame[0].id == 2029 %}
          {% set PID = value_json.frame[0].data[0:5] %}
          {% if PID == [16,13,98,2,189] %}
            {% set AA = value_json.frame[0].data[6] %}
            {% set BB = value_json.frame[0].data[7] %}
        {{ (AA*65536) + (BB*256) }}
        {% endif %}
        {% endif %}
      json_attributes_topic: "wican/e-up/can/rx"
      json_attributes_template: >-
        {% if value_json.frame[0].id == 2029 %} 
          {% set PID = value_json.frame[0].data[0:5] %}
          {% if PID == [16,13,98,2,189] %}
            {% set time = value_json.ts %}
        { "ts": {{ time }} }
          {% endif %} 
          {% endif %}

Same topic is receiving lots of JSON messages, that have slightly different id and data values.
Here is example of one message

{
  "bus": "0",
  "type": "rx",
  "ts": 39329,
  "frame": [
    {
      "id": 2029,
      "dlc": 8,
      "rtr": false,
      "extd": false,
      "data": [
        4,
        98,
        116,
        72,
        4,
        170,
        170,
        170
      ]
    }
  ]
}

I need for my odo1 sensor, value from ts to confirm odo2 sensor (not shown) uses right message (compare timestamps).

I get these errors for each new message puplished in that topic.
Erroneous JSON: error is caused by { "ts": {{ time }} } line in odo1 json_attribute_template but i dont know what is wrong whit it, attribute shows correctly in sensor in UI.

How i can get these error corrected?

2

Your template defines the variable AA only if the value of id is 2029.

If it’s not 2029, the variable AA is undefined. Your template then references the undefined AA variable here (which results in an error message):

        {% if AA | round(0) == 0 %}
          B
        {% elif AA | round(0) == 4 %}
          C
        {% endif %}

You will need to change the template’s logic to handle the situation where id is not 2029. Perhaps it’s as simply as initially setting AA to 0 but I can’t say for sure because I don’t know the sensor’s requirements.

Potentially the same problem. The variable time is defined only if PID is [16,13,98,2,189], otherwise it’s undefined.

3

That i was affaird of…
Well i get lots of JSON messages and my sensor should only change its state if correct message is broadcasted, othervise it should not change.
Sensor should ignore other messages, how can this be done?, or is there some workaround, all i can think of is setting sensor that redirects every sensors message to that specific sensor, so that sensor gets only messages that it wants.

4

For some reason i cant get template editor to work when sending JSON with MQTT Explorer
But would this kind of approach work?

    - name: e-Up_charging_status
      unique_id: eup_charging_status
      icon: mdi:ev-plug-type2
      state_topic: "wican/e-up/can/rx"
      payload_on: "C"
      payload_off: "B"
      value_template: >-
        {% if value_json.frame[0].id == 2029 %}
          {% set PID = value_json.frame[0].data[0:4] %}
          {% if PID == [4,98,116,72] %}
          {% set AA = value_json.frame[0].data[4] %}
          {% endif %}
        {% else %}
          {% set value_json = XX %}
        {% endif %}
        {% if AA | round(0) == 0 %}
          B
        {% elif AA | round(0) == 4 %}
          C
        {% endif %}

Adding {% else %} and setting value_json to something that will not be shown?

5

Well i got most of errors out, those were caused by binary_sensor
Added lines

        {% else %}
        {{ this.state }}

Errors from json_attribute_template was a bit harder atleast for me, got it right with

      json_attributes_template: >-
        {% if value_json.frame[0].id == 2029 %}
          {% set PID = value_json.frame[0].data[0:5] %}
          {% if PID == [16,13,98,2,189] %}
            {"ts": {{value_json.ts}} }
        {% else %}
        {"ts": {{(state_attr('sensor.e_up_odo1','ts'))}} }
          {% endif %} 
        {% else %}
        {"ts": {{(state_attr('sensor.e_up_odo1','ts'))}} }
          {% endif %}

Now i get errors only when there has not been any messages, sending right message once clears issue.

6

I tried to add attribute to mqtt binary_sensor but cant get it to work, attribute is NOT shown in UI and it gives error message Erroneous JSON:.
Exactly same setup for attribute works in sensor but not in binary_sensor.
Can someone help, what is wrong here?

    - name: e-Up_plug_status
      unique_id: eup_plug_status
      icon: mdi:ev-plug-type2
      state_topic: "wican/e-up/can/rx"
      payload_on: "in"
      payload_off: "out"
      value_template: >-
        {% if value_json.frame[0].id == 1999 %}
          {% set PID = value_json.frame[0].data[0:5] %}
          {% if PID == [16,9,98,29,218] %}
            {% set AA = value_json.frame[0].data[6] %}
        {% endif %}
        {% endif %}
        {% if AA is defined %}
          {% if AA | round(0) == 20 %}
          in
        {% else %}
          out
        {% endif %}
        {% else %}
        {{ this.state }}
        {% endif %}
      json_attributes_topic: "wican/e-up/can/rx"
      json_attributes_template: >-
        {% if value_json.frame[0].id == 1999 %}
          {% set PID = value_json.frame[0].data[0:5] %}
          {% if PID == [16,9,98,29,218] %}
           { "lock": {{value_json.frame[0].data[5]}} } 
        {% else %}
        { "lock": {{(state_attr('binary_sensor.e_up_plug_status','lock')) | tojson}} }
        {% endif %}
        {% else %}
        { "lock": {{(state_attr('binary_sensor.e_up_plug_status','lock')) | tojson }} }
        {% endif %}

Error looks like

2024-08-24 23:30:39.283 WARNING (MainThread) [homeassistant.components.mqtt.mixins] Erroneous JSON: { "lock": 3 } 
 
##################################
2024-08-24 23:30:39.302 WARNING (MainThread) [homeassistant.components.mqtt.mixins] Erroneous JSON: { "lock": null }  
##################################
2024-08-24 23:30:39.604 WARNING (MainThread) [homeassistant.components.mqtt.mixins] Erroneous JSON: { "lock": null } 
##################################
2024-08-24 23:30:45.361 WARNING (MainThread) [homeassistant.components.mqtt.mixins] Erroneous JSON: { "lock": 3 }

I tried text locked and true instead of value but allways error.
Binary_sensor is working but attribute is not shown.

7

I have two ´MQTT Binary_sensor` setup, these are allmost identical, uses same data, first one is working but second doesnt work.
Could someone spot problem?

    - name: e-Up_plug_status
      unique_id: eup_plug_status
      icon: mdi:ev-plug-type2
      state_topic: "wican/e-up/can/rx"
      payload_on: "in"
      payload_off: "out"
      value_template: >-
        {% if value_json.frame[0].id == 1999 %}
          {% set PID = value_json.frame[0].data[0:5] %}
          {% if PID == [16,9,98,29,218] %}
            {% set AA = value_json.frame[0].data[6] %}
        {% endif %}
        {% endif %}
        {% if AA is defined %}
          {% if AA | round(0) == 20 %}
          in
        {% else %}
          out
        {% endif %}
        {% else %}
        {{ this.state }}
        {% endif %}
        
    - name: e-Up_plug_lock
      unique_id: eup_plug_lock
      icon: mdi:lock
      state_topic: "wican/e-up/can/rx"
      payload_on: "locked"
      payload_off: "unlocked"
      value_template: >-
        {% if value_json.frame[0].id == 1999 %}
          {% set PID = value_json.frame[0].data[0:5] %}
          {% if PID == [16,9,98,29,218] %}
            {% set AA = value_json.frame[0].data[5] %}
        {% endif %}
        {% endif %}
        {% if AA is defined %}
          {% if AA | round(0) >= 2 %}
            locked
          {% else %}
            unlocked
          {% endif %}
        {% else %}
        {{ this.state }}
        {% endif %}

Binary sensor e-Up_plug_lock does not work, states allways unknown even eup_plug_status changes its state and incoming message is right.
I have tested this with template_editor and it works like it should, but not when configured in YAML.

8

Hi,

Apart from the {% endif %} unbalanced indenting slightly hurting my code-review eyes, the obvious stuff looks OK. Then again, you suggest it has already been linted with the template_editor.

What that doesn’t do is check the incoming MQTT data is as you expect.

  • Cross-check what the broker actually published using https://mqtt-explorer.com/ .
  • Try publishing what you think the message should contain manually.
    Personally, I use mosquitto_pub & mosquitto_sub in Bash scripts to prototype MQTT actions - can be quicker to test stuff like auto-discovery, and gives an immutable test case.
    Using The Mosquitto_pub and Mosquitto_sub MQTT Client Tools- Examples
  • BREAK THE TEMPLATE DOWN
    I’ve written really complex, clever code, and find it usually breaks. Writing provable blocks, then sticking them together works better.
    • Try breaking the code down and extract just a part of the state - PID and AA are a good start to duplicate and publish (copy-pasta the whole thing several times, and strip back) so you can log for a cross-check (sort of a printf() ).

If I’d have to bet, as one block works and the other doesn’t suggests the input data isn’t quite what you think. Go for a walk, then explain the issue to a cat. Works!

If this helps, :heart: this post!

1 Like
9

I have other sensor that logs all JSON messages coming to this topic, there i can verify that i get right message, value in that byte is 0,1,2,3 depending on situation.
And that message is “visible” for atleast 500ms currently.
Im really confused because that other sensor is working, im writing yaml in Notepad++ and yaml is selected as “language”.
Well i have to try removing what you suggested.

e. I changed code abit, removed first if line and then it started to work, but WHY?
Other sensor is able to read that message and that sensor was just copy paste with very little changes.
Now sensor looks like

    - name: e-Up_plug_lock
      unique_id: eup_plug_lock
      icon: mdi:lock
      state_topic: "wican/e-up/can/rx"
      payload_on: "ON"
      payload_off: "OFF"
      value_template: >-
        {% set PID = value_json.frame[0].data[0:5] %}
          {% if PID == [16,9,98,29,218] %}
            {% set AA = value_json.frame[0].data[5] %}
          {% endif %}
        {% if AA is defined %}
          {% if AA | round(0) >= 2 %}
            OFF
          {% else %}
            ON
          {% endif %}
        {% else %}
        {{ this.state }}
        {% endif %}

ee. Now im really confused, after i got it working, i wanted to check what was wrong, entered that if line and used locked and unlocked as payload_on and payload_off messages and it keeps working…
So sensor looks like

    - name: e-Up_plug_lock
      unique_id: eup_plug_lock
      icon: mdi:lock
      state_topic: "wican/e-up/can/rx"
      payload_on: "unlocked"
      payload_off: "locked"
      value_template: >-
        {% if value_json.frame[0].id == 1999 %}
          {% set PID = value_json.frame[0].data[0:5] %}
          {% if PID == [16,9,98,29,218] %}
            {% set AA = value_json.frame[0].data[5] %}
          {% endif %}
        {% endif %}
        {% if AA is defined %}
          {% if AA | round(0) >= 2 %}
            locked
          {% else %}
            unlocked
          {% endif %}
        {% else %}
        {{ this.state }}
        {% endif %}

Could it be just locked and unlocked messages getting it upset, as that logic was wrong way around??
When binary_sensor defined as lock is ON it is unlocked.
Well it works, just wondering what went wrong. And i dont think that indenting made any differense as my yaml has all {% endif %} like that and everything else worked.

10

I’ve seen YAML barf slightly with complex templating, hence the linter thought (ESPhome can be terrible for this - might be the same parser library).

It’s very annoying that HASS sometimes re-formats YAML automations if you touch them with the GUI editor - fine for most things, just not this level of {{templating}}.

I’d suggest splitting this complex automation out into a separate file/ code directory and !include it in YAML to prevent this (YAML can include a whole directory of files).

There are times where I wonder if writing a very small Add-On in Python might be easier!

If this helps, :heart: this post!

11

I havent touch these sensors in UI, i have these sensors in separate file under packages.
Could it be possible that there is too many sensors in one file or folder, this file contains only about 15 sensors, those all are MQTT sensors updating from same source. Should not be problem with my host machine, it uses only 1% cpu, x86 Intel N200.