MQTT Sensor - variable payload throws errors?

Configuration
1

I’ve got a MQTT sensor that is throwing errors because sometimes the payload doesn’t contain the parameters that I’m pulling out.

The sensors are working because it eventually does contain them, but I’d like to not have my log spammed with errors. I do not have control over the source MQTT being published so I can’t modify it to split up the groupings under their own MQTT topics nor ensure that all of them exist in each publishing.

What I’d like to do is if the attribute doesn’t exist in the JSON, just use the last known value (and ALSO don’t throw an error in my log).

Example of sensor:

    - name: "Ecoflow Beeper"
      icon: "mdi:volume-source"
      state_topic: "bridge-ecoflow//app/device/property/DAABZ5ZE8020852"
      payload_on: True
      payload_off: False
      value_template: >-
        {% set value = value_json['params']['pd.beepState'] | int("invalid") %}
        {% if value == "invalid" %}
          {{ states("binary_sensor.ecoflow_beeper") }}
        {% else %}
          {# 1=Mute 0=Sound #}
          {{ value == 0 }}
        {% endif %}

Example of errors:

2022-12-18 11:11:36.573 ERROR (MainThread) [homeassistant.helpers.template] Error parsing value: 'dict object' has no attribute 'pd.beepState' (value: {"params":{"inv.dcInVol":0,"inv.cfgAcWorkMode":0,"inv.cfgAcOutVoltage":120000,"inv.cfgStandbyMin":0,"inv.dcInAmp":0,"inv.cfgAcOutFreq":2,"inv.errCode":0,"inv.outputWatts":0,"inv.dcInTemp":26,"inv.invOutFreq":0,"inv.cfgFastChgWatts":1800,"inv.chargerType":0,"inv.acInAmp":0,"inv.fanState":0,"latestTimeStamp":1671379896479,"inv.cfgAcEnabled":0,"inv.cfgAcXboost":0,"inv.outTemp":25,"inv.invType":0,"inv.acDipSwitch":2,"inv.invOutVol":0,"inv.acInVol":0,"inv.inputWatts":0,"inv.acPassByAutoEn":0,"inv.acInFreq":0,"inv.chgPauseFlag":0,"inv.dischargeType":0,"inv.invOutAmp":0,"inv.sysVer":16843552,"inv.cfgSlowChgWatts":400},"version":"1.0","cmdId":0,"cmdFunc":0,"id":1345560820876055542,"addr":0,"timestamp":1671379896479}, template: {% set value = value_json['params']['pd.beepState'] | int("invalid") %} {% if value == "invalid" %}
  {{ states("binary_sensor.ecoflow_beeper") }}
{% else %}
  {# 1=Mute 0=Sound #}
  {{ value == 0 }}
{% endif %})

Example data:

{
  "params": {
    "inv.dcInVol": 0,
    "inv.cfgAcWorkMode": 0,
    "inv.cfgAcOutVoltage": 120000,
    "inv.cfgStandbyMin": 0,
    "inv.dcInAmp": 0,
    "inv.cfgAcOutFreq": 2,
    "inv.errCode": 0,
    "inv.outputWatts": 0,
    "inv.dcInTemp": 36,
    "inv.invOutFreq": 0,
    "inv.cfgFastChgWatts": 1800,
    "inv.chargerType": 1,
    "inv.acInAmp": 10505,
    "inv.fanState": 0, 
    "latestTimeStamp": 1671242617303,
    "inv.cfgAcEnabled": 0,
    "inv.cfgAcXboost": 0,
    "inv.outTemp": 39,
    "inv.invType": 0,
    "inv.acDipSwitch": 1,
    "inv.invOutVol": 0,
    "inv.acInVol": 111373,
    "inv.inputWatts": 1109,
    "inv.acPassByAutoEn": 0,
    "inv.acInFreq": 60, 
    "inv.chgPauseFlag": 0,
    "inv.dischargeType": 0,
    "inv.invOutAmp": 0,
    "inv.sysVer": 16843552,
    "inv.cfgSlowChgWatts": 400
  },
  "version": "1.0",
  "cmdId": 0,
  "cmdFunc": 0,
  "id": xxxxxxxxx,
  "addr": 0,
  "timestamp": 1671242617303
}

-then later-

{
  "params": {
    "pd.iconWifiMode": 0,
    "pd.iconOverloadState": 0,
    "pd.wattsInSum": 1127,
    "pd.wifiVer": 0,
    "pd.iconLowTempMode": 0,
    "pd.iconGasGenMode": 0,
    "pd.iconBmsParallelMode": 0,
    "pd.iconInvParallelState": 0,
    "pd.typec2Watts": 0,
    "pd.iconTypecState": 0,
    "pd.iconCarMode": 1,
    "pd.iconCoGasState": 0,
    "pd.iconRcState": 0,
    "pd.iconHiTempState": 0,
    "pd.iconHiTempMode": 0,
    "pd.iconEcoMode": 0,
    "pd.usb1Watts": 0,
    "pd.standByMode": 0,
    "latestTimeStamp": 1671242620559,
    "pd.chgPowerDc": 408,
    "pd.dsgPowerDc": 328,
    "pd.iconUsbState": 0,
    "pd.typccUsedTime": 4079,
    "pd.iconBtMode": 0,
    "pd.typec2Temp": 28,
    "pd.carUsedTime": 4493,
    "pd.typec1Watts": 0,
    "pd.soc": 86,
    "pd.iconFactoryState": 0,
    "pd.iconChgStationState": 0,
    "pd.iconBmsParallelState": 0,
    "pd.wirelessWatts": 0,
    "pd.iconCarState": 0,
    "pd.iconSocUpsMode": 0,
    "pd.iconSolarBracketMode": 0,
    "pd.wattsOutSum": 0,
    "pd.iconWifiState": 0,
    "pd.iconWindGenMode": 0,
    "pd.iconUsbMode": 0,
    "pd.iconBmsErrState": 0,
    "pd.iconBmsErrMode": 0,
    "pd.iconWindGenState": 0,
    "pd.iconBtState": 0,
    "pd.iconEcoState": 0,
    "pd.carState": 0,
    "pd.invUsedTime": 169719,
    "pd.iconAcFreqMode": 1,
    "pd.typec1Temp": 27,
    "pd.dsgPowerAc": 2693,
    "pd.dcInUsedTime": 2570,
    "pd.iconLowTempState": 0,
    "pd.model": 1,
    "pd.chgPowerAc": 5154,
    "pd.beepState": 0,
    "pd.iconPackHeaterState": 0,
    "pd.wifiAutoRcvy": 0,
    "pd.remainTime": 26,
    "pd.iconFanState": 0,
    "pd.chgSunPower": 0,
    "pd.carTemp": 0,
    "pd.lcdBrightness": 100,
    "pd.iconRechgTimeState": 0,
    "pd.qcUsb2Watts": 0,
    "pd.lcdOffSec": 300,
    "pd.iconTransSwState": 0,
    "pd.iconTypecMode": 0,
    "pd.sysVer": 16844309,
    "pd.iconOverloadMode": 0,
    "pd.iconRechgTimeMode": 0,
    "pd.sysChgDsgState": 2,
    "pd.iconInvParallelMode": 0,
    "pd.qcUsb1Watts": 0,
    "pd.iconTransSwMode": 0,
    "pd.iconFactoryMode": 0,
    "pd.kit0": 0,
    "pd.kit1": 0,
    "pd.kit2": 0,
    "pd.iconFanMode": 1,
    "pd.usbqcUsedTime": 4089,
    "pd.dcOutState": 0,
    "pd.usbUsedTime": 4197,
    "pd.mpptUsedTime": 80,
    "pd.iconSolarPanelState": 0,
    "pd.iconWirelessChgMode": 0,
    "pd.wifiRssi": 0,
    "pd.iconSocUpsState": 0,
    "pd.iconSolarBracketState": 0,
    "pd.errCode": 0,
    "pd.iconPackHeaterMode": 0,
    "pd.iconCoGasMode": 0,
    "pd.carWatts": 0,
    "pd.iconChgStationMode": 0,
    "pd.iconAcFreqState": 0,
    "pd.iconGasGenState": 0,
    "pd.usb2Watts": 0,
    "pd.iconRcMode": 0,
    "pd.iconWirelessChgState": 0,
    "pd.iconSolarPanelMode": 0
  },
  "version": "1.0",
  "cmdId": 0,
  "cmdFunc": 0,
  "id": xxxxxxxxxxxxxx,
  "addr": 0,
  "timestamp": 1671242620559
}
2 
    - name: "Ecoflow Beeper"
      icon: "mdi:volume-source"
      state_topic: "bridge-ecoflow//app/device/property/DAABZ5ZE8020852"
      payload_on: True
      payload_off: False
      value_template: >-
        {% set value = value_json['params']['pd.beepState'] %}
        {{ not(value|bool) if value is defined else this.state }}
3

Interesting, I think there’s some shorthand there I’m not understanding with the second line.

Could you help me understand what that breaks down to?
{{ not (something something something) if value is something else something }}

Trying to understand because I’ve got about a dozen binary sensors all similar (but some are 0/1 others 1/2) as well as a few dozen sensor (mostly numeric or coded values) that all have this same problem, and all follow the exact same format as the original one I posted for how I break them out (partly why the boolean may be slightly less-efficiently coded, since others are just plain value or math).

4

You did this:

{{ value == 0 }}

It reports true if value is equal to 0, otherwise it reports false.

In contrast, {{ value | bool }} converts 0 and 1 to false and true respectively. However, that’s the opposite state of what you want so that’s the reason for employing not to invert the result.

{{ not(value|bool) }}

So if value_json['params']['pd.beepState'] is defined, its value is converted to boolean and then inverted. If it’s not defined, the Template Sensor’s existing state value is reported (using this.state).

5

Ah, ok - that makes sense then for the inverting/swapping.
this.state sounds like a really handy one I’ll have to remember.

So I’m guessing for a normal numerical sensor (vs boolean) I’d just substitute in not(value|bool) would be where I do like value / 1000.0 to compute the value like voltages and stuff in the value-template accomplishing the same “if it’s available update, otherwise use previous state”?

6

Correct.

Don’t forget to use float() or int() if the value is a numeric string.

{{ (value | float(0) / 1000) if value is defined else this.state }}
7

Is there a reason not to just put the | int('invalid') or whatever I wish in the value-assignment statement and have it down in the lower portion as you were doing?

As a C/C++/Java person it felt more readable to me handing all the “make it a numeric” in the messy assignment and then I just use the value in the template without worrying about conversion. Maybe there’s something I don’t know that’s not the best way in this case?

8

For the same reason you already discovered empirically.

If the key pd.beepState doesn’t exist then
value_json['params']['pd.beepState']
is undefined. The int filter won’t report its default value for an undefined variable.

If you prefer to use your template, then use the default filter. It reports its default value exclusively for an undefined variable.

      value_template: >-
        {% set value = value_json['params']['pd.beepState'] | default("invalid") %}
        {% if value == "invalid" %}
          {{ states("binary_sensor.ecoflow_beeper") }}
        {% else %}
          {# 1=Mute 0=Sound #}
          {{ value | int(0) == 0 }}
        {% endif %}
9

Ok, yeah that helps me understand what’s going on a LOT better what exactly is going on. Thank you for taking the time to explain that!

So it sounds like it’s not the fact I referenced/stored the non-existent key that was the problem, rather it’s that I tried to use/convert the non-existent value that throws the error?

I think for logical flow I may end up doing a hybrid of both, leveraging the this.state would fix a lot of copy-paste errors, and then if I am correctly understanding where to do which conversions could also help. My end goal is to understand it well enough I can use a script to generate the sensors for different serial-number units in the future so I do want to design a maintainable readable formatting that is easy to duplicate in most/all conditions.

10

The non-existent variable.

Copy-paste the following template into the Template Editor.

{% set value_json = {"temperature": {"high": 22, "low": 16} } %}
{{ value_json.temperature.high }}

It will report: 22

Change the template’s second line to this:

{{ value_json.temperature.foo }}

It will report:

UndefinedError: 'dict object' has no attribute 'foo'
1 Like
11

Ah ok, now I think I’m better understanding the difference. Thanks.

I also wasn’t aware I could use a general default statement, and couldn’t figure out how to say “if this exists” before attempting to do something, so I think I’ve now learned several things.

12

You’re welcome!

Please consider marking my first post above with the Solution tag. It will automatically place a check-mark next to the topic’s title which signals to other users that this topic is resolved. This helps other users find answers to similar questions.

For more information about the Solution tag, refer to guideline 21 in the FAQ.

13

I think with combining things here, it looks like the (in my opinion) best balance of readability and flexibility may be doing something of this style (playing with another parameter I have to handle).

I think this will also have the benefit if the key is defined but the private-API OEM changes it to bad data I’ll know because sensors will report “invalid” instead of just freezing. Also makes it easy a few are coded keys and I can trivially throw in elif when I have to handle several values without designing a whole new template.

      value_template: >-
        {% set value = value_json['params']['bmsMaster.temp'] %}
        {% if not value is defined %}
          {{ this.state }}
        {% else %}
          {{ value | int("invalid") }}
        {% endif %}

Unless you spot somewhere I am still not understanding (this appears to not throw errors, but I’m still poking at edge cases).

14

Traditional If.

{% if test %}
  test_true
{% else %}
  test_false
{% endif %}

Inline If.

{{ test_true if test else test_false }}

Immediate If.

{{ iif(test, test_true, test_false) }}

The catch with the last one (iif) is that it always evaluates both “test_true” and “test_false” expressions regardless of the test’s result. Depending on what’s in them, in can cause an error.

For example, the following will cause an error if x is zero because iif always evaluates both expressions and will fail when dividing 100 by zero.

{{ iif(x != 0, 100/x, x) }}

Immediate If is better suited for applications where “test_true” and “test_false” are static values (or expressions that won’t fail).

1 Like