Mqtt.py log warning "No matching payload found for entity"

Aha! Here we actually coming to another level.
I’m currently porting my Kerui D026 door sensor from OMG RF to OMG Pilight.

Here’s the issue: Kerui D026 can send 4 different codes: opened, closed, tampered and low battery.
First two can be classes as “states”, where the last two are more of “attributes” (imho)

I think that it’s a good idea to have states retained, when attributes can be used as signals for automation or to temporarily change the current state.
Therefore, I use MQTT sensor to represent the door sensor’s state and take into account the tampered attribute. In fact, I use an additional template sensor mostly because MQTT sensor does not support icon_template.

So here’s the raw MQTT sensor (you don’t need to add it to any view)

- platform: mqtt
  name: contacts_1st_floor_safe_pilight
  state_topic: 'home/contacts/1st_floor/safe'
  json_attributes_topic: 'home/contacts/1st_floor/safe/attributes'

and its UI representation

- platform: template
  sensors:
    contact_1st_floor_safe:
      friendly_name: !secret 1st_floor_safe_name
      value_template: >
        {% if is_state_attr('sensor.contacts_1st_floor_safe_pilight', 'tamper', 'on') %}
          {{ states('sensor.contact_always_tampered') }}
        {% else %}
          {{ states('sensor.contacts_1st_floor_safe_pilight') }}
        {% endif %}
      icon_template: >
        {% set state = states('sensor.contact_1st_floor_safe') %}
        {% if state == states('sensor.contact_always_opened') %}
          mdi:door-open
        {% elif state == states('sensor.contact_always_closed') %}
          mdi:door-closed
        {% elif state == states('sensor.contact_always_tampered') %}
          mdi:image-broken
        {% else %}
          mdi:cloud-question
        {% endif %}

The thing is my demultiplexer listens to the OMG Pilight topic, extracts known codes if protocol is kerri_d026 and publishes to appropriate topics (home/contacts/1st_floor/safe for opened/closed and home/contacts/1st_floor/safe/attributes for tamper).
It kind of works, but I don’t like 2 things:

  1. Handling of Pilight messages. They come as JSONs, but the main info (unicode, state etc) is within message, something like this:
{"message":"{\"unitcode\":441825,\"state\":\"tamper\"}","protocol":"kerui_D026","length":"","repeats":2,"status":2}

I have to parse them myself in every script as it’s a string and there is no way to use something like value_json,message.state. I yet to find a way to do it nicely. So for now it’s like

known_states = ['opened', 'closed']
known_attributes = ['tamper']
#known_states = stable_states + temporary_states

entity = None
state = None

# remove curly brackets and split into "key:value" strings
params = message.replace('{', '').replace('}', '').split(',')

# clean and store parameters - have to do ALL of them, awful!
for param in params:
    key, value = param.split(':')
    key = key.replace('"','')

    if key == 'unitcode':
        id = int(float(value))
        for e in known_entities.keys():
          if id in known_entities[e]:
            entity = e
            break
    elif key == 'state':
        state = value.replace('"','').lower()

if entity is None:
    logger.debug('unknown entity, ignored')
elif state is None:
    logger.error('no state information provided for %s, ignored', entity)

and only after that I can do something…
By the way, I don’t know who and why created this format of messages, but it would be great to make it either “plain” (one-level) JSON or JSON-able (i.e when message is formed as JSON as well).

  1. The way I handle that tamper attribute. I want the UI sensor to show “tampered” for a number of seconds instead of its state and then go back to it. That way my alarm panel would be activated as it can react not only on opened, but on tamper as well.
    Currently I achieve that by publishing JSON {“tamper”:“on”} and then {“tamper”:“off”} to home/contacts/1st_floor/safe/attributes, but I have to add a delay between these 2 commands to make the change visible for a set amount of time. Don’t like it.

Any ideas how to improve that?

So all in all, there is no single solution, and it’s not that great and bright, but we are getting there :wink:

Hi.
I’m trying to get rid of all those warnings.
I modified some MQTT sensors.
This, for example (the commented lines were the previous, working, configuration):

- platform: mqtt 
    name: "Contatto_Open_Close_2"
    # payload_on: "02ADC3"
    # payload_off: "02ADC9"
    device_class: garage_door
    state_topic: "tele/Sonoff_RF_Bridge/RESULT"
    # value_template: "{{ value_json.RfReceived.Data }}"
    value_template: >-
      {% if value_json.RfReceived.Data == '02ADC3' %}
        {{'ON'}}
      {% elif value_json.RfReceived.Data == '02ADC9' %}
        {{'OFF'}}
      {% else %}
        {{states('binary_sensor.contatto_open_close_2') | upper}}
      {% endif %}

If I publish 02ADC3 value, the sensor does not activate, and in the log I read:

2020-02-19 13:37:14 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: Contatto_Open_Close_2 with state topic: tele/Sonoff_RF_Bridge/RESULT. Payload: 02ADC3, with value template Template("{% if value_json.RfReceived.Data == '02ADC3' %}
  {{'ON'}}
{% elif value_json.RfReceived.Data == '02ADC9' %}
  {{'OFF'}}
{% else %}
  {{states('binary_sensor.contatto_open_close_2') | upper}}
{% endif %}")

I can’t figure out what’s wrong…

Correct, because the payload you’ve published is not in the format the value_template requires.

Look at what the value_template is attempting to extract from the received payload:

value_json.RfReceived.Data

It expects to receive a payload in JSON format, specifically a dictionary containing the keys RfReceived and Data.

To test sensor.contatto_open_close_2, publish this payload to the state_topic:

{"RfReceived":{"Data":"02ADC3"}}

Mmm… ok, now I try, I’m rebooting, but even before applying your “fix”, the payload was value_json.RfReceived.Data, and simply publishing the payload value was working.

it doesn’t work.
I tested it on a less critical, but identical, sensor.

- alias: Segna_sensore_box_ON
    trigger:
      - platform: state
        entity_id: input_boolean.sensore_sezionale
        to: 'on'
    action:
      - service: mqtt.publish
        data:
           topic: 'tele/Sonoff_RF_Bridge/RESULT'
           payload: {"RfReceived":{"Data":"33150A"}}
           
  - alias: Segna_sensore_box_OFF
    trigger:
      - platform: state
        entity_id: input_boolean.sensore_sezionale
        to: 'off'
    action:
      - service: mqtt.publish
        data:
           topic: 'tele/Sonoff_RF_Bridge/RESULT'
           payload: {"RfReceived":{"Data":"33150E"}}

but the sensor state doesn’t change.

Error while executing automation automation.segna_sensore_box_on. Unknown error for call_service at pos 1:
Traceback (most recent call last):
File “/usr/src/homeassistant/homeassistant/components/automation/init.py”, line 397, in async_trigger
await self.action_script.async_run(variables, trigger_context)
File “/usr/src/homeassistant/homeassistant/helpers/script.py”, line 247, in async_run
await self._handle_action(action, variables, context)
File “/usr/src/homeassistant/homeassistant/helpers/script.py”, line 331, in _handle_action
await self._actions[_determine_action(action)](action, variables, context)
File “/usr/src/homeassistant/homeassistant/helpers/script.py”, line 413, in _async_call_service
context=context,
File “/usr/src/homeassistant/homeassistant/helpers/service.py”, line 96, in async_call_from_config
domain, service_name, service_data, blocking=blocking, context=context
File “/usr/src/homeassistant/homeassistant/core.py”, line 1226, in async_call
await asyncio.shield(self._execute_service(handler, service_call))
File “/usr/src/homeassistant/homeassistant/core.py”, line 1251, in _execute_service
await handler.func(service_call)
File “/usr/src/homeassistant/homeassistant/components/mqtt/init.py”, line 681, in async_publish_service
await hass.data[DATA_MQTT].async_publish(msg_topic, payload, qos, retain)
File “/usr/src/homeassistant/homeassistant/components/mqtt/init.py”, line 816, in async_publish
self._mqttc.publish, topic, payload, qos, retain
File “/usr/local/lib/python3.7/concurrent/futures/thread.py”, line 57, in run
result = self.fn(*self.args, **self.kwargs)
File “/usr/local/lib/python3.7/site-packages/paho/mqtt/client.py”, line 1260, in publish
‘payload must be a string, bytearray, int, float or None.’)
TypeError: payload must be a string, bytearray, int, float or None.

Another example: Xiaomi Magic Cube.
I modified the sensor to be like this:

- platform: mqtt
    name: "Cube Left"
    # payload_on: "rotate_left"
    off_delay: 1
    device_class: moving
    state_topic: "zigbee2mqtt/0x00158d00029aaf18"
    # value_template: "{{ value_json.RfReceived.Data }}"
    value_template: >-
      {% if value_json.RfReceived.Data == 'rotate_left' %}
        {{'ON'}}
      {% else %}
        {{states('binary_sensor.cube_left') | upper}}
      {% endif %}

In developer tools-states I can see that if I rotate the cube it sends rotate_left. With the “old” sensor configurationthe HA sensor changes its state to on and my dimming automation is working.
Using the “new” config above, if I rotate the cube I can see the rotate_left in developer tools, but the sensor state doesn’t change to on.

The reason why it fails is because the value you’ve supplied for payload is incorrect. The error message says it all:

TypeError: payload must be a string, bytearray, int, float or None.

Here it is as a string:

payload: "{'RfReceived':{'Data':'33150E'}}"

If you want to publish payloads for testing purposes, just use:

Developer tools > MQTT > Publish a packet

I get the impression you don’t understand the difference between:

value

and

value_json
  • The first makes no assumption about the received data’s format and accepts it “as-is”.
  • The second assumes the received data’s format is JSON.

If the received payload is simply rotate_left then that’s not JSON and so you shouldn’t be creating templates using value_json.

Yes, sorry, I used those 2 automations because they were already there for other purposes, I forgot the MQTT publish section.

I tried.
The following sensor:

- platform: mqtt
    #friendly_name: Porta Giardino 
    name: "Contatto_Open_Close_1"
    # payload_on: "33150A"
    # payload_off: "33150E"
    device_class: door
    state_topic: "tele/Sonoff_RF_Bridge/RESULT"
    # value_template: "{{ value_json.RfReceived.Data }}"
    value_template: >-
      {% if value_json.RfReceived.Data == '33150A' %}
        {{'ON'}}
      {% elif value_json.RfReceived.Data == '33150E' %}
        {{'OFF'}}
      {% else %}
        {{states('binary_sensor.contatto_open_close_1') | upper}}
      {% endif %}

does not change state after publishing “{‘RfReceived’:{‘Data’:‘33150E’}}” or “{‘RfReceived’:{‘Data’:‘33150A’}}”

You are completely right if you think that I’m missing many concepts.
I stopped programming about 25 years ago (C++) and I just started to read and write some code last april after buying a RPI3B+ and installin Hassio, so I’m missin maaaaaany things.

But, and forgive me if I don’t understand, why the cube was working before if the old instruction was

value_template: "{{ value_json.RfReceived.Data }}"

?
Because the payload_on was clearly expressed?

payload_on: "rotate_left"

The sensor configuration you just posted appears to be correct. If you cannot make the sensor change state then it’s due to an error in the testing procedure.

I wish I had more time to explain the fundamentals to you but, unfortunately, I do not. I suggest you review the documentation, starting here:

Good luck.

Sorry, and thanks for the help you gave me until now.
Making and error in copy-paste “{‘RfReceived’:{‘Data’:‘33150A’}}” in MQTT section and clicking “publish” is quite hard, but probably I did it anyway.

If you’re using that utility to publish the payload, the payload’s format is like I first suggested to you:

{"RfReceived":{"Data":"33150A"}}

or even this:

{'RfReceived':{'Data':'33150A'}}

You’ll notice they do not have leading and trailing double-quotes (") because that version was exclusively for use with the payload option of the automation you had created.

Thanks for the additional help.
No luck anyway.

But the sensor stays off anyway.
I think I will stop for the moment.

Ok, after going home, rechecking everything, trying to understand how json works and spending the evening on this… I fixed many little mistakes and now sensors are working.

I’m still having “No matching payload found for entity” warnings and “Error parsing value: ‘value_json’ is undefined” errors, but I will look into it.

Thank you for stimulating me and for pointing me in the right direction.

Mmm… I try to report here the error.
For every sensor I get:

No matching payload found for entity: Pir_Digoo_2 with state topic: tele/Sonoff_RF_Bridge/RESULT. Payload: 33150E, with value template Template("{% if value_json.RfReceived.Data == '1CC3A6' %} {{'ON'}} {% else %} {{states('binary_sensor.pir_digoo_2') | upper}} {% endif %}")
8:05 components/mqtt/binary_sensor.py (WARNING)
Error parsing value: 'value_json' is undefined (value: 33150E, template: {% if value_json.RfReceived.Data == '1CC3A6' %} {{'ON'}} {% else %} {{states('binary_sensor.pir_digoo_2') | upper}} {% endif %})
8:05 helpers/template.py (ERROR)

This happens both for sensor with only open state and sensors with open & close state.
If the payload received matches the one specified, the sensor works.
But if it doesn’t match, it doesn’t use the current sensor state, and the warning “no matching payload found” pops up again

- platform: mqtt
    name: "Pir_Digoo_2"
    off_delay: 5
    device_class: motion
    state_topic: "tele/Sonoff_RF_Bridge/RESULT"
    value_template: >-
      {% if value_json.RfReceived.Data == '1CC3A6' %}
        {{'ON'}}
      {% else %}
        {{states('binary_sensor.pir_digoo_2') | upper}}
      {% endif %}

But I don’t know why, because if I put {{states(‘binary_sensor.pir_digoo_2’) | upper}} in the template checker, it gives correctly OFF.

Another day, same problem. The received payload is not in JSON format. The error message states that very clearly:

No matching payload found for entity: … Payload: 33150E
Error parsing value: ‘value_json’ is undefined (value: 33150E,

Probably I will say another wrong thing, but the payload look a nested JSON to me…

{
    "Time": "2020-02-20T15:56:23",
    "RfReceived": {
        "Sync": 14010,
        "Low": 480,
        "High": 1380,
        "Data": "33150E",
        "RfKey": "None"
    }
}

What you posted looks correct to me as well but whatever Home Assistant is receiving is not understood to be correct:

Error parsing value: ‘value_json’ is undefined

By the way, is the incorrect spacing shown in your example just a mistake when you posted it here or is that the way it is in your configuration file?

- platform: mqtt
    name: "Pir_Digoo_2"
    off_delay: 5

I assume it’s just a copy-paste error.

That’s exactly what Home Assistant is receiving:

And the wrong spacing was just a copy-paste mistake: