Mqtt.py log warning "No matching payload found for entity"

@123 thank you for the solution. I’ve just had a chance to test it and marked it as the solution.

In my (OP) case, since my sensors don’t send JSON the syntax is a bit simpler. e.g. Each sensors goes from this:

  - platform: mqtt
    state_topic: "unosensor"
    device_class: safety
    name: sensor8
    payload_on: "8_ON"
    payload_off: "8_OFF"

To this:

  - platform: mqtt
    state_topic: "unosensor"
    device_class: safety
    name: sensor8
    value_template: >-
      {% if value == '8_ON' %}
        {{'ON'}}
      {% elif value == '8_OFF' %}
        {{'OFF'}}
      {% else %}
        {{states('binary_sensor.sensor8') | upper}}
      {% endif %}

And as always thank you to all including @AhmadK, @Petrica and other who kept the discussion going to get to a good solution.

That said, I still think reversing the PR that originally caused this, or at least lowering the logging severity level is better. But thanks to the creative folks in community @123’s solution is more than good enough.

1 Like

In reply to the new feature in OpenMQTTGateway where you can append the data string to the topic name…

How would I use this with a sensor that sends different codes for on and off?

I cannot see how I can use that in a binary_sensor as it cannot listen to two topics at the same time. It can wildcard everything but then we are back to listening to all topics.

You cannot toggle a binary.sensor in an automation so you would have to make some template sensor and several automations and then you may use some of the other hacks already suggested in this now very long discussion. But have I missed something?

Kenneth

Just want to share my experience.
I tried that feature and have to say it’s not for every situation. For example, it’s ok for PIRs that only send one code, but it gets increasingly complicated for anything else - door sensors, wall switches etc.

So I ended up implementing proper value_template templates as described here.
Believe it or not, there is a good reason behind this warning and you’d be better off sorting your config to avoid any “unusual” behaviour caused by ignoring it.

AhmadK

Your experience matches mine. I changed from Tasmota to OpenMQTTGateway because it had the feature of appending the value to the topic path.

But I found that for my 5 window and door sensors I ended up with having to use input_booleans and template binary sensors and automations to get a clean implementation. It got more ugly than using the value_template hack

The word hack here needs to be understood in a positive mood. You guys that contributed in this thread and other threads with the MQTT multiplexed solutions as well as the value template solution have made a huge positive impact to probably 1000s of RF bridge users. I am deeply impressed with the research you did. And I am personally using the value_template solution. I have 5 window/door sensors and 6 motion sensors and a couple of RF buttons so that solution is acceptable from a performance point of view. I deeply thank you for this solution.

I call it a hack because instead of simply specifying an and off an on payload (and assume anything else is ignored - as it is!) we have to use a complex Jinja code with an if - elif - else structure and setting the sensors to its own value. And we set the sensor to its own value also when we receive codes for different sensors. Luckily HA does not create events when you set the binary_sensor to a value it already has. It is an elegant solution to the problem and I am again deeply thankful that you guys found this solution.

It is not an ERROR to use a common MQTT topic for a radio gateway. It is the most practical solution for a gateway running on an ESP8266. The alternative solutions are

  • The gateway has to be much more complex so that you can define mqtt topics and map multiple codes to the same topic. I am thinking about rewriting OpenMQTTGateway for myself to add a hardcoded map. But it will be a pain to maintain.

  • HA could be expanded so that a binary sensor can support TWO mqtt topics. That would be a good and clean solution already supported by OpenMQTTGateway and I am sure the devs of Tasmota would add this if it was supported by HA.

  • HA should stop reporting the errors (remove 3 lines of code). It works. But it removes a debug opportunity for people that wonder why their sensor does not work.

  • HA could change the logging from errors to warnings and ensure they are not shown in the /dev_info which is an important source of info for dramatic important issues available from the browser.

  • HA could change the logging so it only reports if no sensors matches a payload. This is what the user needs to know

I am happy with my value_template solution. I am just trying to contribute with ideas that helps making HA easier to use. At the moment we count on users finding this 150+ postings thread on the forum for a solution to a common problem. I am not arguing for myself. I am quote OK. Hack till it hurts as a famous Youtuber says :wink:

Example of the value_template solution for users with OpenMQTTGateway

I would like to share a working value_template solution for users that are on OpenMQTTGateway and not Tasmota. There are a few small differences.

First Tasmota returns the data as a string and it is the value in HEX.
OpenMQTTServer returns the value as a number in decimal. This is important as you need to quote the hex string but it will not work if you quote the decimal number
The JSON Payload is different (value_json.value)

First a Binary Sensor for a sensor with both on code and off code (window/door sensors reporting open and closed). In this example I have 3 RF bridges all reporting the sensors in their own topics
OpenMQTTGateway/RFBridge1/433toMQTT
OpenMQTTGateway/RFBridge2/433toMQTT
OpenMQTTGateway/RFBridge3/433toMQTT

I do not care that I get 3 messages that trigger the sensor 3 times. The objective is to ensure the opening or closing of a window is detected. We use the wildcard + in the sensor in HA so it listens on all 3 bridges and triggers 1 to 3 times depending on coverage and noise. Note that the data is not quoted as it is a number and not a string

binary_sensor:
  - platform: mqtt
    name: "Kitchen Window RF"
    state_topic: "OpenMQTTGateway/+/433toMQTT"
    value_template: >-
      {% if value_json.value == 13718474 %}
        {{'ON'}}
      {% elif value_json.value == 13718478 %}
        {{'OFF'}}
      {% else %}
        {{states('binary_sensor.kitchen_window_rf') | upper}}
      {% endif %}

Same setup but here we use a Motion sensor that only sends a code when detecting motion. There is no off code. We use a 180 second time-out

binary_sensor:
  - platform: mqtt
    name: "Hallway Motion RF"
    state_topic: "OpenMQTTGateway/+/433toMQTT"
    value_template: >-
      {% if value_json.value == 13621870 %}
        {{'ON'}}
      {% else %}
        {{states('binary_sensor.hallway_motion_rf') | upper}}
      {% endif %}
    off_delay: 180

Finally an automation triggered by an RF button we have in the kitchen. Here used to trigger a fun message via Alexa. Note that for automations there is no issue with error logging. You can have as many automations you want listening to the same MQTT topic and the payload does not need to be matched. But again note that different value_template needed for OpenMQTTGateway compared to Tasmota.
For this automation I do not use a wildcard as it would trigger the beer message 2-3 times and I would develop an alcohol problem :wink:

automation:
  - alias: 'Black Button'
    initial_state: true
    trigger:
      platform: mqtt
      topic: "OpenMQTTGateway/RFBridge1/433toMQTT"
    condition:
      condition: template
      value_template: '{{ trigger.payload_json.value == 4389249}}'
    action:
  action:
  - service: script.alexa_tts_message
    data:
      message: 'Would you like a little beer?'

I hope this is useful for people.

All these examples also work with Tasmota but then you need a different value_template to decode the JSON payload. value_json.RfReceived.Data and you then quote the hex number in double quotes -“EB67A1”

Kenneth

1 Like

The word ‘hack’, whether used in a positive or negative sense, is misplaced when describing the solution to use a template. The implication is that every time someone uses a value_template (rather than payload_on and payload_off) it’s a ‘hack’. It’s not. Templates are one of the most useful and powerful tools in Home Assistant and they have wide-ranging applications.

With over a dozen devices, my recommendation would be to use Strategy 2: Demultiplexer. All of your sensor configurations will be greatly simplified. For example:

  - platform: mqtt
    name: "Kitchen Window RF"
    state_topic: "home/sensor1"

All it requires is one very simple automation and one python_script. You configure each sensor’s parameters (payload, topic, retain) within the python_script. You don’t have to restart Home Assistant or reload a python_script for changes to take effect.

There’s also less to process for Home Assistant. If you use ‘Strategy 1: value_template’, a single topic shared by a dozen sensors means each received payload is evaluated a dozen times (more in your case because of multiple bridges). The demultiplexer solution evaluates the payload only twice (first by the automation and then second by the targeted sensor).

Couldn’t agree more.

But you definitely need to pursue for your solution to be included in HA as the current (official) implementation is an… abomination :slight_smile:

I also have a pair of Sonoff RF Bridges running tasmota. These each publish events when they receive some RF transmission from a sensor; each RF Bridge publishes to a unique topic.

I ended up building a really simple flow in Node Red to de-duplicate events, and the publish the result to a single MQTT topic that used by Home Assistant in the same way as if you had a single RF Bridge. I was going to hack up a little python script, but it turns out a really simple flow can be defined in Node Red to solve the problem. I already had Node Red doing some other unrelated pre-processing of MQTT messages, so this was an easy addition.

When I return from travel (or can get my VPN session up to my router at home), I’ll see about capturing the details of how I did this using a “deduplicate” node that’s out there as a “contributed” node. I think I used https://flows.nodered.org/node/node-red-contrib-deduplicate-adv to this end, along with some other plumbing.

I think we park the discussion about classification and words here. I still love you, anyway :wink:

As I said above I like both solutions and it is an individual thing which one is best. For I like to understand what I am doing and I have a background with Pascal, C, and Perl but not Python. Yet! The value_template was easy to understand and maintain for me. But reading your script and how brief it is I will study it in near future - if nothing else then to learn as the python script feature in HA seems pretty powerful

From a performance point of view my HA runs on a machine that only runs HA and has plenty of CPU power. And it is also part of the decision that the RF sensor solution is one I plan to use less as it is not that reliable a technology. Soon I will have a few outdoor Motion sensors and a couple of RF buttons.

The performance gain is an interesting thing to dig into.
The value_template as you correctly say means that HA will evaluate each message on the common topic once per sensor. But CPU cycles are cheap compared to I/O. Once the MQTT message is loaded I cannot imagine it takes many CPU cycles to repeat the matches for each sensor.

The multiplexing method however means sending the result of the multiplexing to the MQTT broker as a TCP/IP communication and is also involves a delay in the broker. I bet you need to have 100+ sensors before the multiplexing method is faster. Maybe on a Raspberry Pi the gain is more significant but with an Intel based fast PC it will be the I/O between HA and MQTT broker that becomes the significant delay.

This does not take away the fact that the multiplexing solution is very elegant and performance is probably still so fast that noone notice.

I agree with you that the difference in performance may be difficult to perceive on anything other than a low-performing platform. Nevertheless, when given a choice, it seems prudent to choose the more efficient solution.

Don’t overlook the fact that most everything having to do with Home Assistant is based on python, an interpreted language. In contrast, the mosquitto MQTT broker is written in C and compiled for the target platform. The pub-sub cycle is executed very fast, especially if the client and broker are on the same host. Even if the two are hosted on a separate machines, transit time is negligible on a 100 Mbps or (1 Gbps) ethernet connection (and the payload typically fits into a single TCP packet).

Another advantage of Strategy 2 is that it leaves the door open to additional enhancements. For example, one could implement support for Home Assistant’s MQTT Discovery. That would eliminate the need to manually add sensors to the config file.

I leave enhancements to Strategy 2 an exercise for someone else (I don’t even own a Sonoff RF Bridge).

Another solution could be a differenciation of the RF value published by OMG into 2 , one as an RF sensor identifier and one as the sensor state.
If we add the RF sensor identifier to the topic, and just publish a state payload it should answer the need isn’t it?

Example with a GS WDS07:

  • decimal value when close 2952974 -> unit 61620 / state: closed
  • decimal value when open 2952970 -> unit 61620 / state: open

Will give the following publishing:

  • OpenMQTTGateway/433toMQTT/61620 {state:“close”}
  • OpenMQTTGateway/433toMQTT/61620 {state:“open”}

This approach could also a first step for an MQTT convention support like Homie

Yes. I use the WDS07 as Window sensors. But how will you get that Unit code? The sensor sends two codes. One for closed and one for open (and some codes also for battery low I believe).

But to use it the Gateway software has to know a common ID fingerprint of the two so the open and close are published to the same topic name. Where do you get the 61620 from? I cannot see any data received from the sensor that connects the open and close data to the same physical sensor.

Here is the trick:
2952974 -> bin 1011010000111100001110
2952970 -> bin 1011010000111100001010
The 20th digit correspond to the sensor state.
If we take the first 19 digits we could use it as an identifier, they are indeed a constant depending on the sensor.

61620 and this proposal comes from the pilight decoding method discussed here

Aha! Here we actually coming to another level.
I’m currently porting my Kerui D026 door sensor from OMG RF to OMG Pilight.

Here’s the issue: Kerui D026 can send 4 different codes: opened, closed, tampered and low battery.
First two can be classes as “states”, where the last two are more of “attributes” (imho)

I think that it’s a good idea to have states retained, when attributes can be used as signals for automation or to temporarily change the current state.
Therefore, I use MQTT sensor to represent the door sensor’s state and take into account the tampered attribute. In fact, I use an additional template sensor mostly because MQTT sensor does not support icon_template.

So here’s the raw MQTT sensor (you don’t need to add it to any view)

- platform: mqtt
  name: contacts_1st_floor_safe_pilight
  state_topic: 'home/contacts/1st_floor/safe'
  json_attributes_topic: 'home/contacts/1st_floor/safe/attributes'

and its UI representation

- platform: template
  sensors:
    contact_1st_floor_safe:
      friendly_name: !secret 1st_floor_safe_name
      value_template: >
        {% if is_state_attr('sensor.contacts_1st_floor_safe_pilight', 'tamper', 'on') %}
          {{ states('sensor.contact_always_tampered') }}
        {% else %}
          {{ states('sensor.contacts_1st_floor_safe_pilight') }}
        {% endif %}
      icon_template: >
        {% set state = states('sensor.contact_1st_floor_safe') %}
        {% if state == states('sensor.contact_always_opened') %}
          mdi:door-open
        {% elif state == states('sensor.contact_always_closed') %}
          mdi:door-closed
        {% elif state == states('sensor.contact_always_tampered') %}
          mdi:image-broken
        {% else %}
          mdi:cloud-question
        {% endif %}

The thing is my demultiplexer listens to the OMG Pilight topic, extracts known codes if protocol is kerri_d026 and publishes to appropriate topics (home/contacts/1st_floor/safe for opened/closed and home/contacts/1st_floor/safe/attributes for tamper).
It kind of works, but I don’t like 2 things:

  1. Handling of Pilight messages. They come as JSONs, but the main info (unicode, state etc) is within message, something like this:
{"message":"{\"unitcode\":441825,\"state\":\"tamper\"}","protocol":"kerui_D026","length":"","repeats":2,"status":2}

I have to parse them myself in every script as it’s a string and there is no way to use something like value_json,message.state. I yet to find a way to do it nicely. So for now it’s like

known_states = ['opened', 'closed']
known_attributes = ['tamper']
#known_states = stable_states + temporary_states

entity = None
state = None

# remove curly brackets and split into "key:value" strings
params = message.replace('{', '').replace('}', '').split(',')

# clean and store parameters - have to do ALL of them, awful!
for param in params:
    key, value = param.split(':')
    key = key.replace('"','')

    if key == 'unitcode':
        id = int(float(value))
        for e in known_entities.keys():
          if id in known_entities[e]:
            entity = e
            break
    elif key == 'state':
        state = value.replace('"','').lower()

if entity is None:
    logger.debug('unknown entity, ignored')
elif state is None:
    logger.error('no state information provided for %s, ignored', entity)

and only after that I can do something…
By the way, I don’t know who and why created this format of messages, but it would be great to make it either “plain” (one-level) JSON or JSON-able (i.e when message is formed as JSON as well).

  1. The way I handle that tamper attribute. I want the UI sensor to show “tampered” for a number of seconds instead of its state and then go back to it. That way my alarm panel would be activated as it can react not only on opened, but on tamper as well.
    Currently I achieve that by publishing JSON {“tamper”:“on”} and then {“tamper”:“off”} to home/contacts/1st_floor/safe/attributes, but I have to add a delay between these 2 commands to make the change visible for a set amount of time. Don’t like it.

Any ideas how to improve that?

So all in all, there is no single solution, and it’s not that great and bright, but we are getting there :wink:

Hi.
I’m trying to get rid of all those warnings.
I modified some MQTT sensors.
This, for example (the commented lines were the previous, working, configuration):

- platform: mqtt 
    name: "Contatto_Open_Close_2"
    # payload_on: "02ADC3"
    # payload_off: "02ADC9"
    device_class: garage_door
    state_topic: "tele/Sonoff_RF_Bridge/RESULT"
    # value_template: "{{ value_json.RfReceived.Data }}"
    value_template: >-
      {% if value_json.RfReceived.Data == '02ADC3' %}
        {{'ON'}}
      {% elif value_json.RfReceived.Data == '02ADC9' %}
        {{'OFF'}}
      {% else %}
        {{states('binary_sensor.contatto_open_close_2') | upper}}
      {% endif %}

If I publish 02ADC3 value, the sensor does not activate, and in the log I read:

2020-02-19 13:37:14 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: Contatto_Open_Close_2 with state topic: tele/Sonoff_RF_Bridge/RESULT. Payload: 02ADC3, with value template Template("{% if value_json.RfReceived.Data == '02ADC3' %}
  {{'ON'}}
{% elif value_json.RfReceived.Data == '02ADC9' %}
  {{'OFF'}}
{% else %}
  {{states('binary_sensor.contatto_open_close_2') | upper}}
{% endif %}")

I can’t figure out what’s wrong…

Correct, because the payload you’ve published is not in the format the value_template requires.

Look at what the value_template is attempting to extract from the received payload:

value_json.RfReceived.Data

It expects to receive a payload in JSON format, specifically a dictionary containing the keys RfReceived and Data.

To test sensor.contatto_open_close_2, publish this payload to the state_topic:

{"RfReceived":{"Data":"02ADC3"}}

Mmm… ok, now I try, I’m rebooting, but even before applying your “fix”, the payload was value_json.RfReceived.Data, and simply publishing the payload value was working.

it doesn’t work.
I tested it on a less critical, but identical, sensor.

- alias: Segna_sensore_box_ON
    trigger:
      - platform: state
        entity_id: input_boolean.sensore_sezionale
        to: 'on'
    action:
      - service: mqtt.publish
        data:
           topic: 'tele/Sonoff_RF_Bridge/RESULT'
           payload: {"RfReceived":{"Data":"33150A"}}
           
  - alias: Segna_sensore_box_OFF
    trigger:
      - platform: state
        entity_id: input_boolean.sensore_sezionale
        to: 'off'
    action:
      - service: mqtt.publish
        data:
           topic: 'tele/Sonoff_RF_Bridge/RESULT'
           payload: {"RfReceived":{"Data":"33150E"}}

but the sensor state doesn’t change.

Error while executing automation automation.segna_sensore_box_on. Unknown error for call_service at pos 1:
Traceback (most recent call last):
File “/usr/src/homeassistant/homeassistant/components/automation/init.py”, line 397, in async_trigger
await self.action_script.async_run(variables, trigger_context)
File “/usr/src/homeassistant/homeassistant/helpers/script.py”, line 247, in async_run
await self._handle_action(action, variables, context)
File “/usr/src/homeassistant/homeassistant/helpers/script.py”, line 331, in _handle_action
await self._actions[_determine_action(action)](action, variables, context)
File “/usr/src/homeassistant/homeassistant/helpers/script.py”, line 413, in _async_call_service
context=context,
File “/usr/src/homeassistant/homeassistant/helpers/service.py”, line 96, in async_call_from_config
domain, service_name, service_data, blocking=blocking, context=context
File “/usr/src/homeassistant/homeassistant/core.py”, line 1226, in async_call
await asyncio.shield(self._execute_service(handler, service_call))
File “/usr/src/homeassistant/homeassistant/core.py”, line 1251, in _execute_service
await handler.func(service_call)
File “/usr/src/homeassistant/homeassistant/components/mqtt/init.py”, line 681, in async_publish_service
await hass.data[DATA_MQTT].async_publish(msg_topic, payload, qos, retain)
File “/usr/src/homeassistant/homeassistant/components/mqtt/init.py”, line 816, in async_publish
self._mqttc.publish, topic, payload, qos, retain
File “/usr/local/lib/python3.7/concurrent/futures/thread.py”, line 57, in run
result = self.fn(*self.args, **self.kwargs)
File “/usr/local/lib/python3.7/site-packages/paho/mqtt/client.py”, line 1260, in publish
‘payload must be a string, bytearray, int, float or None.’)
TypeError: payload must be a string, bytearray, int, float or None.

Another example: Xiaomi Magic Cube.
I modified the sensor to be like this:

- platform: mqtt
    name: "Cube Left"
    # payload_on: "rotate_left"
    off_delay: 1
    device_class: moving
    state_topic: "zigbee2mqtt/0x00158d00029aaf18"
    # value_template: "{{ value_json.RfReceived.Data }}"
    value_template: >-
      {% if value_json.RfReceived.Data == 'rotate_left' %}
        {{'ON'}}
      {% else %}
        {{states('binary_sensor.cube_left') | upper}}
      {% endif %}

In developer tools-states I can see that if I rotate the cube it sends rotate_left. With the “old” sensor configurationthe HA sensor changes its state to on and my dimming automation is working.
Using the “new” config above, if I rotate the cube I can see the rotate_left in developer tools, but the sensor state doesn’t change to on.

The reason why it fails is because the value you’ve supplied for payload is incorrect. The error message says it all:

TypeError: payload must be a string, bytearray, int, float or None.

Here it is as a string:

payload: "{'RfReceived':{'Data':'33150E'}}"

If you want to publish payloads for testing purposes, just use:

Developer tools > MQTT > Publish a packet