Mqtt.py log warning "No matching payload found for entity"

[AhmadK (akasma74)]

@Petrica

Best way to solve this is to reverse the PR that introduced the warning in the first place.

I started from there, but read this and my next post.

[AhmadK (akasma74)]

Ok, I think I got it, thanks to @123 .

An MQTT binary sensor HAS to know its state, and it can be done either by providing value_template that returns ON/OFF (default payload_on/payload_off values) OR making sure payload_on and payload_off are the only values that being published to that topic.
Having only payload_on and payload_off defined and allowing something that is not matched by above mentioned payloads to be published to that topic triggers that warning.
So basically that warning means: look, your MQTT switch received a message, but couldn’t react (as there is no corresponding rule in a form of payload_xxx OR value_template ), something is wrong with your configuration.
Pretty obvious, isn’t it.

And there are at least 3 ways to get rid of that warning with binary_sensors:

1. Provide value_template definition that returns only ON/OFF
2. Provide payload_on and payload_off AND make sure these are the only 2 values that are allowed in that topic.
3. Provide value_template definition that returns only payload_on or payload_off values

I’m off to fix my switches

[Petrica (PetricaM)]

@AhmadK

So basically that warning means: look, your MQTT switch received a message, but couldn’t react (as there is no corresponding rule in a form of payload_xxx OR value_template ), something is wrong with your configuration.

But it’s not wrong OpenMQTT binary sensors work fine while removing the four lines in the binary_sensor.py.

Anyway, I can’t test it but for binary sensors that only use one value (switches, PIR sensors) an automation that forwards each payload into its separate mqtt topic could be used and then change the binary sensor topic (for open door sensors or other devices that send two codes it needs templating).

- platform: "mqtt"
  name: "friendly_name"
  state_topic: "home/433toMQTT/code_from_device"
  payload_on: 'code_from_device'
  payload_off: 'off'
  force_update: true
  off_delay: 1


- alias: OpenMQTT forward
  initial_state: True
  trigger:
    - platform: mqtt
      topic: 'home/433toMQTT'
  action:
    - service: mqtt.publish
      data_template:
        topic: 'home/433toMQTT/{{trigger.payload}}'
        payload_template: '{{ trigger.payload}}'

[AhmadK (akasma74)]

@Petrica

OpenMQTT binary sensors work fine while removing the four lines in the binary_sensor.py.

yes, it’s not wrong.
I presume the author just wanted to make easier debugging situations like “Why on Earth my sensor is not working?!” as potentially that can happen.
And yes, you can get rid of these warnings just by fixing your config and not touching the HA code, result!

@Petrica

an automation that forwards each payload into its separate mqtt topic could be used

yes, there are many ways to skin a cat
I use input_booleans and this automation (one per device)

- alias: "[int] pir__catch_code__groung_floor__hall"
  initial_state: true
  trigger:
    platform: mqtt
    topic: !secret rf_bridge__topic
  condition:
    condition: template
    value_template: "{{ (trigger.payload_json['value'] == 13937454) and (states.input_boolean.pir_groung_floor_hall.state == 'off') }}"
  action:
  - service: input_boolean.turn_on
    data:
      entity_id: input_boolean.pir_groung_floor_hall
  - delay: !secret pir__reset_indicator_interval
  - service: input_boolean.turn_off
    data:
      entity_id: input_boolean.pir_groung_floor_hall

but maybe it’s time to enable value as topic suffix and replace it with MQTT binary sensor with off_delay…

UPDATE: here is my version for PIRs - instead of input_boolean and automation I can have just

- platform: mqtt
  name: pir_1st_floor_landing
  state_topic: !secret pir_1st_floor_landing_topic
  value_template: 'ON'
  device_class: motion # this bit can be moved to customize_glob.yaml for all pir_*
  off_delay: !secret pir_off_delay

I like it!

[123 (Taras)]

@Petrica

for binary sensors that only use one value (switches, PIR sensors) an automation that forwards each payload into its separate mqtt topic …

I imagine the automation you posted was meant to represent the concept and not a working example? These lines below will produce a horrendously long MQTT topic because the typical payload is a long string in JSON format.

    - service: mqtt.publish
      data_template:
        topic: 'home/433toMQTT/{{trigger.payload}}'
        payload_template: '{{ trigger.payload}}'

Ideally, you want to demultiplex tele/rf_bridge/RESULT using each sensor’s unique identifier, or its unique payload, instead of its entire payload. I’ve already supplied links to two other relevant examples of demultiplexer automations (see above), but it’s worth repeating at least one of them.

In this post, the assumption is the RF buttons are uniquely identified by Sync.

  - alias: 'sonoff bridge demultiplexer'
    hide_entity: true
    trigger:
      platform: mqtt
      topic: tele/sonoff/RESULT
    action:
      service: mqtt.publish
      data_template:
        topic: "home/button/{{trigger.payload_json.RfReceived.Sync|string }}"
        payload: 'ON'

Binary_sensors are defined for each RF button:

  - platform: mqtt
    name: "Button 1"
    state_topic: "home/button/13550" # <--- 13550 comes from Sync
    off_delay: 1

  - platform: mqtt
    name: "Button 2"
    state_topic: "home/button/13560"
    off_delay: 1

If Sync can’t be used to uniquely identify the RF device, then the recourse is to use the device’s payload which is often represented by Data.

    action:
      service: mqtt.publish
      data_template:
        topic: "home/button/{{trigger.payload_json.RfReceived.Data|string }}"
        payload: 'ON'

  - platform: mqtt
    name: "Button 1"
    state_topic: "home/button/2B4AC"
    off_delay: 1

This works as long as the RF devices only report when they are on and not off such as certain models of motion sensor, RF button, and contact sensor (that’s the same constraint you mentioned for your automation example).

If the RF devices report both their on and off states (two payloads), yet have no unique identifier within the payload, then this simplistic demultiplexing automation is no longer useful. It will produce two topics per sensor and that’s unusable for MQTT Sensor or Binary Sensor.

For this situation, the automation’s data_template requires a more complex template that is aware of each sensor’s Data payload, representing on and off, and forwards it to the sensor’s single topic. It’s not difficult, just more work.

[Petrica (PetricaM)]

@123

I imagine the automation you posted was meant to represent the concept and not be a working example? These lines will produce a horrendously long MQTT topic because the typical payload is a long string in JSON format.

Nope; it is a working automation for use with OpenMQTT 433Mhz gateway (codes received from the 433Mhz devices are up to 10 char in length if not using the JSON functionality for the payload). Agree that using JSON payload in the topic is a no go.

[klogg]

For PIRs with only one state I use the method given by @123

@123

• platform: mqtt name: ‘Hallway Motion’ state_topic: ‘tele/RF_Bridge/RESULT’ value_template: >- {% if value_json.RfReceived.Data == ‘E5D30E’ %} {{‘ON’}} {% else %} {{states(‘binary_sensor.hallway_motion’) | upper}} {% endif %} off_delay: 15 device_class: motion

Am I missing something* or is there a down side to this solution, because I can’t see one? All the other solutions seem to require more code.

*Serious question, not trying to be difficult.

[123 (Taras)]

@klogg

is there a down side to this solution

None that I’m aware of.

FWIW, if there are many sensors, my preferred solution is the demultiplexer because I believe it’s more efficient. Here’s why:

• Imagine you have ten binary_sensors and all are subscribed to one topic: tele/rf_bridge/RESULT.
• Each time the topic has a new payload it’s evaluated ten times (once by each binary_sensor).

Using a demultiplexer:

• Each time the topic has a new payload it’s evaluated by the automation which forwards the payload to a specific topic.
• The binary_sensor subscribed to the topic evaluates the payload.

That’s a maximum of two evaluations for every payload as opposed to ten per payload.

So if you only have 2 or 3 devices then the demultiplexer automation and the template-based solution are equally efficient. However, when there are many devices, the demultiplexer has the advantage.

[klogg]

@123

However, when there are many devices, the demultiplexer has the advantage

Good point. But (and I haven’t entirely thought this through…) how does this method deal with a mix of sensors some of which have two states and some only one. Specifically, what if you wish to retain the states of door/window sensors which have two states but do not want to retain states of PIR sensors which have only one?

The demultiplexer would presumably need a (possibly very) complex template to decide when to retain?

[123 (Taras)]

@klogg

The demultiplexer would presumably need a (possibly very) complex template to decide when to retain?

I offer you the “deluxe” solution.

Let’s say we have three RF sensors:

1. A contact sensor that reports both its on and off states (assume ‘A1A’ = ‘ON’ and ‘A2A’ = ‘OFF’).
2. A motion sensor that reports only its on state (assume ‘B1B’ = ‘ON’).
3. A button that only reports its on state (assume ‘C1C’ = ‘ON’).

• Because the contact sensor reports both of its states, the demultiplexer automation can use retain: true when publishing to the contact sensor’s topic. When Home Assistant restarts, it will re-subscribe to the contact sensor topic’s and receive its current (retained) state from the broker.

• The other two sensors (motion sensor and button) don’t report their off state. We will use the off_delay option to automatically reset them to off. However, this technique does not publish off to their respective MQTT topic (i.e. once set to on the topic will always show on). Therefore the demultiplexer automation should publish to their topics using retain: false. Otherwise, using retain: true, when you restart Home Assistant, it would erroneously set these sensors to on.

To make the automation neater, and avoid elaborate if-elif-else trees, we will use dictionaries.

This one defines which commands belongs to a sensor’s topic.

{ 'A1A':'sensor1', 'A2A':'sensor1', 'B1B':'sensor2', 'C1C':'sensor3' }

This one defines which commands represents ON or OFF.

{ 'A1A':'ON', 'A2A':'OFF', 'B1B':'ON', 'C1C':'ON' }

Finally, this one defines which commands are retained or not.

{ 'A1A':'true', 'A2A':'true', 'B1B':'false', 'C1C':'false' }

Putting it all together, we get this demultiplexer automation. If it receives a command that is not defined in the dictionaries, it will publish it to home/unknown (with retain: false).

- alias: 'RF_Bridge demultiplexer'
  hide_entity: true
  trigger:
    platform: mqtt
    topic: tele/RF_Bridge/RESULT
  action:
    service: mqtt.publish
    data_template:
      topic: >-
        {% set cmd = trigger.payload_json.RfReceived.Data %}
        {% set commands = { 'A1A':'sensor1', 'A2A':'sensor1', 'B1B':'sensor2', 'C1C':'sensor3' } %}
        {% set topic = commands[cmd] if cmd in commands.keys() else 'unknown' %}
        home/{{topic}}
      payload: >-
        {% set cmd = trigger.payload_json.RfReceived.Data %}
        {% set commands = { 'A1A':'ON', 'A2A':'OFF', 'B1B':'ON', 'C1C':'ON' } %}
        {% set payload = commands[cmd] if cmd in commands.keys() else cmd %}
        {{payload}}
      retain: >-
        {% set cmd = trigger.payload_json.RfReceived.Data %}
        {% set commands = { 'A1A':'true', 'A2A':'true', 'B1B':'false', 'C1C':'false' } %}
        {% set retain = commands[cmd] if cmd in commands.keys() else 'false' %}
        {{retain}}

Configuring the binary sensors becomes an easy task:

  - platform: mqtt
    name: 'Bathroom Door'
    state_topic: 'home/sensor1'
    device_class: Door

  - platform: mqtt
    name: 'Hallway Motion'
    state_topic: 'home/sensor2'
    off_delay: 5
    device_class: motion

  - platform: mqtt
    name: 'Button1'
    state_topic: 'home/sensor3'
    off_delay: 1

FWIW, I tested all of this and can confirm it works.

[klogg]

@123

I offer you the “deluxe” solution

Indeed you do!

Thanks, that is not only a great solution but a bit of an education for me too. Tomorrow I was planning on embarking on some

@123

elaborate if-elif-else trees

@123

I tested all of this and can confirm it works.

I’ll let you know if it doesn’t

EDIT: So far, so good! Thank you once again.

[sentur]

This seems like a very ingenious solution. Do we still need to have a sensor that represents the current RFRecieved Data from the SonoffRF unit in order for the the trigger to work?

Ie this…

- platform: mqtt
  state_topic: 'tele/RF_Bridge/RESULT'
  value_template: '{{ value_json.RfReceived.Data }}'
  name: "Sonoff RF Bridge"
  expire_after: 1

[123 (Taras)]

@sentur

Do we still need to have a sensor that represents the current RFRecieved Data from the SonoffRF unit in order for the the trigger to work?

This is the demultiplexer automation’s trigger:

  trigger:
    platform: mqtt
    topic: tele/RF_Bridge/RESULT

No other sensor is needed to trigger it.

[Tdw]

Sincere thanks for sharing this solution. Equally impressive and effective.

[kslb (KSLBe)]

My working and tested solution (for alarming purposes)…

# ALARM - vklopi alarm DSC spodaj (RF ključ C)
- id: '30006'
  alias: 'ALARM - vklopi alarm DSC spodaj (RF ključ C)'
  hide_entity: true
  trigger:
    - platform: mqtt
      topic: "tele/Sonoff-RF-Bridge/RESULT"
    - platform: mqtt
      topic: "tele/Sonoff-RF-Bridge2/RESULT"
  condition:
    condition: and
    conditions:
    - condition: template
      value_template: "{{ trigger.payload_json.RfReceived.Data == '!RF-SECRET-RECEIVED-CODE-KEY-C' }}"
    - condition: state
      entity_id: alarm_control_panel.DSC5020
      state: disarmed
  action:
    service: alarm_control_panel.alarm_arm_away
    data:
      entity_id: alarm_control_panel.DSC5020
      code: !SECRET-DSC-ALARM-CODE

# ALARM - izklopi alarm DSC spodaj (RF ključ D)
- id: '30007'
  alias: 'ALARM - izklopi alarm DSC spodaj (RF ključ D)'
  hide_entity: true
  trigger:
    - platform: mqtt
      topic: "tele/Sonoff-RF-Bridge/RESULT"
    - platform: mqtt
      topic: "tele/Sonoff-RF-Bridge2/RESULT"
  condition:
    condition: or
    conditions:
    - condition: template
      value_template: "{{ trigger.payload_json.RfReceived.Data == '!RF-SECRET-RECEIVED-CODE-KEY-D' }}"
    - condition: state
      entity_id: alarm_control_panel.DSC5020
      state: armed_away
    - condition: state
      entity_id: alarm_control_panel.DSC5020
      state: armed_home
  action:
    service: alarm_control_panel.alarm_disarm
    data:
      entity_id: alarm_control_panel.DSC5020
      code: !SECRET-DSC-ALARM-CODE

[kslb (KSLBe)]

Before I had binary_sensor which was switched ON/OFF via payload ON/OFF…

[123 (Taras)]

I assume you are aware that using a 433MHz RF button to arm/disarm an alarm panel is not very secure. The RF codes are easily detectable and reproducible. You would need to (minimally) use a rolling-code system and, as far as I know, RF bridges based on stock and Tasmota firmware don’t support rolling-code.

If you are aware of the vulnerability and feel the risk is acceptable then all’s well. However, be advised that for a professionally-installed residential, especially commercial, alarm system, it would be unacceptable to use an unencrypted and unchanging RF signal to control it.

[AhmadK (akasma74)]

Thanks for your solution.
I spent last 2 days playing with various ways of handling my PIRs, smoke detectors, door contacts, RF wall switches and remotes.

First I enabled adding code to each mqtt message and changed all my config to work with it.
Have to say it’s great for devices that only send one code (PIRs, smoke detectors) - you just need to define corresponding mqtt binary sensors.
For devices with two or more codes things get complicated as apart from creating mqtt switch/sensors you need automations to forward every command to a corresponding switch/sensor topic.

It worked, but I didn’t like it because of extra automations and need for plenty of !secret constants.

Then I tried your solution. Have to say in that form it’s ok for demonstration, but not suitable if you have a real setup even with a modest number of devices.
First of all, I believe it’s hard to maintain 3 separate but interlinked dicts if you have plenty of devices, many possibilities to make a mistake.
Second, in my setup (Hass.io @ RPi3 with Mosquitto add-on) mqtt messages take a while to appear and therefore reposting mqtt messages makes it even slower (not dramatically, but still).

I went for a proper dict that contains records with code, command and retain flag:

          'sensor/smoke_detector/1st_floor/landing': {'codes': {5997800: {'state': 'detected', 'command': 'ON'}}},
          'wall_switch/1st_floor/master_bedroom': {
            'codes': {
              10952785: {'state': 'on', 'command': 'ON'},
              10952784: {'state': 'off', 'command': 'OFF'}
            }
          }

That means I have to use nested for loops instead of simple lookup:

        {% set devices = {...}
        {% set code = trigger.payload_json.value | int %}
        {% set ns = namespace(topic='unknown') %}
        {% for device, definition in devices.items() %}
          {% for state_code, details in definition.codes.items() if state_code | int == code %}
            {% if loop.first %}
              {% set ns.topic = device %}
            {% endif %}
          {% endfor %}
        {% endfor %}
        home/{{ ns.topic }}

It might add up to slow response as well.
And my major complain is the size of dict AND the fact that I need to replicate it to each template (command, topic and retain). It doesn’t look great and I don’t like to copy it every time I add/change something in devices’ description.

Any thoughts on how it can be improved, or maybe a better solution?

[Petrica (PetricaM)]

@AhmadK

Second, in my setup (Hass.io @ RPi3 with Mosquitto add-on) mqtt messages take a while to appear and therefore reposting mqtt messages makes it even slower (not dramatically, but still).

Migrate to NUC

MicroSD card will fail exactly when you needed it the most, anyway.

[123 (Taras)]

@AhmadK

When I started designing the ‘deluxe’ demultiplexer automation, I thought of using an ‘all-in-one’ dictionary where each value is an array containing topic, payload, and retain.

Example: x

{% set x = { 'A1A':['sensor1','ON','true'], 'A2A':['sensor1','OFF','true'], 'B1B':['sensor2','ON','false'], 'C1C':['sensor3','ON','false'] } %}

The problem is the scope of x is limited to within the option where it’s defined. That means, as you have also discovered, you have to define the dictionary three times: one for topic:, another for payload: and yet another for retain:.

Although Jinja2 supports macros, their scope is also limited to where they are defined. See this thread for more information: Jijnja2 macro in template

Given that I had to define the dictionary three times, I chose to make it compact. Each one of the three dictionaries only contains what’s needed for its associated option:

topic   { 'A1A':'sensor1', 'A2A':'sensor1', 'B1B':'sensor2', 'C1C':'sensor3' }
payload { 'A1A':'ON',      'A2A':'OFF',     'B1B':'ON',      'C1C':'ON' }
retain  { 'A1A':'true',    'A2A':'true',    'B1B':'false',   'C1C':'false' }

Yes, this makes it more challenging to construct because each command’s data is spread over three dictionaries. However, this is a ‘one-time’ editing exercise and the dictionary’s structure is simple (basic key:value pairs).

@AhmadK

First of all, I believe it’s hard to maintain 3 separate but interlinked dicts if you have plenty of devices, many possibilities to make a mistake.

I agree interlinked dictionaries aren’t optimal but, seeing that we’re obliged to have three of them (due to scope limitations) it’s a way of keeping them compact. The alternative is to use the ‘all-in-one’ style of dictionary (shown above) and use three full copies (for topic, payload, and retain). Effectively, this is what you’ve done but with a more complex dictionary design.

Second, in my setup (Hass.io @ RPi3 with Mosquitto add-on) mqtt messages take a while to appear and therefore reposting mqtt messages makes it even slower (not dramatically, but still).

I have not noticed a delay on my system. I’m running Home Assistant on an old laptop and the Mosquitto broker runs on a Raspberry Pi 3B. They’re separate hosts yet MQTT communications are virtually instantaneous.

Any thoughts on how it can be improved, or maybe a better solution?

If you don’t like the demultiplexer solution, there’s always the other solution I offered: create a value_template for each sensor that always returns a valid state.