Mqtt.py log warning "No matching payload found for entity"

I think we park the discussion about classification and words here. I still love you, anyway :wink:

As I said above I like both solutions and it is an individual thing which one is best. For I like to understand what I am doing and I have a background with Pascal, C, and Perl but not Python. Yet! The value_template was easy to understand and maintain for me. But reading your script and how brief it is I will study it in near future - if nothing else then to learn as the python script feature in HA seems pretty powerful

From a performance point of view my HA runs on a machine that only runs HA and has plenty of CPU power. And it is also part of the decision that the RF sensor solution is one I plan to use less as it is not that reliable a technology. Soon I will have a few outdoor Motion sensors and a couple of RF buttons.

The performance gain is an interesting thing to dig into.
The value_template as you correctly say means that HA will evaluate each message on the common topic once per sensor. But CPU cycles are cheap compared to I/O. Once the MQTT message is loaded I cannot imagine it takes many CPU cycles to repeat the matches for each sensor.

The multiplexing method however means sending the result of the multiplexing to the MQTT broker as a TCP/IP communication and is also involves a delay in the broker. I bet you need to have 100+ sensors before the multiplexing method is faster. Maybe on a Raspberry Pi the gain is more significant but with an Intel based fast PC it will be the I/O between HA and MQTT broker that becomes the significant delay.

This does not take away the fact that the multiplexing solution is very elegant and performance is probably still so fast that noone notice.

I agree with you that the difference in performance may be difficult to perceive on anything other than a low-performing platform. Nevertheless, when given a choice, it seems prudent to choose the more efficient solution.

Don’t overlook the fact that most everything having to do with Home Assistant is based on python, an interpreted language. In contrast, the mosquitto MQTT broker is written in C and compiled for the target platform. The pub-sub cycle is executed very fast, especially if the client and broker are on the same host. Even if the two are hosted on a separate machines, transit time is negligible on a 100 Mbps or (1 Gbps) ethernet connection (and the payload typically fits into a single TCP packet).

Another advantage of Strategy 2 is that it leaves the door open to additional enhancements. For example, one could implement support for Home Assistant’s MQTT Discovery. That would eliminate the need to manually add sensors to the config file.

I leave enhancements to Strategy 2 an exercise for someone else (I don’t even own a Sonoff RF Bridge).

Another solution could be a differenciation of the RF value published by OMG into 2 , one as an RF sensor identifier and one as the sensor state.
If we add the RF sensor identifier to the topic, and just publish a state payload it should answer the need isn’t it?

Example with a GS WDS07:

  • decimal value when close 2952974 -> unit 61620 / state: closed
  • decimal value when open 2952970 -> unit 61620 / state: open

Will give the following publishing:

  • OpenMQTTGateway/433toMQTT/61620 {state:“close”}
  • OpenMQTTGateway/433toMQTT/61620 {state:“open”}

This approach could also a first step for an MQTT convention support like Homie

Yes. I use the WDS07 as Window sensors. But how will you get that Unit code? The sensor sends two codes. One for closed and one for open (and some codes also for battery low I believe).

But to use it the Gateway software has to know a common ID fingerprint of the two so the open and close are published to the same topic name. Where do you get the 61620 from? I cannot see any data received from the sensor that connects the open and close data to the same physical sensor.

Here is the trick:
2952974 -> bin 1011010000111100001110
2952970 -> bin 1011010000111100001010
The 20th digit correspond to the sensor state.
If we take the first 19 digits we could use it as an identifier, they are indeed a constant depending on the sensor.

61620 and this proposal comes from the pilight decoding method discussed here

Aha! Here we actually coming to another level.
I’m currently porting my Kerui D026 door sensor from OMG RF to OMG Pilight.

Here’s the issue: Kerui D026 can send 4 different codes: opened, closed, tampered and low battery.
First two can be classes as “states”, where the last two are more of “attributes” (imho)

I think that it’s a good idea to have states retained, when attributes can be used as signals for automation or to temporarily change the current state.
Therefore, I use MQTT sensor to represent the door sensor’s state and take into account the tampered attribute. In fact, I use an additional template sensor mostly because MQTT sensor does not support icon_template.

So here’s the raw MQTT sensor (you don’t need to add it to any view)

- platform: mqtt
  name: contacts_1st_floor_safe_pilight
  state_topic: 'home/contacts/1st_floor/safe'
  json_attributes_topic: 'home/contacts/1st_floor/safe/attributes'

and its UI representation

- platform: template
  sensors:
    contact_1st_floor_safe:
      friendly_name: !secret 1st_floor_safe_name
      value_template: >
        {% if is_state_attr('sensor.contacts_1st_floor_safe_pilight', 'tamper', 'on') %}
          {{ states('sensor.contact_always_tampered') }}
        {% else %}
          {{ states('sensor.contacts_1st_floor_safe_pilight') }}
        {% endif %}
      icon_template: >
        {% set state = states('sensor.contact_1st_floor_safe') %}
        {% if state == states('sensor.contact_always_opened') %}
          mdi:door-open
        {% elif state == states('sensor.contact_always_closed') %}
          mdi:door-closed
        {% elif state == states('sensor.contact_always_tampered') %}
          mdi:image-broken
        {% else %}
          mdi:cloud-question
        {% endif %}

The thing is my demultiplexer listens to the OMG Pilight topic, extracts known codes if protocol is kerri_d026 and publishes to appropriate topics (home/contacts/1st_floor/safe for opened/closed and home/contacts/1st_floor/safe/attributes for tamper).
It kind of works, but I don’t like 2 things:

  1. Handling of Pilight messages. They come as JSONs, but the main info (unicode, state etc) is within message, something like this:
{"message":"{\"unitcode\":441825,\"state\":\"tamper\"}","protocol":"kerui_D026","length":"","repeats":2,"status":2}

I have to parse them myself in every script as it’s a string and there is no way to use something like value_json,message.state. I yet to find a way to do it nicely. So for now it’s like

known_states = ['opened', 'closed']
known_attributes = ['tamper']
#known_states = stable_states + temporary_states

entity = None
state = None

# remove curly brackets and split into "key:value" strings
params = message.replace('{', '').replace('}', '').split(',')

# clean and store parameters - have to do ALL of them, awful!
for param in params:
    key, value = param.split(':')
    key = key.replace('"','')

    if key == 'unitcode':
        id = int(float(value))
        for e in known_entities.keys():
          if id in known_entities[e]:
            entity = e
            break
    elif key == 'state':
        state = value.replace('"','').lower()

if entity is None:
    logger.debug('unknown entity, ignored')
elif state is None:
    logger.error('no state information provided for %s, ignored', entity)

and only after that I can do something…
By the way, I don’t know who and why created this format of messages, but it would be great to make it either “plain” (one-level) JSON or JSON-able (i.e when message is formed as JSON as well).

  1. The way I handle that tamper attribute. I want the UI sensor to show “tampered” for a number of seconds instead of its state and then go back to it. That way my alarm panel would be activated as it can react not only on opened, but on tamper as well.
    Currently I achieve that by publishing JSON {“tamper”:“on”} and then {“tamper”:“off”} to home/contacts/1st_floor/safe/attributes, but I have to add a delay between these 2 commands to make the change visible for a set amount of time. Don’t like it.

Any ideas how to improve that?

So all in all, there is no single solution, and it’s not that great and bright, but we are getting there :wink:

Hi.
I’m trying to get rid of all those warnings.
I modified some MQTT sensors.
This, for example (the commented lines were the previous, working, configuration):

- platform: mqtt 
    name: "Contatto_Open_Close_2"
    # payload_on: "02ADC3"
    # payload_off: "02ADC9"
    device_class: garage_door
    state_topic: "tele/Sonoff_RF_Bridge/RESULT"
    # value_template: "{{ value_json.RfReceived.Data }}"
    value_template: >-
      {% if value_json.RfReceived.Data == '02ADC3' %}
        {{'ON'}}
      {% elif value_json.RfReceived.Data == '02ADC9' %}
        {{'OFF'}}
      {% else %}
        {{states('binary_sensor.contatto_open_close_2') | upper}}
      {% endif %}

If I publish 02ADC3 value, the sensor does not activate, and in the log I read:

2020-02-19 13:37:14 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: Contatto_Open_Close_2 with state topic: tele/Sonoff_RF_Bridge/RESULT. Payload: 02ADC3, with value template Template("{% if value_json.RfReceived.Data == '02ADC3' %}
  {{'ON'}}
{% elif value_json.RfReceived.Data == '02ADC9' %}
  {{'OFF'}}
{% else %}
  {{states('binary_sensor.contatto_open_close_2') | upper}}
{% endif %}")

I can’t figure out what’s wrong…

Correct, because the payload you’ve published is not in the format the value_template requires.

Look at what the value_template is attempting to extract from the received payload:

value_json.RfReceived.Data

It expects to receive a payload in JSON format, specifically a dictionary containing the keys RfReceived and Data.

To test sensor.contatto_open_close_2, publish this payload to the state_topic:

{"RfReceived":{"Data":"02ADC3"}}

Mmm… ok, now I try, I’m rebooting, but even before applying your “fix”, the payload was value_json.RfReceived.Data, and simply publishing the payload value was working.

it doesn’t work.
I tested it on a less critical, but identical, sensor.

- alias: Segna_sensore_box_ON
    trigger:
      - platform: state
        entity_id: input_boolean.sensore_sezionale
        to: 'on'
    action:
      - service: mqtt.publish
        data:
           topic: 'tele/Sonoff_RF_Bridge/RESULT'
           payload: {"RfReceived":{"Data":"33150A"}}
           
  - alias: Segna_sensore_box_OFF
    trigger:
      - platform: state
        entity_id: input_boolean.sensore_sezionale
        to: 'off'
    action:
      - service: mqtt.publish
        data:
           topic: 'tele/Sonoff_RF_Bridge/RESULT'
           payload: {"RfReceived":{"Data":"33150E"}}

but the sensor state doesn’t change.

Error while executing automation automation.segna_sensore_box_on. Unknown error for call_service at pos 1:
Traceback (most recent call last):
File “/usr/src/homeassistant/homeassistant/components/automation/init.py”, line 397, in async_trigger
await self.action_script.async_run(variables, trigger_context)
File “/usr/src/homeassistant/homeassistant/helpers/script.py”, line 247, in async_run
await self._handle_action(action, variables, context)
File “/usr/src/homeassistant/homeassistant/helpers/script.py”, line 331, in _handle_action
await self._actions[_determine_action(action)](action, variables, context)
File “/usr/src/homeassistant/homeassistant/helpers/script.py”, line 413, in _async_call_service
context=context,
File “/usr/src/homeassistant/homeassistant/helpers/service.py”, line 96, in async_call_from_config
domain, service_name, service_data, blocking=blocking, context=context
File “/usr/src/homeassistant/homeassistant/core.py”, line 1226, in async_call
await asyncio.shield(self._execute_service(handler, service_call))
File “/usr/src/homeassistant/homeassistant/core.py”, line 1251, in _execute_service
await handler.func(service_call)
File “/usr/src/homeassistant/homeassistant/components/mqtt/init.py”, line 681, in async_publish_service
await hass.data[DATA_MQTT].async_publish(msg_topic, payload, qos, retain)
File “/usr/src/homeassistant/homeassistant/components/mqtt/init.py”, line 816, in async_publish
self._mqttc.publish, topic, payload, qos, retain
File “/usr/local/lib/python3.7/concurrent/futures/thread.py”, line 57, in run
result = self.fn(*self.args, **self.kwargs)
File “/usr/local/lib/python3.7/site-packages/paho/mqtt/client.py”, line 1260, in publish
‘payload must be a string, bytearray, int, float or None.’)
TypeError: payload must be a string, bytearray, int, float or None.

Another example: Xiaomi Magic Cube.
I modified the sensor to be like this:

- platform: mqtt
    name: "Cube Left"
    # payload_on: "rotate_left"
    off_delay: 1
    device_class: moving
    state_topic: "zigbee2mqtt/0x00158d00029aaf18"
    # value_template: "{{ value_json.RfReceived.Data }}"
    value_template: >-
      {% if value_json.RfReceived.Data == 'rotate_left' %}
        {{'ON'}}
      {% else %}
        {{states('binary_sensor.cube_left') | upper}}
      {% endif %}

In developer tools-states I can see that if I rotate the cube it sends rotate_left. With the “old” sensor configurationthe HA sensor changes its state to on and my dimming automation is working.
Using the “new” config above, if I rotate the cube I can see the rotate_left in developer tools, but the sensor state doesn’t change to on.

The reason why it fails is because the value you’ve supplied for payload is incorrect. The error message says it all:

TypeError: payload must be a string, bytearray, int, float or None.

Here it is as a string:

payload: "{'RfReceived':{'Data':'33150E'}}"

If you want to publish payloads for testing purposes, just use:

Developer tools > MQTT > Publish a packet

I get the impression you don’t understand the difference between:

value

and

value_json
  • The first makes no assumption about the received data’s format and accepts it “as-is”.
  • The second assumes the received data’s format is JSON.

If the received payload is simply rotate_left then that’s not JSON and so you shouldn’t be creating templates using value_json.

Yes, sorry, I used those 2 automations because they were already there for other purposes, I forgot the MQTT publish section.

I tried.
The following sensor:

- platform: mqtt
    #friendly_name: Porta Giardino 
    name: "Contatto_Open_Close_1"
    # payload_on: "33150A"
    # payload_off: "33150E"
    device_class: door
    state_topic: "tele/Sonoff_RF_Bridge/RESULT"
    # value_template: "{{ value_json.RfReceived.Data }}"
    value_template: >-
      {% if value_json.RfReceived.Data == '33150A' %}
        {{'ON'}}
      {% elif value_json.RfReceived.Data == '33150E' %}
        {{'OFF'}}
      {% else %}
        {{states('binary_sensor.contatto_open_close_1') | upper}}
      {% endif %}

does not change state after publishing “{‘RfReceived’:{‘Data’:‘33150E’}}” or “{‘RfReceived’:{‘Data’:‘33150A’}}”

You are completely right if you think that I’m missing many concepts.
I stopped programming about 25 years ago (C++) and I just started to read and write some code last april after buying a RPI3B+ and installin Hassio, so I’m missin maaaaaany things.

But, and forgive me if I don’t understand, why the cube was working before if the old instruction was

value_template: "{{ value_json.RfReceived.Data }}"

?
Because the payload_on was clearly expressed?

payload_on: "rotate_left"

The sensor configuration you just posted appears to be correct. If you cannot make the sensor change state then it’s due to an error in the testing procedure.

I wish I had more time to explain the fundamentals to you but, unfortunately, I do not. I suggest you review the documentation, starting here:

Good luck.

Sorry, and thanks for the help you gave me until now.
Making and error in copy-paste “{‘RfReceived’:{‘Data’:‘33150A’}}” in MQTT section and clicking “publish” is quite hard, but probably I did it anyway.

If you’re using that utility to publish the payload, the payload’s format is like I first suggested to you:

{"RfReceived":{"Data":"33150A"}}

or even this:

{'RfReceived':{'Data':'33150A'}}

You’ll notice they do not have leading and trailing double-quotes (") because that version was exclusively for use with the payload option of the automation you had created.

Thanks for the additional help.
No luck anyway.

But the sensor stays off anyway.
I think I will stop for the moment.

Ok, after going home, rechecking everything, trying to understand how json works and spending the evening on this… I fixed many little mistakes and now sensors are working.

I’m still having “No matching payload found for entity” warnings and “Error parsing value: ‘value_json’ is undefined” errors, but I will look into it.

Thank you for stimulating me and for pointing me in the right direction.