433toMQTTto433 Bidirectional Arduino Gateway

EAN · March 7, 2018, 1:21pm

kanga_who · April 28, 2018, 12:14am

I have been using this project for some time, fantastic bit of kit. I do have an issue however.

I have recently added 3 door sensors that send both open, and closed codes. That part works perfectly. The issue is when HA is restarted, if a door is open, it will revert back to closed on reboot. The open status does not seem to be retained.

This is the binary_sensor I have set up for one of the doors.

- platform: mqtt
  name: "Front Door"
  qos: 1 # I have tried 0 for this as well
  state_topic: "home/433toMQTT"
  payload_on: "12510986"
  payload_off: "12510990"
  device_class: door
  retain: true

I have even tried using an automation to publish the state on change to attempt to have it retained, no joy.

- alias: Set State of Front Door
  trigger:
    platform: state
    entity_id: binary_sensor.front_door
    to: 'on'
  action:
    service: mqtt.publish
    data:
      topic: 'home/433toMQTT'
      payload: '12510986'
      retain: true

Is there something obvious I am doing wrong?

Petrica · April 28, 2018, 6:38pm

Hi,

There is nothing wrong with your settings. OpenMQTT is set to publish without retain flag.

Your automation to publish the state is on the wright track, however you might want to use another topic for publishing in order not to get confused as the home/433toMQTT gets overwritten each time a new message is received (and it is not retained, thus when restarting HA it is blank). There are a few components that support restore state however binary sensor is not among them.

- alias: Set State of Front Door on
  trigger:
    platform: state
    entity_id: binary_sensor.front_door
    to: 'on'
  action:
    service: mqtt.publish
    data:
      topic: 'home/devices/frontdoor'
      payload: 'on'
      retain: true

- alias: Set State of Front Door off
  trigger:
    platform: state
    entity_id: binary_sensor.front_door
    to: 'off'
  action:
    service: mqtt.publish
    data:
      topic: 'home/devices/frontdoor'
      payload: 'off'
      retain: true

and then the binary sensor (the one that you have would become an intermediate sensor and you can hide it from HA):

- platform: mqtt
  name: "Front Door Modified"
  qos: 0 
  state_topic: 'home/devices/frontdoor'
  payload_on: "on"
  payload_off: "off"
  device_class: door

Actually, you can use a single automation in order to reduce number of entities managed by HA (as the trigger would look for changes in the binary sensor or not set an intermediate binary sensor after all, but in this case it needs the condition part to look for either payloads on or off - 12510986 or 12510990). Are you comfortable with that?

kanga_who · April 29, 2018, 2:03am

@Petrica Thanks for the reply. I had assumed that the retain flag wasn’t being sent. This is something that would be very beneficial to be added in, however.

I will adjust my automation and see how I go. Thanks again.

kanga_who · April 29, 2018, 4:38am

@Petrica I think I have it sorted, thanks for the suggestions. Here is my config, just using one door sensor as an example. I have a single automation covering all the open/close events for all 3 doors (I think)

binary_sensor.yaml

- platform: mqtt
  name: "frontdoor"
  qos: 0
  state_topic: "home/433toMQTT"
  payload_on: "12510986"
  payload_off: "12510990"

- platform: mqtt
  name: "Front Door"
  qos: 0
  state_topic: 'home/door'
  payload_on: "12510986"
  payload_off: "12510990"
  device_class: door
  retain: true

automation.yaml

# DOOR STATE #
- alias: Door State
  initial_state: True
  trigger:
    platform: mqtt
    topic: home/433toMQTT
  action:
    service: mqtt.publish
    data_template:
      topic: "home/door"
      payload_template: '{{ trigger.payload }}'
      retain: true

Petrica · April 29, 2018, 4:58am

Hi,

Yes, one automation to rule them all

Except you would need to change the other two mqtt binary sensors to monitor “home/door” topic too. There is no need for any binary sensor to monitor “home/433toMQTT” topic anymore (you end up with one automation as the sole overhead).

Look to Mqtt.py log warning "No matching payload found for entity" for description of the log warnings that you will going to have

kanga_who · April 29, 2018, 5:05am

Yes, I have all 3 doors using the /home/door topic, I think that is what you mean?

The problem now is I need to write an automation to have the home/433toMQTT binary sensor grab the state of the /home/door binary sensor as when I reboot twice, the state is lost again as the 433toMQTT binary sensor resets at boot. I hope that makes sense.

Petrica · April 29, 2018, 5:09am

You only need to keep the automation from above.

The binary sensors will become as following:

- platform: mqtt
  name: "Front Door"
  qos: 0
  state_topic: 'home/door'
  payload_on: "12510986"
  payload_off: "12510990"
  device_class: door
  retain: true

- platform: mqtt
  name: "Other Door 1"
  qos: 0
  state_topic: 'home/door'
  payload_on: "11111"
  payload_off: "22222"
  device_class: door
  retain: true

- platform: mqtt
  name: "Other Door 2"
  qos: 0
  state_topic: 'home/door'
  payload_on: "33333"
  payload_off: "44444"
  device_class: door
  retain: true

Petrica · April 29, 2018, 5:19am

But again, this is a quick and dirty fix (the downside is that if the doors are opening/closing while HA is restarting then the state will be in topic “home/433toMQTT”; because automation is not working until HA is up again the topic “home/door” is not being updated).

The best way would be to have OpenMQTT publishing with Retain flag (and thus to have binary sensors monitor “home/433toMQTT” again).

kanga_who · April 29, 2018, 5:23am

@Partybug can we make this happen at all?

Partybug · June 5, 2018, 8:18pm

I have no clue… That needs to be directed toward @1technophile as he is the expert.

ha-username · August 22, 2019, 3:39pm

First of all thank you for the project.
I’ve built it from scratch for the nodeMCUv2 with the PiLight option enabled and ALL the other libraries included. So the define is basically on for PiLight and off for everything else - since RF is not supported at the same time.
However, I can only see RF switches, which work with the RF library anyway, but not weather sensors. I have sensors which use the Nexus protocol and is otherwise successfully decoded with nexus_433 and rtl_433.
Any idea what could be missing? I could attach some screenshots, logs, but the configuration seems ok and I’m looking at all the MQTT messages.

1technophile · August 22, 2019, 5:24pm

Hello and welcome!

the weather stations needs to be compatible with Pilight to be supported, is it the case for yours?
https://wiki.pilight.org/devices

Here is the tested devices compatible with OMG:

ha-username · August 28, 2019, 5:08pm

Thank you for the quick reply. I finally had time to play with the code and PiLight and can confirm that my weather sensor is not supported by EspPiLight.
It follows a similar protocol to Digoo R8H, so I will try to find time to implement it into PiLight.

There was a post on this forum: https://community.openmqttgateway.com/t/digoo-dg-r8h-temperature-sensor/539/5 and I’ve followed similar instructions to build a “ZgatewayRF3” decoder. Unfortunately, the HomeGW code is not stable enough and has a lot of false positives and erroneous data, as it doesn’t do any kind of error checking. It could be made to be a bit more robust, but, I think it would be a waste of time as the RF pin needs to be dedicated and the protocol fully reimplemented, including the discovery subscription model. Making a PiLight decoder should be the way to go.

The other options I’ve looked at:

rtl_433 - sometimes triggers low voltage on the rPi and does not have an optimized antenna for 433 Mhz. It also uses about 20% CPU constantly. It does however decode everything perfectly.
various Arduino/ESP8266 bridges - do not decode all protocols and have much much worse integration (such as MQTT and captive portal)
nexus_433 - works fine for nexus but uses a lot of CPU from the rPi and does not decode other protocols.

1technophile · August 28, 2019, 6:03pm

I agree. Pilight is the solution to GO, in my point of view the open source solution with the bigger list of supported devices. And the most extensible one.

ha-username · August 29, 2019, 7:08pm

I have a few questions, if you or someone else wants to answer:

How to best integrate PiLight into HA? I could think that a demultiplexer could work - one which parses the JSON and auto-generates switches/sensor - but perhaps there are better ways.

–

How to tell the OMG PIO project to the ESPilight library from the lib folder (synced to a local git) instead of the one from github? I’ve changed the library include to be ./lib/ESPiLight but sometimes it is copied to my .pio/libdeps/nodemcuv2-test folder, sometimes the GitHub version is used.
Even with that approach, if I could get it working, I still have to manually delete the libdeps/…/ESPiLight subfolder before a build so that my library changes are recognized.
The other option is to modify directly into the subfolder and risk the chance that it might get overwritten with the GitHub version by an update/clean.

1technophile · August 29, 2019, 7:16pm

I will let HA specialist answer on this one.

Maybe there is remaining links, did you try to clean your .pio folder, here is some information that can help:

ha-username · September 4, 2019, 1:42pm

I’ve started working last night on the Nexus sensor integration but it proved to be quite challenging. According to some documentation (I did not do my own measurements) the sensors send 12 bursts of 36 bits i.e. 12 x 36 x 2 = 864 pulses. I haven’t seen any PiLight example that is able to deal with so many pulses and catch an incomplete train of them. Within so many pulses there are bound to be errors, typically spurious shorter pulses within the normal ones (e.g. 100us interspersed with the usual 500, 1000 and 2000us).
Essentially, one must be able to capture at least 2 trains, preferably 3 and see that they all carry the same data. This would not be so hard if there was a consistent way to detect the start and end of a train.

I will leave this aside for now until I can gather some more programming tips, it was not just a simple matter of adjusting timings.

1technophile · September 4, 2019, 6:59pm

Thanks for the feedback!

ha-username · September 28, 2019, 7:46pm

Just returned from vacation and had time to play a bit some more. The decoding was quickly done but the biggest issue was that ESPiLight had a minimum delay of 5100ms between messages. Changing this to 3700 allowed the nexus messages to be properly detected, as the length of the sync/repeat pulse is ~4000ms.
After cleaning up of the old code (I created a processor for a train instead of a single sequence) I will try to create a pull request for ESPiLight.

Question: currently all messages get published together under the home/OpenMQTTGateway_ESP8266_RF/PilighttoMQTT topic, regardless of the device being used. So I have both:

{"message":{"id":157,"channel":0,"battery":0,"temperature":22.6,"humidity":60},"protocol":"nexus","length":"157","repeats":2,"status":2}

and

{"message":{"id":16,"systemcode":341,"unit":7,"state":"off"},"protocol":"daycom","length":"16","repeats":4,"status":2}

under the same topic.

I’ve enabled Zmqttdiscovery but it doesn’t seem to do anything useful for the PiLight integration. I’ve tried adding a custom section at the end of ZgatewayPilight.pilightCallback() to try autodiscovery but I didn’t research exactly how it’s supposed to work.
My idea was to check if msg[“temperature”] is available in the resulting JSON and create a few discovery topics (similar to DHT). Otherwise, if there is a “state” message member, it’s probably a switch/binary sensor.

I don’t get it how the OMG + PiLight is supposed to integrate with HA. The only way I see it, an MQTT demultiplexer should be written which handles this: MQTT - topic - help
Is there any easier way?