433toMQTTto433 Bidirectional Arduino Gateway

I have no clue… That needs to be directed toward @1technophile as he is the expert.

First of all thank you for the project.
I’ve built it from scratch for the nodeMCUv2 with the PiLight option enabled and ALL the other libraries included. So the define is basically on for PiLight and off for everything else - since RF is not supported at the same time.
However, I can only see RF switches, which work with the RF library anyway, but not weather sensors. I have sensors which use the Nexus protocol and is otherwise successfully decoded with nexus_433 and rtl_433.
Any idea what could be missing? I could attach some screenshots, logs, but the configuration seems ok and I’m looking at all the MQTT messages.

Hello and welcome!

the weather stations needs to be compatible with Pilight to be supported, is it the case for yours?
https://wiki.pilight.org/devices

Here is the tested devices compatible with OMG:

1 Like

Thank you for the quick reply. I finally had time to play with the code and PiLight and can confirm that my weather sensor is not supported by EspPiLight.
It follows a similar protocol to Digoo R8H, so I will try to find time to implement it into PiLight.

There was a post on this forum: https://community.openmqttgateway.com/t/digoo-dg-r8h-temperature-sensor/539/5 and I’ve followed similar instructions to build a “ZgatewayRF3” decoder. Unfortunately, the HomeGW code is not stable enough and has a lot of false positives and erroneous data, as it doesn’t do any kind of error checking. It could be made to be a bit more robust, but, I think it would be a waste of time as the RF pin needs to be dedicated and the protocol fully reimplemented, including the discovery subscription model. Making a PiLight decoder should be the way to go.

The other options I’ve looked at:

  • rtl_433 - sometimes triggers low voltage on the rPi and does not have an optimized antenna for 433 Mhz. It also uses about 20% CPU constantly. It does however decode everything perfectly.
  • various Arduino/ESP8266 bridges - do not decode all protocols and have much much worse integration (such as MQTT and captive portal)
  • nexus_433 - works fine for nexus but uses a lot of CPU from the rPi and does not decode other protocols.

I agree. Pilight is the solution to GO, in my point of view the open source solution with the bigger list of supported devices. And the most extensible one.

1 Like

I have a few questions, if you or someone else wants to answer:

How to best integrate PiLight into HA? I could think that a demultiplexer could work - one which parses the JSON and auto-generates switches/sensor - but perhaps there are better ways.

How to tell the OMG PIO project to the ESPilight library from the lib folder (synced to a local git) instead of the one from github? I’ve changed the library include to be ./lib/ESPiLight but sometimes it is copied to my .pio/libdeps/nodemcuv2-test folder, sometimes the GitHub version is used.
Even with that approach, if I could get it working, I still have to manually delete the libdeps/…/ESPiLight subfolder before a build so that my library changes are recognized.
The other option is to modify directly into the subfolder and risk the chance that it might get overwritten with the GitHub version by an update/clean.

I will let HA specialist answer on this one.

Maybe there is remaining links, did you try to clean your .pio folder, here is some information that can help:

1 Like

I’ve started working last night on the Nexus sensor integration but it proved to be quite challenging. According to some documentation (I did not do my own measurements) the sensors send 12 bursts of 36 bits i.e. 12 x 36 x 2 = 864 pulses. I haven’t seen any PiLight example that is able to deal with so many pulses and catch an incomplete train of them. Within so many pulses there are bound to be errors, typically spurious shorter pulses within the normal ones (e.g. 100us interspersed with the usual 500, 1000 and 2000us).
Essentially, one must be able to capture at least 2 trains, preferably 3 and see that they all carry the same data. This would not be so hard if there was a consistent way to detect the start and end of a train.

I will leave this aside for now until I can gather some more programming tips, it was not just a simple matter of adjusting timings.

Thanks for the feedback!

Just returned from vacation and had time to play a bit some more. The decoding was quickly done but the biggest issue was that ESPiLight had a minimum delay of 5100ms between messages. Changing this to 3700 allowed the nexus messages to be properly detected, as the length of the sync/repeat pulse is ~4000ms.
After cleaning up of the old code (I created a processor for a train instead of a single sequence) I will try to create a pull request for ESPiLight.

Question: currently all messages get published together under the home/OpenMQTTGateway_ESP8266_RF/PilighttoMQTT topic, regardless of the device being used. So I have both:

{"message":{"id":157,"channel":0,"battery":0,"temperature":22.6,"humidity":60},"protocol":"nexus","length":"157","repeats":2,"status":2}

and

{"message":{"id":16,"systemcode":341,"unit":7,"state":"off"},"protocol":"daycom","length":"16","repeats":4,"status":2}

under the same topic.

I’ve enabled Zmqttdiscovery but it doesn’t seem to do anything useful for the PiLight integration. I’ve tried adding a custom section at the end of ZgatewayPilight.pilightCallback() to try autodiscovery but I didn’t research exactly how it’s supposed to work.
My idea was to check if msg[“temperature”] is available in the resulting JSON and create a few discovery topics (similar to DHT). Otherwise, if there is a “state” message member, it’s probably a switch/binary sensor.

I don’t get it how the OMG + PiLight is supposed to integrate with HA. The only way I see it, an MQTT demultiplexer should be written which handles this: MQTT - topic - help
Is there any easier way?

Depending on the code modified you should better submit it in the pilight repository directly.

Yes, mqtt discovery for pilight is not implemented. And currently we don’t how what would be the best way to do it.

There was some discussions here :

Maybe @ahmadk or @Petrica have some ideas.

Thanks for the answer!

Ugh, I realized this as I was cleaning the code. I think I copied the files for the protocol directly into the ESPilight repo folder, so my changes are now untracked. Need to figure out how to best do this under Windows, as it’s supposed to work with links. My C/C++ + VS skills are not that strong, I would rather let someone else handle all this and do a pre-review.

I started it but it proved to be a bit of a can of worms:

  • ESPiLight doesn’t send the device type or name back to the callback
  • Just adding the hardware type to the callback breaks all previous code
  • Default parameters cannot be implemented in a class, but it could be done by adding another callback
  • the protocol structure is not public so it cannot be returned - was thinking of opening up findProtocol()
  • while trying to do auto-detection back into OMG, I got unexpected resets a few times

It would be probably easier to do this “heavy” development on an ESP32 instead, as one can use OpenOCD for ICD. AFAIK this cannot be done with NodeMCU.

I started attacking this topic and managed to get it working in 1-2h. It’s easier to “develop” in YAML and do a quick automation reload. It’s ugly and has hardcoded bits but it does the job fine for now:

- alias: OMG Pilight demultiplexer
  id: 1a9ac126b34641c09824f583c33fd9f5
  trigger:
    platform: mqtt
    topic: home/OpenMQTTGateway_ESP8266_RF/PilighttoMQTT
  condition:
    - condition: template
      value_template: '{% if trigger.payload_json[''message''][''temperature''] %} true {% endif %}'
  action:

    - service: mqtt.publish
      data_template:
        topic: '{{ ''home/sensor/PiLight'' + trigger.payload_json.protocol|string + trigger.payload_json.message.id|string + ''/state'' }}'
        payload: '{{trigger.payload}}'

    - service: mqtt.publish
      data_template:
        topic: '{{ ''homeassistant/sensor/PiLight'' + trigger.payload_json.protocol|string + trigger.payload_json.message.id|string + ''_temperature/config'' }}'
        payload: >
          {"name": "PiLight {{trigger.payload_json.protocol|string}} Sensor Id:{{trigger.payload_json.message.id|string}} channel {{trigger.payload_json.message.channel|string}} Temperature","device_class": "temperature","state_topic": "home/sensor/PiLight{{trigger.payload_json.protocol|string}}{{trigger.payload_json.message.id|string}}/state","unit_of_measurement": "°C","value_template": "{{"{{ value_json['message']['temperature'] }}"}}","expire_after": 300}

    - service: mqtt.publish
      data_template:
        topic: '{{ ''homeassistant/sensor/PiLight'' + trigger.payload_json.protocol|string + trigger.payload_json.message.id|string + ''_humidity/config'' }}'
        payload: >
          {"name": "PiLight {{trigger.payload_json.protocol|string}} Sensor Id:{{trigger.payload_json.message.id|string}} channel {{trigger.payload_json.message.channel|string}} Humidity","device_class": "humidity","state_topic": "home/sensor/PiLight{{trigger.payload_json.protocol|string}}{{trigger.payload_json.message.id|string}}/state","unit_of_measurement": "%","value_template": "{{"{{ value_json['message']['humidity'] }}"}}","expire_after": 300}

The automations.yaml snippet above catches everything that’s published by OMG and creates separate state topics for each temperature sensor. It then publishes two new auto-discovery topics, for temperature and humidity, pointing back to the state topic:
image

Third topic: I ordered some time ago a relatively 433MHz PIR sensor from Amazon and thought it would be detected by PiLight. As it seems, it’s not, so I will probably have another protocol on the way. I’ve already decoded the transmission type (with URC) and just need to figure out how channel selection works. Would like to know if there’s anyone willing to ffer help for a code review before a pull request.

As signals from all devices come to one gateway and it does not separate them into individual mqtt topics, a demultiplexer is needed if you don’t want to duplicate a chunk of YAML every time you add a new device.
I personally use a simple automation and a python script to do the job for DIGOO R8S sensors:

- alias: pilight2sensor demultiplexer
  initial_state: true
  trigger:
    - platform: mqtt
      topic: !secret pilight_bridge_state_topic
  condition:
    condition: template
    value_template: "{{ trigger.payload_json.protocol == 'tfa'}}"
  action:
    - service: python_script.set_values_from_pilight_message
      data_template:
        id: "{{ trigger.payload_json.message.id }}"
        temperature: "{{ trigger.payload_json.message.temperature }}"
        humidity: "{{ trigger.payload_json.message.humidity }}"
        battery: "{{ trigger.payload_json.message.battery }}"

but your use case might be different and require another approach. I do not use any autodicsovery features at all, don’t see any point so far, at least in OMG case (are there any btw?)

I’m in the same boat with Sonoff PIR2 - works fine with RF module, but is unknown to Pilight :frowning:
Happy to do something to get it supported but don’t know where exactly to start…

Do you mean that ESPilight PR? I have some general C++/C background, but not sure it would help in such a specific case…
Anyway, I think you’re safe to make a PR and if something is not quite right, they’ll let you know :wink:

I use several 433 Mhz PIR and open sensors with OMG RF on Arduino Mega and ethernet shield, then a demultiplexer in HA. The OMG RF is, by far, the most stable smart device I have (the initial setup was done a long time ago and everything is manually setup in HA). Although I tried OMG PiLight, I found multiple decoding errors or two different codes for some of the devices successfully identified by OMG RF so I will stick with it.

For instance, the following two posts occur simultaneously when a PIR sensor triggers (from what I’ve found ev1527 is a generic protocol used by many 433 Mhz devices; also, I don’t own any Kerui D026 sensor).

home/OpenMQTTGatewayPilightTest/PilighttoMQTT
{"message":{"unitcode":34499,"state":"opened"},"protocol":"kerui_D026","length":"","repeats":2,"status":2}
Creating RF PiLight buffer
Pub json into:
home/OpenMQTTGatewayPilightTest/PilighttoMQTT
{"message":{"unitcode":507038,"state":"closed"},"protocol":"ev1527","length":"","repeats":2,"status":2}
Creating RF PiLight buffer

For environment sensors (temperature, humidity and light) I use different solutions:

  • ESP8266 with DHT22&LDR with OMG code for all rooms except bathroom (due to high moisture I didn’t wanted to go anywhere near the mains); each sensor posts the readings in its own topic so no need for a demultiplexer;
  • Digoo R8H weather sensors with RFLink (currently not supported by PiLight) for bathroom and inside the fridge :slight_smile: For the bathroom I use another R8H with the humidity sensor de-soldered and substituted with a LDR (humidity scale is 0 to 100% which was much easier to work with compared to the temperature scale published by the temperature sensor).

Cool setup.

You say that PiLight gives errors but I see that you pasted an OMG Pilight trace.
I’ve initially thought that RF would be the way to go but PiLight actively supports a lot more protocols and is just not worth trying to implement each from scratch. Personally, I’ve found that rtl_433 supports everything I own, but I had issues with undervoltage, CPU usage and randomly dying decoder process.

How do you power them? The biggest advantage, for me, for these R8H sensors is that they are self-contained and batteries last for >1 year. One of them has been sitting outside in the rain for months now and still works.

Why LDR? I thought humidity is a more practical application for a bathroom.

Does the demultiplexer look similar to what I’ve posted above and the one by @AhmadK ?

Yeap, one of the issues I have with PiLight concerns decoding: for one PIR sensor is decoding two payloads, while for another PIR (of the same brand) is only showing only one.

I use standard USB phone chargers (nothing fancy, however I try to avoid knockouts). I literally have one such device in every room except for the bathroom.

However, for the RF gateway I use an Asus powerbank as it supports passthrough charging and cost is lower than of an UPS.

My R8H batteries are only good for 7-8 months (I tried with both rechargeable and regular batteries).

There was a misunderstanding :slight_smile: : I use two R8H units in the bathroom (one untouched that provides temperature and humidity and one modified that provides light intensity readings). Although payloads from R8H is received at each 30 seconds which I consider sufficient, the LDR sensors in the other rooms send the payloads based on changes in the light intensity immediately so that the automation for the lights can resend the payload if bulbs didn’t react.

Pretty much similar, except in my case the RF output is not in json format so it only deals with simple payloads.

Hi there Petrica,
can you tell me how do you receive data from DIGOO DG R8H please ?
Do you use the OMG Pilight ? with witch hardware ?

Thanks a lot
Multumesc mult :slight_smile:

Hi Denis,

Currently I’m using RF Link (Arduino Mega) to receive data from R8H (most of the time it is pretty much stable but I would like to move all 433Mhz receiving part to an independent device). I know that Florian is working to add support for it in the OMG, however I haven’t been able to closely follow the topic. I will then update my oldest and most reliable smart device, which is a 4 years running Arduino Mega with one of the very early 433Mhz OMG builds.

Although R8H are pretty nice devices and very cheap, I’m more fond of an all-in-one solution consisting of NodeMCU + DHT22 + LDR with OMG for running sensor nodes in each room. With passthrough powerbanks, these devices run for months (one of them even for 27 months) without any reboot being needed whatsoever.

I mostly use R8H in environments with high humidity, where I don’t want to fiddle with electric related dangers, such as in the bathroom and the fridge :slight_smile: After replacing the humidity sensor with a LDR (I could have replaced the temperature sensor but the humidity uses a 0-100% scale more easy to work with, rather that dealing with both positive and negative values), drilling the case and using lots of hot glue I also ended up with a reliable light intensity sensor; it ain’t gonna win a beauty prize but it is perfect to be used with motion sensor in low light conditions (I could have placed LDR directly on the PIR motion sensor in order to trigger it only in low light but it would limit PIR usage with an alarm system).

You’re welcome! (Cu placere :slight_smile: )

1 Like

Dear Petrica, thanks a lot for your answer.
This can be a solution, but cause I’m so new in this world, can you tell me about powerbanks,
in order to make it work for about one year please ? how much batts (18650?) need it ?
Or if you have a link that explain me the combination of NodeMCU + DHT22 + LDR with OMG please.

Here is the link where Florian tell us that DIGOO R8H has been added to Pilight:

But I don’t understand so well… maybe we must whait some type of compilation or release of the Pilight

Thanks a lot again
Denis

Hi,

I think I might need to clarify what my previous note means:

It’s not that the NodeMCU with all the sensors can run for 27 months of a single powerbank charge (regardless of powerbank capacity). It means that the powerbank can prevent any situation where a reboot of the NodeMCU device is needed (either due to an error within the code or due to a mains power failure). The powerbank runs in passthrough mode, thus the 5V needed for the board is still fed by the mains, though a USB charger.

In order to be able to run a NodeMCU for a long period on a battery then a sketch with deep sleep enabled functionality would be needed, which is beside the point made by me above as the NodeMCU would sleep for most of the time and only wake up and send data under certain conditions which are time related (ie. wake up once per hour/day and send sensors data to MQTT) or triggered by an external module (this is the functionality of the battery operated Tuya devices which employ a secondary MCU that will wake up the ESP82xx when the sensors are triggered, then ESP82xx connects to wifi and sends its payload; however, this process takes several seconds and it is not really suitable for cases where immediate feedback is needed). It should also be noted that wifi is quite power hungry and cannot be expected to run for long periods of small batteries if continuous connection with the smart home software is needed (not even 18650s could provide power for extended periods).

In respect of powerbank brand, they should specifically include passthrough mode in their description in order to be able to both charge the internal accumulator and provide power to external devices (I use 10,000 mAh Asus devices and I know Xiaomi has several in their portfolio). There are 18650 modules on the Chinese sites that claim similar functionality but it is always difficult to find a reputable seller who did not cut corners.

Here you can find the necessary documentation for building an OMG with Platformio (it’s not that difficult, at least not when compared to the Arduino build, where you didn’t easily knew what software library version was needed for which project :slight_smile: )

https://docs.openmqttgateway.com/upload/pio.html

My personal use case is for the data from the LDR sensor (light intensity data) to be sent to HA immediately after a certain delta from previous value is registered (I use this data in order to adjust motion triggering lighting, so, even if the PIR sensor triggers, the bulbs only turn on if there is insufficient light). For the humidity and intensity data I only need them once per 5 minutes and I use these data for monitoring the environment condition in my house (to turn on heat - with a relay connected to a NodeMCU - or AC - with IR blaster and IR receiver connected to NodeMCU). As humidity and temperature they don’t change very often it this is ok for me; however, there might be cases where much lower time delays would be needed (I helped someone build a greenhouse monitoring system consisting of about 40 NodeMCUs with DHT22 sensors sending data on a 10 seconds interval).

If you only started going the smart home route I suggest reading a lot of the documentation in multiple topics. It is quite a steep learning curve; however, once things start moving on, I think that time invested is totally worth it.

I think that only Florian can advise on the status of the implementation.