Project: RFLink to MQTT using ESP8266 and arduino IDE

Share your Projects!
#1

With the demise of ninjablocks, I had all these RF433 sensors that I wanted to make use of. Especially temperature sensors that use Manchester encoding, which the RCSwitch library cannot handle.

So I found RFLink online and was impressed that they had a large and growing number of protocols supported. The only thing is it assumes you are connected via USB to a computer that listens to its output and deals with it.

I have taken a slightly more agnostic approach to using the RFLink RF433 reciever/transmitter. Rather than trying to build a component in HASS to handle every possibility, I wanted to just have the data from the RFLink exposed on the MQTT broker and then HASS (or anything else) can decide what to do with it.

I have treated the RFLink as a black box - the people there have an excellent number of protocols supported including Manchester encoding (which you cant do with RCSwitch and I need for temperature sensors etc).

This little bridge connects to the arduino Mega running RFLink to a NodeMCU V1.0 via software serial. The NodeMCU (ESP8266) takes the output of the RFLink and converts it to a JSON payload in MQTT for consumption from anyone who has access ā€¦ which in my case is HASS!

The ESP8266 is programmed using arduino IDE - the arduino file and relevant HASS yaml sections are includid in my github project.

The arduino code works but likely needs a little touching up.

Open to constructive comments.

Assumes you have the hardware - Arduino Mega 2560, RF433 RX/TX per RFLink requirements, NodeMCU ESP8266. If you have different hardware, you may need to tweak.

Phil

3 Likes
433toMQTTto433 Gateway Device List
RFLink on Robotdyn ATMEGA 2560 with embedded ESP8266 and ESPEasy
433toMQTTto433 Bidirectional Arduino Gateway
433toMQTTto433 Bidirectional Arduino Gateway
#2

Nice. I designed my own mqtt rf gateway years ago and adapted it to HA when I started using it. In fact I dumbed it down because previous to HA it ran my whole automation. HA now does that but I am limited in codes and protocols and itā€™s a pain to add new ones quickly. I also wrote an mqtt receiver for my Oregon Scientific v3 sensors to be able to use the data in HA but theyā€™re 2 separate units. I have seen the RFLink software but didnā€™t like the fact that itā€™s serial. All mine are ethernet/mqtt. I like what youā€™re doing though and will follow closely. May be a nice replacement one day :smiley:

EDIT: Saw your payloads. The other thing I did was to do most of the raw processing in the RF gateway and only pass basic data down to HA. That way I didnā€™t have to template in HA, rather do it in C in the Arduino and send clean data to HA sensors etc. Also I used really weird RF switches when I first started. They used a rolling code and couldnā€™t find anything to decode them so I modified RCSwitch and added my protocol. I nearly have replaced all of them with Sonoff switches now and almost use the gateway to receive mainly.

#3

Thanks for the feedback Kman

Interested in your experience

I am trying to keep the ESP work as light as possible so it does not miss another RF code coming in ā€¦ And I am consciously leaving RFLink alone as a stock ā€œblack boxā€ so that changes can be uploaded to the mega it runs on if you want those changes to protocols.

My arduino/MQTT temp sensors I already use do post a separate topic per reading (ie one for temp, one for hum), but given the array of data coming out of the RFLink, I decided to jump the shark and embrace JSON with the idea it might be more efficient to build a ā€œblobā€ of data and post it in one. It also then keeps the data together in a time sense.

I have not bumped into rolling codes yet - hence my MQTT topic section being name-ID concatenated. Was trying to have a way for some persistence of MQTT messages for HAS digestion. Putting it all in a single branch per name would not work for me with 4 WT450 devices reporting temp/hum as different IDs of the same name.

The RFLink gets pretty busy when you have several chatty RF devices! I have a digitor weather station transmitting every 48 sec, plus 2-4 WT450 doing the same, plus 3 plant moisture monitors. I think it is why OTA refuses occasionally - because something distracts the ESP at a crucial point in the loading process.

Open to improvements ā€¦ especially for sending commands to the RFLink for execution via callback in the code.

#4

Oh I know. Thatā€™s the other reason I wrote a separate weather sensor receiver. On windy days, the Oregon sensors go crazy, and if you add rain to that ā€¦ WooHoo. When you have a dedicated weather station receiving the info itā€™s fine because itā€™s doing nothing else, but when itā€™s an all-in-one receiver and transmitter and Doorbells, PIRā€™s Smoke Detectors were going off as well as transmitting, there were times when it would miss events so decided to make them separate especially the Oregon that was doing all the Manchester stuff. 1 bit wrong and you get some weird results. Iā€™d sometimes get wind gusts of 59Km/h and think WTF until I realized what was happening.

Again, great what your doing though. Will watch with anticipation :smiley:

#5

@phileep thanks for sharing this. do you have any guide, diagram or some pictures on how to build this gateway? sorry, new to this.

#6

I really want to try out this RFLink to MQTT gateway but I donā€™t understand how to build it.

For example, if I already have NodeMCU, Superheterodyne 433MHz RF Receiver + Transmitter Kit do I still need to get Arduino Mega 2560?

#7

Hi

In theory you could port the code to a nodemcu. However the software from rflink is designed to be dropped on to an Arduino mega 2560.

I couldnā€™t work out how to change it, so I installed on the Arduino mega using their installer, attached the receiver, transmitter etc as per their instructions and then just added the nodemcu and my code to interpret the serial Comms from the rflink into the nodemcu and pass that on to mqtt.

So, for wiring etc, follow the rflink Instructions and it will work as a black box. Then just attach the Tx/Rx serial pins of the mega to the relevant pins on the nodemcu.

The advantage is you can load updates to the rflink system straight on to the Arduino mega if you stick to their recommended setup, which seems a more efficient way to go if you are after leveraging their development.

The exact pins to link the nodemcu to are in the instructions on my GitHub (donā€™t have at hand at the moment), but it is effectively only 2 wires between the Arduino mega setup and the nodemcu.

Hope this helps

Phil

#8

Ahā€¦ I get it. So basically I need to build the USB gateway as per RFlink guide. Then add the nodemcu as per your instructions.

Will try it out soon. Thanks

#9

Hi @masterkenobi

I am using a ESP8266 to make the Arduino MEGA wireless, you could also do the same with a wifi shield.

I have created a pull request to add the extra ESP8266 esp-link detail to the RFLink components page.

Pull Request:
https://github.com/home-assistant/home-assistant.github.io/pull/2292

#10

Hi @phileep,

Iā€™m trying to use you RFlink to MQTT with an ESP-12, but can you confirm that itā€™s working the R46 ? In test mode or in normal mode I never received data in a topic.

I only receive the RF Status in MQTT :

RF/status online

Thx for the help

#11

hi @Minims

I have looked at my downloads - looks like I am using R45 (and originally used R44)

I thought I checked their site the other day and noted that there was no new version - but looks like it has been there since March!

Will try and have a look tomorrow.

I cant think why it would not work due to version.

Have you had success with an earlier RFLink version?

Do you see codes via USB on your computer after flashing RFLink only on the mega (ie not havng the ESP in the mix)

Also watch out what pins you use for the serial link to the ESP ā€¦ some are reserved for serial data etc so may cause clashes if you are trying to monitor via USB at the same time as using the soft serial link to the ESP. What kind of ESP-12 are you using? I have used a NodeMCU V1.0 board in my case.

Let me know if you solve it. I will try and upload the new version in my morning (about 8 hours from now)

Regards

Phil

#12

Iā€™m using it since about 1 month.
It is currently connected via USB to RPi3 and itā€™s working well in HASS with this configuration :

rflink:
    port: /dev/serial/by-id/usb-1a86_USB2.0-Serial-if00-port0

I would like convert it to MQTT. So I remove usb cable, I use a 12v power on my RFLink. Connect the wires on 0,1 to D5,D6 and upload the sketch with updated SSID, password and mqtt host.

When I was in testing mode, Iā€™ve tried to copy/paste some of your example in Arduino IDE Serial to ESP but there is no actions :
20;0D;UPM_Esic;ID=test;TEMP=00df;HUM=67;BAT=OK;

I may do something wrong, but I donā€™t know what :confused:

It a NodeMCU 1.0 with ESP-12E version.

Thx for checking, I keep searching on my side.

#13

@Minims

Have you tried d5 and d6 switched?

Might be just the Tx/Rx are wrong way about

Also, check you are getting enough power to the esp. I have it powered by usb seperately to the Arduino in the first instance as I have had dropouts if the power source was not supplying enough to power both.

#14

Many thanks, D5/D6 are wrong way on my ESP.
It works well now :-). So I can say R46 is compatible.

1 Like
#15

RX is OK, but Iā€™m unable to sent a command there is no action :

RX:

RF/NewKaku-ID=0110b000 {"raw":"20;08;NewKaku;ID=0110b000;SWITCH=2;CMD=OFF;\r","SWITCH":2,"CMD":"OFF"}
RF/NewKaku-ID=0110b000 {"raw":"20;09;NewKaku;ID=0110b000;SWITCH=2;CMD=ON;\r","SWITCH":2,"CMD":"ON"}

To send Iā€™ve tried many thing without success :

TX:

[email protected]:~$ mosquitto_pub -h 127.0.0.1 -t RF/command -m '20;NewKaku;0110b000;2;ON\n'
[email protected]:~$ mosquitto_pub -h 127.0.0.1 -t RF/command -m '20;NewKaku;0110b000;02;ON\n'
[email protected]:~$ mosquitto_pub -h 127.0.0.1 -t RF/command -m '20;NewKaku;0110b000;02;ON'

do I missed something ?

I think ā€˜20ā€™ is for RX and ā€˜10ā€™ for TX, so I change to :

[email protected]:~$ mosquitto_pub -h 127.0.0.1 -t RF/command -m '10;NewKaku;0110b000;2;ON;\n'
[email protected]:~$ mosquitto_pub -h 127.0.0.1 -t RF/command -m '10;NewKaku;0110b000;02;ON;\n'
[email protected]:~$ mosquitto_pub -h 127.0.0.1 -t RF/command -m '10;NewKaku;0110b000;02;ON;'
[email protected]:~$ mosquitto_pub -h 127.0.0.1 -t RF/command -m '10;NewKaku;0110b000;2;ON\n'
[email protected]:~$ mosquitto_pub -h 127.0.0.1 -t RF/command -m '10;NewKaku;0110b000;02;ON\n'
[email protected]:~$ mosquitto_pub -h 127.0.0.1 -t RF/command -m '10;NewKaku;0110b000;02;ON'

But nothing.

Iā€™ve tried with another ESP, same problem.
The Arduino has is LED RX blinking on mosquito command so I think wiring is OK.

I Have some TriState and EV1527 switch too, but same problem.

#16

hmm

beware using multiple espā€™s with the same firmware - you have to make their mqtt client id unique - which is why I added ā€œtestā€ mode as I would test on an esp first before rolling it onto the ā€œproductionā€ esp. If you have a second esp trying to connect to mqtt with the same id, it will be refused.

is 127.0.0.1 a valid mqtt server? I assume it is from the command line on the machine your mqtt server runs on? maybe try the address you would use from external. I tried equivalent to your command line and got no response either ā€¦ but the HASS config below works!

The other ā€œfeatureā€ of the arduino code is that a ā€œmalformedā€ command will make it reset - hence adding the \n to the end to ā€œcorrectā€ the package. So beware that you might be making the esp reset, then sending a ā€œcorrectā€ command whilst it is still restoring from a reset sp it is being missed. (Maybe I should work for Microsoft or Apple with that subtle shift of blame for my poor coding to the user!)

so my HASS config for a wattsclever socket based upon the raw data collected is:

  - platform: mqtt
    name: "Watts1"
    state_topic: "RF/Eurodomest-ID=02df0d"
    command_topic: "RF/command"
    qos: 0
    payload_on: "10;Eurodomest;02d0f2;00;OFF\n"
    payload_off: "10;Eurodomest;02d0f2;03;OFF\n"

Have you ever transmitted successfully from the RFLink? did you try from their load tool to confirm transmit is working? I have an antenna on my receiver, but not on my transmitter, so sometimes range to the device to be controlled is an issue also.

#17

I have many esp, and yes the client id is unique. 127.0.0.1 is ok my RPi, iā€™m using 192.168.1.10 in the ESP config.
So at this point I make the same configuration as you.

My RFLink can transmit when connected via USB, this part is OK. But I think I canā€™t when iā€™m using the TX Pin on the Arduino.
Iā€™ve tried a minimal sketch on the Arduino to check both RX and TX Pins, and it works. So itā€™s strange.

Iā€™m quite sure the problem is now TX on the Arduino, I think I will check a R44 or R45 to see if I have the same behavior.

ā€“
Last news, I tried R44, R45,R46, same issue.
On the RFLink Loader, Iā€™ve tried to send a command like 10;PING; it works.

Now Iā€™ve tried esp-link. I can received code. When I send data I never get an answer.

So Iā€™ve connected the MEGA to an UNO, link RX and TX between each other to be sure a 2nd time with another sketch. RX & TX are OK in both way.
RX TX Check i use

Any ideas why Iā€™ve got this problem? On Mega 0 is RX and 1 is TX

This problem is solved, I connect I CP1202 on RX/TX and I can send data.
But I still canā€™t with ESP. So I put my serial between TX ESP and RX Arduino.
So if I copy paste data from serial ESP to Arduino, itā€™s ok.
But it doesnā€™t work directlyā€¦

#18

hmm (again)

well - if you have definitely made the RFLink mega transmit successfully using usb and an uno

What about the mega not liking 3.3v logic? do you have a logic level shifter? My mega works ok without one at the moment.

Also, have you tried different pins for tx from the esp? maybe the pin you are using is not liked??

Also, this uses softwareserial library - have you checked for issues with that library with esp?

#19

thx @phileep for all your help and your work, I finally have a working RFLink to MQTT.

I have clean flash on ESP and Arduino, re upload sketch, power the Arduino with 12v, used and ESP-12F (generic one) with 3.3v from Mega. Wirong has not changed.
I canā€™t say exactly why it works now, but it works and I have my first switch in HASS :wink: Next step, hass config for all my RF.

#20

maybe check your power up sequence

I think if the arduino mega boots, reboot the esp (I use a node mcu) after the mega is up - has caused me issues in the past on recollection.

Anyway - enjoy setting up your mqtt sensors & switches!!