433mhz, infrared IR to and from MQTT on ESP8266

jxjhheric · August 25, 2017, 7:21am

good news，i have at-09，and i want to know can i flash the fireware throug the nodemcu not arduino uno？

Petrica · August 25, 2017, 8:03am

Yes, it works (I flashed v540 again, this time using nodemcu)

Just remember to modify the sketch for CCLoader.ino in order to use nodemcu pins:

the sketch uses pins 4,5 and 6 on the Arduino however pin 6 is not connected on nodemcu;
modify section “// Debug control pins & the indicate LED” to the actual pins on the nodemcu (for convenience purposes I added a D in front of each pin from Arduino):
int DD = D6;
int DC = D5;
int RESET = D4;

Also, it’s no need to look for the v540 firmware on that malware infested site mentioned in the video (it is already included in ccloader repository)

jxjhheric · August 25, 2017, 8:17am

great，thanks，i’ll try that

could you share an example of bt yaml file

try this to fix the led，and try v546 firmware
https://forum.arduino.cc/index.php?topic=393655.msg3137812#msg3137812

Petrica · August 25, 2017, 12:55pm

Actually the led is not the issue on >v540, it’s the board there is no reply from it (semi-brick). I can re-flash it with v540 afterwards.

I’m not a software engineer so there might be easier (and better) ways to do it, but here it is.

The expire_after option sets the sensor value to unknown after a reasonable amount of time (you need to set yours based on the measurement of seconds between beacons). I’ve then added the binary sensor component as it can be used with “to: (on or off) for: minutes: x” (sensors cannot be used with “for”) in automations.

I guess further adjustments could be used (such as a binary sensor with a state “leaving home” if the sensor’s value drops below -80 or “coming home” if it’s above -75/-70 and increasing) however this method I think is not 100% accurate and it should be connected with other presence sensors.

I. Create a mqtt sensor:

platform: mqtt
state_topic: “home/BTtoMQTT/BTADDRESS”
name: btdevice1
expire_after: 300

II. Create a binary sensor based on the previous mqtt sensor:

platform: template
sensors:
btpresence1:
friendly_name: BT presence Phone
value_template: >
{% if is_state(‘sensor.btdevice1’, ‘unknown’) %}
true
{% else %}
false
{% endif %}

michaelyasay · August 25, 2017, 3:15pm

Nice, I’ve been waiting for this all the time. How would you implement the Signal Strength of the Beacon in yaml file. I would like to try. The Skybeacon I have i can set the TX power to them.

Petrica · August 25, 2017, 7:08pm

Again, not an IT professional

A) Sensors

- platform: mqtt
  state_topic: "home/BTtoMQTT/BTADDRESS"
  name: btdevice1test
  expire_after: 300
- platform: template
  sensors:
    testbt
      value_template: >
        {% if is_state('sensor.btdevice1test', 'unknown') %}
          -100
        {% else %}
          {{states.sensor.btdevice1test.state | float}}
        {% endif %}

I guess it could be done in just one step in order not to have 2 sensors for the same job, but I prefer to keep things separate to know where the bad smell came from (when the device is offline/away the RSSI value will be a small negative number, -100 or -120 for example). Also, having this second sensor would allow for easier calibration (add/subtract any number from the RSSI value determined in first instance).

- platform: statistics
  entity_id: sensor.btdevice1
  name: btdevice1stats
  sampling_size: 10

B) Binary sensor:

- platform: template
  sensors:
      thresholdbtdevice1:
        friendly_name: thresholdbtdevice1
        value_template: '{{ states.sensor.btdevice1stats_mean.attributes["average_change"] | float > 0 }}'

When the average change is negative (the device moves away from the BT gateway, based on the sampling size of 10 beacons) the sensor is off; when the average is positive it means that the RSSI value is increasing and the sensor is on. As mentioned before, having a binary sensor would allow “to & for” in the automation.

C) Then throw all the mess under the carpet (keep in view only the threshold sensor):

  customize_glob:
    "sensor.btdevice1*":
      hidden: true

jxjhheric · August 25, 2017, 8:01pm

Yes, you are professional

clyra · August 26, 2017, 4:33am

Is there any chance of a hm-10 clone work with the mqtt gateway? I just got a (not so) cheap one and discovered that it is a MLT-BT05. Not sure if it has all the necessary AT commands to make it work. I found some information about this clone here: https://blog.yavilevich.com/2017/03/mlt-bt05-ble-module-a-clone-of-a-clone/

thanks!

1technophile · August 26, 2017, 5:44am

It could, try to connect to it by serial. Test the necessaries command to have a list of ble beacons and if successfull put these commands into ZgatewayBT instead of the standard ones.

Petrica · August 26, 2017, 9:13am

Use serial connection with “Both NL&CR” instead of “No line ending” and 9600 baud rate on the original firmware and you can interrogate the module. Depending on the model, it might have some specific procedure for putting it into AT mode.

For clones, only flashing v540 mentioned also in the youtube video works as this firmware includes also the bootloader (the firmware for other versions above 540 from the manufacturer has only 248KB and I think it is due for original HM-10 modules).

Anyway, v540 on clone works with MQTT gateway.

Petrica · August 26, 2017, 11:11am

Ignore my previous post about versions above 540 not flashing on clones.

Just need to run the specific command from HMConfigAssistant.exe to put it into update mode (AT+SBLUP). Then HMSoft.exe will be able to run (and firmware without bootloader can be uploaded).

clyra · August 26, 2017, 2:57pm

Hey, that’s great! I had it on serial and had limited success in sending AT commands. Which commands should I use to scan/list ble beacons? @Petrica tried the AT+SBLUP and got nothing, so I’m not sure if it understand this command or not. Anyway, i will try the firmware upgrade.

Petrica · August 26, 2017, 6:55pm

Not really I wasn’t paying attention to the instructions on how to update after flashing HM-10 thus I was stuck on v540 (command AT+SBLUP, which I was omitting, is the missing link as it is needed to put the device in upgrade mode on v540, thus after replacing original firmware).

In conclusion, steps are as following:
I. Initial flashing of HM-10 firmware
Solder the three pins to Arduino/Nodemcu and flash v540. All steps are described in the thread below. Nodemcu can be used instead of Arduino Uno/Mega, just need to modify pins in the CCLoader.ino sketch so that they match the ones from Nodemcu.

or in the Youtube video: https://www.youtube.com/watch?v=ez3491-v8Og

If everything worked fine, you now have a device with HM-10.

II. Updating HM-10 from v540
Soldering those three pins was needed to have access to the entire device memory and replace the original firmware with HM-10. Firmware v540 is needed for this as it is the only one available that includes also bootloader. Flashing any version above v540 with the above process will cause the device to brick (it can be re-flashed with v540, though; then the whole update process can be started again). For updates there is no further need for bootloader to be included with the firmware as it was already flashed with v540.

a) Download last firmware version from the manufacturer’s site (sometimes there are virus alerts when downloading the files…). Currently, the last version is v550.
The archive with the firmware includes also the two executable files needed for the flash (HMConfigAssistant - needed for the serial connection and HMSoft - for the flashing process). On the manufacturer’s site there’s also the pdf documentation of the upgrading procedure.
b) After playing around with AT commands in HMCofigAssistant.exe (select the proper com port and then click Open Com) send AT+SBLUP so it will enter update mode (note: the module will not respond to any other AT command until flashing is complete and led will stop blinking). Then close the Com port.
c) Open HMSoft.exe, select new firmware file, Com port and then click Load image.

The problem was that I haven’t read literally the instructions related to upgrading from the post mentioned above :

For future update, the procedure are easier as you can now update the HM-10 via an USB to Serial converter, the firmware and the procedure are on the manufacturer website: www.jnhuamao.cn

Sole current issue is that the bluetooth module with HM-10 causes the host Nodemcu to restart (there is no trace in the serial connection of Nodemcu). I have several gateways in addition to the one with BLE (one with RF gateway on an Arduino Mega with ethernet shield and the others with IR gateway on Nodemcu for controlling AC/receiver/Xbox/etc) and another component for controlling MiLight bulbs (also on Nodemcu). For some 433mhz signals (from PIR sensors and light switches) or some commands sent to the light bulbs, the Nodemcu with BLE gateway restarts itself. Some time it works fine for a few days, then it restarts about 3-4 times per hour. I observed this on both v540 (which I had for some time) and v550.

clyra · August 26, 2017, 10:37pm

Thanks for the instructions! I did the first part: read the arduino forum instructions, soldered the pins and used a Arduino Uno to upload the CC2541hm10v540.bin file. The flashing seemed to work, but right after that the red led on HM-10 clone turned off. I tried to connect it directly to the usb-serial and it didnt respond to AT commands. A re-flashing procedure seems to work, and the ccloader reports success, also the led turned on during the flashing and then off again at the end. I didnt did the voltage divider and there was people saying it wasnt needed (the clone seems to have the voltage regulator). Maybe this could be the problem…

Petrica · August 27, 2017, 5:25am

What settings did you used for the serial monitor after flashing v540? If using serial monitor from Arduino IDE, with the original firmware it needed “Both NL & CR” however after flashing v540 it should be “No line ending”. In both cases a 9600 baud rate should be chosen.

Maybe the device works after all (have you tried using a bluetooth sniffer? after flashing it shows as HMSoft) and only the led is off. There is a post mentioned above referencing a problem with the led.

jxjhheric · August 27, 2017, 7:56am

when upload the OpenMQTTGateway.ino

get this error：
Arduino:1.8.3 (Linux),：“Generic ESP8266 Module, 80 MHz, 40MHz, DIO, 115200, 4M (3M SPIFFS), nodemcu, Serial1, None”

In file included from /home/jxjhheric/桌面/OpenMQTTGateway-master/OpenMQTTGateway/OpenMQTTGateway.ino:42:0:
User_config.h:200: error: ‘D7’ was not declared in this scope
#define BT_RX D7 //ESP8266 RX connect HM-10 TX
^
/home/jxjhheric/桌面/OpenMQTTGateway-master/OpenMQTTGateway/ZgatewayBT.ino:39:27: note: in expansion of macro ‘BT_RX’
SoftwareSerial softserial(BT_RX, BT_TX);
^
User_config.h:201: error: ‘D6’ was not declared in this scope
#define BT_TX D6 //ESP8266 TX connect HM-10 RX
^
/home/jxjhheric/桌面/OpenMQTTGateway-master/OpenMQTTGateway/ZgatewayBT.ino:39:34: note: in expansion of macro ‘BT_TX’
SoftwareSerial softserial(BT_RX, BT_TX);
^
exit status 1
‘D7’ was not declared in this scope

Petrica · August 27, 2017, 8:05am

My post was referring to modifying sketch needed for flashing HM-10 on bluetooth clones (the sketch is CCLoader.ino). Were you trying to modify the pins from Arduino in OpenMQTTGateway.ino to use it for flashing the bluetooth module?

jxjhheric · August 27, 2017, 8:11am

no，i successful flash the CC2541hm10v540.bin follow your post

i just flash a new nodemcu with the OpenMQTTGateway.ino，and get the error

that‘s about OpenMQTTGateway.ino not CCLoader.ino

Petrica · August 27, 2017, 8:37am

Do you have all the files (7 * ino and User_config.h) from the repository in the same folder? I needed to add IRremote.h and IRremoteESP8266.h from the Arduino library in the same folder with the sketches otherwise it won’t compile.

Did you modify also pubsub to allow for larger MQTT payloads (MQTT_MAX_PACKET_SIZE to 1024/512 in Arduino\libraries\pubsubclient\src\PubSubClient.h)?

When all files are loaded in Arduino IDE it should look something like this.

jxjhheric · August 27, 2017, 8:50am

Do you have all the files (7 * ino and User_config.h) from the repository in the same folder?

YES there are all in the same folder

Did you modify also pubsub to allow for larger MQTT payloads (MQTT_MAX_PACKET_SIZE to 1024/512 in Arduino\libraries\pubsubclient\src\PubSubClient.h)?

Yes i modify MQTT_MAX_PACKET_SIZE to 512

Arduino IDE it now looks like

i don’t know why got the error

my nodemcu is V3