ESP32 based BLE tracking for mqtt_room

ptrsnja · October 2, 2018, 8:52pm

Hello folks,

I’ve been using the fantastic room-assistant project to do BLE tracking on a room-to-room basis. However, living in Canada, it gets expensive pretty quickly trying to pick up more than a couple Pi Zero W’s, so I decided to take on porting functionality of room-assistant over to an ESP32.

You can check out the project on github. I’ll be adding to it as I get the chance, with a goal towards moving all my sensor nodes to ESP32.

Cheers,
James

randytsuch · October 2, 2018, 10:29pm

Very cool, thanks for doing this.

The pizero w version of this was on my list of things to do, but now I think I’d like to try your esp32 version.

I have a couple of the now retired lolin32 boards, hopefully I can use them for this project.

When I actually try it, will probably have some questions for you, but will try not to bug you too much.

Randy

techwithjake · October 2, 2018, 10:49pm

Does this need dedicated BLE beacons or can it track your phone as well; similar to monitor?

ptrsnja · October 2, 2018, 11:35pm

If you’re using an app like Beacon Toy, it can track your phone by emitting an advertising packet. However, it won’t work with most phones via general BLE because they randomize their Bluetooth MAC address.

It’s subject to the same limits as room assistant or happy bubbles. Most new phones are BLE capable, you can pick up generic iBeacons on eBay or Ali Express for about 10 bucks, and you can also track some other devices like fitness bands.

I’m planning on bringing in support for the Mi Flora sensor as well, as I’ve got a few of those scattered around. I have no idea how much time I’ll have to devote to this project, but I’m at the minimum viable product stage now.

roofuskit · October 3, 2018, 12:20am

When compiling I get:
Arduino: 1.8.7 (Windows 10), Board: “Node32s, 80MHz, 921600”

D:\Google Drive\Projects\ESP32-mqtt-room\ESP32-mqtt-room.ino: In member function 'virtual void MyAdvertisedDeviceCallbacks::onResult(BLEAdvertisedDevice)':

ESP32-mqtt-room:90:7: error: 'StaticJsonBuffer' was not declared in this scope

       StaticJsonBuffer<500> JSONbuffer;

       ^

ESP32-mqtt-room:90:29: error: 'JSONbuffer' was not declared in this scope

       StaticJsonBuffer<500> JSONbuffer;

                             ^

ESP32-mqtt-room:205:21: error: 'class ArduinoJson::JsonObject' has no member named 'printTo'

         JSONencoder.printTo(JSONmessageBuffer, sizeof(JSONmessageBuffer));

                     ^

exit status 1
'StaticJsonBuffer' was not declared in this scope

This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.

ptrsnja · October 3, 2018, 12:33am

It looks like it’s a problem with the ArduinoJSON library. Do you have it installed? And you’ve gone through adding the board through the boards manager? Did you clone the entire directory, or did you just copy the sketch file?

Just some basic guesses to get you started.

roofuskit · October 3, 2018, 12:39am

Latest version JSON Library, I updated to the latest version of the Arduino IDE, and I have cloned the entire directory.

ptrsnja · October 3, 2018, 1:01am

What version of ArduinoJSON are you using? I’m on 5.13.2 and it’s working properly on my DOIT ESP32 Devkit V1.

taikapanu · October 3, 2018, 6:55am

If you “blindly” press update in arduino library manager for arduinoJSON library you might get the beta release. Take another look in the library manager that you have non beta version.

roofuskit · October 3, 2018, 2:47pm

That was it. I was having a different issue before, updated my libraries to eliminate those when troubleshooting and must have updated to the beta without realizing it. What a poorly designed software feature for something that’s intended for beginners.

Thank you both!

clyra · October 3, 2018, 8:19pm

Hi,

Nice. I’d like to try, but my first attempt failed. I’m not used to use platformio but I got it working here with ATOM and was able to download the libraries, but compile fails with this error:

Processing esp32doit-devkit-v1 (platform: espressif32; board: esp32doit-devkit-v1; framework: arduino)
--------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
PLATFORM: Espressif 32 &gt; DOIT ESP32 DEVKIT V1
SYSTEM: ESP32 240MHz 320KB RAM (4MB Flash)
DEBUG: CURRENT(minimodule) EXTERNAL(minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny)
Converting ESP32-mqtt-room.ino
Library Dependency Finder -&gt; http://bit.ly/configure-pio-ldf
LDF MODES: FINDER(chain) COMPATIBILITY(soft)
 Collected 26 compatible libraries
Scanning dependencies...
 Dependency Graph
 |-- <WiFi>
 |-- <ArduinoJson> 5.13.2
 |-- <ESP32 BLE Arduino> 0.4.16
 |-- <PubSubClient> 2.6
Compiling .pioenvs/esp32doit-devkit-v1/src/ESP32-mqtt-room.ino.cpp.o
Compiling .pioenvs/esp32doit-devkit-v1/libe20/WiFi/ETH.cpp.o 
Compiling .pioenvs/esp32doit-devkit-v1/libe20/WiFi/WiFi.cpp.o
Compiling .pioenvs/esp32doit-devkit-v1/libe20/WiFi/WiFiAP.cpp.o
Compiling .pioenvs/esp32doit-devkit-v1/libe20/WiFi/WiFiClient.cpp.o
Compiling .pioenvs/esp32doit-devkit-v1/libe20/WiFi/WiFiGeneric.cpp.o
Compiling .pioenvs/esp32doit-devkit-v1/libe20/WiFi/WiFiMulti.cpp.o
/Users/lyra/Documents/PlatformIO/Projects/181003-163949-esp32doit-devkit-v1/src/ESP32-mqtt-room.ino: In function 'void reconnect()':
/Users/lyra/Documents/PlatformIO/Projects/181003-163949-esp32doit-devkit-v1/src/ESP32-mqtt-room.ino:73:58: error: 'uint64_to_string' was not declared in this scope
 if (client.connect(uint64_to_string(ESP.getEfuseMac()), mqttUser, mqttPassword )) {
 ^
 *** [.pioenvs/esp32doit-devkit-v1/src/ESP32-mqtt-room.ino.cpp.o] Error 1
 /Users/lyra/.platformio/packages/framework-arduinoespressif32/libraries/WiFi/src/WiFiClient.cpp: In member function 'virtual uint8_t WiFiClient::conn
 ected()':
 /Users/lyra/.platformio/packages/framework-arduinoespressif32/libraries/WiFi/src/WiFiClient.cpp:440:13: warning: unused variable 'res' [-Wunused-variable]
int res = recv(fd(), &amp;dummy, 0, MSG_DONTWAIT);
^
/Users/lyra/.platformio/packages/framework-arduinoespressif32/libraries/WiFi/src/WiFiMulti.cpp: In member function 'uint8_t WiFiMulti::run(uint32_t)'
/Users/lyra/.platformio/packages/framework-arduinoespressif32/libraries/WiFi/src/WiFiMulti.cpp:129:27: warning: variable 'mac' set but not used [-Wun
used-but-set-variable]
uint8_t * mac;
^

any hints? what I missed?

ptrsnja · October 3, 2018, 8:33pm

I have not yet implemented PlatformIO, so you’ll need to open the sketch in the Arduino editor and compile and upload it there.

Beware there is a nagging watchdog timeout issue that I’m working on, but likely won’t have a fix before next weekend.

fillwe · October 3, 2018, 8:44pm

Exited for this, have a couple of ESP32 devices laying around, a great use for them!

clyra · October 3, 2018, 8:58pm

ahhh, ok. I misunderstood the instructions :-(.

ptrsnja · October 5, 2018, 2:46pm

A little bit of bad news: I managed to kill two of my three ESP32 modules yesterday evening, along with a breadboard power supply. I’ve order more units, but they tend to take about 2 months to arrive. I’ve still got one remaining dev board that I can use for testing and development, but progress will be a little bit slower than I’d anticipated.

There’s a known issue of watchdog timer causing the board to stop responding after a few minutes. I’m working on it, and I’ll be updating the repository when I’ve found a fix. I may end up moving the entire project into a different platform such as mongoose to try and address the issue.

Cheers,
James

randytsuch · October 5, 2018, 3:10pm

Curious how you killed your boards?
I don’t want to repeat what you did.

ptrsnja · October 5, 2018, 3:24pm

I was using the carrier board with the bare ESP32 module, and had a dead short across the power supply rails on one of them, due to terrible soldering by the user (me).

When I plugged in the second board, it fried the linear power regulator on my power supply, which proceeded to dump 12v straight into the other board that was still connected.

Moral of the story: unplug things that are working before plugging in things that are not.

randytsuch · October 5, 2018, 3:35pm

Oh, well that sucks.
I usually check my soldering with a meter for shorts and opens, especially on power wires, before I plug them in. And yeah, if you just worked on it, plug in the board alone.

Randy

eric3616191 · October 6, 2018, 1:33am

Thanks for sharing. Can you provide all library names?

After writing esp8266, how do you complete the configuration in homeassistant?

Can IOS devices be detected?

Thank you. Cheers. This is a good solution for indoor positioning.

ptrsnja · October 13, 2018, 10:05pm

Release v0.0.1 is now available on the github repo. A couple of improvements in this release:

PlatformIO support! You can now use PlatformIO to build and upload the code. In fact, you’ll need to use it, as I’ve stopped development and support for the Arduino IDE. Big thanks to Kyle Gordon for adding a pull request with all the required frameworks in place to get it functional. It was a great bonus, as it helped with the second big change, which is…
Async MQTT! Sounds unimpressive, but I was getting issues with watchdog timers firing exceptions, which was making the whole system entirely unreliable and useless. Since we’re using PlatformIO, you don’t have to worry about installing the required libraries, just build and upload. You may want to change the specified upload_port in platformio.ini, as it was attempting to use my USB hub as the default and I had to manually set the correct port.

Please migrate to the new code and let me know if you come across any issues. I’m keen to hear back from folks about any bugs lurking.