ESP32 based BLE tracking for mqtt_room

Hi, I have a problem. when I run the build I see the following in the status window

Compiling .pio\build\esp32\lib480\BLE\BLEAdvertising.cpp.o
Compiling .pio\build\esp32\lib480\BLE\BLEBeacon.cpp.o
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino: In function 'void WiFiEvent(system_event_id_t)':
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_SCAN_DONE' not handled in switch [-Wswitch]
   switch(event) {
         ^
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_STA_CONNECTED' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_STA_AUTHMODE_CHANGE' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_STA_LOST_IP' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_STA_WPS_ER_SUCCESS' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_STA_WPS_ER_FAILED' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_STA_WPS_ER_TIMEOUT' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_STA_WPS_ER_PIN' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_STA_WPS_ER_PBC_OVERLAP' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_AP_START' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_AP_STOP' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_AP_STACONNECTED' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_AP_STADISCONNECTED' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_AP_STAIPASSIGNED' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_AP_PROBEREQRECVED' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_GOT_IP6' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_ETH_START' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_ETH_STOP' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_ETH_CONNECTED' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_ETH_DISCONNECTED' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_ETH_GOT_IP' not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:207:9: warning: enumeration value 'SYSTEM_EVENT_MAX' 
not handled in switch [-Wswitch]
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino: In lambda function:
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:493:50: warning: unknown conversion type character 0xa in format [-Wformat=]
       Serial.printf("Progress: %u% \n\r", percent);
Compiling .pio\build\esp32\lib480\BLE\BLECharacteristic.cpp.o
                                                  ^
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino: In function 'bool handleMqttDisconnect()':
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:173:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino: In function 'bool reportDevice(BLEAdvertisedDevice)':
Compiling .pio\build\esp32\lib480\BLE\BLECharacteristicMap.cpp.o
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:409:18: warning: 'distance' may be used uninitialized in this function [-Wmaybe-uninitialized]
     Serial.printf("%s exceeded distance threshold %.2f\n\r", mac_address.c_str(), distance);
                  ^
C:/Users/shvedko/Desktop/ESP32-mqtt-room-master/ESP32-mqtt-room.ino:283:8: note: 'distance' was declared here
  float distance;
        ^
Compiling .pio\build\esp32\lib480\BLE\BLEClient.cpp.o

Compiling .pio\build\esp32\lib480\BLE\BLESecurity.cpp.o
C:\Users\shvedko\.platformio\packages\framework-arduinoespressif32\libraries\BLE\src\BLERemoteService.cpp: In member function 'void BLERemoteService::getCharacteristics(std::map<short unsigned int, BLERemoteCharacteristic*>*)':
C:\Users\shvedko\.platformio\packages\framework-arduinoespressif32\libraries\BLE\src\BLERemoteService.cpp:246:89: warning: parameter 'pCharacteristicMap' set but not used [-Wunused-but-set-parameter]
 void BLERemoteService::getCharacteristics(std::map<uint16_t, BLERemoteCharacteristic*>* pCharacteristicMap) {

The settings file is full. tell me how to solve my problem

Those appear to be compilation warnings, but I do not see an error entry. Does it fail to compile or upload?

Eventually compiles successfully. I thought that these were errors)))) thanks for the clarification

I have another question.
is it possible to configure esp 32 so that it sends an MQTT message only about my BLE devices (register MAC addresses directly in the code)?

Not currently, but that’s an interesting idea. Why would you want to restrict the data being collected?

If you’re concerned about performance, the messages shouldn’t be much of an issue for your mqtt server, and unless you configure a sensor for it, Home Assistant doesn’t check for it.

Great work here! I’ve skimmed through the posts and can’t see how the initial struggles that were occurring when simultaneously trying to do wi-fi and bluetooth were resolved?

It was a lot of trial-and-error, testing, and use of different libraries to manage connections to WiFi and MQTT. Essentially, connection failures to MQTT or WiFi are ignored during an active bluetooth scan, since the antenna is not available. There is a reconnection timer set for either/each of these, in order to re-establish connection when it fails. Finally, I’m allowing WiFi to disconnect when needed, and recovering once the antenna is available.

I haven’t documented any of that here, but if you’re interested, take a look at the source code in the GitHub repo. It’ll allow you to see all the event handlers and async libraries used. This is one of the main reasons why I moved toPlatformIO - I can set specific dependencies and it will auto-install them for the user.

Curious why you ask - are you working on an ESP32 BLE project?

Heh no. If I were to, I don’t know why I wouldn’t use yours! :slight_smile:
I’m interested for two reasons:

  • I’m an electrical engineer so I try to keep an eye on what hardware can do what
  • I’ve toyed with the idea of having BLE trackers in the house, and I assume that ESP32’s would be the most economical (but honestly don’t have much of a use case, so it isn’t high on my list)

Has anyone noticed any battery impacts from the android iBeacon apps?

1 Like

I’ve had some trouble with keeping the background service running in Android 9 on my OnePlus 5. There are bluetooth issues on the current build of my phone’s firmware, and it will regularly kill the process somehow. I use bluetooth for other things quite often, so it’s always enabled on my phone. If you’re always running bluetooth, I would imagine that it’s not much extra, as it’s a simple low-energy advertisement. However, if you’re a battery miser and you turn everything off when you’re not using it, you may find an impact.

I picked up a load of Trackr Pixel BLE beacons on clearance for 4 bucks a piece, and they work reliably for tracking internally (although the app is terrible). I now know where my dog is when I’m not home, although I haven’t come up with anything interesting to do with that data. Maybe ensure lights are on for him after dark.

Hi, I’m having trouble Building this. I don’t think it;s ESP_MQTT_room itself, but I’ve tried on MAC OS and Ubunto and get the same errors.

PACKAGES: toolchain-xtensa32 2.50200.80 (5.2.0), framework-arduinoespressif32 2.10004.191002 (1.0.4), tool-esptoolpy 1.20600.0 (2.6.0)
Converting ESP32-mqtt-room.ino
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Looking for ArduinoJSON library in registry
Found: https://platformio.org/lib/show/64/ArduinoJson
LibraryManager: Installing id=64 @ ^6

ArduinoJson @ 6.14.1 is already installed
Found 31 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <ESP32 BLE Arduino> 1.0.1
|-- <AsyncMqttClient> 0.8.2
|   |-- <AsyncTCP> 1.1.1
|-- <AsyncTCP> 1.1.1
|-- <ArduinoOTA> 1.0
|   |-- <Update> 1.0
|   |-- <WiFi> 1.0
|   |-- <ESPmDNS> 1.0
|   |   |-- <WiFi> 1.0
|-- <WiFi> 1.0
Building in release mode
Compiling .pio/build/esp32/src/ESP32-mqtt-room.ino.cpp.o
Generating partitions .pio/build/esp32/partitions.bin
xtensa-esp32-elf-g++: error: ESP32-mqtt-room.ino.cpp: No such file or directory
xtensa-esp32-elf-g++: fatal error: no input files
compilation terminated.
Compiling .pio/build/esp32/lib17b/BLE/BLE2902.cpp.o
Compiling .pio/build/esp32/lib17b/BLE/BLE2904.cpp.o
Compiling .pio/build/esp32/lib17b/BLE/BLEAddress.cpp.o
Compiling .pio/build/esp32/lib17b/BLE/BLEAdvertisedDevice.cpp.o
Compiling .pio/build/esp32/lib17b/BLE/BLEAdvertising.cpp.o
*** [.pio/build/esp32/src/ESP32-mqtt-room.ino.cpp.o] Error 1
Compiling .pio/build/esp32/lib17b/BLE/BLEBeacon.cpp.o
========================== [FAILED] Took 2.78 seconds ==========================

Has anyone seen this before, and have a sugested solution. I’m banging my head against the wall trying to sort this out.

Thanks, T

It looks like there’s an issue in one of the libraries that’s brought in by PlatformIO during the build process as it’s compiling the BLE beacon library. You may try deleting that folder (.pio/build/esp32/lib17b/BLE) and then recompile and upload.

I’ve not run into that problem before, and a quick check of the github repo issues doesn’t show anything necessarily showing up.

It seems to me that if the ESP sent information only about the necessary devices, the ether would be less busy. BLE sees many devices in an apartment building. and accordingly sends information about each of them to MQTT.

Thank you ptrsnja for the pointer. I did that and the following error appears:

Scanning dependencies...
Dependency Graph
|-- <ESP32 BLE Arduino> 1.0.1
|-- <AsyncMqttClient> 0.8.2
|   |-- <AsyncTCP> 1.1.1
|-- <AsyncTCP> 1.1.1
|-- <ArduinoOTA> 1.0

|   |-- <Update> 1.0
|   |-- <WiFi> 1.0
|   |-- <ESPmDNS> 1.0
|   |   |-- <WiFi> 1.0
|-- <WiFi> 1.0
Building in release mode
Compiling .pio/build/esp32/src/ESP32-mqtt-room.ino.cpp.o
Generating partitions .pio/build/esp32/partitions.bin
Compiling .pio/build/esp32/lib17b/BLE/BLE2902.cpp.o
Compiling .pio/build/esp32/lib17b/BLE/BLE2904.cpp.o
Compiling .pio/build/esp32/lib17b/BLE/BLEAddress.cpp.o
Compiling .pio/build/esp32/lib17b/BLE/BLEAdvertisedDevice.cpp.o
Compiling .pio/build/esp32/lib17b/BLE/BLEAdvertising.cpp.o
Compiling .pio/build/esp32/lib17b/BLE/BLEBeacon.cpp.o
Compiling .pio/build/esp32/lib17b/BLE/BLECharacteristic.cpp.o
/Users/xxx/Documents/PlatformIO/Projects/ESP32-mqtt-room/ESP32-mqtt-room.ino:32:25: fatal error: ArduinoJSON.h: No such file or directory
 
*********************************************************************
* Looking for ArduinoJSON.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:ArduinoJSON.h"
* Web  > https://platformio.org/lib/search?query=header:ArduinoJSON.h
*
*********************************************************************
 
compilation terminated.
*** [.pio/build/esp32/src/ESP32-mqtt-room.ino.cpp.o] Error 1
========================== [FAILED] Took 3.44 seconds ==========================

This is the error that I get consistently across Atom-PlatformIO on MAC OS and Ubuntu.

Strange as when I delete

/Users/xxx/Documents/PlatformIO/Projects/ESP32-mqtt-room/.pio/libdeps/esp32/ArduinoJson_ID64

it is loaded straight away by PlatformIO.

I know this is not specific to ESP32_MQTT_room, seems to be a platformIO issue. I’m wondering has anyone come across this issue before?

I’m very keen to get ESP32_MQTT_room going, I have Home Assistant handling the whole house and 6 ESP32s ready to go.

I should point out, I’m new to platformIO. I normally use Adruino to compile and flash.

In ESP32-mqtt-room.ino change #include <ArduinoJSON.h> to #include <ArduinoJson.h>

That way I got it to compile.

Jarnose, thank you, that sorted the ArduinoJSON errors. It now compiles and uploads.

Thanks for the suggestion. I’ll make the change in the repository so others don’t run into the same difficulty.

1 Like

tell me what are the optimal parameters scan wait. so that the lighthouse is detected instantly?
I currently have the following options
scan dur 5
wait dur 5

Those settings are usually sufficient for quick detection. It’s important to have a wait duration defined, as this is when the device gets reported (bluetooth and wifi cannot use the antenna concurrently, so there needs to be some separation of use).

If you have many beacons to detect, I recommend bumping those numbers up a bit, but for general purposes, it should be sufficient to detect as quickly as possible. It will never be “instant” as this is a polling-type architecture, so there are no push notifications from the beacon to the ESP32 that it has changed location.

New release!

I’ve just published v0.0.7 of the project, which introduces (optional) i2c sensor support for the HTU21D temperature/humidity sensor. You can attach the sensor via the hardware i2c bus (SDA 21, SCL 22) and enable the sensor in the Settings.h file.

I’ve been running the test code for a few months with only one module using the sensor. I don’t expect it to introduce problems for anyone who does not want to use the sensor, as you won’t need to make any changes to your configuration.

For more info, take a look at the GitHub pages.

Enjoy!

Random thought but I was considering extending my presence detection to a different physical property. At the moment that property does not have HA running, but it probably will at some point.

How much work do you think it would be to change this from a mqtt publish to hitting a Web endpoint with the payload?

For the moment that’s the easiest way I can think of to solve the problem.