ESP32 based BLE tracking for mqtt_room

Thanks man! Just did the update to my 7 ESP32s and all are working and reporting the position of people!
I’ve set the scan frequency to 1s and it is working.

I do see a LOT of disconnected from MQTT in the logs tho(After each scan). It does reconnect immediately and keeps working. But thought I’d let you know

That’s great news, actually. I’ve been waiting for MONTHS for ESP32s to arrive (ordered in September when I started this project). If you’re willing to help me test, can you open an issue on the GitHub repository?

The devices are likely crashing regularly, it’s a known bug in the Bluetooth/WiFi coexistence. I’ve managed to get it to a point where it should self-recover, so seeing the announcement and disconnection messages gives me some idea of the scope of the problem. You can try adjusting the scan duration to see if that has any impact, although a shorter scan in a busy environment may not give you optional results.

Hopefully the OTA will facilitate some testing. Unfortunately, I’m running out of memory space, so a full web UI won’t be possible from the ESP itself without some serious redesign.

Man, I am waiting to get this to work since months!
And I can’t buy multiple Zero Ws since I am in canada too.
I will be more than happy to assist you!

I have 7 of them So I can test on all of them to make sure it is not hardware issues when i encounter a bug.

1 Like

I love it already due to the easy use of platformIO! Finally my ESP32 is doing something!

Unfortunately, the Arduino framework implementation (used by PlatformIO) done by espressif appears to have a bug with the wifi/bluetooth coexistence. This means that your ESP32 will be doing something, but it’ll reboot itself regularly every time it crashes. Keep an eye on your network to make sure that it is able to self-recover.

More news!

I’ve finally gotten a few more ESP32 modules to work with. This means I’ll be able to put some load testing on these devices, just in time for the holidays. I expect to be putting in some time next week to get things to be a little more stable.

Sadly, what my testing has revealed is that it’s unlikely to be possible to get a solid, stable build that will maintain a WiFi connection (complete with MQTT listeners) while at the same time running a BLE scan. I’m hopeful that this will not be a fatal problem, as I can ideally maintain an open MQTT session while allowing the WiFi to take a back seat to the BLE scan. However, it adds complexity, and I’m not a “real” developer in C/C++, so you’ll have to bear with me.

What does this mean for you?

Likely, nothing. I’m going to continue chipping away at the bugs, and hopefully they’ll behave a little better. I expect to have a new release ready within the first couple weeks of the new year.

1 Like

Hello,

I am pretty new to Atom Platformio but I think I figured out how to load it into PlatformIO. However, when trying to build the project I am getting errors.

Could you help me out?

@JHeijting Looks like it’s an issue with AsyncTCP. I’m actually surprised anyone else is using this project!

I’ll take a look at the code. I’ve got a pull request that likely fixes the issue, so I can ideally get a new version published today.

@ptrsnja, Thanks a lot! I am really looking forward to use this project for presence detection on room level. Would be fun to make music follow you through the house and also control the heating (personalized) and stuff like that :slight_smile:

Okay, a new release has been published. v0.0.4 includes a fix for the breaking changes in AsyncTCP in the latest version of the espressif IDF.

If you run into difficulty building or uploading, you may need to remove the .piolibdeps folder and rebuild the project. This will clear out any older libraries brought in at the time you built the project.

1 Like

Cool, thanks a lot!

I have removed the .piolibdeps folder and did a ‘build’. However, I am getting different errors now:

ip_api_call_data*)'

.piolibdeps\[email protected]\src\AsyncTCP.cpp:259:12: note: class type 'tcpip_api_call' is incomplete

struct tcpip_api_call call;

^

.piolibdeps\[email protected]\src\AsyncTCP.cpp: In function 'esp_err_t _tcp_bind(tcp_pcb*, ip_addr_t*, uint16_t)':

.piolibdeps\[email protected]\src\AsyncTCP.cpp:382:63: error: invalid conversion from 'err_t (*)(tcpip_api_call*) {aka signed char (*)(tcpip_api_call*)}' to 'tcpip_api_call_fn {aka sign

ed char (*)(tcpip_api_call_data*)}' [-fpermissive]

tcpip_api_call(_tcp_bind_api, (struct tcpip_api_call*)&msg);

^

.piolibdeps\[email protected]\src\AsyncTCP.cpp:382:63: error: cannot convert 'tcpip_api_call*' to 'tcpip_api_call_data*' for argument '2' to 'err_t tcpip_api_call(tcpip_api_call_fn, tcp

ip_api_call_data*)'

.piolibdeps\[email protected]\src\AsyncTCP.cpp:259:12: note: class type 'tcpip_api_call' is incomplete

struct tcpip_api_call call;

^

.piolibdeps\[email protected]\src\AsyncTCP.cpp: In function 'tcp_pcb* _tcp_listen_with_backlog(tcp_pcb*, uint8_t)':

.piolibdeps\[email protected]\src\AsyncTCP.cpp:397:65: error: invalid conversion from 'err_t (*)(tcpip_api_call*) {aka signed char (*)(tcpip_api_call*)}' to 'tcpip_api_call_fn {aka sign

ed char (*)(tcpip_api_call_data*)}' [-fpermissive]

tcpip_api_call(_tcp_listen_api, (struct tcpip_api_call*)&msg);

^

.piolibdeps\[email protected]\src\AsyncTCP.cpp:397:65: error: cannot convert 'tcpip_api_call*' to 'tcpip_api_call_data*' for argument '2' to 'err_t tcpip_api_call(tcpip_api_call_fn, tcp

ip_api_call_data*)'

.piolibdeps\[email protected]\src\AsyncTCP.cpp:259:12: note: class type 'tcpip_api_call' is incomplete

struct tcpip_api_call call;

^

.piolibdeps\[email protected]\src\AsyncTCP.cpp: At global scope:

.piolibdeps\[email protected]\src\AsyncTCP.cpp:624:6: error: prototype for 'void AsyncClient::_dns_found(ip_addr_t*)' does not match any in class 'AsyncClient'

void AsyncClient::_dns_found(ip_addr_t *ipaddr){

^

In file included from .piolibdeps\[email protected]\src\AsyncTCP.cpp:24:0:

.piolibdeps\[email protected]\src\AsyncTCP.h:84:10: error: candidate is: void AsyncClient::_dns_found(_ip_addr*)

void _dns_found(struct _ip_addr *ipaddr);

^

.piolibdeps\[email protected]\src\AsyncTCP.cpp:946:6: error: prototype for 'void AsyncClient::_s_dns_found(const char*, ip_addr_t*, void*)' does not match any in class 'AsyncClient'

void AsyncClient::_s_dns_found(const char * name, ip_addr_t * ipaddr, void * arg){

^

In file included from .piolibdeps\[email protected]\src\AsyncTCP.cpp:24:0:

.piolibdeps\[email protected]\src\AsyncTCP.h:164:17: error: candidate is: static void AsyncClient::_s_dns_found(const char*, _ip_addr*, void*)

static void _s_dns_found(const char *name, struct _ip_addr *ipaddr, void *arg);

^

*** [.pioenvs\esp32\libc9d\[email protected]\AsyncTCP.cpp.o] Error 1

 [ERROR] Took 28.12 seconds

Sorry if I am doing something wrong, I am very new to Atom and PlatformIO.

Try running the command pio update in the project directory.

I ran the command pio update from the terminal in the project directory:
Updating tool-scons @ 2.20501.7 [Up-to-date] Updating tool-unity @ 1.20403.0 [Up-to-date] Updating contrib-pysite @ 2.27.181225 [Up-to-date] Updating contrib-piohome @ 2.0.0 [Up-to-date]

However, after I ‘build’, I get the same error.

I’m not sure what’s going wrong there. Try removing the .piolibdeps folder again and building the project once more.

I can’t reproduce the problem here.

Just wanted to say thanks for this. I wanted to expand my room presence, originally done with happy bubbles (RIP), and this is easy and reliable (not to mention cheap, ~$10 per room). Haven’t given OTA a shot yet, but it should make messing with stuff far easier. The only issue I ran into was #include <ArduinoJSON.h> needed changed to #include <ArduinoJson.h>. Thanks again!

1 Like

So this is a pretty great project. I have an old rpi 3 that used to run HA before i moved away from it, and a RpiZeroW, both running room assistant, and i wanted to use this to supplement as a i have a few esp32s laying around (who does that?)

Ive got it working as expected, except that it justeither not detect some of the devices that room assistant does, or report significantly different distance values.

Any hints ? Or do they just not play nicely together (ie do i need all rpis, or all esp32s?)

Also – can anyone give me some insight into how mqtt_room actually works ?
Is it all based on first detection and then timeout ? Or does smallest distance win ? I seem to be getting things move all over the place when detected my multiple sensors, which happens a bit.

@sh00t2kill I think the main difference between the ESP32s and the rPi boards would be the antenna, which would explain the different distance calculations. I was previously using a combination, but I found that due to the inconsistency of the boards, it was causing erroneous reporting. I’ve moved my rPi 3 to be a media server, and my rPi Zero Ws are running volumio in my kids’ rooms.

If you’re comfortable with writing code, you could potentially implement a distance offset (either a constant or a percent increase/decrease). The distance calculation is exactly the same as exists in the room assistant project, so I can only imagine that hardware differences would be causing the variation you’re seeing.

As for detection of devices, try playing with the scan duration parameter. Very short scans can miss devices, but longer scans means a longer delay between detection and reporting. You can also play with the max distance parameter (the units are meters, but it’s a best guess anyway).

My best understanding is that mqtt_room uses a combination of last seen and nearest distance. You can set a couple of parameters in the sensor entry, which can help keep the random room jumps to a minimum. I’m using these values, stuck in a local variable so I only have to modify it in one place. As far as I can tell, timeout is used to determine how long the last measured distance from a given sensor should remain valid, while away_timeout sets how long from the last time the beacon was seen until it is reported as not_home.

- platform: mqtt_room
  device_id: "2f234454cf6d4a0fadf2f4911ba9ffa6-1-0"
  name: 'Nexus 5 bt_room'
  <<: &ROOM_PARAMS
    state_topic: 'room_presence'
    timeout: 15
    away_timeout: 120

- platform: mqtt_room
  device_id: "fda50693a4e24fb1afcfc6eb07647825-5-4"
  name: 'Keys bt_room'
  <<: *ROOM_PARAMS
1 Like

I’ve been playing with the away timeout, I guess it’s just about finding the sweet spot.

I was playing around with that value last night and it’s definitely better than it was.

It’s entirely possible that my dogs just move around A LOT during the night - the main driver for this project is to track their movements when I’m not at home, as well as normal presence tracking for automation etc
Thanks!

I’m doing the same with my dog, and I’ve set up a couple automations to notify me if I’ve forgotten to leave the radio on for him. I can respond to the notification and it will automatically start it up. I’ve also got a voice integration through the dialog flow component so I can tell it what station I want to hear, and where. I’ll eventually put together a video series going through each part of the system, as it’s somewhat cross-platform integration heavy.