Area/Room Presence Detection for Mobile Devices using ESP32-based BLE Beacons

Share your Projects!
1

Hello,
I’d like to introduce to you a project I have been working on during the last few weeks. Its focus is the localization of mobile devices using low-cost COTS hardware, i.e. the ESP32. By configuring and deploying multiple ESP32’s as iBeacon transmitters into a region (e.g. an apartment, house, etc.), a mobile app on the device can detect in which room/area of the region the device currently is. It then uses MQTT to publish the determined region to an MQTT broker. The current room/region is estimated from the RSSI values obtained from all beacons in range.

Why?
I started the project because I wanted a BLE solution to detect in which room of my flat my devices are. Among the existing solutions, I found monitor and room-assistant to be the most promising ones.
One drawback for me was the required hardware. My apartment has six rooms, and equipping each of the rooms with an RPi Zero was not feasible, mainly due to the limited deliverable quantities here in Germany (only 1 per order).

How?
I found the ESP32 to be a promising alternative. It has Wifi and BLE build-in, is available for 6-8€, depending on quantity and supplier, and can easily be converted to an iBeacon using espHome. Furthermore, I also use it for other sensors (e.g. moisture, light, temperature, etc.). So I still had a bunch of them lying around.
My first idea was to utilize the ESP32’s as passive BLE sensors to monitor the advertisements of devices. But, as some may know from the discussions surrounding the current COVID-19 tracing apps, almost all mobile device manufacturers periodically change the broadcast MAC address, making it hard to identify and track devices. While surely some (maybe complicated) tricks are possible, I didn’t want to work around privacy-preserving mechanisms.
Being fully aware of the power consumption drawback of on-device solutions, I still decided to try and see if it works and how large the battery impact truly is.
Regarding the processing, I decided to go with a Machine Learning-based approach. By doing so, overlapping coverage areas of beacons are beneficial instead of harmful. The additional diversity helps to provide more accurate, and stable predictions. Hence beacon positions can be chosen more freely, e.g. near power outlets at convenient locations.
Here is a quick illustration with three BLE beacons and two devices performing self-positioning. Devices can report their estimated positions with MQTT.

explanaition
explanaition1048×900 72 KB

Setup procedure
The initial setup procedures for the system will be as follows:
First, one has to set up the iBeacon identities (UUID, Major ID and, Minor ID) and configure the areas (or rooms, if used for room presence detection). Note, that it is not necessary to provide any map or location of the beacons.
Second, using the App, the wireless propagation characteristics of the iBeacons (currently only the RSSI unfortunately), are collected for each room. This step requires you to move around in the respective rooms while the app performs beacon ranging. Depending on the size of the overall region, this step takes the longest. From my experience, collecting samples in each room for 5-10 minutes should be fine.
Next, the data must be exported from the app to a local computer. Using two simple Python scripts, the data is used to train a model, which is then re-deployed to the app. The model can be deployed to different devices without the need to perform the time-consuming data collection in the second step for each device.
That’s it. The app can now estimate the current room in which the device is. By configuring an MQTT broker and topic of choice, each room change will be pushed to the broker.

Demo
Here a short demo GIFs showing the capabilities to control light scenes in combination with some simple Home Assistant automation (room labels are in german). Many more use cases are possible, e.g. triggering automation if the device is placed on a shelf/table.
The delay is usually in the order of 1-2 seconds.

Open Issues
Of course, there are also some issues. The most important one is the power consumption. Constant beacon ranging in the background will drain the device battery quicker. However, I consider this an open topic for further optimizations. Simple measures can be implemented to counteract. For example, the app will suspend all measurements if the device is not moving or if the device leaves the specified beacon region (e.g. the house).
Additionally, there are many use-cases where non-stop measurements in the background are not required.
Another issue is instability in the transition between two open areas where no physical separation (e.g. a wall or door) exists. There are techniques to stabilize the predictions in those cases, but implementing them will take further time.

So, why do this?
Honestly, I just wanted to know if it works and went from there. Now it is at a point where it fits my use personal use case and works reliably. Which makes me wonder, whether it will also be useful for others?
Therefore, the next step is to find a small circle of willing testers, who want to try it in their own home and help me improve it.
Currently, the app works only on iOS devices with version 13 and above (I did not test/build for other versions yet).
If you are interested, you will be given access to the app to help me test and develop it further. Once it is confirmed to work for others and the remaining bugs are fixed, I plan to submit it to the Appstore for review and, if all works out, release it for free.

Thanks for sticking with me until here :wink:
Please let me know what you think. Looking forward to your replies, feedback, and comments.

TL;DR Another way to perform room presence detection based on low-cost BLE beacons for mobile devices using an app and MQTT. Currently iOS only. Interested? Let me know below.

27 Likes
2

Nice project! I’ve got a couple esp32’s and would be willing to test the app.

1 Like
3

I’ve been working through exactly the same issues as well and discovering the limitations. Would be happpy to test it out as well. I already have a few RPi devices transmitting as iBeacon in my apartment.

4

That is dope and super fast. Faster then with room-assistant (uses similar method ble). Could i use my apple watch 5 series instead of iphone? i would love to test!!! been searching for something like this forever

1 Like
5

Wow. Great job.

1 Like
6

Excellent project! Unfortunately I can’t assist with testing right now but will be following your progress with interest.

How about only beacon ranging if in a home zone and the accelerometer / gyro has indicated significant movement?

1 Like
8

Really cool project. While it reminds me of FIND3 your planned features are something which I am missing from the (Android only) app.

1 Like
9

Thanks to you and the others willing to help me test :smile: I will release more details soon and get in touch.

Unfortunately no, at least not yet. I have looked into it and hope to release a simplified version for the Apple Watch.

Thanks for the ideas. Actually I already had implemented both of them a few days back but as I had already drafted the post I didn’t care to change it :smile:. Now it only performs the measurements, when the device moved 1m in the horizontal plane and stops completely when the beacon region is exited. So now it can also indicate, whether the device is home and publish a corresponding message via MQTT.

Thanks for that reference. From a quick look I’d say it seems really similar. If I see correctly, its Android only. Maybe there is an opportunity for collaboration. Will definitely check it out in more detail.

10

This is exactly what I have been looking for! I’ve been using Geofency (with beacons) with some measure of succes, but would love to have more granularity (knowing in what room you are) instead of just home/not home. and more reliability would also be great.

Good reason to order some more ESP32’s :wink: count me in! Love to help test!

1 Like
11

Sounds exactly the sort of thing I am looking for and I have a couple of ESP’s floating around. Happy to help test

1 Like
12

I use 4 esp32s runnng esp32-mqtt-room and HA mqtt room to do exactly this - seems to work well.

I used to use room assistant on rpi zeros but I found it too slow.

1 Like
13

i have a couple of beacons deployed and woudl very much like to test this, too.

1 Like
14

If I understand it correctly you do all the calculations on the device. Do you have any idea if it would be beneficial to do them centrally. Or do you think the battery usage is mainly in the constant detection?

If there would be an Android version I would be happy to test as well :slight_smile:

1 Like
15

Thank you all so much for the feedback. To everyone who has expressed willingness to test, I will get an Apple Developer Access to use Testflight to distribute the app. It’s my first iOS app, so I still have to figure some stuff out. It might take a few days before I can get around to it, so please be patient :wink:

Yes, I was aware of esp32-mqtt-room but forgot to mention it in the intro post, sorry. There are already many good solutions/approaches in the field. Unfortunately, esp32-mqtt-room doesn’t work for iOS users, as Apple does not allow devices to advertise/act as an iBeacon device in the background. Hence, it was unsuitable for me to use. However, performing beacon measurements in the background is possible. It is essentially the same approach but inversed.

I have used Xcode to perform the debugging and check usage system resources. Here is what I found:

  • The biggest impact is on the battery (as expected). Both the BLE monitoring and using the WIFI connection for the MQTT reports require the largest share. The battery impact of the processing is minimal, as in not even shown. It is just a few tensor multiplications, which are not very resource-intensive. As expected, the wireless hardware requires the most power. Here are two screenshots to show background activity and how it looks when a prediction happens (two predictions in the screenshot). The “Overhead” in red refers to the power required to power up and configure the Wifi hardware.
    What I still find suspicious is, if the device is not moved, the power should go down even further as the measurements should be suspended by iOS. As currently it is not fully suspended, I suspect there is still some bug in the implementation.
  • The central processing (i.e. predicting which room the device is in) has almost no footprint in the CPU usage.
  • Maximum RAM usage is 40MB.

powcons_background
powcons_background1000×610 35.7 KB

Figure 1: Power consumption in the background. No prediction happens.
powcons_prediction
powcons_prediction1000×610 37.6 KB

Figure 2: Power consumption in the background with two prediction events. The red “Overhead” parts show the power required setup and configure the wireless hardware for the MQTT report.

16

I would also be happy to test for you, this could be really useful for me.

17

im still interested!

18

Given that the approach requires an app, I wonder why didn’t you go with the other direction: app BLE broadcasting and esp32 listening ? MAC address for broadcast is not an issue since you can broadcast stable a string that is then interpreted by ESP32.
I use this approach and it works just fine.

2 Likes
19

That’s an interesting point. Certainly, at the time I started reading up on it, I found it simply wasn’t possible to advertise as an iBeacon on iOS in the background. As I moved along, the idea with a unique, constant string per device (I assume this is what you mean by stable?) also popped into my mind. At the time I was thinking around whether to completely rework the approach or stay with a device-centric solution.
For me, the main drawback of the unique-string approach was, that it would effectively make the device trackable for others, as anyone can simply follow the BLE messages with the unique string. Of course, it would be possible to mitigate this by anonymization of the unique string. Similar to a car remote rolling key, where a new sequence is used after each time the button is pressed. So complexity would increase, as for each new device I would require a unique key generator, which would have to be synced with the ESP32, increasing complexity of the system.
Plus I like how the device-centric approach is fully privacy-preserving. The user has control over his location at all times and decides how to share it.
Here are some other arguments I thought about:

  • Simplified processing, no additional server required: All processing happens on the device, no need to fuse data from multiple sensors at a server.
  • Lower latency: After processing the beacons measurements and making the prediction, the only step left is to publish an MQTT message to the broker.
  • Apple review acceptance: Given that Apple always states how they are very protective about customer privacy, I was unsure whether they would allow an app that potentially makes the device trackable.

In addition, I was also not sure, which solution is truly better energy-wise and how efficiently iOS can perform the ranging. If you have any materials on the matter, I’d love to check them out.
I’d also like to emphasize, that the above list is not complete in any sense. A better one can be found at [1]. The above is simply a collection of the arguments that mattered the most to me.

[1] -Crownstone | Indoor Localization

20

Thanks for sharing thought process !
I use Android and it is not clear to me how things are on iOS. W.r.t. energy I guess it will be similar.
I’ve experimented with ble scanning on phone first and it was definatelly noticable from the battery perspective. I don’t have any concrete material on this. Technology-wise I don’t think constant BLE scan is actually energy consuming since I would guess that phones are in constant BLE listen mode (e.g. consider how BLE headphones get paired automatically), but I would guess actually actioning on the scan (i.e. responding to BLE advertisement from ESP32) might be actually an issue since it will not let the phone to sleep properly. Again, no hard evidence just guesses.
Also, at least on Android, doing BLE scan was causing intermittent connection issues with other BT devices connected to phone.

The privacy argument is definatelly true, hence it is more of a personal choice. This is not that a big deal for more. W.r.t. detecting whether someone is at home I already have bunch for Mi Flora devices and Mi Scale that perform broadcast over BLE. If a thief was really into technology they would be able to detect my presence using other means. In particular, physical surveillance would be likely much easier :slight_smile:

I don’t think latency is really an issue here. Phone->ESP32->HA should be order of tens of milliseconds if everything working in push fashion. With end user device doing broadcast you also have control over the frequency of broadcasts, hence potentially even further decreasing latency.
The additional benefit is that you can do ML on the server and improve the algorithm over time without need to push something to the end user device.

The final argument that was the most important to me is that this approach works not only with devices that have the app, but with any BLE devices that performs broadcast. I personally have Android phone with free Beacon Toy app where I can configure BLE broadcast (stable string, while MAC is changing). My wife uses iNode BLE beacon. There is plenty of BLE trackers on the marker that would likely work. I’ve recently bought Oura Ring which seems to do constant BLE broadcast which would be ultimate wearable for BLE broadcast if it turns out to work.

1 Like
21

Thanks so much for the feedback. I agree with you, regarding privacy it’s much more a personal preference :wink:. Other decisions are based on what’s possible under iOS and acceptable for the Apple app review process. Again, I feel that the approach I am presenting is much more of an addon and not a replacement. The biggest benefit, is without doubt, for iOS users, as solutions as esp32-mqtt-rooms are simply not available there.
The latency argument is probably also true for most cases, though with an increasing number of ESP32s it will get noticeable (e.g. due to the shared BT/Wifi hardware on the ESP32, CSMA/CA medium access). But yes, probably not an issue.

Yes, also a good point. I like to train my models using sklearn and/or Keras. The current implementation allows importing a new model with a single button from a URL :smile:.

Definitely a good point, and one of the biggest drawbacks of the device-centric approach.
From what I understand, all approaches do some sort of AI-based classification using RSSI values. I do not expect significant improvements/drawbacks among them. RSSI is not a real high-value wireless channel feature, just the best we can get with cheap hardware and reasonable effort at the moment.
A real game-changer would be to use significantly more expensive SDR hardware and extract more advanced wireless channel features for the learning, i.e. multipath-parameters, or time-difference of arrival.
I hope this is not taking it to far off-topic, but I really appreciate the input and wanted to explain my thought-process as thoroughly as possible.