I’m not sure but it may have to do with the last few lines of this where they mention rssi.
“After the ground work here is done, it should be relatively simple to mirror the changes to ble_rssi as well.”
You could try asking the devs on the ESPHome Discord server if you don’t get a response here.
For whatever reason, the ble_rssi component does not yet support iBeacons. I wrote a little workaround, that also adds filtering. Details here: ESPHome component for ESPresense-like room tracking
So the workaround approach appeals…though to be fair I feels like I have no idea what I’m doing…so it’s probably true.
With my Studio Code addon I created custom_components folder under esphome, and a lib folder under that. I placed your ble_dist.h in custom_components and OneEuro under the lib sub folder.
I then tooled up a yaml that I thought might work:
esphome:
name: example
includes:
- custom_components/lib/OneEuro.h
- custom_components/ble_dist.h
on_boot:
then:
lambda: |-
addTracker("bt_s20_ble", "0d591b07-5d2c-4551-a8cc-39a38968e462");
esp32:
board: esp32dev
framework:
type: arduino
api:
ota:
#Set WiFi
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
logger:
level: DEBUG
logs:
esp32_ble_tracker: INFO
esp32_ble_tracker:
scan_parameters:
duration: 60s
active: false
on_ble_advertise:
then:
- lambda: |-
parseAdvertisement(x);
sensor:
- platform: template
name: "BT S20 BLE (Lounge)"
id: lounge_bt_s20_ble_dist
update_interval: 30s
unit_of_measurement: "ft"
device_class: ""
state_class: "measurement"
accuracy_decimals: 1
entity_category: "diagnostic"
lambda: |-
return getTracker("bt_s20_ble").get_dist();
It is slightly different to your reference example…it just complained with that one that:
‘board’ is a required option for [ESP32]
And when I added that detail is wasn’t happy with the Platform: ESP32 bit…so…snipping done, edit window happy, lets install!
Then I get this error:
Compiling /data/example/.pioenvs/example/FrameworkArduino/esp32-hal-rmt.c.o
Compiling /data/example/.pioenvs/example/FrameworkArduino/esp32-hal-sigmadelta.c.o
Compiling /data/example/.pioenvs/example/FrameworkArduino/esp32-hal-spi.c.o
Compiling /data/example/.pioenvs/example/FrameworkArduino/esp32-hal-time.c.o
/data/cache/platformio/packages/framework-arduinoespressif32/cores/esp32/esp32-hal-spi.c: In function 'spiTransferBytesNL':
/data/cache/platformio/packages/framework-arduinoespressif32/cores/esp32/esp32-hal-spi.c:922:39: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
uint8_t * last_out8 = &result[c_longs-1];
^
/data/cache/platformio/packages/framework-arduinoespressif32/cores/esp32/esp32-hal-spi.c:923:40: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
uint8_t * last_data8 = &last_data;
^
Compiling /data/example/.pioenvs/example/FrameworkArduino/esp32-hal-timer.c.o
Compiling /data/example/.pioenvs/example/FrameworkArduino/esp32-hal-touch.c.o
Compiling /data/example/.pioenvs/example/FrameworkArduino/esp32-hal-uart.c.o
Compiling /data/example/.pioenvs/example/FrameworkArduino/libb64/cdecode.c.o
Compiling /data/example/.pioenvs/example/FrameworkArduino/libb64/cencode.c.o
Compiling /data/example/.pioenvs/example/FrameworkArduino/main.cpp.o
In file included from src/main.cpp:30:0:
src/ble_dist.h: In member function 'void BeaconTracker::update(int, int)':
src/ble_dist.h:61:14: error: 'to_string' is not a member of 'std'
s += std::to_string(b) + ", ";
^
src/ble_dist.h:61:14: note: suggested alternatives:
In file included from src/esphome/core/preferences.h:6:0,
from src/esphome/core/application.h:6,
from src/esphome/components/api/api_connection.h:4,
from src/esphome.h:2,
from src/main.cpp:3:
src/esphome/core/helpers.h:398:20: note: 'esphome::to_string'
inline std::string to_string(const std::string &val) { return val; }
^
src/esphome/core/helpers.h:398:20: note: 'esphome::to_string'
In file included from src/esphome/components/sensor/sensor.h:3:0,
from src/esphome/core/application.h:16,
from src/esphome/components/api/api_connection.h:4,
from src/esphome.h:2,
from src/main.cpp:3:
src/ble_dist.h: In function 'BeaconTracker& getTracker(std::__cxx11::string)':
src/esphome/core/log.h:117:99: warning: format '%s' expects a matching 'char*' argument [-Wformat=]
esp_log_printf_(ESPHOME_LOG_LEVEL_WARN, tag, __LINE__, ESPHOME_LOG_FORMAT(format), ##__VA_ARGS__)
^
src/esphome/core/log.h:151:28: note: in expansion of macro 'esph_log_w'
#define ESP_LOGW(tag, ...) esph_log_w(tag, __VA_ARGS__)
^
src/ble_dist.h:105:3: note: in expansion of macro 'ESP_LOGW'
ESP_LOGW(TAG, "BeaconTracker %s not recognized, returning the first tracker (hopefully one exists)");
^
Compiling /data/example/.pioenvs/example/FrameworkArduino/stdlib_noniso.c.o
Compiling /data/example/.pioenvs/example/FrameworkArduino/wiring_pulse.c.o
Compiling /data/example/.pioenvs/example/FrameworkArduino/wiring_shift.c.o
*** [/data/example/.pioenvs/example/src/main.cpp.o] Error 1
========================== [FAILED] Took 4.33 seconds ==========================
I’m sure it is something simple…but I don’t know much about this code…
Help?
Thanks,
CP.
Ah, I guess it’s not compatible with the arduino framework as written. I actually have a ESP32C3 board which is much better supported on the esp-idf platform, so I haven’t tested on arduino.
Looks like there’s 3 issues.
ESP_LOGW(TAG, "BeaconTracker %s not recognized, returning the first tracker (hopefully one exists)", n.c_str());. The difference is the extra argument at the end. This is a silly mistake on my part, not sure how I didn’t catch it.Oh I just noticed you have a C3 part as well. My esp32 and esphome sections read as:
esphome:
name: env-2
includes:
- custom_components/lib/OneEuro.h
- custom_components/ble_dist.h
on_boot:
then:
lambda: |-
addTracker("phone", "uuid-xxx-xxx");
esp32:
board: esp32-c3-devkitm-1
variant: esp32c3
framework:
type: esp-idf
sdkconfig_options:
CONFIG_BT_BLE_50_FEATURES_SUPPORTED: y
CONFIG_BT_BLE_42_FEATURES_SUPPORTED: y
CONFIG_BOOTLOADER_WDT_ENABLE: n
CONFIG_BOOTLOADER_WDT_DISABLE_IN_USER_CODE: y
CONFIG_ESP_INT_WDT: n
CONFIG_ESP_TASK_WDT: n
CONFIG_INT_WDT: n
CONFIG_TASK_WDT: n
The various sdkconfig_options are workarounds for my setup, you may or may not need them too. If you’ve successfully uploaded a build with bluetooth without them, you can ignore them. More details here: Cannot use esp32_ble_tracker on ESP32C3 · Issue #2941 · esphome/issues · GitHub
OK…progress! But not totally…of course…
I did these things:
Replaced one instances of std::to_string with esphome::to_string
Replaced line 105 with `ESP_LOGW(TAG, “BeaconTracker %s not recognized, returning the first tracker (hopefully one exists)”, n.c_str());
Changed my framework to esp-idf
Changed line 53 to “dist_buf.insert(dist_buf.begin(), pow(10.0, (p-r)/(10.0 * PATH_LOSS)));” removing the conversion to ft (sorry, but SI units rule) ;-p
I have changed my yaml to:
substitutions:
devicename: esp32study
esphome:
name: $devicename
includes:
- custom_components/lib/OneEuro.h
- custom_components/ble_dist.h
on_boot:
then:
lambda: |-
addTracker("study_bt_s20_ble", "0d591b07-5d2c-4551-a8cc-39a38968e462");
esp32:
board: esp32dev
framework:
type: esp-idf
api:
ota:
#Set WiFi
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
manual_ip:
static_ip: 192.168.20.150
gateway: 192.168.20.1
subnet: 255.255.255.0
dns1: 192.168.1.6
logger:
level: DEBUG
logs:
esp32_ble_tracker: DEBUG
esp32_ble_tracker:
scan_parameters:
duration: 60s
active: false
on_ble_advertise:
then:
- lambda: |-
parseAdvertisement(x);
sensor:
- platform: template
name: "BT S20 BLE (Study)"
id: study_bt_s20_ble_dist
update_interval: 30s
unit_of_measurement: "ft"
device_class: ""
state_class: "measurement"
accuracy_decimals: 1
entity_category: "diagnostic"
lambda: |-
return getTracker("bt_s20_ble").get_dist();
it now compiles, uploads and boots. I can’t however detect my beacon…I’ve tried it as:
addTracker("study_bt_s20_ble", "0d591b07-5d2c-4551-a8cc-39a38968e462");
addTracker("study_bt_s20_ble", "0d591b07-5d2c-4551-a8cc-39a38968e462-100-1");
addTracker("study_bt_s20_ble", "iBeacon:0d591b07-5d2c-4551-a8cc-39a38968e462-100-1");
None allow it to be detected…yet it does work in that iBeacon format when I set up a sensor in HA for ESPresence (the other one I’m playing with). All I get back is:
[14:23:23][D][esp32_ble_tracker:214]: Starting scan...
[14:23:51][W][ble_dist:105]: BeaconTracker bt_s20_ble not recognized, returning the first tracker (hopefully one exists)
[14:23:51][D][sensor:125]: 'BT S20 BLE (Study)': Sending state nan ft with 1 decimals of accuracy
Is there a way to get it to show you what it does find? I suspect it just doesn’t like the formatting of my beacon id…
CP.
Ah damn, I totally forgot to document this. The iBeacon UUID decoding is byte reversed. Details here: https://github.com/esphome/issues/issues/2445
You can either follow my instructions there to modify the source in your esphome library (this will be overwritten when you upgrade esphome) or set the tracker UUID as the byte reversed version of the real UUID. for example A1B2-C3 becomes C3B2-A1.
Wow…that is a pretty major bug - “Oops - we accidentally turned it inside out!”
Still no joy on my side…reversed or not reversed I can never get past the whole “not recognised” issue for the UUID. I just cannot land the correct way to write in the string so it gets detected…
Is there a way to get it to return a lit of detected beacons?
Yeah, set the logging for the tracker component to very verbose, it’ll spit out details for every BLE advertisement it sees. Not every advertisement will be sent during a scan period, so it might take some time to catch the right one. According to the issue I linked, the UUID in the tracker logs will be reversed from what you should use.
As a note, the format should match the first example you gave, no major/minor numbers and no iBeacon: prefix.
Also, it looks like your get_tracker() call is using a different name than your add_tracker() call. It returns the first tracker by default so it shouldn’t make a difference with only one tracker,
Yup…just noticed I have a name mismatch…
[D][sensor:125]: 'BT S20 BLE (Study)': Sending state nan m with 1 decimals of accuracye
…that’s fixed now but still getting NaN…so still not pinning the beacon.
I’ll switch logging level and see what I find…back soon.
Now I’m seeing the UUID getting pciked up:
[13:12:03][VV][esp32_ble_tracker:544]: Adv data: 03.03.9F.FE.17.16.9F.FE.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00 (28)
[13:12:04][VV][esp32_ble_tracker:494]: Parse Result:
[13:12:04][VV][esp32_ble_tracker:511]: Address: 6E:B2:09:94:71:33 (RANDOM)
[13:12:04][VV][esp32_ble_tracker:513]: RSSI: -68
[13:12:04][VV][esp32_ble_tracker:514]: Name: ''
[13:12:04][VV][esp32_ble_tracker:528]: Manufacturer data: 02.15.0D.59.1B.07.5D.2C.45.51.A8.CC.39.A3.89.68.E4.62.00.64.00.01.C5 (23)
[13:12:04][VV][esp32_ble_tracker:531]: iBeacon data:
[13:12:04][VV][esp32_ble_tracker:532]: UUID: 62E46889-A339-CCA8-5145-2C5D071B590D
[13:12:04][VV][esp32_ble_tracker:533]: Major: 100
[13:12:04][VV][esp32_ble_tracker:534]: Minor: 1
[13:12:04][VV][esp32_ble_tracker:511]: Address: 46:E2:37:12:13:42 (RANDOM)
[13:12:04][VV][esp32_ble_tracker:513]: RSSI: -53
[13:12:04][VV][esp32_ble_tracker:514]: Name: ''
[13:12:04][VV][esp32_ble_tracker:525]: Service UUID: 0xFE9F
[13:12:04][VV][esp32_ble_tracker:539]: Service data:
And with this in my yaml:
addTracker("study_bt_s20_ble", "62E46889-A339-CCA8-5145-2C5D071B590D");
I now get something:
[13:26:27][D][sensor:125]: 'BT S20 BLE (Study)': Sending state 3.94919 m with 1 decimals of accuracy
[13:26:27][VV][api.service:140]: send_sensor_state_response: SensorStateResponse {
key: 3329046011
state: 3.94919
missing_state: NO
}
It says 3.9m…but it is more like 3.9cm as the phone is right next to the ESP32…but at least it is progress… Changing debug level now as it keeps losing connection/crashing…will play for a bit more…
Happy it works, thanks for working through debugging as my first beta tester lol.
You should put the phone 1m from the ESP32 and monitor the RSSI values printed to the log. Set the lowest repeated value you see as the TX power in the HA companion app, the default is probably quite a bit off depending on what power setting you’re using. Once that’s set, the distances should be more accurate, but the path loss constant is another knob to tweak if the reported distance doesn’t match actual distance once you move past 1m. The wikipedia page has some example values.
Once the TX power is calibrated, you can tweak the filter constants, number of retained samples, and timeout to get the behavior you want.
Nice! So I have done a bit of tinkering just with the RSSI readings…I put it exactly 1m away from the ESP32 and then I let it run for a while so I could get an average reading of the RSSI…which turned out to be about -78. I’ve updated the sensor config in the app and now I’m getting a reasonable value of 0.8m at 1m…which I think is going to be good enough given all the variables such as interference, positioning, and whether or not I’ve got the phone in my pocket or hands.
Ultimately I just want to set a distance range at which I would consider the beacon to be ‘in the room’ or ‘out of the room’ so it only need to be good enough to get a <5m type reading…plus once I have more of them I’ll take the ‘closest’ reading to zero in on the most likely location. So this is great now!
What is the syntax for adding additional trackers? Obviously I can simply duplicate the sensor block, but I’m not sure how to set the on_boot section…for example:
esphome:
name: $devicename
includes:
- custom_components/lib/OneEuro.h
- custom_components/ble_dist.h
on_boot:
then:
lambda: |-
addTracker("study_bt_s20_ble", "62E46889-A339-CCA8-5145-2C5D071B590D");
addTracker("study_mp_s20_ble", "9584DD79-FFBE-589A-8B14-AAE3E3BE33E3");
Thanks!
Figured it out…use a comma, semicolon at the end. So it should be:
on_boot:
then:
lambda: |-
addTracker("study_bt_s20_ble", "62E46889-A339-CCA8-5145-2C5D071B590D"),
addTracker("study_mp_s20_ble", "9584DD79-FFBE-589A-8B14-AAE3E3BE33E3");
And that just works!
Yay. Time to soak test and she if it is stable.
That should work fine for multiple trackers, just make sure you pass the right name when calling get_tracker().
edit: huh that’s weird, lambdas should be straight C++ so a semicolon was correct (and what I have in my config) and a comma should error out.
I’ve been trialing Espresense lately and for some reason I’m getting very inaccurate distance readings.
I’ve tried different values in “Factor used to account for absorption, reflection, or diffraction” but still all over the place.
I’ve heard that esphome has problems with ble tracking when it comes to Android devices so I haven’t tried it.
Curious to see if anyone here can compare the 2?
As it happens I run both ESPresence and the custom ESPHome component from Raj (rpatel300 → ESPHome component for ESPresense-like room tracking). This is because I’m not ready to implement BLE room sensing yet (too busy) but I’ve got enough time to run a couple of naked ESP32s in two different rooms to see how they go…
My observations as follows:
At the moment I’m still very much on the fence. It is likely that I will continue to run two platforms all the way through the sensor build and then see if one stands out…I have NodeRed, so the wrangling of BLE data from multiple sources can be done on the back end…so the server-side code for ESPPresence it not a deal maker for me (in fact I’d have preferred a Node Red package).
A bit of a ramble…but hopefully you’ll find some value in it.
CP.
As CP said, distance is gonna be pretty rough because of environmental effects, not much you can do besides filtering the shit out of the raw signal and dealing with the resulting lag. The way the filtering works, it should pick up new presence fairly quickly and it’ll take a bit to drop off once you leave.
The default path loss value should be good enough, what you should tune is the 1 meter reference distance transmitted with the advertisements (this is described above). Both ESPresence and ESPHome are identical in this, they use the same BLE hardware and software stack to receive advertisements.
Personally, I want to use motion detectors and other instant read sensors to detect entering a space and then BLE to determine whether I’m still there even if I’m sitting still so motion won’t capture anything. I’m a little busy to set up anything more complicated than just BLE_presence = BLE_distance < threshold but eventually I’ll add a template sensor or Bayesian sensor or automation to do it this way.
The idea of triangulation with multiple nodes is cool, but setting up the coordinates of each node and room and tuning everything with noisy distance values feels like way too much work for so little gain.
I know this is slightly old but I’m trying to ge this working, I have everything set up right I think, I’m getting nan on the sensor so I guessing it’s just the device it’s looking for is wrong, but everything else is set up right?
I’ve tried my uuid forwards and backwards but neither seem to give me a reading, any additional help would be great
Edit: sorted it out, I had an error in my uuid. Seems to be working great now. Thanks