Bermuda - Bluetooth/BLE Room Presence and tracking [custom integration]

The wiki has some good suggestions for C3 devices: ESPHome Configurations · agittins/bermuda Wiki · GitHub

If the tag includes a name in its broadcast, the name should be easy to find in the device list in Bermuda’s config flow.

Failing that, I find it really helpful to use an app like NRFConnect on a phone (I use Android) as it will list all the devices around you that are advertising, and finding the strongest signal when holding the tag up to the phone will usually get you the info you need.

If the tag changes its MAC address (like real AirTags do) you might have some trouble, but sometimes even the cheap tags might have a way to change them to iBeacon mode.

1 Like

I will test the settings you suggested, but the esp or WIFI connection is OK.
The esp has been connected to the usb since I posted here and nothing in the log indicates a disconnection.

Using 'COM16' as serial port.
Showing logs:
[10:37:47]ets Jun  8 2016 00:22:57
[10:37:47]
[10:37:47]rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[10:37:47]configsip: 0, SPIWP:0xee
[10:37:47]clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
[10:37:47]mode:DIO, clock div:2
[10:37:47]load:0x3fff0030,len:1184
[10:37:47]load:0x40078000,len:13132
[10:37:47]load:0x40080400,len:3036
[10:37:47]entry 0x400805e4
[10:37:47][I][logger:156]: Log initialized
[10:37:47][C][safe_mode:079]: There have been 0 suspected unsuccessful boot attempts
[10:37:47][D][esp32.preferences:114]: Saving 1 preferences to flash...
[10:37:47][D][esp32.preferences:143]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed
[10:37:47][I][app:029]: Running through setup()...
[10:37:47][C][esp32_ble:032]: Setting up BLE...
[10:37:47][C][wifi:047]: Setting up WiFi...
[10:37:47][C][wifi:060]: Starting WiFi...
[10:37:48][C][wifi:061]:   Local MAC: delete
[10:37:48][D][wifi:481]: Starting scan...
[10:37:48][W][component:157]: Component wifi set Warning flag: scanning for networks
[10:37:48][D][esp32_ble:279]: Enabling BLE...
[10:37:48][D][esp32_ble_tracker:270]: Starting scan...
[10:37:51][D][wifi:496]: Found networks:
[10:37:51][I][wifi:540]: - 'JUNIOR 2.4B' (delete) ▂▄▆█
[10:37:51][D][wifi:541]:     Channel: 1
[10:37:51][D][wifi:542]:     RSSI: -54 dB
[10:37:51][I][wifi:540]: - 'JUNIOR 2.4B' (delete) ▂▄▆█
[10:37:51][D][wifi:541]:     Channel: 6
[10:37:51][D][wifi:542]:     RSSI: -63 dB
[10:37:51][D][wifi:545]: - 'DIRECT-SQJUNIOR-NOTEmsTT' (delete) ▂▄▆█
delete wifi 
[10:37:51][I][wifi:312]: WiFi Connecting to 'JUNIOR 2.4B'...
[10:37:53][I][wifi:616]: WiFi Connected!
[10:37:53][C][wifi:427]:   Local MAC: delete
[10:37:53][C][wifi:432]:   SSID: 'JUNIOR 2.4B'
[10:37:53][C][wifi:435]:   IP Address: 192.168.0.238
[10:37:53][C][wifi:439]:   BSSID: delete
[10:37:53][C][wifi:440]:   Hostname: 'display-proxy'
[10:37:53][C][wifi:442]:   Signal strength: -54 dB ▂▄▆█
[10:37:53][C][wifi:446]:   Channel: 1
[10:37:53][C][wifi:447]:   Subnet: 255.255.255.0
[10:37:53][C][wifi:448]:   Gateway: 192.168.0.1
[10:37:53][C][wifi:449]:   DNS1: 192.168.0.1
[10:37:53][C][wifi:450]:   DNS2: 0.0.0.0
[10:37:53][D][wifi:625]: Disabling AP...
[10:37:53][C][api:025]: Setting up Home Assistant API server...
[10:37:53][I][app:062]: setup() finished successfully!
[10:37:53][W][component:170]: Component wifi cleared Warning flag
[10:37:53][W][component:157]: Component api set Warning flag: unspecified
[10:37:53][I][app:100]: ESPHome version 2024.6.2 compiled on Jul  6 2024, 10:37:29
[10:37:53][C][wifi:599]: WiFi:
[10:37:53][C][wifi:427]:   Local MAC: delete
[10:37:53][C][wifi:432]:   SSID: 'JUNIOR 2.4B'
[10:37:53][C][wifi:435]:   IP Address: 192.168.0.238
[10:37:53][C][wifi:439]:   BSSID: delete
[10:37:53][C][wifi:440]:   Hostname: 'display-proxy'
[10:37:53][C][wifi:442]:   Signal strength: -53 dB ▂▄▆█
[10:37:53][C][wifi:446]:   Channel: 1
[10:37:53][C][wifi:447]:   Subnet: 255.255.255.0
[10:37:53][C][wifi:448]:   Gateway: 192.168.0.1
[10:37:53][C][wifi:449]:   DNS1: 192.168.0.1
[10:37:53][C][wifi:450]:   DNS2: 0.0.0.0
[10:37:53][C][logger:185]: Logger:
[10:37:53][C][logger:186]:   Level: DEBUG
[10:37:53][C][logger:188]:   Log Baud Rate: 115200
[10:37:53][C][logger:189]:   Hardware UART: UART0
[10:37:53][C][bluetooth_proxy:088]: Bluetooth Proxy:
[10:37:53][C][bluetooth_proxy:089]:   Active: YES
[10:37:53][C][esp32_ble:383]: ESP32 BLE:
[10:37:53][C][esp32_ble:385]:   MAC address: delete
[10:37:53][C][esp32_ble:386]:   IO Capability: none
[10:37:53][C][esp32_ble_tracker:653]: BLE Tracker:
[10:37:53][C][esp32_ble_tracker:654]:   Scan Duration: 300 s
[10:37:53][C][esp32_ble_tracker:655]:   Scan Interval: 1100.0 ms
[10:37:53][C][esp32_ble_tracker:656]:   Scan Window: 1100.0 ms
[10:37:53][C][esp32_ble_tracker:657]:   Scan Type: ACTIVE
[10:37:53][C][esp32_ble_tracker:658]:   Continuous Scanning: True
[10:37:53][C][captive_portal:088]: Captive Portal:
[10:37:53][C][mdns:115]: mDNS:
[10:37:53][C][mdns:116]:   Hostname: display-proxy
[10:37:53][C][esphome.ota:073]: Over-The-Air updates:
[10:37:53][C][esphome.ota:074]:   Address: display-proxy.local:3232
[10:37:53][C][esphome.ota:075]:   Version: 2
[10:37:53][C][esphome.ota:078]:   Password configured
[10:37:53][C][safe_mode:018]: Safe Mode:
[10:37:53][C][safe_mode:020]:   Boot considered successful after 60 seconds
[10:37:53][C][safe_mode:021]:   Invoke after 10 boot attempts
[10:37:53][C][safe_mode:023]:   Remain in safe mode for 300 seconds
[10:37:53][C][api:139]: API Server:
[10:37:53][C][api:140]:   Address: display-proxy.local:6053
[10:37:53][C][api:142]:   Using noise encryption: YES
[10:38:19][D][api:102]: Accepted 192.168.0.6
[10:38:19][W][component:170]: Component api cleared Warning flag
[10:38:19][D][api.connection:1375]: Home Assistant 2024.6.4 (192.168.0.6): Connected successfully
[10:38:47][I][safe_mode:041]: Boot seems successful; resetting boot loop counter
[10:38:47][D][esp32.preferences:114]: Saving 1 preferences to flash...
[10:38:47][D][esp32.preferences:143]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed
[10:42:48][D][esp32_ble_tracker:270]: Starting scan...
[10:47:48][D][esp32_ble_tracker:270]: Starting scan...
[10:48:59][W][component:237]: Component esp32_ble_tracker took a long time for an operation (53 ms).
[10:48:59][W][component:238]: Components should block for at most 30 ms.
[10:52:48][D][esp32_ble_tracker:270]: Starting scan...
[10:57:48][D][esp32_ble_tracker:270]: Starting scan...
[11:02:48][D][esp32_ble_tracker:270]: Starting scan...
[11:04:05][W][component:237]: Component esp32_ble_tracker took a long time for an operation (52 ms).
[11:04:05][W][component:238]: Components should block for at most 30 ms.
[11:07:48][D][esp32_ble_tracker:270]: Starting scan...
[11:12:48][D][esp32_ble_tracker:270]: Starting scan...
[11:16:42][W][component:237]: Component esp32_ble_tracker took a long time for an operation (55 ms).
[11:16:42][W][component:238]: Components should block for at most 30 ms.
[11:17:48][D][esp32_ble_tracker:270]: Starting scan...
[11:22:49][D][esp32_ble_tracker:270]: Starting scan...
[11:27:49][D][esp32_ble_tracker:270]: Starting scan...
[11:32:49][D][esp32_ble_tracker:270]: Starting scan...
[11:37:49][D][esp32_ble_tracker:270]: Starting scan...
[11:42:49][D][esp32_ble_tracker:270]: Starting scan...
[11:47:49][D][esp32_ble_tracker:270]: Starting scan...
[11:52:49][D][esp32_ble_tracker:270]: Starting scan...

Edit:

bluetooth_proxy:
  active: True

esp32_ble_tracker:
  scan_parameters:
    interval: 320ms
    window: 300ms 

Did not help.
The problem is somewhat random, sometimes it happens when I use the Restart Home Assistant option and sometimes when I use the Rebbot System option, but in all cases it only seems to happen when the Smartphone is out of range of the esp32, even when bringing the smartphone close from esp32 the integration remains unavailable.

Any idea?

Nothing in the logs:

2024-07-09 13:23:55.317 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration arpscan_tracker which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-07-09 13:23:55.318 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration frigate which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-07-09 13:23:55.318 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-07-09 13:23:55.318 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration spook which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-07-09 13:23:55.319 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration nodered which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-07-09 13:23:55.319 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration reversotts which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-07-09 13:23:55.320 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration platerecognizer which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-07-09 13:23:55.320 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration browser_mod which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-07-09 13:23:55.320 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration next_holiday which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-07-09 13:23:55.320 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration proxmoxve which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-07-09 13:23:55.321 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration spook_inverse which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-07-09 13:23:55.321 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration bermuda which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant

the esp or WIFI connection is OK. The esp has been connected to the usb since I posted here and nothing in the log indicates a disconnection.

Interesting - perhaps things have changed in esphome of late. If you start having issues with lost packets or connectivity I’d look there first.

Can you confirm the versions of HA and Bermuda that you’re running? I recall there being a similar issue some time ago with sensors not coming to life until after a reload of the integration, but I am sure it was fixed a while ago.

Also can you send a screenshot of the two settings pages in Bermuda? The global opts and devices pages. Feel free to blur out the individual MACs if you prefer, as long as I can see properly how it’s configured.

Burmuda v0.6.7
Esphome 2024.6.6
HA:

  • Core2024.6.4
  • Supervisor2024.06.2
  • Operating System12.4
  • Frontend20240610.1

This one?

I configured it using the UUID, there is no MAC on the device page. Should there be one?

I have an iPhone but i will try some apps and follow your tip to closely hold the device to the tag.

Turn a device in iBeacon mode means the mac does no longer change randomly?

And another question:
I observed that my Bermuda log is growing fast and is getting huge over day. Especially if the device swichtes often from unavailable to detected room when the RSSI is not the best.
Is it possibel to turn of the logbook or to limit the logbook entries?

1 Like

I have an iPhone

NRFConnect is available on iOS but I haven’t tried it myself. I think LightBlue and BLE Scanner 4.0 might be good options, too.

iBeacon mode means the mac does no longer change randomly?

Well… probably. But more accurately, if it’s sending iBeacon packets then it won’t matter if it changes its MAC or not, since we’ll be looking for any broadcasts containing that iBeacon’s UUID, regardless of mac address.

log is growing fast and is getting huge […] limit the logbook entries

Yes, the logbook acts as a “view” of the data stored by the recorder integration. If you want to reduce the disk space used by Bermuda (it can be a LOT, especially if you enable some of the extra sensors) you can configure the recorder to exclude sensors from being stored in the db.

If you just want to reduce clutter in the Logbook, while still having all the data stored, you can configure the logbook integration to filter out those entries.

I’ve just written a wiki page on how the whole storage / history stuff works and how to configure it, let me know if it needs any clarification: Logs, Recorder and Database size · agittins/bermuda Wiki · GitHub

1 Like

This one?

Sorry I was a bit vague. I mean this one from Bermuda’s configuration:
image
eg:

What might be more useful though is if you can go to Developer Tools, Services and select Bermuda BLE Trilateration: Dump Devices and past the options section for one of the entries. Using a screenshot rather than copy/paste will make redaction easier if you want to remove mac addresses (just leave enough intact so I can see what’s going on, like this):

I found the old issue where sensors weren’t recovering from unavailable until after the integration was reloaded, but it only applied to device_tracker sensors, and was fixed: device_tracker for out-of-range devices don't recover after restart · Issue #121 · agittins/bermuda · GitHub

Since it’s hard to keep track of things here (and makes for a noisier thread) would you mind creating an issue on github for this? Something like “Entities not leaving Unavailable state until after integration reload” or similar. Create Issue

Done.
I am only tracking one device and I only use 3 boards with proxy, so I only sent the information for those 3 boards. But let me know if you need any more information.

Today my iBeacon arrived, but I’m facing a problem.
The 3 devices have different MACs, but the same UUID.

This is making the integration a bit “crazy”, device A being recognized as B and vice versa. Or one of the devices not being detected at all.

How can I fix this?

https://www.aliexpress.com/item/1005006301631031.html?spm=a2g0o.order_list.order_list_main.53.364218021gWssd

edit:
Reading the faq, I’m falling into this category, but not on purpose :disappointed_relieved::

  • Bermuda considers every UUID/Major/Minor version to uniquely identify a given iBeacon. That means the MAC address can change, or you can have multiple beacons transmitting the same uuid/major/minor and they’ll all be one single device - but don’t do that though, IMO it’s silly.

Edit 2:

Fixed

I downloaded this application:

I entered the password aa14061112 and modified the last number of the UUID and took the opportunity to give the devices a name.

2 Likes

Reading the faq, I’m falling into this category, but not on purpose

Haha it’s the manufacturer, not you :slight_smile:

On the product page it has the steps you need to do to change the configuration of the device. In a nutshell, you install the “HolyIOT-beacon” app on Android, or I think “HolyIOT Beach”(?!) app on iOS, connect to the device (default password is aa14061112, and you can then configure it to be different beacon types - choose iBeacon, then change the name, uuid, major, minor etc to whatever you like.

You might choose to use the same UUID on each device, but just use different minor/major numbers. That’s probably what I’d do if I were manually configuring a bunch of them.

It’s likely that it’s also possible to set the parameters using another BLE app, like NRFConnect - it’s less easy than using the holyiot app but is a likely option if you don’t want to install yet another app. They usually implement the config as a set of read/write services using standard BLE GATT (insert more jargon as required!). Apps like NRFConnect can usually show you those services and let you access them, it just might not be obvious what each one does until you poke it.

The other great thing about those devices is that one can program and flash them with one’s own software, if that floats one’s boat. The NRF51822’s are great little chips, and that’s a pretty reasonable price for ones done up in a case with battery etc.

The cheapest I’ve found on a bare breakout board (no battery clip, no case etc) is AU$2.82 ea when buying 10.

Edit: Ah! You beat me to it! Well done :slight_smile:

2 Likes

I’m excited to start using Bermuda. Thanks for this project. I’ve been browsing the documentation, and…

But if you are automating a trap-door in your hallway, you might want a delay of a few seconds to ensure your favourite guest really is half-way down the hall before you trigger the yawning portal to their doom. This is because re-setting a trap-door is “expensive” in time terms, and it’s always harder to get someone to re-enter the hallway after they’ve already seen the sharks.

I wasn’t expecting that! Thanks for the laugh.

4 Likes

Is the Calibration page on the wiki up to date? It says, “In the attributes section of the transmitter’s Distance sensor, watch the “Area rssi” value,” but when I look at that entity, I can’t find any such attribute. The only attribute I see is “Current MAC”.

Will all this data writing wear out my SSD? RU you all Discord? thanks again

Where is the log file that Bermuda writes too?
I have a lot of other programs with hi log outputs running.
Can it just be deleted or limited in size using generic HAOS parameters?
Thanks

How long do the batteries last on these things? (in this sort of mode checking all the time) thanks

@solarjunkie
I have no idea, I’ve only had them working for 22 days.
I bought 3 devices but I’m only using 2 so far, one is reporting 96% and the other 99%.

@agittins
Totally off topic.
The Holyiot app reports battery status and if I’m not mistaken there is a feature request on github and I completely understand if the feature is not implemented. But do you know of any way to track battery status using HA?

Ahh yes, that is out of date. The rssi value is now only on its own sensor (“Nearest RSSI”), which is disabled by default. This was done to reduce sensor log churn. The last line of the instructions is still current, though:

Repeat this procedure until you decide nothing works, the universe is pure chaos and it’s time to give up.

A proper calibration procedure is being integrated into the configuration flow currently - this will walk you through setting it up and (hopefully) make it a lot easier.

For now, you could enable that “Nearest RSSI” sensor and use that value to find the ref_power, or you could have the “Distance” sensor open in a separate window (or on a phone etc) and through the configuration alter the ref_power and save until you get the distance to close in on 1m.

The calibration flow changes are probably less than a day or so away, and calibration really isn’t all that helpful in broader terms at least until we have trilateration.

2 Likes

Yes, Bermuda is only using HA’s built-in logging and history recording. There shouldn’t be much in the way of log output specifically (unless you turn on debugging, in which case brace for the firehose!), but the recorder/lts/logbook/history stuff can be pretty significant. There’s a page on the wiki that goes through what goes on and how to manage it: Logs, Recorder and Database size · agittins/bermuda Wiki · GitHub

The above includes a config snippet to show how to exclude the most volatile sensors (rssi, distance) from being stored in the recorder db.

I can’t speak for those devices specifically, but will note that using Bermuda does not add any battery drain, as we just passively listen for broadcasted data. It should be completely possible for a beacon using a single CR2032 cell to last for a year or more sending broadcasts every second, but depends a bit on the individual device’s hardware design and firmware.

Yeah it’s a good question. For things like BTHome it (can be) built-in and automatic, but lots of devices will not advertise their battery level, but instead you need to connect to it and query the battery status. I think there’s a standard (GATT?) service for that, so I think it should be possible for an integration to offer that for any known / desired device supporting the spec-compliant way of doing it, but I don’t know of one. I think it is usually left up to the individual integration that supports a given device to handle this. That’s fine for a weather station or door sensor where you want some other integration to support features, but for beacons there’s not much other than Bermuda or some other passive listener that you would normally need.

I might be gradually talking myself into something here, so I’d better shut up. :rofl:

So… as it stands, I don’t know of any “generic” way of getting BLE battery stats into HA other than via device-specific integrations.

If you want, feel free to jump into NRFConnect on your phone, “connect” to the device and see what services etc it offers. If you pop some screenshots in here I’m happy to look at what it’s doing/offering and have a bit of think about what options there might be.