Bermuda - Bluetooth/BLE Room Presence and tracking [custom integration]

If you provide your “download diagnostics” info and tell me which device you’re seeing issues with I can take a look. I suggest doing “reload integration” first, wait 30 seconds, then download diagnostics - otherwise if your system has been up a while the diagnostics might take a long time (or crash).

One would need evidence to support that conclusion, so I think it’s wise to keep an open mind on the “why” until you get a good picture of “what” :slight_smile: The history tool and a diagnostics are the way, see my post to Alexandre for info on using the history tool.

I just tried to install the integration.

After adding it to HACS under Settings/Devices&Services the integration does not appear. It only shows up on Settings/Devices&Services/Devices


but there is no way to configure the integration.

What did I do wrong? I am using the latest Version of HomeAssistant.

So you’ve added via HACS and you can see it under the HACS integration in Settings - Devices & Services. All good

But have added the integration itself to the integrations page?
Click the big Blue + Add Integration button and search for Bermuda and then add it.

You can then configure it.

Great. Works.

I have the same issue with the skull and crossbones on an ESP proxy device.

All I did was change the IP Address in the yaml.

I have reloaded the integration and restarted HA, but it hasn’t picked up the scanner on the new IP even though I can ping it and connect to it via ESPHome dashboard and see it in the ESPHome integration.

How do you get Bermuda to see this proxy again?

UPDATE: put the IP Addy back to the original and updated the proxy thru ESPHome dashboard, reloaded Bermuda and it all came back!!

UPDATE2: After a few minutes the proxy went skull & Crossbones again so it didn’t come back.

So how do you scan for proxies that change?
How does Bermuda know if a proxy is added? (I haven’t tried that yet)

I am running the latest beta Bermuda - if that’s relevant

Any idea what is causing some proxies to show the death sign? They are fixed on static IP addesses -
image

And any idea how to reduce the bouncing around from proxies in the house?

Bermuda simply looks at Home Assistants’ list of bluetooth scanners. If it’s not there, then it should be an issue between the proxy and home assistant itself.

Bermuda uses an event listener that is triggered when HA makes any changes to the “device registry”. When Bermuda sees that, it re-scans HA’s bluetooth back-end to check for any new/removed/altered proxies.

You can go into the ESPHome integration and click “enable debug logging”, wait for 10 seconds or so and then click “disable debug logging”. You’ll be prompted to download the debug log. It should contain hundreds lines that include: Got message of type BluetoothLERawAdvertisementsResponse. Each one is prefixed with the proxy’s name and IP. If your proxy is working it should show up several times a second in that log. If it doesn’t, then HA is not receiving BLE data from that proxy, which is beyond Bermuda’s control.

It doesn’t matter in this particular instance, but please never say “latest”, it takes fewer characters to just put in the actual version, and saves the people helping you from having to explain that “latest” is not a version number :slight_smile:

If you are having trouble going through the debug logfile, feel free to email it to me at [email protected] - it’s better not to upload it publicly since it will contain IP addresses etc.

1 Like

Bermuda shows the skull and crossbones if that proxy has not forwarded any advertisements for 10 or more seconds. Looks like Verandah hasn’t for 2.5 hours, and Bathroom for 5 hours.

While it’s possible that those proxies simply haven’t received any traffic, the earlier diagnostics you provided via github showed that those two proxies were working well at that time (it seems my original review of that got lost at the draft stage before I passed out, sorry!).

Are they still showing the same now? You could upload a fresh diagnostics to the github issue if you like, and I’ll take a look at what’s going on. Your last diags looked pretty good, with update intervals all around a second or so.

Can you re-do that history screenshot, but use + Choose device to add all the sensors for that device, and click the date selector and narrow it down to the last hour or two ten minutes? It’s particularly useful to see the distance and area sensors together, over a small enough time-frame to see the individual updates - like this:

Thanks for the info. That explains things very well.

In this case, I had forgotten that I had migrated the proxy from my test system to my production system and hadn’t stopped the ESPHome stuff on my test system.
Once I stopped the ESPHome add-on and disabled the ESPHome integration in my test system the proxy was again seen by Burmuda on my production instance.
I intended to respond to my own post as above but you have beaten me to it
So, lesson learnt regarding 2 instances of ESPHome and once again thanks for your response.
I then managed to change the IP address of the proxy without a problem when updating the yaml.

Yes I was just being lazy with the version number as it was getting Very late at night here in blighty and since there are only 2 beta versions …anyway point taken.

Again many thanks

Ahh yes, it’s not obvious anywhere but esphome only supports sending bluetooth data to a single HA instance. Been bitten by that myself :slight_smile: AFAIK it’s fine to have multple esphome dashboards (the add-on) but the proxy can only be configured in one HA integration’s instance.

1 Like

Yeah, they’re still showing the same. The way I have fixed that is by going into the ESP home dashboard, going into edit those devices and then pressing the install button again. That seems to unload the packages and my config to the proxie again and then it is online once more. Not sure if that is the best way to be doing things though? Its strange that those two randomly have gone offline, they are on static IP addresses. Hmm. I will send you another diagnostic with now via the github. Thanks Ashley, you’re a legend!!

On the bouncing around - Is this what you’re after? Here is a snap shot of 10 mins from 8am this morning. As you can see it is bouncing around, and for example, the kitchen, tv and guest bathroom are all at least 5-7 metres apart from eachother in physical distance. So, my guess is it would be great if I could limit the pings to a smaller radius but I am hesitant to start trying to configure that without your advice first!

Thanks again!

Hey Ashley! Sorry I think I replied to my own comment, still working these forums out a bit! Haha

New Release v0.7.0 - Calibrated Expectations!

It’s been a while coming, but we have a fresh release to go along with HA 2024.11!

:balance_scale: Calibration

The big change is the official release of Calibration options! The first two have been in the beta for a while, however…

  • Global Calibration, where you use one device and one proxy to set your global defaults
  • Per-scanner calibration, where you can tweak the offset for each scanner (proxy) to account for differences in proxy receivers and antennae
  • Per-device calibration, using a number entity on the device page, allowing you to tweak each device based on its signal strength.

Instructions on how to do the calibration process will be forthcoming, but for now, know that you should do them in the order stated above, and once you click “CONFIGURE” the config flow will give you plenty of instructions on how to do it.

The per-device calibration can be left at 0 to use the global default, or you can enter a specific ref_power for that device. The distance measurement updates immediately on changing the value, so you can dial in the expected distance.

Huge thanks to @Lash-L for their contributions on the calibration features!

:globe_with_meridians: The “Global” Bermuda Device

This is a device that shows up amongst the other Bermuda devices, but gives you some overall info on the system, like how many proxies are active, and how many bluetooth devices are visible. Another contribution from @Lash-L!

:dart: Easier Device Selection

  • @Lash-L has added the device manufacturer name (where available) to the device name selector, making it a little easier to track down your desired devices when configuring Bermuda. These are based on the assigned manufacturer service UIDs.

:adhesive_bandage: Fixes

  • fix: Entities not updating if away at startup @agittins (#269)
  • fix: Absent devices now show as “Away” on startup @agittins (#270)
  • Stability and performance clean-ups @agittins (#358) which should help with some performance issues / lockups on some systems.

Full release notes at Releases · agittins/bermuda · GitHub

3 Likes

@Duke_box @mterry63 @sam3487 You might like to know that I’ve released a beta version, v0.7.1rc1 that includes fixes for proxies that have been removed or are no longer acting as proxies. Would be good to know if that solves the haunting for you :slight_smile:

It solved it for me. Thanks!

2 Likes

just want to say a big thanks to @agittins your integration is excellent. I was using espresense and this a much simpler and cleaner integration to do ble tracking. Especially when i had a bunch of esp32 bluetooth proxies deployed already.

I have one question though, in the configure options it displays the bluetooth scanners and the last advertisement, is there anyway those stats can be exposed as an entitity so I can use it to do automations. I want to setup a notification if any of the bluetooth scanners stop advertising for x mins. I noticed sometimes my bluetooth proxies stop scanning and needs a reboot.

right now I just have an automation to reboot the bluetooth proxy every week.

thanks

Thanks Ashley! I see that it gives me a device called ‘bermuda global’ which has some sensors that show the number of active proxies. I assume the fixes to those that go offline are in the background though? Maybe by restarting the integration or something is the way it will try to fix them?

Also, did you end up having any suggestions for me to reduce the bounce rates from the proxies around the house? At a last resort I might just reduce them from being in each room of the house to something a less.

That’s where I landed. I have one on each floor and one in basement. Then, I have a dashboard where only certain parts of it are visible depending on floor. I originally tried having 6 and there was too much bouncing especially on 1st floor where the kitchen ESP32 and Living Room one were at most 30ft apart.

After updating to v0.7.1rc1 the Log is flooded with errors:

2024-11-08 12:56:34.543 ERROR (MainThread) [custom_components.bermuda] Failed to find scanner hci0 (xxxxxxxx) (xxxxxxxx) in Device Registry (21 previous messages suppressed).

This BT is from my mini PC, and at first I don’t even intend to use it with Bermuda, I want to leave only the esp32 boards.

Any ideas?

Yes, the Failed to find scanner hci.... in Device Registry errors are a bug I introduced when refactoring the scanner set-up. I’ll have a fix soon in the next rc / release.

The esphome integration already has a “watchdog” system in place so that it can request proxies to restart if they stop sending advertisements. The main idea of these particular sensors is to give you a good idea of whether all your proxies are working, and to help identify if some are not sending updates as often as they should - this is often indicative of bad firmware settings, but causes may vary.

There is a feature request open currently to provide more detailed diagnostic entities that will be a lot easier to interpret, and can give deeper insights into the troubleshooting process. Some will be on the global device, others on the individual scanner’s device pages. This will also allow for setting up automations/alerts if anything goes awry, and also allowing historical analysis to identify trends etc.
Some initial thoughts on what the sensors will measure and what those things mean are at FR: Add Diagnostic Entities on Scanners · Issue #363 · agittins/bermuda · GitHub

Yes, your diags look good as far as proxy health goes - your intervals are typically around 1s or so most of the time. Your distances however are pretty large, and looking at your phone in the kitchen over the last 10 seconds of that diags, the distance bounces from 5m to 21m, which is fairly variable.

How close do you think your phone was likely to have been to the kitchen proxy at that time? If the calibration is way off, it might be worth sorting that out - it usually doesn’t matter much, but having it dialled in will help the max_velocity function do it’s job properly (it matters because the velocity filter works in metres/second which is a linear measurement, while rssi is a logarithmic measurement - so if our conversion is off then these absolute comparisons don’t work as well).

Some steps to get more data and make improvements:

  • concentrate on just one device for now, such as your iPhone
  • in Bermuda’s device page for your iPhone, enable the distance to... sensors for a few of the proxies - maybe choose the ones that give you the most trouble (Kitchen, Study, Bath?)
  • Go through the calibration process:
    • Do the Global Calibration:
      • put your phone 1m away from one of the proxies, and adjust ref_power until you get a fairly reliable 1m measurement from it. It’s important you start with 1m, not 1.5 etc.
      • Move your phone to say 5m or 10m (basically the furthest you can get with a wide free space between proxy and phone), and adjust only the attenuation until the measurement reads the actual distance that your phone is at.
    • You can probably stop there, but you might want to do a quick test and see that all the other proxies also read about 1m when your phone is 1m from them. If they vary substantially, adjust the per-proxy offset for each of them until they’re about right.

Now that you’re calibrated, leave the phone in a typical “in the kitchen” place for a few minutes, then go into the History page in HA. Choose + Choose Device and select your phone - that way you’ll get all the various sensors from your phone on one page of history graphs - like previously but with more data! Sexy. In the time/date range selector, set it to only display the last 10 minutes or so - as long as you can clearly see each update taking affect. You can turn off the “Estimated Distance” sensor from Private BLE Device, and turn off the “Distance” sensor as well, since it will just be overlaying on of the “Distance to…” sensor’s values. Grab a screenshot, and we’ll take a look at it.

If your distances are still all over the place, we can then experiment with the max_velocity setting to filter out the weaker signals, and the smoothing_samples to alter the upward slopes on the smoothed distance sensors to avoid crossing the streams so much :slight_smile: But grab that screenshot first to ensure we have a good baseline to start with.

As far as positioning the proxies, what you really want for your typical loitering locations (kitchen bench, couch etc) is that the closest proxy is much closer than the next closest proxy. If one is 3m away under the TV, and the other is 3.5m away on the other side of that wall, well… it’ll be rubbish. But move the TV proxy so it’s right near the lounge, and suddenly it’s reading 1m, and Bermuda will have an easier time choosing it versus the 3.5m bedroom proxy. It’s less about the distance between the proxies and more about the difference in distance between you and the two closest proxies. You want one proxy to be a clear winner in the places you tend to loiter - ie, place the proxies near the places you hang out, rather than places that are corners of rooms etc.

Note that this advice is for the current simplistic area matching that Bermuda does. Once trilateration is ready, we’ll probably lean towards having proxies more spread out. Maybe. We’ll know when we get there, I guess :slight_smile:

1 Like