Bermuda - Bluetooth/BLE Room Presence and tracking [custom integration]

Share your Projects! Custom Integrations
89

I’ll give it a shot! Does Bermuda also consider distance value in the calculations? Another thought popped in my mind, would it be possible to keep area unchanged until there’s enough confirmations for the area determination? Ie. Wait for 5 confirmations/data points before changing device area? Or would it be possible to publish new locations to MQTT directly from Bermuda?

Second thought I had was adjusting tx power of the tag itself. Unfortunately RuuviTags I have requires manually compiled firmware, as there’s no software adjustments as there is on BlueUp tags. I would assume lowering the TX power and adding more proxies/anchors should make tracking much more accurate :thinking:

I’ll call BlueUp tomorrow and see if their tags are easily available for testing :sweat_smile: For proxies I’ll stick with the Shelly plan, but may play around with battery powered proxies as well.

There’s also mesh protocol called Wirepas, which works on bluetooth, also the proxies/anchors.

EDIT: BlueUp not easily available, but the tags seems very versatile and easy to config. Wirepas is commercial bluetooth mesh, which operates purely on battery devices (pretty flexible), but isn’t very fast with updates. Gone through a rabbit hole of different locating technologies and all of them have their pros/cons and in the end it boils down to the planned use case. There’s not a lot of open source or community development around this topic, most of indoor location systems are commercial services aimed for niche market, none of them is consumer friendly.

@agittins hats off for embarking on this path of a pioneer! It is truly remarkable work :ok_hand: It seems like AoA/AoD, Mesh tech stack is been kept as hostage by few software development companies, even if the hardware is quite freely available for consumers. I know we are not going there (yet), but I’m sure this project will inspire others to push the boundaries :wink:

1 Like
90

I have been watching the RuuviTags now, and maybe it is RuuviTag issue… They seem to send updates only when one of the sensors change (temperature, humidity, pressure, battery voltage, movement detected). Maybe I have been looking this issue wrong all the time and root cause is the tags. If so, it does make sense from battery management perspective. Which actually made me thing about the MQTT idea, solution for this could be automation that triggers on Bermuda area change, ignores unavailable status and posts all other updates to MQTT topic, which I could turn into more stable sensors for HA dashboard? :thinking:

EDIT: Contacted RuuviTag team, above assumption was not completely correct. RuuviTags doed broadcast MAC with 1285 ms interval, which is much higher than iBeacon normally do, but still less than 10000 ms that is said to be the upperbound for iBeacons. It is still lower than 2.8s threshold.

Shelly logs didn’t show continuous or repeating connection issues (3 disconnect events in past 24h which IMHO is fine for WiFi). Neither did Shellys on device logs show anything weird.

I looked into Bermuda addon debug logs and used the service dump, but I can’t see any error errors, but I see some duplicate creation messages and beacon type “not a beacon” parameters in the dump.

Strange this is, this exactly same behaviour is also happening with HA Android Companion App tracker (my mobile). That would point again that RuuviTag may not be the culprit after all, then again Shelly logs (device or integration) doesn’t show any errors. Native Shelly integration works through RPC as it seems, not sure if other protocol could be more stable.

TLDR; Something seems to be off with the Shelly as a proxy or bluetooth_le_tracker config that causes devices to go “Unavailable” all the time.

Logs: Bermuda Add-On Debug Log2024-04-23 14:28:49.045 WARNING (SyncWorker_0) [home - Pastebin.com

Bluetooth from configuration.yaml:

bluetooth:
device_tracker:
platform: bluetooth_le_tracker
track_new_devices:true
track_battery: true
track_battery_interval: 3600

1000055848
10000558481440×3216 152 KB

1000055850
10000558501438×2803 151 KB

1000055847
10000558471440×2791 255 KB

1 Like
91

I use the addon to track who is at home with my G Tag key fob. This works very well with Shelly and bluetooth proxy. When I restart Home assistant and I am not at home, the message not aviable always appears. Can’t I save the Mac address somewhere so that this doesn’t happen when I restart?

92

Hmm. Good question. HA’s thinking on this generally is that if we don’t “know” something, we set it to unknown or unavailable. But in our case, our only way of knowing something is “away” is simply not hearing from it, which logically is no different to “not heard from yet”.

I could change the logic so that “configured” devices default to being “away” at startup.

The downside to this is that automations based on away status would trigger at startup, as would automations set to trigger on transition from away to home, once a packet is detected.

Do devices stay stuck at “unknown” until you return home, or do they change to “away” after the configured away timeout? The latter is what it probably should do.

93

That makes sense. It would be good if he could remember the last status? The status remains unknown until I return home.

It would be best if I could enter the Mac address somewhere in the config so that the system would still recognize it when I restart and then set it to present if it is reachable or to absent if it is not reachable.

The problem now is that it no longer recognizes the Mac address after a restart. Or can I save it somewhere now? I may not have seen that.

94

Hmmm… that doesn’t sound right. Can you post some screenshots of what you are seeing and how it’s configured?

If you are getting actual Bermuda sensors for something, then it implies that it must either be set up in the configuration under Settings, Devices&Services, Bermuda, Configure, “Configured Devices” - or it must be a Private BLE Device in which case it will be saved in the config of that integration.

If it’s a random MAC address then it’s either sending iBeacon adverts (in which case it will be added via “Configured Devices” using the iBeacon UUID/Minor/Major, and/or via the Private BLE integration. Neither should require you to keep track of the MAC address, because it should just update the configured sensor whenever it knows something.

Unless of course you’re actually looking at a sensor provided by the iBeacon integration, in which case all of the above three posts can be ignored, and you need to add your thing via Bermuda, Configure, Configured Devices.

Some screenshots of what you’re seeing, and of your Bermuda configuration might be helpful if there’s any confusion

95

So bluetooth_le_tracker is completely separate from Bermuda, and you don’t need any of those entries in configuration.yaml for Bermuda to work (in fact, the poor state of bluetooth_le_tracker is one of the reasons Bermuda exists, that plus I knew it wouldn’t be a good fit for core for at least a while).

I can see you have the Devtracker Timeout in seconds to consider a device as "Not Home" set to (the default of?) 30 seconds. This is fairly short, and while I find that 180 seconds is pretty good, you could bump yours up as much as you need to (perhaps 4 minutes (240s) or more) to stop Bermuda from setting them as away as eagerly. Arrivals should still be instant, it’s only the “departure” that gets delayed.

In your dump_devices service output, you can see this block that shows the number of seconds between each received packet for that device (updated only every update_interval, so usually won’t show less than 1s):

     hist_interval:
        - 2.5520358320209198
        - 77.1560656990041
        - 2.591035165998619
        - 2.5540346179623157
        - 77.11702469200827
        - 2.5480331380385906
        - 2.5510331269470043
        - 3.8650500990333967
        - 73.26307638001163
        - 3.7490643459605053

The newest entry is at the top. So our last packet arrived 2.5 seconds after the previous one we saw, but before that it was over a minute (77 seconds) since we heard from that device. Then 2.5 seconds twice more, and 77 seconds again, and so on.

Bermuda gets this info by looking in Home Assistant’s internal cache of bluetooth packets, which (except for ones from usb bluetooth adaptors) will have timestamps on them supplied by either esphome or HA (not sure which part applies the stamps).

Given you are regularly getting 77 second gaps between packets, and you have the devtracker timeout in Bermuda set to the default 30 seconds, I think you’d probably find a big improvement by setting the timeout to at least 180, since “away” tends to not need to be as time-sensitive a status as “home”.

Is that likely to improve the situation for you?

96

Sorry for the delay in my response, combinations of real life and adhd mean I sometimes think I’ve already replied, or it slips my mind etc!

I plan for Bermuda to be an HA-native integration, so I don’t plan on doing any mqtt-specific things, but since they are just sensors on HA I expect they should all be available via the mqtt integration, if I understand correctly.

Re ideas for algorithmic improvements, there’s a few threads that are pretty much mandatory reading at this stage :slight_smile: I’ve tried to spell out the reasoning behind how I’ve set it up, so that people can evaluate that and provide ideas with the background knowledge on why it is currently how it is. I think most of the points you raise are well-covered in those, but let me know what you think after taking a read through:

Generally speaking, for Bermuda’s purposes, the maximum TX power is best, as it allows Bermuda to “hear” your devices from further away, maximising the opportunity to use multiple receivers (proxies) to make decisions about your device’s location.

If you have a fairly simple setup and just want to reduce the radius in which bermuda will detect the device, you can try lowering the power, but this is only a reasonable strategy when you have very few proxies. If you have more proxies the device should be “moved” to a closer area instead - but given the gaps in reception you are seeing it might be a reasonable option for you, at least until we have trilateration implemented.

97

v0.6.4 Released!

Very much a maintenance release, but with a few important bug fixes.

Thanks to @skrashevich and @itsteddyyo for their much-appreciated contributions via PRs merged into this release, and to those who have generously offered financial support to keep Bermuda improving. As well as the amazing community support these things make it that much more enjoyable to further this thing that I made to scratch my own itches :slight_smile:

Most of these changes have been sitting in the main branch of the repo for about two weeks, so you may already have these changes if you’re tracking that branch rather than official releases.

This release sees:

  • fixes to the dump_devices service call
  • making sure that changing the Area on a scanner applies properly
  • an important fix for devices that are both Private_ble and iBeacon (ie, all Android HA Companion app beacons) that would stop updating after a time ( @ingranu it might be worth checking if this was affecting your system)
  • some tidying / hardening of calculations
  • a process that “prunes” stale mac addresses from the in-memory cache to keep only the last 1000 un-tracked devices over the past 3 days. This may offer some CPU usage improvements, after 6 days my server went from 0.004 seconds per update cycle to 0.100s per cycle. Now it remains pretty stable around 0.005. It may improve memory usage but I didn’t see much difference there on my machine.
  • :rotating_light: Device names are now filtered for whitespace and binary cruft before and after the name. I have a device that calls itself ‘UC96BLE\0\0\0\0’ which makes it very unpopular with my long-term stats database :slight_smile: Note that you might find some device names may change if they had whitespace or non-alphanum chars in their names previously.

Changes

Full release notes at Github, update available via HACS.

2 Likes
98

I really love this integration. I’m having great success with shelly minis as bluetooth proxies. They’re cheap and can install behind an existing switch or outlet. The shelly integration is local and really nice. Set them up and turn on bluetooth and bermuda picks them up right away. I have 2 dogs that not everyone loves, so my main purpose for Bermuda is tracking them to know if they’re inside or outside the house. They have Blue Charm Beacons on their collars. I have a wifi pet door from High Tech pets integrated with HA and some cameras outside. Couple all of that with Bermuda, and I can know where they are at all times, with no battery life issues or expensive gps subscriptions. If guests are coming and I’m not home, I can keep them outside if they’re already there, or I can send them outside if they’re inside and close the door behind them. The dogs even have their own person card now! I really enjoy that on my mobile dashboard. It is useful, and just fun. Thanks @agittins!
image

4 Likes
99

Heads-up - reconfiguring in v0.6.4 causes HA lockup!

@sdholden28 discovered a bug in the new version v0.6.4 that causes the HA user interface to lock up if you change any of Bermuda’s config settings and click “Submit”.

I’ve been able to replicate the issue on my system, so should hopefully have a fix sorted out “tomorrow” - my brain is fried and overdue for sleep right now :slight_smile:

If you encounter this issue I’d suggest rolling back to v 0.6.3 until a fix is provided.
More info at After installing v0.6.4, changes to configuration page causes HA lockup · Issue #185 · agittins/bermuda · GitHub - and other observations welcome.

If you get the UI lock-up, you will probably need to restart HA - either by SSH’ing into your server, or if using a supervised install it looks like it will restart it automatically after perhaps 10 minutes.

Apologies for the inconvenience!

2 Likes
100

Cool system! Love the dog cards with pictures!

How do you “send them outside” if you are not home? Do you have some cool way to remotely deliver them treats outside with a sound alert so they run outside to get the treats?

101

They are both well trained and well behaved. They understand the “outside” command and go through the dog door whenever it is issued. A couple of the cameras are two-way audio so I can just tell them to go outside if I know where they are in the house. Now that I think of it, I really should just record the command and set up a button on the mobile dashboard to broadcast it. That would be easier.

Honestly, given the hours and money spent on training long ago, “the cool way to remotely deliver treats” would have been probably been cheaper and easier. :rofl:

3 Likes
102

Wow, those dogs are well-trained. Nice work on the HA system and also the dogs! :dog2: :dog:

1 Like
103

I am totally stoked to hear how it’s useful to you, that’s awesome! I also love the cards you have set up, they look amazing!

104

Release v0.6.5

This release fixes the nasty bug from v0.6.4 detailed in #185

Big thanks to @sdholden28, @terryhonn and @tbrasser for their contributions in narrowing down the fault, and again to @skrashevich for some more safety checks :+1:t3:

Changes

  • :warning: fix: Resolve infinite loop on config reload in 2024.5 @agittins (#189)
  • :safety_pin: fix(bermuda): Improved condition checks and handling of MAC addresses @skrashevich (#186)
  • :books: chore (docs): Refactoring docs over to wiki @agittins (#187)
  • :books: chore (docs): Fix link to FAQ from readme @agittins (#188)

Update via HACS recommended, full release info here

1 Like
105

Hello can you tell me exactly which model of Blue Charm Beacons you have?

106

I have the “BC04P-MultiBeacon”, one for each dog. I used rivets to attach them to their collar. One dog is 90 lbs, one 10lbs. For the small dog, i thought the physical size was maybe too big when I put it on, but she doesn’t mind. They’ve been wearing them since 1st of April now. I bought them on Amazon.

Blue Charm Beacons - Long Range Phy (300m) Water-Resistant BLE iBeacon (BC04P-MultiBeacon) - w/Adjustable Motion Sensor https://a.co/d/0y1T2ek

1 Like
107

perfect thanks for the information.

Do you also have the problem that if you restart HA and the dogs are not at home at that moment, the status in Bermuda is set to unknown? And then as soon as the trackers are back in range the status is not updated, but the Bermuda integration has to be restarted first?

108

I haven’t seen that particular behavior. I have a large back yard and if the dogs are outside and far enough from the house, distance and “last seen proxy” go unknown, as expected, and the device_tracker for each dog goes to away. As soon as they are back in range of a scanner then all values are updated and their location changes. I don’t have to restart Bermuda or anything like that.