Shelly Plus Bluetooth Proxy set up for presence detection - problems!

alastaid · March 22, 2024, 7:27pm

If you get that working here is my automation, that fires every time there is a state change on the Private BLE device, it looks to see if the source has changed, if so, match the mac address to the real room, and post that to the mqtt_device. I have been trying to work out in the code what triggers the update of the source attribute of the Private BLE, when the Shelly integration is getting multiple updates per second, and where the logic is deciding which is the nearest Shelly device.

###############################################################################
## Apple watch location
###############################################################################

- id: apple_watch_location
  alias: apple_watch_location
  trigger:
    - platform: state
      entity_id:
        - device_tracker.alastairs_watch
  condition:
    - condition: template
      value_template: >
        {{ trigger.from_state and
           trigger.to_state.attributes.source !=
           trigger.from_state.attributes.source }}
  action:
    - service: input_text.set_value
      data_template:
        entity_id: input_text.alastairs_location
        value: > 
          {% set beacons = {
            'xx:xx:xx:xx:xx:xx':'Kitchen',
            'xx:xx:xx:xx:xx:xx':'Lounge',
            'xx:xx:xx:xx:xx:xx':'Bedroom' } %}
          {{ beacons.get(state_attr('device_tracker.alastairs_watch','source')) }}
    - service: mqtt.publish
      data_template:
        topic: homeassistant/device_tracker/alastairs_location
        payload: >
          {% set beacons = {
            'xx:xx:xx:xx:xx:xx':'Kitchen',
            'xx:xx:xx:xx:xx:xx':'Lounge',
            'xx:xx:xx:xx:xx:xx':'Bedroom' } %}
          {{ beacons.get(state_attr('device_tracker.alastairs_watch','source')) }}

50bbx · March 22, 2024, 11:05pm

This is really good. Unfortunately I only see “Bluetooth LE” as the source (on mobile).

Is this part different for you?

alastaid · March 23, 2024, 8:16am

Yes, this is what I get

50bbx · March 27, 2024, 2:08pm

Can you confirm that you are using a Shelly Plus device as bluetooth proxy using the Bluetooth LE integration?

alastaid · March 27, 2024, 7:12pm

I think so, I have a bunch of Shelly UK plugs, added via the Shelly integration, then used the configure option in the Shelly integration for these devices and selected the Active Bluetooth scanning, which as far as I can make out adds and enables the bluetooth scanning script on the Shelly device. I then added my Apple watch as a Private BLE device and the info shown before started to populate. My HA instance is running on a Pi 4, not sure as such this makes the Shelly devices “proxying” the bluetooth. As far as I can make out the Shellys are sending the results of their bluetooth scans from the script using rpc over http via wifi direct to the HA instance running the Shelly integration, but I could have got this completely wrong.

vulcanjedi · April 1, 2024, 1:42pm

Much thanks I still dont know why my Mac only shows my Apple Watch and can’t get the IRK from my phone.
My issue is the Private BLE integration for me only has state changes of home/away and so it doesnt change unless I leave. I have multiple Shelly 2PM +s and was hoping to use w/ more granular presense detection but from what I can tell this attribute doesnt change unless the home/away state does which is far less useful to know which proxy detected I came first.

alastaid · April 1, 2024, 2:09pm

So just to confirm, you have pressed the configure on the device in the Shelly integration page, and chosen active scanning. Then gone on to the web page of the Shelly device, chosen scripts, and made sure the script is running. Also, turn on the debug in the panel below the script on the shelly device and you should see the output of the scan running, it should be very verbose. I can only think that the script isn’t running, and that would stop the attribute info I am seeing on HA instance being displayed.

vulcanjedi · April 1, 2024, 2:42pm

I believe I am only doing passive scanning atm. I don’t have any BT devices I want ported into HA so was currently only trying to leverage for presence detection.
The shelly script is running…etc I see the attribute but it just doesnt change unless the home/away state changes. If I walk thru the house the attribute doesnt change say if i was right next to a different shelly from what I can tell.

alastaid · April 1, 2024, 3:38pm

You maybe where I am then, it takes about 20 minutes before I get the attribute to change to reflect which room I am in. I have been trying to trace in the code what conditions define when it changes in the attribute but it is slow going,

vulcanjedi · April 1, 2024, 6:18pm

I havent had the need/opportunity to mess with much exposed via the ‘attributes’ til lately but this. But from the logbook I never see additional updates unless the wholesale state changes from home/away. so intra house moves dont show up as such as an additional ‘home’ entry so not sure if Id validate such as you say 20 mins later if the attribute is in fact changing. But currently no way its usable as room detection. Didnt realize it supported flashing esphome but I’m too lazy to pull them out to UART flash them and dont really want to esphome flash them anyhow and rather keep native shelly code on em. Frustrating cause in the shelly script log you can see the exposed data is there.

vulcanjedi · April 2, 2024, 2:12pm

Agree i got into Grafana seems like it just polls 20mins?

agittins · April 20, 2024, 5:45am

For anyone looking for room/area-based BLE detection, the Bermuda integration (disclosure: I wrote it and have all the biases) works with Shelly Plus devices (and esphome proxies) and gives fast responses to area changes and home/away status.

I don’t have any Shelly’s myself (yet) but others are using it successfully. The Shelly’s do seem to have some limitations on how much broadcast traffic they’re willing to send (looks like they cache 3 seconds worth as of v1.0) but if you’ve multiple devices in an area that would probably even out OK.

It updates every second, as long as the proxies have sent new data.

It has support for Private BLE Device and iBeacons, so you can get iOS and Android devices working in it, too.

I’m interested in giving better support to those who are trying to use Shelly devices so any input on how to optimise the advert forwarding is welcome - although a lot of it seems to be well-covered in this thread already, so I might point people here for specific help.

Community thread: Bermuda - Bluetooth/BLE Room Presence and tracking [custom integration]

Github: GitHub - agittins/bermuda: Bermuda Bluetooth/BLE Triangulation / Trilateration for HomeAssistant

alastaid · May 1, 2024, 1:04pm

@agittins thanks for the nudge (and for writing Bermuda), I had looked at Bermuda a while ago but I think at the time it didn’t support Private BLE, however, I have installed Bermuda now and it works perfectly, approx 1-2 seconds latency typically from arriving in a room to it displaying in HA! So for anyone else, using Apple Watch, Shelly Plus devices, Private BLE and the Bermuda integration you can get rock solid and responsive room presence.

agittins · May 2, 2024, 4:21am

Yeah, Private BLE was a more recent addition That’s great to have confirmation that it’s working for you particularly re Shelly and Apple Watch

jptuomi · May 22, 2024, 7:27am

Recently installed and started using bermuda and now while troubleshooting found this thread as well. First off @agittins, hats off for you and your great bermuda, the engagement in the issues from contributors is nice to see as well, looking forward to the progress.

I have about 25 regular shelly devices where approx half are gen 2, the rest are dimmers. Further I have 2 shelly 1 pluses flashed to ESPHome to handle active connections.

So I’ve found something interesting, when I got to wanting to track our phones I hit a little snag, something that’s not well documented in either the issues or the documentation of private_ble_device is that it is possible to track other stuff than just iPhones and Apple Watches, just by BT-pairing them to my MacBook I could follow the same procedure.

This is not the only problem however. I currently have 3 devices added to bermuda, 1 robo mower via the regular bluetooth integration and 2 phones via private_ble_tracker.

The first gets picked up no matter where it goes from all of the shelly 2nd gen devices with active shelly proxy mode, haven’t tried passive as I’m going for best compatibility.

The 2 phones added via private_ble_tracker are just picked up by the two ESPHome proxies and you’re not gonna be able to get good triangulation from that in the long run when 2d-plane and even 3d-plane is supported.

So could it be that private_ble_tracker is only making use of what ESPHome and a local USB-dongle could provide @Jc2k ?

agittins · May 22, 2024, 9:27am

Great to hear you are having some wins with Bermuda!

I’ll have a think about updating the docs for Bermuda on IRK devices, I just haven’t got around to it (plus I wasn’t ready to take on too much support for that side of things, since Private BLE Device has its own docs on how to do the basics). What other devices did you have success with by pairing on the macbook? I know that iWatches have been successfully paired that way.

Interesting about the IRK devices only being picked up via the esphome proxies. I suspect that the issue is probably that the Shelly API is not sending through advertisements from random MAC addresses (ie, the “resolvable” addresses that IRK uses).

If it were, then Private BLE Device would reflect the updates, and even if it didn’t, Bermuda would because we only use PBD to find out the current MAC address of the unit, then we check each scanner for updates from that address.

However… I am pretty sure that others have IRK devices showing up via shelly units fine, so I wonder if it’s the firmware or config on your gen2 shellys that is the issue? Are you using the ble script directly installed via the shelly integration in HA, or did you use one of the scripts posted elsewhere?

Something that might help you troubleshoot in bermuda at least is to use the dump_devices service to see what it shows for the scanners list on the relevant devices. In the service call you can specify only the irk keys or mac addresses you are interested in (click the “fill example data” button to get hints).

Jc2k · May 22, 2024, 9:59am

From my side, if your system is set up correctly I’d expect it to just work. All Bluetooth scanners feed into the same API, so even if a new scanner was added to HA tomorrow private_ble_device would support it without any changes.

I don’t know much about Shelly’s. All mine run esphome. I know not all of them support Bluetooth tracking. And I think it’s off by default. But if it’s not something simple like that I’m not sure what’s happening.

jptuomi · September 19, 2024, 6:46am

Coming back to this topic to report that I’ve recently made progress.
I recently went through the hassle of enabling internet for the IOT-vlan to check for updates and lo and behold, I had missed some for my shellys.

So my issue was / is two-fold, first the non-esphome shelly devices were running firmware 1.0.8 which don’t do BT-proxying as well as the latest version of 1.4.2, go figure …

Further the signals from my different devices, Robo-mower, Mi Band 5 and Smartphones as Private Ble Devices added with the help of the Companion settings in the Home Assistant app, vary quite much in their strength leading to a hard time configuring RSSI for the receivers even with the latest RC… However it seems like there would need to be something specific to the device being tracked as well.

Anyway, this makes it a whole lot more usable and all of my devices are now seen by all shellys, esphome or not. Looking forward to future upgrades in functionality and actually being able to map out where a device is located relative to the trackers as shellys most often are hidden in the perimeter-walls to a room and not in the center of them…

zollak · September 24, 2024, 9:12pm

Hi, Bermuda BLE Trilateration is a very good alternative to ESPresence, especially for Shelly Plus (gen2) devices. I integrated it into HA a few days ago and I have to say it is very easy. So it works perfectly with my already installed Shelly devices and I don’t have to dedicate a separate ESP32 to it. Although, the HA “Private BLE Device” integration is required for iPhone devices and I was able to obtain the IRK in the easiest way with ESPresense. Now I “just” need the fine-tuning, because the Shelly devices sometimes flap between two rooms.

bobo2bobo · September 30, 2024, 1:52pm

Same here… Setup is great and works a charm! Except the sporadic “flap” as you called it between rooms… Have you had any success in stopping this?