Shelly Plus Bluetooth Proxy set up for presence detection - problems!

felixmxr · January 7, 2024, 5:57pm

So i got something working with Shelly’s BLE and the mqtt_room integration.

configuration.yaml

- platform: mqtt_room
  device_id: "8c64a22cde82"
  name: "Nordbuds"
  state_topic: "espresense/rooms"
  timeout: 10
  away_timeout: 120

Shelly script

// mqtt_room BLE script
BLE.Scanner.Subscribe(function (ev, res) {
    if (ev === BLE.Scanner.SCAN_RESULT) {
      let room = "office";
      let maxDistance = 3;
      let minRSSI = ( maxDistance * -15 ) - 40;
      let topic = "espresense/rooms/"+room;
      
      if (res.rssi > minRSSI){
        let id = res.addr.split(":").join("");
        let distance = ( res.rssi + 40 )/ -15;
        distance = distance < 0.1 ? 0.1 : distance;
        console.log(topic, id, distance);
        if (MQTT.publish(topic, '{ "id": "'+id+'", "distance":'+distance+' }')){
          console.log('published')
        }
      }
    }
});

BLE.Scanner.Start({
    duration_ms: -1,
    active: false,
    interval_ms: 320,
    window_ms: 30,
});

console.log('Script started')

if (MQTT.isConnected()){
  console.log('MQTT is connected')
}

The distance calculation and filtering is crudely based on RSSI.

The Shelly should be connected to the homeassistant MQTT server
and bluetooth should be enabled on the Shelly

This still cannot track my android phone however. It seems shelly only receives BLE advertisements

christf · February 26, 2024, 9:21pm

building on this, I installed a BLE peripheral simulator on my android phone.
It pretends to be a device having a battery and announces via BLE.
After activating it, I very much was able to see my phone, confirming your theory.

For easier reading it is possible to output the device name:

    console.log(topic, id, distance, BLE.GAP.parseName(res.advData));

I am not sure where to go from here. Having to have a scanner app open seems a bit inconvenient.

Edit:
https://www.youtube.com/watch?time_continue=241&v=p7C2QvmsM8M&embeds_referring_euri=https%3A%2F%2Fduckduckgo.com%2F&feature=emb_logo says to use a different tracker than the phone and I might try with a fitbit.

alastaid · March 2, 2024, 4:55pm

Just to resurrect this thread, I have recently switched all my smart plugs to the Shelly plugs, which enabled me to try this out. The good news is that using the new Private BLE you can use it to track iPhones or Apple watches, however I am struggling with how it is working and how it updates, as it is not very useful atm. I created an automation to read the attributes of the source of the Private BLE device when it changes, and then using mqtt_room map that to the correct room in the house. So the problem is it updates very slowly, even though it looks like the script on the shelly devices is running every 300ms. I wanted to understand a bit what is happening from an architecture pov, the shelly’s are all sending updates every 300ms (with 25 devices this is very chatty!) using rpc over http to the shelly integration (I assume), this is then being analysed somewhere in HA (where?) and passed to the Private BLE device as a distance measurement with an update of the mac address of the source. It takes sometimes 20 minutes to register a room change, the data is correct, I would just like to make this update happen a lot quicker.

vulcanjedi · March 22, 2024, 3:07am

How do you get the IRK from phone, I have ancient mac but only had one id and appears to be my only my also ancient watch?

vulcanjedi · March 22, 2024, 6:25pm

Can you share? I dont see anything in the attributes or anywhere suggesting which shelly provided what info. Just ambiguous signal reading for only the watch is what i can find IRK from.

alastaid · March 22, 2024, 7:09pm

Hi,

You get the IRK for the phone or watch by following the instructions in the Private BLE here . It’s a bit fiddly, especially when getting the IRK out of the XML stream, but it works. Once you have the Private BLE device showing up in Settings->Devices&Services->Private BLE, go into the device, click on the device in the Diagnostic panel, in the panel that opens up, you can pull down the Attributes tab, and the second source entry at the bottom will be the MAC address of the Shelly device that detected it.

alastaid · March 22, 2024, 7:27pm

If you get that working here is my automation, that fires every time there is a state change on the Private BLE device, it looks to see if the source has changed, if so, match the mac address to the real room, and post that to the mqtt_device. I have been trying to work out in the code what triggers the update of the source attribute of the Private BLE, when the Shelly integration is getting multiple updates per second, and where the logic is deciding which is the nearest Shelly device.

###############################################################################
## Apple watch location
###############################################################################

- id: apple_watch_location
  alias: apple_watch_location
  trigger:
    - platform: state
      entity_id:
        - device_tracker.alastairs_watch
  condition:
    - condition: template
      value_template: >
        {{ trigger.from_state and
           trigger.to_state.attributes.source !=
           trigger.from_state.attributes.source }}
  action:
    - service: input_text.set_value
      data_template:
        entity_id: input_text.alastairs_location
        value: > 
          {% set beacons = {
            'xx:xx:xx:xx:xx:xx':'Kitchen',
            'xx:xx:xx:xx:xx:xx':'Lounge',
            'xx:xx:xx:xx:xx:xx':'Bedroom' } %}
          {{ beacons.get(state_attr('device_tracker.alastairs_watch','source')) }}
    - service: mqtt.publish
      data_template:
        topic: homeassistant/device_tracker/alastairs_location
        payload: >
          {% set beacons = {
            'xx:xx:xx:xx:xx:xx':'Kitchen',
            'xx:xx:xx:xx:xx:xx':'Lounge',
            'xx:xx:xx:xx:xx:xx':'Bedroom' } %}
          {{ beacons.get(state_attr('device_tracker.alastairs_watch','source')) }}

50bbx · March 22, 2024, 11:05pm

This is really good. Unfortunately I only see “Bluetooth LE” as the source (on mobile).

Is this part different for you?

alastaid · March 23, 2024, 8:16am

Yes, this is what I get

50bbx · March 27, 2024, 2:08pm

Can you confirm that you are using a Shelly Plus device as bluetooth proxy using the Bluetooth LE integration?

alastaid · March 27, 2024, 7:12pm

I think so, I have a bunch of Shelly UK plugs, added via the Shelly integration, then used the configure option in the Shelly integration for these devices and selected the Active Bluetooth scanning, which as far as I can make out adds and enables the bluetooth scanning script on the Shelly device. I then added my Apple watch as a Private BLE device and the info shown before started to populate. My HA instance is running on a Pi 4, not sure as such this makes the Shelly devices “proxying” the bluetooth. As far as I can make out the Shellys are sending the results of their bluetooth scans from the script using rpc over http via wifi direct to the HA instance running the Shelly integration, but I could have got this completely wrong.

vulcanjedi · April 1, 2024, 1:42pm

Much thanks I still dont know why my Mac only shows my Apple Watch and can’t get the IRK from my phone.
My issue is the Private BLE integration for me only has state changes of home/away and so it doesnt change unless I leave. I have multiple Shelly 2PM +s and was hoping to use w/ more granular presense detection but from what I can tell this attribute doesnt change unless the home/away state does which is far less useful to know which proxy detected I came first.

alastaid · April 1, 2024, 2:09pm

So just to confirm, you have pressed the configure on the device in the Shelly integration page, and chosen active scanning. Then gone on to the web page of the Shelly device, chosen scripts, and made sure the script is running. Also, turn on the debug in the panel below the script on the shelly device and you should see the output of the scan running, it should be very verbose. I can only think that the script isn’t running, and that would stop the attribute info I am seeing on HA instance being displayed.

vulcanjedi · April 1, 2024, 2:42pm

I believe I am only doing passive scanning atm. I don’t have any BT devices I want ported into HA so was currently only trying to leverage for presence detection.
The shelly script is running…etc I see the attribute but it just doesnt change unless the home/away state changes. If I walk thru the house the attribute doesnt change say if i was right next to a different shelly from what I can tell.

alastaid · April 1, 2024, 3:38pm

You maybe where I am then, it takes about 20 minutes before I get the attribute to change to reflect which room I am in. I have been trying to trace in the code what conditions define when it changes in the attribute but it is slow going,

vulcanjedi · April 1, 2024, 6:18pm

I havent had the need/opportunity to mess with much exposed via the ‘attributes’ til lately but this. But from the logbook I never see additional updates unless the wholesale state changes from home/away. so intra house moves dont show up as such as an additional ‘home’ entry so not sure if Id validate such as you say 20 mins later if the attribute is in fact changing. But currently no way its usable as room detection. Didnt realize it supported flashing esphome but I’m too lazy to pull them out to UART flash them and dont really want to esphome flash them anyhow and rather keep native shelly code on em. Frustrating cause in the shelly script log you can see the exposed data is there.

vulcanjedi · April 2, 2024, 2:12pm

Agree i got into Grafana seems like it just polls 20mins?

agittins · April 20, 2024, 5:45am

For anyone looking for room/area-based BLE detection, the Bermuda integration (disclosure: I wrote it and have all the biases) works with Shelly Plus devices (and esphome proxies) and gives fast responses to area changes and home/away status.

I don’t have any Shelly’s myself (yet) but others are using it successfully. The Shelly’s do seem to have some limitations on how much broadcast traffic they’re willing to send (looks like they cache 3 seconds worth as of v1.0) but if you’ve multiple devices in an area that would probably even out OK.

It updates every second, as long as the proxies have sent new data.

It has support for Private BLE Device and iBeacons, so you can get iOS and Android devices working in it, too.

I’m interested in giving better support to those who are trying to use Shelly devices so any input on how to optimise the advert forwarding is welcome - although a lot of it seems to be well-covered in this thread already, so I might point people here for specific help.

Community thread: Bermuda - Bluetooth/BLE Room Presence and tracking [custom integration]

Github: GitHub - agittins/bermuda: Bermuda Bluetooth/BLE Triangulation / Trilateration for HomeAssistant

alastaid · May 1, 2024, 1:04pm

@agittins thanks for the nudge (and for writing Bermuda), I had looked at Bermuda a while ago but I think at the time it didn’t support Private BLE, however, I have installed Bermuda now and it works perfectly, approx 1-2 seconds latency typically from arriving in a room to it displaying in HA! So for anyone else, using Apple Watch, Shelly Plus devices, Private BLE and the Bermuda integration you can get rock solid and responsive room presence.

agittins · May 2, 2024, 4:21am

Yeah, Private BLE was a more recent addition That’s great to have confirmation that it’s working for you particularly re Shelly and Apple Watch