Connect Shelly BLU Door & Window directly to ESPHome driven ESP32 via Bluetooth

ESPHome
1

Hi all!

I recently flashed my Shelly Plus Plug S with ESPHome (this topic).

My goal is to create a washing machine monitoring plug with all the logic on this system. The device should only provide the actual status to home assistant when integrated. The ESPHome YAML file is ready and works fine. (I can share the complete code if needed. :slight_smile:)

The ESP monitors the power consumption and set the status from “Idle” to “Washing” and after this to “Ready”.

Here the Shelly BLU Door & Window comes into play. This sensor should be placed on the washing machine door. The Shelly Plus Plug S (with ESPHome on its ESP32 with Bluetooth) should connected directly to this sensor and receive the state updates. If the door is opened while the status is “Ready” the status should change back to “Idle”.

It is possible to connect the BLU Door & Window to Home Assistant and setup the logic there. But this is not what I want. Why? Aesthetics.

I spent many hours with ESPHome, the documentation, BLE logging apps but now it is time to ask for help. :slight_smile:

  • The Bluetooth Proxy is working (but not needed, I want to connect directly)
  • The esp32_ble_tracker is also working
  • The MAC address is known
  • The Service UUID is known
  • Many Service Characteristic UUIDs are known but not shure wich one is the right
  • Firmware of the BLU Door & Window is updated via Shelly BLU Debug App to “20230826-195045/v1.0.9-rc1@1fb04462”
  • ESPHome version is 2023.8.3
  • I am not able to get the information from the sensor
esp32_ble_tracker:
  scan_parameters:
    active: false
#bluetooth_proxy:
#  active: true
ble_client:
  - mac_address: 8C:6F:B9:2E:4D:80
    id: doorSensor
sensor:
  - platform: ble_client
    name: "Bullauge"
    id: frontDoor
    type: characteristic
    ble_client_id: doorSensor
    service_uuid: "180f"  # Anpassen
    characteristic_uuid: "2a19"  # Anpassen
#    service_uuid: "1D14D6EE-FD63-4FA1-BFA4-8F47B42119F0"  # Anpassen
#    characteristic_uuid: "F7BF3564-FB6D-4E53-88A4-5E37E0326063"  # Anpassen
    notify: true
    icon: "mdi:washing-machine"
    on_notify:
      then:
         - logger.log: "Notifiy sensor updated"
text_sensor:
  - platform: ble_client
    ble_client_id: doorSensor
    name: "Test"
    service_uuid: "DE8A5AAC-A99B-C315-0C80-60D4CBB51225"
    characteristic_uuid: "0BA9"
    on_notify:
      then:
         - logger.log: "Notifiy text-sensor updated"

Here is the log output if I open the configured sensor. I tried many different Service Characteristic UUIDs but the result is always the same.

[20:15:54][D][esp32_ble_client:048]: [0] [8C:6F:B9:2E:4D:80] Found device
[20:15:54][D][esp32_ble_tracker:214]: Pausing scan to make connection...
[20:15:54][I][esp32_ble_client:064]: [0] [8C:6F:B9:2E:4D:80] 0x00 Attempting BLE connection
[20:15:54][D][esp-idf:000]: W (2051537) BT_APPL: gattc_conn_cb: if=3 st=0 id=3 rsn=0x13

[20:15:54][D][esp-idf:000]: W (2051544) BT_HCI: hcif disc complete: hdl 0x0, rsn 0x13

[20:15:54][W][ble_sensor:037]: [Bullauge] Disconnected!
[20:15:54][D][sensor:094]: 'Bullauge': Sending state nan  with 0 decimals of accuracy
[20:15:54][W][ble_text_sensor:040]: [Test] Disconnected!
[20:15:54][D][text_sensor:064]: 'Test': Sending state ''
[20:15:54][W][esp32_ble_client:134]: [0] [8C:6F:B9:2E:4D:80] Connection failed, status=133
[20:15:54][D][esp32_ble_tracker:246]: Starting scan...
2

This is what another device with bluetooth proxy enabled is logging when the Shelly BLU Door & Window is configured directly in Home Assistant and forwards the data from the sensor to Home Assistant.

[19:35:45][W][component:204]: Component esp32_ble_tracker took a long time for an operation (0.05 s).
[19:35:45][W][component:205]: Components should block for at most 20-30ms.
[19:35:51][D][ble_adv:055]: New BLE device
[19:35:51][D][ble_adv:056]:   address: 8C:6F:B9:2E:4D:80
[19:35:51][D][ble_adv:057]:   name: SBDW-002C
[19:35:51][D][ble_adv:058]:   Advertised service UUIDs:
[19:35:51][D][ble_adv:062]:   Advertised service data:
[19:35:51][D][ble_adv:064]:     - 0xFCD2: (length 14)
[19:35:51][D][ble_adv:066]:   Advertised manufacturer data:
[19:35:51][D][ble_adv:068]:     - 0x0BA9: (length 13)
[19:35:57][D][ble_adv:055]: New BLE device
[19:35:57][D][ble_adv:056]:   address: 8C:6F:B9:2E:4D:80
[19:35:57][D][ble_adv:057]:   name: SBDW-002C
[19:35:57][D][ble_adv:058]:   Advertised service UUIDs:
[19:35:57][D][ble_adv:060]:     - DE8A5AAC-A99B-C315-0C80-60D4CBB51225
[19:35:57][D][ble_adv:062]:   Advertised service data:
[19:35:57][D][ble_adv:066]:   Advertised manufacturer data:
[19:35:57][D][ble_adv:068]:     - 0x0BA9: (length 13)
[19:35:58][D][ble_adv:055]: New BLE device
[19:35:58][D][ble_adv:056]:   address: 8C:6F:B9:2E:4D:80
[19:35:58][D][ble_adv:057]:   name: SBDW-002C
[19:35:58][D][ble_adv:058]:   Advertised service Uacturer data:
[19:35:58][D][ble_adv:068]:     - 0x0BA9: (lenUIDs:
[19:35:58][D][ble_adv:060]:     - DE8A5AAC-A99B-C315-0C80-60D4CBB51225
[19:35:58][D][ble_adv:062]:   Advertised service data:
[19:35:58][D][ble_adv:066]:   Advertised manufgth 13)
[19:35:58][D][ble_adv:055]: New BLE device
[19:35:58][D][ble_adv:056]:   address: 8C:6F:B9:2E:4D:80
[19:35:58][D][ble_adv:057]:   name: SBDW-002C
[19:35:58][D][ble_adv:058]:   Advertised service UUIDs:
[19:35:58][D][ble_adv:060]:     - DE8A5AAC-A99B-C315-0C80-60D4CBB51225
[19:35:58][D][ble_adv:062]:   Advertised service data:
[19:35:58][D][ble_adv:066]:   Advertised manufacturer data:
[19:35:58][D][ble_adv:068]:     - 0x0BA9: (length 13)
3

It seems a bit like a necro-posting but I couldn’t find a better thread, so here’s what worked for me and what I plan to check.

I was able to decode messages from Shelly BLU Door & Window (uses BTHome format) locally on Shelly Plus 1 using the following configuration:

esp32_ble_tracker:
  on_ble_service_data_advertise:
    - mac_address: 60:EF:AB:43:02:74
      service_uuid: "FCD2"
      then:
        - lambda: |-
            ESP_LOGI("ble_adv", "Advertised service data:");
            for (int i=1; i<x.size(); i++) {
              switch(x[i]) {
              case 0x00:
                ESP_LOGI("ble_adv", "pid: %02x", x[i+1]);
                i += 1; // = { n: "pid", t: uint8 };
                break;
              case 0x01:
                ESP_LOGI("ble_adv", "battery: %02x", x[i+1]);
                i += 1; // = { n: "battery", t: uint8, u: "%" };
                break;
              case 0x02:
                ESP_LOGI("ble_adv", "temperature: %02x%02x", x[i+2], x[i+1]);
                i += 2; // = { n: "temperature", t: int16, f: 0.01, u: "tC" };
                break;
              case 0x03:
                ESP_LOGI("ble_adv", "humidity: %02x%02x", x[i+2], x[i+1]);
                i += 2; // = { n: "humidity", t: uint16, f: 0.01, u: "%" };
                break;
              case 0x05:
                ESP_LOGI("ble_adv", "illuminance: %02x%02x%02x", x[i+3], x[i+2], x[i+1]);
                i += 3; // = { n: "illuminance", t: uint24, f: 0.01 };
                break;
              case 0x21:
                ESP_LOGI("ble_adv", "motion: %02x", x[i+1]);
                i += 1; // = { n: "motion", t: uint8 };
                break;
              case 0x2d:
                ESP_LOGI("ble_adv", "door/window: %02x", x[i+1]);
                id(test_door).publish_state(x[i+1] > 0);
                i += 1; // = { n: "door/window", t: uint8 };
                break;
              case 0x3a:
                ESP_LOGI("ble_adv", "button: %02x", x[i+1]);
                i += 1; // = { n: "button", t: uint8 };
                break;
              case 0x3f:
                ESP_LOGI("ble_adv", "rotation: %02x%02x", x[i+2], x[i+1]);
                i += 2; // = { n: "rotation", t: int16, f: 0.1 };
                break;
              default:
                ESP_LOGI("ble_adv", "unknow");
                break;
              }
            }

This does not require bonding and I believe is not secure. But might be enough for the problem described initially (monitor washing machine). I would like to try to make it secure and my current plan is to try to implement the following:

  • Configure Shelly BLU Door & Window in ble_client as auto_connect: false
  • Provide a way to trigger the following action from Home Assistant:
    • Disable BLE scan
    • Connect to Shelly BLU Door & Window
    • Write passcode to Shelly BLU Door & Window as described in Encryption | Shelly Technical Documentation
    • Read encryption key (same reference as above)
    • Save encryption key in global variable
    • Re-enable BLE scan
  • Use encryption key from a global variable to decode Shelly BLU Door & Window messages.

I hope this will work then as follows:

  • Factory reset Shelly BLU Door & Window
  • Enable pairing mode in Shelly BLU Door & Window
  • Start the procedure described above

If everything works well this should result in encrypted configuration of Shelly BLU Door & Window which Shelly Plus 1 should be able to decode.

4

@badrpc did you manage to use encrypted communication between Shelly BLU devices and ESPhome ?

I recently discovered the cheap shelly BLU devices (Motion) and I’d like to get them connected to the Shelly gateway but also to ESPhome with BThome protocol through encrypted communication.
With the shelly BLE debug app on Iphone IOS, I could easily get the encryption key and then it would be quite easy to copy it to ESPhome, but how to decrypt data in ESPhome ?
Thansk for your help

5

I made a working proof of concept. I’m trying to find time to create a new component to process both encrypted and clear-text bthome messages directly in esphome but so far I haven’t been very successful in it (finding time that is).

I can dump PoC here later with some notes if you’re interested in trying it out but it’s nowhere close to copy and paste config and will require some editing including C++ lambdas.

If you can get the key in the app then you should be able to remove part of the code that’s responsible for enabling encryption and reading the key and just hardcode the key in the decryption code.

Also keep in mind that my PoC is missing protection against replay attacks and dose not discard duplicate packets. I plan adding both of these to the component but I just skipped this part to save time in PoC.

6

Here’s the decryption part (sorry it’s ugly - it was not intended for sharing or any kind of real use but rather only to prove the idea):

esp32_ble_tracker:
  on_ble_service_data_advertise:
    - mac_address: 60:EF:AB:43:02:74
      service_uuid: "FCD2"
      then:
        - lambda: |-
            const uint8_t *data = x.data()+1;
            size_t data_len = x.size()-1;
            uint8_t cleartext[x.size()];

            // Decrypt starts here
            mbedtls_ccm_context ctx;
            mbedtls_ccm_init(&ctx);
            int ret = mbedtls_ccm_setkey(&ctx, MBEDTLS_CIPHER_ID_AES, id(test_door_sensor_key), 16 * 8);
            if (ret) {
              ESP_LOGE("ble_service_data lambda", "mbedtls_ccm_setkey() failed: %04x", ret);
              mbedtls_ccm_free(&ctx);
              return;
            }

            // Nonce:
            //                   Device data
            // MAC          UUID |  Counter
            // ------------ ---- -- --------
            // MAC 6 bytes
            // UUID 4 bytes
            // Device data 1 byte
            // Counter 4 bytes
            uint8_t nonce[13] = {0x60, 0xEF, 0xAB, 0x43, 0x02, 0x74, 0xD2, 0xFC, 0x00, 0x00, 0x00, 0x00, 0x00};
            nonce[8] = x[0];
            nonce[9] =  x[x.size()-8];
            nonce[10] = x[x.size()-7];
            nonce[11] = x[x.size()-6];
            nonce[12] = x[x.size()-5];

            uint8_t tag[4]; // = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
            tag[0] = x[x.size()-4];
            tag[1] = x[x.size()-3];
            tag[2] = x[x.size()-2];
            tag[3] = x[x.size()-1];

            ret = mbedtls_ccm_auth_decrypt(&ctx, x.size()-9, nonce, 13, NULL, 0, x.data()+1, cleartext, tag, 4);
            if (ret != 0) {
              ESP_LOGE("ble_service_data lambda", "mbedtls_ccm_auth_decrypt() failed: %04x", ret);
              mbedtls_ccm_free(&ctx);
              return;
            }

            mbedtls_ccm_free(&ctx);

            data = cleartext;
            data_len = x.size()-9;
            // Decrypt ends here

            ESP_LOGI("ble_adv", "Advertised service data:");
            for (int i=0; i<data_len; i++) {
              switch(data[i]) {
              case 0x00:
                ESP_LOGI("ble_adv", "pid: %02x", data[i+1]);
                i += 1; // = { n: "pid", t: uint8 };
                break;
              case 0x01:
                ESP_LOGI("ble_adv", "battery: %02x", data[i+1]);
                i += 1; // = { n: "battery", t: uint8, u: "%" };
                break;
              case 0x02:
                ESP_LOGI("ble_adv", "temperature: %02x%02x", data[i+2], data[i+1]);
                i += 2; // = { n: "temperature", t: int16, f: 0.01, u: "tC" };
                break;
              case 0x03:
                ESP_LOGI("ble_adv", "humidity: %02x%02x", data[i+2], data[i+1]);
                i += 2; // = { n: "humidity", t: uint16, f: 0.01, u: "%" };
                break;
              case 0x05:
                ESP_LOGI("ble_adv", "illuminance: %02x%02x%02x", data[i+3], data[i+2], data[i+1]);
                i += 3; // = { n: "illuminance", t: uint24, f: 0.01 };
                break;
              case 0x21:
                ESP_LOGI("ble_adv", "motion: %02x", data[i+1]);
                i += 1; // = { n: "motion", t: uint8 };
                break;
              case 0x2d:
                ESP_LOGI("ble_adv", "door/window: %02x", data[i+1]);
                id(test_door).publish_state(data[i+1] > 0);
                i += 1; // = { n: "door/window", t: uint8 };
                break;
              case 0x3a:
                ESP_LOGI("ble_adv", "button: %02x", data[i+1]);
                i += 1; // = { n: "button", t: uint8 };
                break;
              case 0x3f:
                ESP_LOGI("ble_adv", "rotation: %02x%02x", data[i+2], data[i+1]);
                i += 2; // = { n: "rotation", t: int16, f: 0.1 };
                break;
              default:
                ESP_LOGI("ble_adv", "unknow");
                break;
              }
            }

In this PoC test_door_sensor_key holds the key. I have it defined as

globals:
  - id: test_door_sensor_key
    type: uint8_t[16]
    restore_value: yes

but that’s only because I had another part that was configuring encryption and storing the key. If you know the key, you can just hardcode it. Please note that key is not the same as the PIN I configure in Shelly BLE Debug app (I have Android). Key is 16 bytes long and once the encryption is enables it can be read from

    service_uuid: "DE8A5AAC-A99B-C315-0C80-60D4CBB51225"
    characteristic_uuid: "eb0fb41b-af4b-4724-a6f9-974f55aba81a"

Ideally decryption should be conditional on the corresponding bit in device information byte of the packet:

  bool encryption;
  bool trigger_based;
  uint8_t bthome_version;

  v = service_data.data[0];
  encryption = v & 0b000000001;
  trigger_based = (v & 0b00000100) >> 2;
  bthome_version = (v & 0b11100000) >> 5;

  if (encryption) {
      // decrypt here
  }
  // process both encrypted and non-encrypted identically here.

Some references:

7

Thanks a lot !!!

8

Initial version of BTHome component is now available at esphome/esphome/components/bthome at dev · badrpc/esphome · GitHub. It can be added as an external component:

external_components:
  - source:
      type: git
      url: https://github.com/badrpc/esphome
      ref: v0
    components: [ bthome ]

And then configured in the following way:

esp32_ble_tracker:

bthome:
  - id: blu_door_window
    mac_address: AA:BB:CC:DD:EE:FF # MAC-address of BLU sensor
    encryption_key: "00112233445566778899AABBCCDDEEFF" # 128-bit key

binary_sensor:
  - platform: bthome
    id: blu_door_window
    window:
      id: test_blu_window
      name: "Test blu window"
      # on_press:
      #   then:
      #     - switch.turn_on: relay
      # on_release:
      #   then:
      #     - switch.turn_off: relay

sensor:
  - platform: bthome
    id: blu_door_window

    angle:
      id: test_blu_angle
      name: "Test blu angle"

    battery_level:
      id: test_blu_battery_level
      name: "Test blu battery level"
      entity_category: "diagnostic"

    illuminance:
      id: test_blu_illuminance
      name: "Test blu illuminance"

If encryption_key is not set component will decode unencrypted packets. With encryption key set it will ignore unencrypted packets. It is possible to enable encryption and obtain the key with the Shelly BLE Debug app (I used Android version, I believe there should be an IOS version too). After enabling encryption the app showed “Device found” message with json dump of device parameters, including encryption key.

On the lower level encryption key can be read from eb0fb41b-af4b-4724-a6f9-974f55aba81a UUID but this can only be done from a paired device (Encryption | Shelly Technical Documentation). Technically it is possible to enable encryption and read key directly in ESPHome and I have a proof of concept code to do that but it’s not in this component.

Currently this component only supports Shelly BLU door/window sensor. I would like to make this component a generic BTHome decoder/parser. This will probably mean significant change of the configuration format in the future to make it more generic.

There’s currently the same drawback as esp32_ble_tracker - after rebooting ESP state of the BLU device will be lost until it send next advertisement. This can be further improved by making ESPHome send an explicit query to BLU device after booting but this (as well as enabling encryption and reading a key) is not defined in BTHome standard and rather is specific to BLU devices. Possible direction for future work - a dedicated component based on bthome to fully support BLU devices (pairing, encryption management, refresh state on boot, device specific configuration).

9

Hi badrpc

Thansk for the excellent work !!!

I managed to adapt to include other BThome components and OIDs :

  • Shelly BLU H&T

    • humidity (0x2e)
    • temperature (0x45)

  • Shelly BLU Motion

    • motion (0x21)
    • illuminance (0x05)

It works quite well, but obviously I had to add some lines of code to define the new OiDs.

However I also had to fix or change some lines in the common part of the code :

all python .py files

I replaced the enumerated

from esphome.const import CONF_xxxx, CONF_yyyyy, ....

by

from esphome.const import *

and I added only what was not defined in esphome.const such as CONF_WINDOW

init.py :
line 35 : to make encryption optional (without encryption key the compilation failed)

    if CONF_ENCRYPTION_KEY in config:
         cg.add(var.set_encryption_key(config[CONF_ENCRYPTION_KEY]))

bthome.h :

line 161 : compilation failed without the "static_cast float "

    return FloatValue {
      .value = static_cast<float>(read_sint(sr.value_size, sr.value_ptr)) * f_num / f_denom,
      .next_ptr = sr.next_ptr,
    };

lines 166-168 were missing : (compilation failed without this statement)

   SensorPublisher *new_publisher(sensor::Sensor *sensor) const {
    return new SensorPublisher(oid, OIDSFixedPoint::read, sensor);
  }

some remarks :

10

Hi! Glad to see so much effort on bthome integration!

Has anyone of you already looked in BLU RC?
I mean the four button remote seen here: Shelly BLU RC Button 4 – Shellyparts.de

Anything I can provide to help?