Bluetooth/BLE Scanner - Docker Container

11-09-2022: Container documentation updated for newer versions of Home Assistant (2022.11.1 as of this update.)

Contains Bluez command line tools and Python. Scans for Bluetooth devices and publishes to MQTT with json payload. Does not support encrypted MQTT and requires functional MQTT endpoint. I use this for Home Assistant but included Bluez tools for troubleshooting. Requires functional Bluetooth in the host operating system. Currently only amd64 builds available.

Based on the Python script written by @Bit-River on Home Assistant Community forums.
Image based on HilscherAutomation/netPI-bluetooth

Base image is bitnami/minideb:buster
Python 3.7.3 (debian package)
PahoMQTT 1.5.1 (pip package)
PyBluez 0.41 (pip package)
Bluepy 1.3.0 (pip package)
gattlib 0.20201113 (pip package)
Bluez 5.54 (compiled from source)

Script will scan for:

  • all visible BLE devices and publish to MQTT with as MQTT_CLIENTID/MAC { addrType, location, rssi }
  • known Bluetooth devices and publish to MQTT as MQTT_CLIENTID/MAC { name, location }

addrType can be “public” or “random”
name is device-provided
Location is always “home”

Docker Requirements

Requires --cap-add=SYS_ADMIN, --cap-add=NET_ADMIN and --net=host

There is a shell in this image as /bin/sh and Bluez command line tools located in /usr/bin
For troubleshooting or pairing bluetooth devices, access shell within container: docker exec -it pybtble /bin/sh

Entrypoint is Python script /root/pybtble.py

Environment

MQTT_SERVER: IP/FQDN of MQTT broker
MQTT_PORT: port of MQTT broker above
MQTT_USER: username for MQTT broker
MQTT_PASS: password for username above
MQTT_CLIENTID: MQTT client ID or autogenerated pybtble-[5chars] (ie: pybtble-6bd6b)
BT_KNOWN_MACS: Quoted, comma-separated list of Bluetooth MAC addresses.
BLE_PUB_RANDOM: Publish randomized BLE addresses to MQTT. True/False. If not set, defaults to True.

MQTT_SERVER, MQTT_PORT, MQTT_USER and MQTT_PASS are mandatory values.

Configuring Bluetooth (non-BLE) devices

Bluetooth devices only can be scanned when in pairing mode which is not useful. This container has the required tools to execute the pairing process below. Do not use this for BLE devices which are passively scanned. Once paired with your BT dongle/controller, a direct name lookup to the device will succeed when the device is in range.

Put your BT device into pairing mode and then pair it to your BT dongle/controller by executing the following within the container:

hcitool cc 11:11:11:11:11:11 
hcitool auth 11:11:11:11:11:11 
hcitool dc 11:11:11:11:11:11 

Where 11:11:11:11:11:11 is the MAC address of the Bluetooth device. On Android devices you’ll need to be in Bluetooth Settings to see the pair request. On Apple devices, a dialog should pop up regardless of which screen you’re in. Accept the pair request and you can then scan for the device using this container. Device names are encoded/decoded as ASCII to overcome the apparent Unicode strings presented by Apple devices.

The Bluetooth devices configured here should then be added to the BT_KNOWN_MACS docker environment variable.

Home Assistant

While you can use the MQTT data for device_tracker in Home Assistant, in my config, I use it as a binary_sensor instead:

binary_sensor:
  - platform: mqtt
    name: Bluetooth Device
    state_topic: "pybtble/11:11:11:11:11:11"
    payload_on: "home"
    value_template: '{{ value_json.location }}'
    off_delay: 180
    qos: 0

Docker Compose

services:
  pybtble:
    container_name: pybtble
    image: flatline69/pybtble:latest
    environment:
      MQTT_SERVER: "172.20.10.100"
      MQTT_PORT: "1883"
      MQTT_USER: "username"
      MQTT_PASS: "password"
      MQTT_CLIENTID: "pybtble"
      BT_KNOWN_MACS: "11:11:11:11:11:11,22:22:22:22:22:22,33:33:33:33:33:33"
      BLE_PUB_RANDOM: "false"
    restart: unless-stopped
    network_mode: host
    cap_add:
    - SYS_ADMIN
    - NET_ADMIN

Logging Output

Found BLE device: a4:c1:38:55:d5:c0
Found BLE device: a4:c1:38:b4:20:2c
Scanning for BT devices...
Found known BT device: 11:11:11:11:11:11
Found known BT device: 22:22:22:22:22:22
Found known BT device: 33:33:33:33:33:33
Found BLE device: 54:bd:79:0e:90:9b
Found randomized BLE device: 7f:be:af:b3:de:e3 - not published
Found BLE device: d0:03:df:ed:ce:1f
Found BLE device: 68:72:c3:03:9c:6b
Found randomized BLE device: 53:d5:b3:03:a7:d1 - not published
Found BLE device: 9c:20:7b:e6:8b:14

–ORIGINAL POST–

Hello Community!

I’ve been using Home Assistant for a few years now, posted a few times here and there on the community and helped where I could. This time I present something I feel might be really useful for folks (as it was for me.) If you check my post history around this topic, it’ll probably make more sense (see here: MQTT parsing help - #4 by chrisw)

Like most of you, I have BLE and BT devices that I ingest into Home Assistant and then act upon. For example, I have BLE Tiles, BT on my family’s phones, etc. Problem was (for me) that getting BT/BLE working in Home Assistant wasn’t too problematic but it took a lot of doing most of the time. I’ve seen posts from folks having issues getting it working in order to track their devices as well.

Some time ago, @Bit-River posted a python script which I originally used to track my Tiles (and various other devices) which I later extended to use MQTT. This worked great and I used it for a good deal of time but it too was problematic and would randomly crash as I had to run it on my host machine and package updates, etc could end up clobbering the scripts execution.

So what I’ve done is created a small docker container that scans for known BT MACs and all visible BLE devices. It also includes command-line tools for troubleshooting. It then logs what it sees to MQTT which can then be ingested into Home Assistant as you wish.

I’m hoping this will help someone out there, it made my life easier moving it to a container and I ended up learning quite a bit about docker in the process. I’ve tried to document as much as possible to ensure successful run of the container.

You can find it here:

https://hub.docker.com/r/flatline69/pybtble

2 Likes

Hello and thanks for your work!

Can’t get it. I’ve used this command to deploy:

sudo docker run -d \
    --net=host \
    --privileged \
    --cap-add=SYS_ADMIN \
    --cap-add=NET_ADMIN \
    --name="BLE2MQTT" \
    -e MQTT_SERVER="_____" \
    -e MQTT_USER="mqtt" \
    -e MQTT_PASS="_____" \
    -e MQTT_PORT="1883" \
    -e MQTT_CLIENTID="pybtble" \
    -e BT_KNOWN_MACS="_____" \
    flatline69/pybtble:latest

I’ve replaced some secure information with ____. So at this conteiner i’m getting logs:

Scanning for BT devices...
Found randomized BLE device: _____
Found randomized BLE device: _____
Scanning for BT devices...

etc. And at mosquitto i’m getting repeated:

1611860855: New connection from _____ on port 1883.
1611860855: New client connected from _____ as pybtble (p2, c1, k60, u'mqtt').
1611860855: Client pybtble disconnected.

and i can’t understand, is connection OK or not. So i tried to add a binary sensor at HA:

- platform: mqtt
  name: MiBand B2M
  state_topic: "pybtble/_____"
  payload_on: "home"
  value_template: '{{ value_json.location }}'
  off_delay: 180
  qos: 0

and it’s always off so i think there’s a connection problem but i can’t understand, why. Any ideas?

Well, the reason was i created a pwd file with -H sha512-pbkdf2 as recommended, but mqq said “Invalid password hash for user mqtt, removing entry.” so anonymous clients keeped connected, but not this one.

I’ve managed to start it, but still have 2 questions:

  1. Is it ok that it keeps connecting, pushing data, disconnecting and round?
  2. Is is going to give HA known_devices or device_tracker by itself? I’ve only managed to get binary sensor using the example above.

Upd. MiBand can be discoverable, it’s a setting which can be made in app. It worked for a month or 2… but now i get Unknown error: Failed to execute management command ‘pasvend’ and container exits with errcode 0.

Upd2 - got error above cause host had lost a bluetooth adapter. Now OK.

1 Like

Hi there.

I’m looking at your posts and it seems the first one where you show output from the container and mosquitto, it’s working as it should. This is how my logs look.

I think the issue here (other than perhaps configuration changes to MQTT that you did) is that your HA sensor isn’t picking up the value. I would suggest testing with the sensor example I gave above and see how it operates. If it turns on/off based on conditions (my go-to test for testing BT/BLE is getting some tinfoil and wrap the device such as a Tile or turn BT off on a phone) you can artificially create.

It does connect/reconnect, that’s normal too. I suppose I could have it stay connected but I saw no impact on my machine doing it this way (and honestly, this is exactly how I did it in the precursor to this work without issue.)

Again, I think you had it right the first time it’s just your HA sensor isn’t working as expected and again suggest you test with the binary_sensor I provided in the OP above to ensure it’s working then figure out where yours isn’t working as expected (the idea being you know the data is good, the retrieval is the problem.)

Hope that helps.

EDIT: Upon re-reading (I apologize, I should have read it more than twice) I see you did indeed use my example. But I also noticed it’s a MiBand. I’m not sure if these devices work different than any band I’ve ever used but if it’s paired to a phone, you can’t see it. If you disconnect it from the phone, it becomes visible. I’ve never had any luck with these types of devices in any other configuration so I stopped tracking bands. I have no idea about high-end devices though.

MiBand has a broadcast option in app so it can be seen.

Now my turn to apologise - i was not reading forums cause all worked OK. Well, now not. Synology gave made an upgrade to their OS named DSM and stopped support for usb devices. I managed to readd them using commands

/sbin/modprobe usbserial
/sbin/modprobe ftdi_sio
/sbin/modprobe cdc-acm
retyping em like a monkey, and cc2531 started well, but your cointainer now fails with error

BLE_PUB_RANDOM is not configured. Will publish randomized BLE addresses to MQTT.
BLE_PUB_RANDOM is TRUE. Will publish randomized BLE addresses to MQTT.
Scanning for BLE devices…
Unknown error: Management not available (permissions problem?)

even it is --privileged and --cap-add=SYS_ADMIN --cap-add=NET_ADMIN --net=host

1 Like

Hi there Chris, I got a bluetooth explorer usb adapter that I am trying to get working but am having some issues. My device address is /dev/ACM0, i wasn’t sure if I need to pass this through via docker, and what I should use inside the container.

I keep getting this error:
Unknown error: Failed to execute management command ‘le on’ (code: 17, error: Invalid Index)

Connecting to XXX.XXX.XXX.XX port: 1883

Using credentials username password: password

No known Bluetooth addresses defined or malformed in environment.

BLE_PUB_RANDOM is FALSE. Will not publish randomized BLE addresses to MQTT.

Scanning for BLE devices...

Unknown error: Failed to execute management command 'le on' (code: 17, error: Invalid Index)

The container never actually starts, it just keeps rebooting and I can’t get into the console at all. Any advice would be appreciated. Thanks

Hi There!

The issue you have looks very similar to the issues I ran into with my host OS while building this tool. Make sure your host OS has a functional Bluetooth stack installed (and assuming Linux here) and hcitool does indeed see your adapter, ie:

[email protected]:/# hcitool dev
Devices:
        hci0    xx:xx:xx:xx:xx:xx

With your userland tools (again assuming Linux,) conduct some manual scans with hcitool lescan and hcitool scan and see if results are returned. If not, the container probably won’t work.

Ensure the container is running with SYS_ADMIN and NET_ADMIN capabilities and host-mode. I do know while troubleshooting the container when I changed my host OS I found it would crash and restart just as you described when it either didn’t have enough permission or couldn’t access a host-device (no Bluetooth stack installed.)

Hope this helps!

Hi NoAdo,

I’m not sure if this ended up working out for you (seems it didn’t even with upd2) but if not there’s not much one can do that I’m aware of outside of Synology’s ecosystem/walled-garden. I could be very wrong here. It’s unfortunate they stopped supporting USB devices but maybe there’s some open community out there you might find a better answer for this.

At the time of the original post, I was running OpenMediavault as my NAS and now I’m running TrueNAS SCALE and there are enough differences between the two alone but fortunately Bluetooth worked like a charm once the userland was installed on TrueNAS SCALE.

I’ve closed this question for me buying a ESP32 device with BLE instead of fighting Synology.

Hey there! Do you have the source/Dockerfile for this published somewhere? I’d love to be able to build it for ARM64

Somewhere in a backup from 2 yr ago. I’m still looking for it so I can update the container :slight_smile:

Way less necessary (for me) now that ESPHome has the Bluetooth proxy capability.