[monitor] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

Using the hassio addon. I was wondering if anyone has an example of how to combine this MQTT device tracker with nmap or ping. I’m slightly confused with the example https://www.home-assistant.io/integrations/device_tracker.mqtt#complete-example-configuration . Does the JSON code go in config.yaml as well?

Hello folks,

My iBeacon goes to “confidence 0” way too often. What could be the issue?
I’m running monitor on Raspberry Pi Zero on Wifi with internal BT+Wifi enabled (no dongles). My iBeacon is broadcasting at +4dB power/1000ms interval only some 3m away from the monitor-RPi inside my car. When I arrive home with my car the iBeacon is usually detected 10m away from the monitor-RPi. This iBeacon drops to “confidence 0” on average once every hour. PREF_BEACON_EXPIRATION is set to 120(s)

I’m monitoring 2 iPhones and 3 iBeacons and have blacklisted every unwanted broadcasting BT device/MAC I can detect. I’m living in rural/country area and my blacklist has some 10 mac addresses.
My iPhones are in known_static_addresses and iBeacons in known_beacon_addresses.

EDIT: After leaving home and coming back with a car (with the iBeacon inside) monitor has been showing solid “confidence 100” for 20h now. It’s the same distance (2m - 3m) from monitor.
Is there a way to see all the beacon advertisements in terminal, e.g. when the beacon is broadcasting every 1000ms?

Well, you don’t combine it directly. What you’d do is use a group to combine the state of the two. Then if either shows home the group is `home.

Sorry, I guess I’m still confused. I did a clean install on a Pi 4, and it’s not showing up on known devices.yaml (it’s blank), whereas when I added nmap, it showed up in known devices. I checked the Bluetooth Presence logs and it detects my phone, but I’m still confused on how to add mqtt as a device tracker. There’s also the JSON from the example and I have no clue on where to add that as well.

# Complete configuration.yaml entry
device_tracker:
  - platform: mqtt
    devices:
      paulus_oneplus: 'location/paulus'
      annetherese_n4: 'location/annetherese'
    qos: 1
    payload_home: 'present'
    payload_not_home: 'not present'
    source_type: bluetooth

{
  "topic": "location/paulus",
  "payload": "home"
}

That JSON is unrelated to the device tracker entry :wink:

Your device tracker config should be a lot simpler:

platform: mqtt
source_type: 'bluetooth'
devices:
  person2_bt_mobile: 'monitor/first floor rear/person2_mobile/device_tracker'

That’s one of mine, where I have the following in mqtt_preferences:

mqtt_topicpath=monitor
mqtt_publisher_identity='first floor rear'

The device’s alias in known_static_addresses is person2_mobile

I’ve been using iBeacons (radbeacon) for about a year, and my experience is that BT presence is good for triggering “arrival events” but not reliable for sustained “I am still here” state.
For example, I have an automation that is triggered when I wake up in the morning and go to the bathroom. Logic is like: If person has entered the bathroom zone for the first time today and it’s after 6am, then trigger automation. This automation works reliably, every day.
Another example: If person has left the house and if person returns to the house (iBeacon in front door area) after >10 minutes of being away, then trigger “welcome back home” automation. This trigger is faster than using wifi to detect when I arrive home.

Bad example of BT for room presense: If person has left the room, turn off the lights. This doesn’t work well because I can be in the room, but BT room presence falsely reports that I’m away from the room. Better logic might be something like: If person has left the room, set a 10 minute timer before turning off the lights. And if person returns to the room before 10 minutes expires, cancel timer and cancel turning off the lights.

@andrewjfreyer is there a reason your RSSI detection sets -99 for a variety of reasons?

I’m currently intending to update your scripts to simply ignore failed connections so that I can develop a form of proximity sensor, as well as update the RSSI publishing to also expose the RSSI that’s currently embedded in beacon payloads.

Was wondering if there was a reason you had elected to make these design choices that I’m not seeing a reason why. The confidence and device_tracker presence are both set correctly, we’re just unable to retrieve the rssi for whatever reason; but rather than persist the previous setting in those scenarios, we’re introducing extreme noise to the history.

History with default settings

Part 1/2 due to posting limits

Part 2/2 due to posting limits

History (from different sensor) excluding values of -99 in publish_rssi_message

This current hack isn’t the proper fix as it doesn’t account for 1 or 2 failures in the triple test, but it seems to suggest the results will be as desired.

UPDATE: Opened a PR to fix this issue - https://github.com/andrewjfreyer/monitor/pull/362

My other question was, is there a reason that placing an entry in the known_beacon_address file forces that device to reporting using their MAC address instead of their alias? If I only have things in known_static_address, then the alias is used, but if I use the beacon file, I get the below

# ---------------------------
#
# BEACON MAC ADDRESS LIST; REQUIRES NAME
#
# Format: 00:00:00:00:00:00 Nickname #comments
# ---------------------------

c8:80:b4:14:cd:8b fitbit_peter
cb:81:8b:f2:48:ee fitbit_amber

58:e6:ba:3a:ba:d3 iphone_peter
7c:d9:5c:a4:50:9e pixel3_amber

And from the monitor.sh -V -b

> blacklisted device: 50:DE:06:62:C8:E0
> 58:E6:BA:3A:BA:D3 confidence topic: home/bedroom/iphone_peter (has previously connected to hci0)
> 58:E6:BA:3A:BA:D3 device_tracker topic: home/bedroom/iphone_peter/device_tracker [not_home or home]
> 7C:D9:5C:A4:50:9E confidence topic: home/bedroom/pixel3_amber (has previously connected to hci0)
> 7C:D9:5C:A4:50:9E device_tracker topic: home/bedroom/pixel3_amber/device_tracker [not_home or home]
> known beacon: C8:80:B4:14:CD:8B publishes to: home/bedroom/C8:80:B4:14:CD:8B
> known beacon: CB:81:8B:F2:48:EE publishes to: home/bedroom/CB:81:8B:F2:48:EE
> known beacon: 58:E6:BA:3A:BA:D3 publishes to: home/bedroom/58:E6:BA:3A:BA:D3
> known beacon: 7C:D9:5C:A4:50:9E publishes to: home/bedroom/7C:D9:5C:A4:50:9E
> log listener pid = 6036

UPDATE: Opened a PR to fix this issue - https://github.com/andrewjfreyer/monitor/pull/361

I’m having some Homer moment because it’s not working out for me. I’m using the Hassio addon, the config is a tiny bit different, but short story of it is that it’s not showing up as an entity, so I can’t use it as a device tracker.

Is there a way to manually enter the info into known devices? Here’s my config from the addon. Detects everything perfectly judging from the logs. I also tried adding the mac address into the device tracker config, but that didn’t work either… Adding MQTT as a device tracker is so confusing! I’m clearly missing something, but I have no clue what that could be.

mqtt:
  broker: 127.0.0.1
  port: 1883
  username: username
  password: password
  topic_root: monitor
  publisher: first floor rear
known:
  beacons: []
  static:
    - '00:00:00:00:00:00 a'
    - '00:00:00:00:00:00 b'
    - '00:00:00:00:00:00 c'
    - '00:00:00:00:00:00 d'
    - '00:00:00:00:00:00 e'
blacklist: []
  1. Have you installed the Mosquito broker on your instance?

  2. Have you tried connecting to the MQTT broker to check the messages are being sent properly?

I used MQTT Explorer on Windows as an easy way to observe the queue - you should be able to login with the username and password you listed above.

  1. Have you connected the MQTT integration on your instance and configured it?

If you’ve done these things, and are correctly seeing entries, you may still need to add some configuration to enable things in HA.

You can either add sensors like this:

- platform: mqtt
  state_topic: 'home/hallway/peter_fitbit'
  value_template: '{{ value_json.confidence }}'
  unit_of_measurement: '%'
  name: 'Peter Fitbit'

Or, if there’s a device_tracker topic in MQTT, then you can directly add device_trackers like this:

  - platform: mqtt
    devices:
      hallway_phone_peter:  'home/hallway/iphone_peter/device_tracker'
      hallway_phone_amber:  'home/hallway/pixel3_amber/device_tracker'
      hallway_fitbit_peter: 'home/hallway/fitbit_peter/device_tracker'
      hallway_fitbit_amber: 'home/hallway/fitbit_amber/device_tracker'
      
      bedroom_phone_peter:  'home/bedroom/iphone_peter/device_tracker'
      bedroom_phone_amber:  'home/bedroom/pixel3_amber/device_tracker'
      bedroom_fitbit_peter: 'home/bedroom/fitbit_peter/device_tracker'
      bedroom_fitbit_amber: 'home/bedroom/fitbit_amber/device_tracker'
      
      lounge_phone_peter:  'home/lounge/iphone_peter/device_tracker'
      lounge_phone_amber:  'home/lounge/pixel3_amber/device_tracker'
      lounge_fitbit_peter: 'home/lounge/fitbit_peter/device_tracker'
      lounge_fitbit_amber: 'home/lounge/fitbit_amber/device_tracker'
    payload_home: 'home'
    payload_not_home: 'not_home'
    source_type: bluetooth

Great! Thanks! I was missing the quotes and didn’t subscribe to a topic in integration. Working now!

Only taken me a few hours but finally got this working. As I’m pretty much totally ignorant of all of this home assistant stuff and I am learning as I go along I would like a tiny bit of advice please.

So three bed house, typical UK size, living room, kitchen.

I have a pi zero running Monitor upstairs in my bedroom on my desk, where I tend to be most of the time. I have Monitor through the docker container on my hass instance downstairs in the living room. Both are in suitable proximities to anywhere in the house on that level. Don’t feel I need more monitor instances. I am just tracking three phones, myself, my partners and my sons.

So far I have not changed any behaviour preferences I am just looking at MQTT Explorer’s topics and I have set up device trackers for my hass monitor but not my pi zero monitor.

I have run monitor.sh with no extra arguments. I have read most of this thread, all 1992 posts but to be fair my eyes glazed over at one point.

Should I have any extra arguments? All I want to do is track home or not home, for quite rudimentary reasons - making sure all the lights have been turned off if we go out for a dog walk, or turning the heating off if it’s on and we’re out all day. Without any extra arguments, am I effectively spamming and causing inteference with the 2.4ghz wifi (so mgiht have zigbee issues?)

I said tiny bit of advice…

edit: I’ve been out, and zero monitor correctly reported my departure and arrival but the hass instance of monitor did not. same settings as far as I can tell

The answer is … complicated

I have it so that it does arrival scans automatically (and on demand), but departure scans only on demand. I use a door sensor (and some other logic) to trigger those.

So, you can use extra arguments to control that, but you don’t have to.

This is my setup as last documented, though I do need to do a minor update.

1 Like

If I don’t have to and it won’t degrade the system too much then I won’t bother. I don’t have 1000s of automations running anyway so one would hope I won’t slow anything down too much. I’ll check your config out, probably will change my mind haha

Anyone ever experienced their Monitor script go sort of haywire? Mine’s done it once or twice, today at 14:38 the script started posting declining confidence for both me and my wife, and soon set us to ‘0’ confidence and we were marked as not_home by Home Assistant.
Over the course of the rest of the afternoon the bluetooth dongles which are normally steady (so long as we’re home) have gone up and down in confidence like a yoyo.

Worked out (I think!) what was going on…

My MQTT sensors are sensors not binary sensors and when in a group Home Assistant sets the group status to unknown if the monitor sensor goes to unknown even if there is another group member home within the group.

Has anyone worked out a way to avoid this problem?

Thanks!

3 posts were split to a new topic: Enabling AppDaemon in HACS

Hi guys,
I’m running into some confusion to get multiple monitors setup.
I have 1 currently, using defaults from the github page.
mqtt_address=127.0.0.1
And the normal sudo bash monitor.sh
How do I make my first floor the main monitor?
And how do I add other monitors to my home?
I read stuff about different arguments like -b and so on, but not sure which to set and where to even set them?

I have it working for 1 node and 2 devices currently.
And its reporting in HA 100% for both devices when im close enough and 0 when i disable bluetooth.
But I’d like to make it more expansive and add more Rpi’s next week for better coverage

Is there any good guides on how to easily setup multiple Rpi’s for monitor and get them all talking to each other and to HA?
(sorry new to this)

Hi all, i’m using monitor now since a few months.
I trigger ARRIVAL and DEPART scans only via MQTT from a Fibaro Home-Center with Door-Sensors.
There are 3 Devices in the known_static_addresses file.

It does trigger Arrivals and Departs of all 3 devices correctly, but when only the 3rd device (3rd in list of known_static_addresses File) comes home (ARRIVAL, no other Devices home yet) there is a Delay of about 20 secondy until monitor detects the Device.
For the first Devices (first in list of known_static_addresses File) the delay is only about 2 seconds, sometimes faster.

Is it somehow possible to speed up the detection? Maybe by lower the delay between detecting the devices in the known_static_addresses list?

Thank you in advance for your help.

EDIT: I just took a closer look to the monitor.sh script and tried the command "hcitool name " by myself.
Even if i terminate the hcitool with Ctrl+C after e.g. 2 seconds of scanning a not present Device and immediately start a new scan of a present device, the present device will be discovered only after about 4 more seconds.
So i think that there is a 5 second timeout between every “hcitool name”-Scan.

Is this by design and hardware-related or can this somehow be changed?

EDIT2: i now have downloaded an old blueZ Library (5.40) where hcitool was included and changed the timeout there to 2.5 seconds and recompiled it. Now an Arrivalscan is much faster. I will look at my logfiles the next days if everything is working as expected.