[monitor] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

cnrd · December 12, 2018, 11:03am

Anyone who can get departure scans to work with iPhones?
I have 2 different iPhones: XS Max and SE, but I can’t seem to get reliable departure scans.

In the log below [IPHONE1MAC] was not present, but it was not marked as away by the script.

Dec 12 10:10:43 raspberrypi bash[422]: 0.1.748 10:10:43 am [CMD-INFO]        **** Started departure scan. [x3 max rep] ****
Dec 12 10:10:43 raspberrypi bash[422]: 0.1.748 10:10:43 am [CMD-SCAN]        (No. 1) [IPHONE1MAC] departure?
Dec 12 10:10:46 raspberrypi bash[422]: 0.1.748 10:10:46 am [CMD-SCAN]        (No. 1) [IPHONE2MAC] departure?
Dec 12 10:10:49 raspberrypi bash[422]: 0.1.748 10:10:49 am [CMD-INFO]        **** Completed departure scan. ****

Am I just misunderstanding how it is supposed to work?

forums2012 · December 12, 2018, 7:18pm

Bringing up an old question, but I found the answer.
From the forums, it seems like the way Google Home works is by scanning for BLE broadcasts. BLE broadcasts give you rssi values.
However BLE broadcast only works on phones when they’re actively searching for devices, unlike beacons which transmit BLE advertisements every second generally.

The way [Monitor] works is by waiting for random (phones keep changing this) bluetooth advertisements from a phone which occur frequently from the phone. Once it sees any random advertisements. It then starts a name search asking specifically if there’s a “xyz” iphone out there transmitting random advertisements (by using the pre-entered MAC address) (Unfortunately this is a yes/no protocol rather than giving the exact rssi value.)

I hope it helps anyone else out there.

Just wanted to put it out there. As after a lot of reading online, Monitor seems to be one of the easiest to setup solution possible with a phone for presence detection.

forums2012 · December 12, 2018, 7:21pm

I don’t think this means that they are out of range.
It just checked if they’re out, and they weren’t.

If you’re trying to test by turning off bluetooth then make sure you turn off bluetooth from the settings app not the control center. (Control center just turns off connecting to devices however transmission of bluetooth advertisements still takes place. Learnt it the hard way.)

forums2012 · December 12, 2018, 7:22pm

See if this answers your question?

I think the only workaround to this is using an android phone with those BLE emulating apps.

forums2012 · December 12, 2018, 7:23pm

AFAIK, it’s automatic internally through the monitor script.

forums2012 · December 17, 2018, 11:15am

After experimenting with other alternatives it seems like I am going to reset my pi zero to run Monitor.

However considering the cost, I would love to be able to use the same pi to add a Temperature and PIR motion sensor to the same pi zero w. Is that possible?

Sorry if this sounds pretty stupid, really new to this stuff. However I have definitely learned a lot.

skywise · December 17, 2018, 4:51pm

I just came across this script and I’m very interested in tracking who is home and away.

First, does Apple Watch work well? If I’m awake, I have the watch on so it’d be a perfect tracking agent.

Second, can/does the script monitor signal level from any given phone/watch? If so, it shouldn’t be too hard to track a location within the house. (do a measuring run by wandering to each room and get general signal levels from various monitor units)

Third, does monitor.sh work only with Pi Zero W? I can only buy single units (for $12) and then pay another $12 for shipping.

I think that’s it for now, I’m sure lots more questions to come.

andrewjfreyer · December 17, 2018, 6:04pm

Yes, monitor works fine with an Apple Watch (tested with my Series 0 and my Series 4), although some users have reported issues. I use my phone primarily.

For some devices, an RSSI is reported yes. However, the error in the signal varies so much with environment that distance/triangulation measurements are not at all reliable. Disappointing, but true.

I’ve written monitor to require as few dependencies as possible. Anything that has bluetooth hardware that can also run btmon and/or bluez should work just fine. This is most *nix boxes.

mattpitts74 · December 17, 2018, 10:15pm

@andrewjfreyer I just wanted to say thanks for all the massive amount of work you and other must have put into this project. Also thanks for the step by step instructions which I was able to follow without too much trouble. I now have monitor setup in a pi zero next to my front door. Using my phone for tracking and hoping this will trigger home more quickly than my previous setup which has been just using the bluetooth device tracker on my main raspberrypi running hassio. I’m also wondering if you would suggest running monitor on the pi running hassio to? is that possible? I have very limited understanding of the hassio setup and Docker so wouldn’t know where to begin. I also have a fairly small house and suspect its unnecessary.

I currently have no door sensor to send the depart message. but I assume that monitor sends depart after the timeout? as I see the confidence dropping over several mqtt messages once I turn off bluetooth. Although it takes quite some time to finally drop to 0. Is it possible to reduce this time?

Also I’m running with the default settings, which i assume is best for my setup?

andrewjfreyer · December 18, 2018, 11:16pm

Yes, by default depart scanning will occur after at least one “random” advertising device is not seen for a period of time.

anasazi · December 19, 2018, 7:44pm

HI,

What have you done to make it work with your Apple Watch 4?
I have tried to add the MAC address for the watch but also by scanning with sudo bash monitor.sh -b -g but none will work.

Thank you for a great script!

jbudd21 · December 20, 2018, 5:32pm

Thanks for the restart code, one question though.

If I am using this address (monitor/raspberrypi/38:53:9C:XX:XX:XX) to track my device, would I need to change the “location/scan/restart” topic in your restart action to “monitor/raspberrypi/scan/restart” in my case?

XeoSpeed · December 21, 2018, 1:25am

@anasazi sudo monitor.sh -g detects my Apple Watch. I’m inserting the Bluetooth mac for it into my known_beacon_addresses.

gth758y · December 21, 2018, 1:59am

I believe it would be monitor/scan/restart in your case, but I’ll let @andrewjfreyer confirm

andrewjfreyer · December 21, 2018, 2:35am

Confirmed!

jbudd21 · December 21, 2018, 2:59am

@andrewjfreyer and @gth758y thank you both for the quick responses!

steeldragon · December 21, 2018, 5:07am

I’m at my wits end trying to get Monitor to communicate with my mosquito broker. I keep getting “connection refused” in the Monitor logs (repeatedly) and in my hass.io mosquito broker I get “socket error on client ”. I’m running Monitor on a RPI3 B+ with Stretch and Hass.io on a separate RPI3 B. On my hass.io I have smartthings-mqtt-bridge communicating just fine with mosquito, so I know my broker is working just fine. I’m at a loss.

mqtt_preferences config:

mosquitto PREFERENCES

#---------------------------

#IP ADDRESS OF MQTT BROKER
mqtt_address=192.168.1.62

mqtt BROKER USERNAME (OR BLANK FOR NONE)
mqtt_user=

mqtt BROKER PASSWORD (OR BLANK FOR NONE)
mqtt_password=

mqtt PUBLISH TOPIC ROOT
mqtt_topicpath=monitor

#PUBLISHER IDENTITY
mqtt_publisher_identity=‘raspberrypi’

mqtt PORT
mqtt_port=‘1883’

mqtt CERTIFICATE FILE (LEAVE BLANK IF NONE)
mqtt_certificate_path=‘’

Hass.io Mosquito add-on config

{
“logins”: ,
“anonymous”: true,
“customize”: {
“active”: false,
“folder”: “mosquitto”
},
“certfile”: “fullchain.pem”,
“keyfile”: “privkey.pem”
}

I’m not using a username and password, however, I have tried using them and I get the same results.

Home Assistant sensor.yaml config

platform: mqtt
state_topic: ‘monitor/raspberrypi/xx:xx:xx:xx:xx’
value_template: ‘{{ value_json.confidence }}’
unit_of_measurement: ‘%’
name: ‘iPhone’

The errors I get look like this:

Hass.io mosquito broker logs:

1545366772: New connection from 192.168.1.63 on port 1883.
1545366772: Socket error on client <unknown>, disconnecting.
1545366773: New connection from 192.168.1.63 on port 1883.
1545366773: Socket error on client <unknown>, disconnecting.
1545366773: New connection from 192.168.1.63 on port 1883.
1545366773: Socket error on client <unknown>, disconnecting.

And the Monitor logs look like this:

Anyone have any ideas?

benjimatt · December 21, 2018, 11:22am

If I remember correctly you need to pair the iPhone with the Pi if you want rssi to be seen

benjimatt · December 21, 2018, 11:57am

This is part of the scripts section.

mqtt_depart:
  sequence:
  - delay: 00:00:10
  - data_template:
      topic: location/scan/depart
    service: mqtt.publish
  - delay: 00:00:20
  - data_template:
      topic: location/scan/depart
    service: mqtt.publish
  - delay: 00:00:30
  - data_template:
      topic: location/scan/depart
    service: mqtt.publish
  - delay: 00:00:40
  - data_template:
      topic: location/scan/depart
    service: mqtt.publish
  - delay: 00:00:50
  - data_template:
      topic: location/scan/depart
    service: mqtt.publish
  - delay: 00:00:60
  - data_template:
      topic: location/scan/restart
    service: mqtt.publish

This is Part of the Automation Section> State 22 is door open and state 23 is door closed

- id: trigger_depart_scan
  alias: trigger depart scan
  trigger:
  - entity_id: sensor.front_door
    platform: numeric_state
    above: 22
  action:
  - entity_id: script.mqtt_depart
    service: script.turn_on
- id: trigger_depart_scan_cancel
  alias: trigger depart scan cancel
  trigger:
  - entity_id: sensor.front_door
    platform: numeric_state
    below: 23
  action:
  - entity_id: script.mqtt_depart
    service: script.turn_off

benjimatt · December 21, 2018, 12:07pm

@andrewjfreyer

Im currently using 0.1.770

It looks like you have done much better in this version than the 675 in regards to bandwidth usage. Thanks