[monitor] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

andrewjfreyer · September 23, 2018, 1:24am

What version of iPhone?

tc23 · September 23, 2018, 1:30am

iPhone S, version 11.3.1

J_IO_B · September 23, 2018, 6:09am

Wow! Thanks for the update and the work your putting in on this project! Would I be right in saying that to overcome the issues I was having previously I should use -tad flag?

jwelter · September 23, 2018, 1:13pm

I run HA on a NUC. Is it possible to also run this on the NUC as the master server as it’s located centrally on main floor.

I would add 2 RPI’s for basement and upstairs in slave mode.

NUC is running Ubuntu 18.04 and has built in BT.

andrewjfreyer · September 23, 2018, 3:19pm

Yes, the script will run on any UNIX or Linux box with all requirements and dependencies installed.

andrewjfreyer · September 23, 2018, 3:20pm

I would suggest to use -td, so that the only time a depart scan occurs is after a trigger is received

J_IO_B · September 23, 2018, 3:23pm

Thanks for the suggestion, would you suggest -td on both Pi Zeros or running this on one and leave my main one running with no flags?

andrewjfreyer · September 23, 2018, 3:32pm

Can you be more specific? iPhone 3GS? iPhone 4s? iPhone 5s? iPhone 7S? iPhone 7S plus?

If her phone is old, or her software is not updated, it is entirely possible that monitor or presence will not be able to detect her phone.

Try to detect her phone manually using hcitool name [mac address]. If nothing is returned from this command when her phone screen is off, then neither presence nor monitor will be able to detect it.

tc23 · September 23, 2018, 3:46pm

Sorry, it’s an iPhone 6s. I do think she needs to update her software. I saw a notification in there when I was looking for the Bluetooth Mac. Ok great, I will try and scan for it

andrewjfreyer · September 23, 2018, 3:47pm

I have been using -td on all my nodes with good results. However, I had to increase my trigger delay from 10 seconds to 20 seconds. This is because I also increased the arrival repetitions on my main node (3) and decreased the arrival repetitions (1) on my remote nodes.

I’m still experimenting to try and find the best settings, but this is working for now.

andrewjfreyer · September 23, 2018, 3:50pm

6s should work fine even without a software update from iOS 11.

I know the following may sound like a stupid question, but I’m asking because I ran into the same thing with a family member: are you sure she’s only turning off the screen not the entire phone?

tc23 · September 23, 2018, 4:04pm

Yeah, I was the one who was actually testing it. I turned Bluetooth off and then and on, and then received a 100 confidence. Then I turned the screen off and saw the messages coming in, dropping down to 50, 25,12, and then 0. When I turned the screen back on, the confidence did not increase back to 100

apt · September 23, 2018, 4:24pm

I’ve noticed since I’ve updated to latest build and added a couple of bluetooth beacons I am only receiving 1 MQTT message (for an iPhone) from Monitor in MQTT.fx. Looking at the output from sudo bash monitor.sh I’m seeing this early on:

Error: A network protocol error occurred when communicating with the broker.

I’ve reset to default settings (sudo bash monitor.sh -d) and updated (sudo bash monitor.sh -u) but the issue persists.

Any ideas?

Odianosen25 · September 23, 2018, 6:36pm

Well @andrewjfreyer, seems its no longer available as at the latest. At the same time it doesn’t seem to queue the messages.
is there something I am missing?

Odianosen25 · September 23, 2018, 6:43pm

I am sorry I am responding so late.

I use Appdaamon and my script is here.

Honestly I keep updating it as @andrewjfreyer makes modes to the system

But basically what I do is, I have my departure scans reduced from default 4 to 3. So when a device sends a confidence of 0, I have an extra timeout of 10 secs, within this time I send out an Arrive scan again.

The scan is completed within the 10 secs, and it checks again to confirm the device is not available before declaring the user is not home.

Regards

andrewjfreyer · September 24, 2018, 12:36am

Oh damnit. I’m sorry - stupid bug on my part. Updated. Make sure to set:

PREF_MQTT_REPORT_SCAN_MESSAGES=true

in the behavior preferences file.

Odianosen25 · September 24, 2018, 6:26am

Many thanks, its been fixed

daneboom · September 25, 2018, 3:50pm

I’m a trifle confused. In the write up, Andrew J Freyer says

“the first floor pi then scans for the fixed address of our cell phones. These addresses are are stored in the known_static_addresses file. If one of those devices is seen, an mqtt message is sent to Home Assistant reporting that the scanned phone is “home” with a confidence of 100%. In addition, an mqtt message is sent to the second and third floor to trigger a scan on those floors as well.”

What’s the point of scanning the other floors when a device arrives? I know there will be a point, I just don’t know what that point is!

He then says

“When we leave the house, we use either the front door or the garage door to trigger an mqtt trigger of [topic_path]/scan/depart […] to trigger a departure scan of our devices.”

I’m wondering if the only reason to do an MQTT departure scan trigger is precisely IF you have multiple Pis operating as a group and set to “-t” … ? If you only have one Pi, you wouldn’t need to do this, right?

andrewjfreyer · September 25, 2018, 4:35pm

Great question - the point is to increase the range and reliability of the entire system!

For example, in a 1-node setup that does not include a trigger for depart scanning (-td), if I spend more than a few moments on the third floor of my house, the first floor is likely to think that I’ve left because it can no longer scan my phone. That’s bad.

I can attempt to solve this problem in at least two ways: (1) only trigger depart scans when particular doors open or close (front door, garage), and (2) include more scanning nodes.

It may not be immediately obvious but solution (1) presents an undesirable edge case if my wife and I are both home. If my wife is on the third floor when I leave to walk the dog, the 1-node will begin a depart scan and it will find that both of us have left. I’m out of range (out of the house) and she’s out of range (third floor).

If we implement (2), we’ll have at least one node reporting that my wife is still home. By combining the output from our multiple nodes, (e.g., via max_min or mean; I’m now using max_min), we can obtain a non-zero confidence that my wife is in fact home despite that the first floor node doesn’t detect her when she’s on the third floor.

So, what’s the best of both worlds? Combine the strategies: use both (1) and (2). That’s what monitor and presence are both designed to do.

The reason I also send an arrival scan is so that all nodes can, hopefully, update their respective internal database with respect to which devices are home and which devices are not. This also prevents an edge case where one device is scanning for arrival right after another device has reported a departure. This might cause HA to stutter on presence detection, rapidly shifting from home-away-home-away.

If you only have one pi, the triggering of other nodes neither hurts nor helps. But, by using only one node - regardless of how your scans are triggered - you’re susceptible to the edge case described above.

andrewjfreyer · September 25, 2018, 7:48pm

Update: 0.1.661

Change Log:

Breaking Change: conform online/offline status reports with home assistant MQTT sensor
Only reports known beacons (not all public beacons observed) if the known_beacons file includes content
Option for single-topic reporting with -s flag. All presence updates are sent to the same topic based on your publisher identity and topic path.
Various bugfixes