[presence] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

Thank you for the quick reply.
I’ll reply to your answers:
1-The huawei is a P9, and I have checked that the bluetooth radio is turned off.
2-I have setup an automation triggering the scan/depart when the main door is closed, the arrival/depart detection has improved a lot. Thank you!
3-I did experience lots of interference with presence.sh, so I’ll not decrease the suggested values.
4-No idea how to check this, but if you need further testing feel free to give me detailed instructions.
5-I tested both ways (faraday cage and leaving the house). After one day I saw some strange behaviours:
Last night the scripts detected several disconnection. I suppose it is a result of some kind of deep sleep in the phone. The device tracker that checks devices connected to the router worked ok:


(The two top lines are existing device trackers, the orange lines is the calculated location based on device trackers -with transitions to just left and just arrived states-. 4th and 5th lines show the monitor.sh sensors.
This afternoon after leaving home, one of the two instances of monitor.sh hasn’t marked me away until an hour later even having triggered the scan/depart. I’ll check the scan/depart automation to avoid repetitive scans if I open the door just after leaving.

Anyway, I’ll keep using your great script. Thank you for your efforts.

1 Like

So last night I tried cloning the SD card. I wouldn’t say it was a great success. When I booted from the cloned card in a new Pi Zero, Monitor wasn’t working and was grumbling about its preferences file being corrupt. So I uninstalled it and reinstalled it and that seems to have cured the issue. Not sure what went wrong.

When next that happens, try running ‘sudo bash monitor.sh -d’ so it creates the files again and try running.

Regards

@andrewjfreyer

Im trying to get back on monitor and had a couple questions. I basically modified the way you currently scan with something that makes more sense in my situation and I am hoping to replicate it but adding my GF’s BT LE wristband for when she works out. Bascially if she leaves her phone and leaves the house with only her wristband she needs to use the key to get back in the house.

Anyways this is what I currently have which is essentially in a script written in powershell (the only language I know well)
Yes the script is a little wasteful but ive noticed that a name scan only takes 1 second per device. So if im home and my GF isnt it will only scan for her until the door opens again then it will scan for both of us for at least 2 minutes.

  • will Monitor -t -b allow me to essentially do what i’m currently doing (obviously without the ping)?
  • If using Monitor -t and 1 of 2 devices are found will it continue doing name scans on the other device until that one is found?

1 Like

Can anyone confirm what the correct topics are for this new version of monitor? I’ve tried taking out "owner’ but topic seems to have changed from location/ to monitor/ ?

1 Like

Topic should be defined in your “mqtt_preferences” file e.g. “mqtt_topicpath=location” identity is defined here too e.g. “mqtt_publisher_identity=‘upstairs’” so those become the MQTT topic of location/upstairs/00:00:00:00:00:00.

This has stayed consistent for me after updates, I did previously need to remove /owner/ from the topic.

Hope this helps.

1 Like

The default topic was changed a few versions ago, but this topic path can be configured in your MQTT settings. git pull will not overwrite your settings.

Monitor will respond to posts to those MQTT topics by default - no need to use the trigger mode. Trigger mode is reserved when you want a node to only respond to triggers.

Else, your flowchart is very similar to the flow of Monitor with default settings.

@all - most everyone should transition to monitor.sh. I’ll start a new topic on the HA forums since 700 messages is getting ridiculous.

presence has been updated to 0.5.1 to remove nearly all features apart from standard, basic, device scanning. All other feature requests and/or options should be made with respect to/have been included in monitor

Thanks!

6 Likes

Hi @andrewjfreyer, I have a NUT device and run with monitor version 0.1.621, I filled the nut mac to the known_static_addresses, in this case is CF:51:A4:D8:A7:59, in the log I got this:

pi@devzero:~/monitor $ sudo bash monitor.sh -g
Starting monitor.sh (v. 0.1.621)...
> generic bluetooth beacon reporting mode enabled
Warning: monitor.service not installed. Install service? (y/n)
n
Warning: Variable mqtt_publisher_identity does not appear in: mqtt_preferences. Using hostname: devzero.
> preference: delay between scans = 3
> preference: periodic arrive/depart check interval = 15
> preference: periodic arrive interval = 45
> preference: periodic depart interval = 90 
> preference: database refresh interval = 35
> preference: max arrival scan attempts = 2
> preference: max depart scan attempts = 4
> preference: random advertisement expiration = 45
> preference: beacon rssi change required for reporting = 10 
> preference: bluetooth environmental report frequency = 300
> preference: forced departure check interval = 240
> preference: interval until beacon is considered expired = 145
> preference: trigger a departure scan at other nodes below [x] confidence = 25
> preference: preferred HCI device = hci0
0.1.621 05:14:42 pm [CMD-RAND]	CF:51:A4:D8:A7:59 ADV_IND -35 dBm
0.1.621 05:14:42 pm [CMD-INFO]	**** Started arrival scan. [x2 max rep] **** 
0.1.621 05:14:42 pm [CMD-SCAN]	(No. 1) CF:51:A4:D8:A7:59 arrival? 
0.1.621 05:14:51 pm [CMD-SCAN]	(No. 2) CF:51:A4:D8:A7:59 arrival? 
0.1.621 05:14:59 pm [CMD-INFO]	**** Completed arrival scan. **** 
0.1.621 05:15:04 pm ZeroW/devzero/CF:51:A4:D8:A7:59 
{
	retain: false
	version : 0.1.621
	confidence : 0
	name : CF:51:A4:D8:A7:59
	timestamp : Tue Sep 11 2018 17:15:03 GMT+0800 (CST)
	manufacturer : Unknown
	type : KNOWN_MAC
}
0.1.621 05:15:04 pm [CMD-NAME]	CF:51:A4:D8:A7:59 CF:51:A4:D8:A7:59  Unknown
0.1.621 05:15:44 pm [CMD-RSSI]	CF:51:A4:D8:A7:59 CF:51:A4:D8:A7:59 RAND RSSI: -52 dBm (Moderate Motion Departing) 
0.1.621 05:15:47 pm [CMD-RSSI]	CF:51:A4:D8:A7:59 CF:51:A4:D8:A7:59 RAND RSSI: -70 dBm (Moderate Motion Departing) 
0.1.621 05:16:19 pm [CMD-RSSI]	CF:51:A4:D8:A7:59 CF:51:A4:D8:A7:59 RAND RSSI: -28 dBm (Fast Motion Approaching) 
0.1.621 05:16:51 pm [CMD-RSSI]	CF:51:A4:D8:A7:59 CF:51:A4:D8:A7:59 RAND RSSI: -69 dBm (Fast Motion Departing) 
0.1.621 05:17:28 pm [CMD-RAND]	6C:0B:75:88:CB:BF ADV_IND -77 dBm
0.1.621 05:17:28 pm [CMD-INFO]	**** Started arrival scan. [x2 max rep] **** 
0.1.621 05:17:28 pm [CMD-SCAN]	(No. 1) CF:51:A4:D8:A7:59 arrival? 
0.1.621 05:17:36 pm [CMD-SCAN]	(No. 2) CF:51:A4:D8:A7:59 arrival? 
0.1.621 05:17:45 pm [CMD-INFO]	**** Completed arrival scan. **** 
0.1.621 05:17:50 pm [CMD-NAME]	CF:51:A4:D8:A7:59 CF:51:A4:D8:A7:59  Unknown
0.1.621 05:17:55 pm [CMD-RSSI]	CF:51:A4:D8:A7:59 CF:51:A4:D8:A7:59 RAND RSSI: -55 dBm (Moderate Motion Approaching) 
0.1.621 05:18:25 pm [CMD-RSSI]	CF:51:A4:D8:A7:59 CF:51:A4:D8:A7:59 RAND RSSI: -71 dBm (Moderate Motion Departing) 

Looks like the monitor scans a nut mac and report the RSSI payload, but it always reports confidence 0, why is that happen? Could you please guide me through this? Thanks

1 Like

Other users have also reported issues with NUT beacons. I’m working on a fix.

They should be excluded from your known devices list because, as you can see from the posted log, the MAC address is being advertised by the NUT as a random address. The address will change over time.

… unless of course NUT decided to break the BTLE spec by advertising a static mac address as a random address. Can you report back if that address stays at CF:51:A4:D8:A7:59 through power cycles and over time?

@kevinshane, well… after a little research, it appears that NUT may well be breaking the BTLE spec, reporting its mac address as random when it is actually status. At least according to this github project for domoticz, the mac address seems static. No wonder this has been such a pain!

I’ve updated to 0.1.622 to include a known beacons file. Place your beacon’s address in this file, along with a name, and please report back. This should your beacon as a GENERIC beacon (use the -g flag), despite that it reports itself as a random beacon.

Hi Andrew,

I have updated to the newest version yesterday - I have four devices in the config, the iphones seem to work 100%, but the Apple Watches (Series 3) statys with a confidence on 0. I figured it might be an issue that the watch have and active pair with the phone, so I switched off BT on the phone to break the connection, and open the bluetooth scan on the watch - All of which had no effect on the confidence of the watch being picked up. Not a train smash for me, but just thought I report it.

Also I have TILE devices around, which are not being picked up - I have the -b flag on, also tried it with the -b & -g flags on.

Thank you for all your effort with this.

Will these fixes help with the Tile issue? I’m going to try regardless ill let you know if I have any luck.

EDIT: Worked a charm!! Thank you @andrewjfreyer!

1 Like

I made some changes early this morning, and pushed them out this afternoon. Update again and let me know if your Tile is still unresponsive.

1 Like

I have also noticed that my Apple Watch (series 0) inconsistently responds to name requests. I’m working on it!

EDIT – Also, it seems that the transmit strength the Apple Watch is intentionally restricted. It may not be a good indicator of presence simply due to limited range. Anecdotally, I think this may be the reason that the Apple Watch takes a second or so to ping for it’s iPhone… It needs to turn up the power first.

Great to hear! Thanks for the report.

Hi, Thankz - I updated again, had to scan on the comman line to find the TILE MAC addresses - monitor wouldn’t start up with an empty known_beacons file. I added them to the file, and they show up like this:

[CMD-RSSI] D5:XX:XX:XX:XX:6A Undiscoverable Device Name RAND RSSI: -75 dBm (Moderate Motion Departing) (x2)

But I do not receive any MQTT messages for them.

Also, I added my AppleTV to the blacklist, but I still see it reported in the logs like this:

[CMD-RSSI] 08:XX:XX:XX:XX:7E Undiscoverable Device Name PUBL RSSI: -48 dBm (Moderate Motion Approaching) (x2)

Thank you again.

Two errors on my part. I’ll get these fixed. Thanks for the report!

Should be fixed now!

Any advice on what device to buy to attach to my wife’s keychain so that I can use her presence in my home automation setup? She doesn’t have an iPhone so I’d like to be able to buy a beacon which can be tracked by monitor just like my iPhone - ie passively looking out for “advertisements” rather than actively probing for a response. I’ve been bowled away by the range of the BT radio in my iPhone…any chance of something approaching that in a cheap keychain beacon? Thanks!

@keithh666 @J_IO_B @Odianosen25 any of you experimented with adding conditions so that the bluetooth trigger for an absent beacon DOESN’T fire if it helps say between 1am and 6am? I figure one might avoid false positives caused by flat batteries in the middle of the night, that way. Though I appreciate most folks have their devices on charge at night…