[presence] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

Is the beacon flag enabled? If so, you’ll also see the iBeacon advertisements.

The above is what I see in monitor with the -b option, also this…

0.1.481 10:59:36 pm onds PUBL_NUM: 3
0.1.481 10:59:40 pm [CMD-RAND] xx:xx:xx:xx:6D:28 ADV_IND RAND_NUM: 3
0.1.481 10:59:40 pm onds PUBL_NUM: 3
0.1.481 10:59:43 pm [CMD-RAND] xx:xx:xx:xx:6D:28 ADV_IND RAND_NUM: 3
0.1.481 10:59:43 pm onds PUBL_NUM: 3
0.1.481 10:59:46 pm [CMD-RAND] xx:xx:xx:xx:6D:28 ADV_IND RAND_NUM: 3
0.1.481 10:59:46 pm onds PUBL_NUM: 3
0.1.481 10:59:50 pm [CMD-RAND] xx:xx:xx:xx:6D:28 ADV_IND RAND_NUM: 3
0.1.481 10:59:50 pm onds PUBL_NUM: 3
0.1.481 10:59:53 pm [CMD-RAND] xx:xx:xx:xx:6D:28 ADV_IND RAND_NUM: 3
0.1.481 10:59:53 pm onds PUBL_NUM: 3
0.1.481 10:59:56 pm [CMD-RAND] xx:xx:xx:xx:6D:28 ADV_IND RAND_NUM: 3
0.1.481 10:59:56 pm onds PUBL_NUM: 3
0.1.481 10:59:59 pm [CMD-RAND] xx:xx:xx:xx:6D:28 ADV_IND RAND_NUM: 3
0.1.481 11:00:00 pm onds PUBL_NUM: 3

This likely means that the Nut BTLE tag is not conforming to the iBeacon specification. I’ll see if I can add support for devices like this.

1 Like

Andrew, i’m running precense on one PI Zero right now, with plans on using 6 to cover house plus garage. But with just one running, my wife complained that her BT keyboard was acting strange and those problems dissapeared when I shutdown the Pi.

To make things worse, her office is just next to the entry door to the house, so cant avoid covering that area for precense detection.

How would you solve that issue if you had the same prob?

1 Like

Hello @andrewjfreyer,

New fan of your script thanks to @PianSom.

Please a quick question:

Is it possible for one to estimate the distance of the Bluetooth presence system, from the Bluetooth device based on the confidence?

If it is, could you kindly share some data on your finding? I know it depends on several things like walls in between and type of Bluetooth device, but I just need any little data you might have worked out already if you had considered it before.

Thanks and regards

Hey @Odianosen25 - I’m sure you don’'t want to read what is nearly 500 posts now, but Andrew did mention up here that he has not had very much luck at all with this.

Hey @andrewjfreyer (and any Appdaemon users) - if you haven’t been watching it, you might like to have a look over in the Appdaemon section here at some wonderful coding that @Odianosen25 has just put together for monitor

1 Like

Oh thanks bro, yeah you right, wasn’t really in the mode of reading via the entire threads :smile:.

I was looking at doing some triangulation, but even at that, i think it I will still look at it later to I could at least allow for possible location of which floor a device is.

Like interfacing it with Snips and asking “where is my phone?”. It could give a possible location based on the different readings, even if its not so accurate.

Well a project for another time.

Many thanks again @PianSom, much appreciated :facepunch:t5:

1 Like

I’d make the switch to the monitor script. With low interval values, the presence script has the tendency to interfere with 2.4 GHz devices, including Bluetooth and Wi-Fi.

@PianSom is totally right. The variability in Bluetooth response times and RSSI calculations is far too large for any meaningfully accurate estimations of distance. The best I have been able to do is a plus or minus of 30 to 50 feet. Triangulation with maximum error of 100 feet is really not that useful, haha

I’m sure someone smarter than me is trying to figure out an answer to this question right now

I’m currently on presence without any interference, would you recommend upgrading to monitor? If so, are the install instructions the same as presence? Thanks so much for the amazing work!

One thought occurs to me - are the response times/RSSI at least consistent over time, in your experience? Or do they vary with, for example, atmospheric conditions?

I don’t know about you, but when at home I am usually in one of relatively few locations - in bed, asleep; slumped in front of the TV; in my study; in the bathroom, etc. So I wonder if it might be possible to infer a room location from a database of times/RSSI?

It seems to me that they vary with environment, processing load of the target device, processor load of the scanning device, and just randomly. If we added some kalman filters and a large database, we might be able to infer a location within a house. That’s a bit out of scope for a bash script though.

1 Like

I did not experience interference at all, but have moved to monitor exclusively. The performance is at least the same as presence, although there’s some variability. Monitor is consistently faster to detect that I have left, but can sometimes lag by a few seconds when I arrive.

Monitor is more elegantly written and I’m more likely to actively maintain it. In a few days, I’ll be cutting certain features from presence in order to encourage more people to move to monitor and to clean up some bad code in monitor.

Can someone assist is running ‘monitor’? I’ve changed from presence and the service is running now but I get no MQTT messages other than ‘offline’ ones when I stop or restart the monitor.service.

I have copied across my MAC addresses into known_static_devices. I also occasionally see this error.

0.1.482 10:13:59 am [ERROR] Attempting to correct HCI error: Set scan parameters failed: Input/output error

If I run sudo bash monitor.sh -b, I do not see any of the house BT MAC addresses reported.

Everything was working great in presence except I was seeing lots of MQTT disconnects which prompted by migration to monitor but I can’t get it working.

EDIT: Just checked my log in the Hassio MQTT add-on and like presence, I’m seeing lots of connects and disconnects.

1534456638: New connection from 10.0.1.51 on port 1883.
1534456638: New client connected from 10.0.1.51 as mosqsub|11910-pi-presen (c1, k60, u'').
1534456642: New connection from 10.0.1.52 on port 1883.
1534456642: New client connected from 10.0.1.52 as mosqsub|9509-pi-presenc (c1, k60, u'').
1534456679: Socket error on client mosqsub|9509-pi-presenc, disconnecting.
1534456686: Socket error on client mosqsub|11910-pi-presen, disconnecting.
1534456721: New connection from 10.0.1.51 on port 1883.
1534456721: New client connected from 10.0.1.51 as mosqsub|534-pi-presence (c1, k60, u'').
1534456732: New connection from 10.0.1.52 on port 1883.
1534456732: New client connected from 10.0.1.52 as mosqsub|529-pi-presence (c1, k60, u'').
1534457329: Socket error on client mosqsub|534-pi-presence, disconnecting.
1534457391: New connection from 10.0.1.51 on port 1883.
1534457391: New client connected from 10.0.1.51 as mosqsub|9992-pi-presenc (c1, k60, u'').
1534457401: Socket error on client mosqsub|529-pi-presence, disconnecting.
1534457405: New connection from 10.0.1.52 on port 1883.
1534457405: New client connected from 10.0.1.52 as mosqsub|8913-pi-presenc (c1, k60, u'').
1534457528: Saving in-memory database to /data/mosquitto.db.
1534457639: New connection from 10.0.1.51 on port 1883.
1534457639: New client connected from 10.0.1.51 as mosqsub|13796-pi-presen (c1, k60, u'').
1534457950: Socket error on client mosqsub|13796-pi-presen, disconnecting.
1534457951: Socket error on client mosqsub|9992-pi-presenc, disconnecting.
1534457964: New connection from 10.0.1.51 on port 1883.
1534457964: New client connected from 10.0.1.51 as mosqsub|29479-pi-presen (c1, k60, u'').
1534457980: New connection from 10.0.1.51 on port 1883.
1534457980: New client connected from 10.0.1.51 as mosqsub|30082-pi-presen (c1, k60, u'').

Make sure that no instance of presence is still running. You can kill all of them by running:

sudo pkill -f presence.sh

Also kill monitor so we know that nothing is using the bluetooth hardware:

sudo pkill -f monitor.sh

and/or

sudo systemctl stop monitor.service

It may also be helpful to try and clear your broker’s retained messages from monitor/presence. Then, we can run monitor again and see what happens.

Note that the nature of a mosquitto publication (at least from bash) is a one-off connection. It’s normal to see connection and disconnection from the broker for messages, at least.

Thanks. Ran both commands and sudo bash monitor.sh -C to clear messages. Change broker to a backup one to test, still nothing other than offline messages when turning my phones BT on and off.

location/owner/kitchen
{"status":"offline"}

pi@pi-presenceup:~/monitor $ sudo bash monitor.sh
Starting monitor.sh (v. 0.1.482)…

preference: delay between scans = 5
preference: periodic arrive/depart check interval = 90
preference: periodic arrive interval = 90
preference: periodic depart interval = 120
preference: database refresh interval = 35
preference: max arrival scan attempts = 2
preference: max depart scan attempts = 4
preference: random advertisement expiration = 120
stopping other instances of ‘monitor.sh’
0.1.482 11:05:30 am [CMD-RAND] 4C:56:9B:55:A3:77 ADV_NONCONN_IND RAND_NUM: 1
0.1.482 11:05:33 am [CMD-RAND] 54:CD:01:8F:04:B8 ADV_NONCONN_IND RAND_NUM: 2
0.1.482 11:05:36 am [CMD-RAND] 74:BD:39:7B:BE:5D ADV_NONCONN_IND RAND_NUM: 3
0.1.482 11:05:39 am [CMD-RAND] 54:E9:37:3B:24:18 ADV_NONCONN_IND RAND_NUM: 4
0.1.482 11:05:42 am [CMD-RAND] 67:10:EB:7D:C6:59 ADV_NONCONN_IND RAND_NUM: 5
0.1.482 11:05:45 am [CMD-RAND] 4F:F4:E9:48:16:63 ADV_NONCONN_IND RAND_NUM: 6
0.1.482 11:06:05 am [CMD-RAND] 54:CD:01:8F:04:B8 ADV_NONCONN_IND RAND_NUM: 6
0.1.482 11:06:08 am [CMD-RAND] 4C:56:9B:55:A3:77 ADV_NONCONN_IND RAND_NUM: 6
0.1.482 11:06:11 am [CMD-RAND] 74:BD:39:7B:BE:5D ADV_NONCONN_IND RAND_NUM: 6
0.1.482 11:06:14 am [CMD-RAND] 54:E9:37:3B:24:18 ADV_NONCONN_IND RAND_NUM: 6
0.1.482 11:06:17 am [CMD-RAND] 67:10:EB:7D:C6:59 ADV_NONCONN_IND RAND_NUM: 6
0.1.482 11:06:20 am [CMD-RAND] 4F:F4:E9:48:16:63 ADV_NONCONN_IND RAND_NUM: 6

This is how I have my known_static_devices laid out.

08:21:EF:D4:xx:xx #David A5 2016
70:70:0D:63:xx:xx #Ali's iPhone
24:00:BA:AF:xx:xx #Huawei Watch
E4:B3:18:7B:xx:xx #X260 Laptop
f4:f5:db:e2:xx:xx #Pete's Mi A1
C4:7C:8D:60:xx:xx #Plant Sensor

Seems to be working now!

Error in docs? Known MAC addresses need to be in known_static_addresses not known_static_devices?

Good catch! I’ll be sure to get that fixed today

I’ve been running the presence script for a couple of days and seems to function pretty reliably with my android phone and mi band, I decided to catch up with the development so read this entire thread and realised you’ll likely be switching your efforts over to the monitor script, I’ve therefore upgraded but am having some issues…

I can only get monitor.sh to recognise the mi band when I run it with the flags -b -P and even then it will drop the confidence down to 0 after a couple of minutes, there are also issues with the android phone, if I disable Bluetooth it will drop to 0 and when I re-enable it it doesn’t come back up.

What flags do I need to run monitor.sh with in order to get the same behaviour as running presence.sh with beacon scanning enabled?

Could the issue with the mi band be related to it not having a name? It’s in known_static_addresses but only shows up with - P and its line in the log starts [CMD-PUBL].

I ordered some BTLE devices for testing and ran into a similar problem with PUBL/RAND reporting. I’m working on it for monitor!