It seems to me that they vary with environment, processing load of the target device, processor load of the scanning device, and just randomly. If we added some kalman filters and a large database, we might be able to infer a location within a house. That’s a bit out of scope for a bash script though.
1 Like
I did not experience interference at all, but have moved to monitor exclusively. The performance is at least the same as presence, although there’s some variability. Monitor is consistently faster to detect that I have left, but can sometimes lag by a few seconds when I arrive.
Monitor is more elegantly written and I’m more likely to actively maintain it. In a few days, I’ll be cutting certain features from presence in order to encourage more people to move to monitor and to clean up some bad code in monitor.
Can someone assist is running ‘monitor’? I’ve changed from presence and the service is running now but I get no MQTT messages other than ‘offline’ ones when I stop or restart the monitor.service.
I have copied across my MAC addresses into known_static_devices. I also occasionally see this error.
0.1.482 10:13:59 am [ERROR] Attempting to correct HCI error: Set scan parameters failed: Input/output error
If I run sudo bash monitor.sh -b, I do not see any of the house BT MAC addresses reported.
Everything was working great in presence except I was seeing lots of MQTT disconnects which prompted by migration to monitor but I can’t get it working.
EDIT: Just checked my log in the Hassio MQTT add-on and like presence, I’m seeing lots of connects and disconnects.
1534456638: New connection from 10.0.1.51 on port 1883.
1534456638: New client connected from 10.0.1.51 as mosqsub|11910-pi-presen (c1, k60, u'').
1534456642: New connection from 10.0.1.52 on port 1883.
1534456642: New client connected from 10.0.1.52 as mosqsub|9509-pi-presenc (c1, k60, u'').
1534456679: Socket error on client mosqsub|9509-pi-presenc, disconnecting.
1534456686: Socket error on client mosqsub|11910-pi-presen, disconnecting.
1534456721: New connection from 10.0.1.51 on port 1883.
1534456721: New client connected from 10.0.1.51 as mosqsub|534-pi-presence (c1, k60, u'').
1534456732: New connection from 10.0.1.52 on port 1883.
1534456732: New client connected from 10.0.1.52 as mosqsub|529-pi-presence (c1, k60, u'').
1534457329: Socket error on client mosqsub|534-pi-presence, disconnecting.
1534457391: New connection from 10.0.1.51 on port 1883.
1534457391: New client connected from 10.0.1.51 as mosqsub|9992-pi-presenc (c1, k60, u'').
1534457401: Socket error on client mosqsub|529-pi-presence, disconnecting.
1534457405: New connection from 10.0.1.52 on port 1883.
1534457405: New client connected from 10.0.1.52 as mosqsub|8913-pi-presenc (c1, k60, u'').
1534457528: Saving in-memory database to /data/mosquitto.db.
1534457639: New connection from 10.0.1.51 on port 1883.
1534457639: New client connected from 10.0.1.51 as mosqsub|13796-pi-presen (c1, k60, u'').
1534457950: Socket error on client mosqsub|13796-pi-presen, disconnecting.
1534457951: Socket error on client mosqsub|9992-pi-presenc, disconnecting.
1534457964: New connection from 10.0.1.51 on port 1883.
1534457964: New client connected from 10.0.1.51 as mosqsub|29479-pi-presen (c1, k60, u'').
1534457980: New connection from 10.0.1.51 on port 1883.
1534457980: New client connected from 10.0.1.51 as mosqsub|30082-pi-presen (c1, k60, u'').Make sure that no instance of presence is still running. You can kill all of them by running:
sudo pkill -f presence.sh
Also kill monitor so we know that nothing is using the bluetooth hardware:
sudo pkill -f monitor.sh
and/or
sudo systemctl stop monitor.service
It may also be helpful to try and clear your broker’s retained messages from monitor/presence. Then, we can run monitor again and see what happens.
Note that the nature of a mosquitto publication (at least from bash) is a one-off connection. It’s normal to see connection and disconnection from the broker for messages, at least.
Thanks. Ran both commands and sudo bash monitor.sh -C to clear messages. Change broker to a backup one to test, still nothing other than offline messages when turning my phones BT on and off.
location/owner/kitchen
{"status":"offline"}
pi@pi-presenceup:~/monitor $ sudo bash monitor.sh
Starting monitor.sh (v. 0.1.482)…
preference: delay between scans = 5
preference: periodic arrive/depart check interval = 90
preference: periodic arrive interval = 90
preference: periodic depart interval = 120
preference: database refresh interval = 35
preference: max arrival scan attempts = 2
preference: max depart scan attempts = 4
preference: random advertisement expiration = 120
stopping other instances of ‘monitor.sh’
0.1.482 11:05:30 am [CMD-RAND] 4C:56:9B:55:A3:77 ADV_NONCONN_IND RAND_NUM: 1
0.1.482 11:05:33 am [CMD-RAND] 54:CD:01:8F:04:B8 ADV_NONCONN_IND RAND_NUM: 2
0.1.482 11:05:36 am [CMD-RAND] 74:BD:39:7B:BE:5D ADV_NONCONN_IND RAND_NUM: 3
0.1.482 11:05:39 am [CMD-RAND] 54:E9:37:3B:24:18 ADV_NONCONN_IND RAND_NUM: 4
0.1.482 11:05:42 am [CMD-RAND] 67:10:EB:7D:C6:59 ADV_NONCONN_IND RAND_NUM: 5
0.1.482 11:05:45 am [CMD-RAND] 4F:F4:E9:48:16:63 ADV_NONCONN_IND RAND_NUM: 6
0.1.482 11:06:05 am [CMD-RAND] 54:CD:01:8F:04:B8 ADV_NONCONN_IND RAND_NUM: 6
0.1.482 11:06:08 am [CMD-RAND] 4C:56:9B:55:A3:77 ADV_NONCONN_IND RAND_NUM: 6
0.1.482 11:06:11 am [CMD-RAND] 74:BD:39:7B:BE:5D ADV_NONCONN_IND RAND_NUM: 6
0.1.482 11:06:14 am [CMD-RAND] 54:E9:37:3B:24:18 ADV_NONCONN_IND RAND_NUM: 6
0.1.482 11:06:17 am [CMD-RAND] 67:10:EB:7D:C6:59 ADV_NONCONN_IND RAND_NUM: 6
0.1.482 11:06:20 am [CMD-RAND] 4F:F4:E9:48:16:63 ADV_NONCONN_IND RAND_NUM: 6
This is how I have my known_static_devices laid out.
08:21:EF:D4:xx:xx #David A5 2016
70:70:0D:63:xx:xx #Ali's iPhone
24:00:BA:AF:xx:xx #Huawei Watch
E4:B3:18:7B:xx:xx #X260 Laptop
f4:f5:db:e2:xx:xx #Pete's Mi A1
C4:7C:8D:60:xx:xx #Plant Sensor
Seems to be working now!
Error in docs? Known MAC addresses need to be in known_static_addresses not known_static_devices?
Good catch! I’ll be sure to get that fixed today
I’ve been running the presence script for a couple of days and seems to function pretty reliably with my android phone and mi band, I decided to catch up with the development so read this entire thread and realised you’ll likely be switching your efforts over to the monitor script, I’ve therefore upgraded but am having some issues…
I can only get monitor.sh to recognise the mi band when I run it with the flags -b -P and even then it will drop the confidence down to 0 after a couple of minutes, there are also issues with the android phone, if I disable Bluetooth it will drop to 0 and when I re-enable it it doesn’t come back up.
What flags do I need to run monitor.sh with in order to get the same behaviour as running presence.sh with beacon scanning enabled?
Could the issue with the mi band be related to it not having a name? It’s in known_static_addresses but only shows up with - P and its line in the log starts [CMD-PUBL].
I ordered some BTLE devices for testing and ran into a similar problem with PUBL/RAND reporting. I’m working on it for monitor!
Regarding the android phone issue, I think it is working correctly I was just used to the pretty much instant reporting of the presence script, monitor just seems to take a little longer.
Thank you for all the work you’re doing on this by the way, I’ve been wanting to get bluetooth presence set up for a while but none of the current implementations were suitable for one reason or another.
I noticed today that one of my Pi Zero’s wasn’t responding. I checked the logs and found the below.
Aug 18 16:57:07 pi0ute sudo[30975]: pam_unix(sudo:session): session closed for user root
Aug 18 16:57:12 pi0ute sudo[30992]: root : TTY=unknown ; PWD=/home/pi/monitor ; USER=root ; COMMAND=/usr/bin/timeout --signal SIGINT 30 hcitool lescan
Aug 18 16:57:12 pi0ute sudo[30992]: pam_unix(sudo:session): session opened for user root by (uid=0)
Aug 18 16:57:22 pi0ute sudo[30992]: pam_unix(sudo:session): session closed for user root
Aug 18 16:57:27 pi0ute sudo[31009]: root : TTY=unknown ; PWD=/home/pi/monitor ; USER=root ; COMMAND=/usr/bin/timeout --signal SIGINT 30 hcitool lescan
Aug 18 16:57:27 pi0ute sudo[31009]: pam_unix(sudo:session): session opened for user root by (uid=0)
Aug 18 16:57:37 pi0ute sudo[31009]: pam_unix(sudo:session): session closed for user root
Aug 18 16:57:42 pi0ute sudo[31024]: root : TTY=unknown ; PWD=/home/pi/monitor ; USER=root ; COMMAND=/usr/bin/timeout --signal SIGINT 30 hcitool lescan
Aug 18 16:57:42 pi0ute sudo[31024]: pam_unix(sudo:session): session opened for user root by (uid=0)
Aug 18 16:57:52 pi0ute sudo[31024]: pam_unix(sudo:session): session closed for user root
Aug 18 16:57:58 pi0ute sudo[31072]: root : TTY=unknown ; PWD=/home/pi/monitor ; USER=root ; COMMAND=/usr/bin/timeout --signal SIGINT 30 hcitool lescan
This Zero had an uptime of 21 days, so I restarted and then checked the logs again:
Aug 18 17:00:58 pi0ute systemd[1]: Started Monitor Service.
Aug 18 17:00:58 pi0ute bash[203]: Starting monitor.sh (v. 0.1.482)...
Aug 18 17:01:00 pi0ute bash[203]: > preference: delay between scans = 3
Aug 18 17:01:00 pi0ute bash[203]: > preference: periodic arrive/depart check interval = 90
Aug 18 17:01:00 pi0ute bash[203]: > preference: periodic arrive interval = 90
Aug 18 17:01:00 pi0ute bash[203]: > preference: periodic depart interval = 120
Aug 18 17:01:00 pi0ute bash[203]: > preference: database refresh interval = 120
Aug 18 17:01:00 pi0ute bash[203]: > preference: max arrival scan attempts = 2
Aug 18 17:01:00 pi0ute bash[203]: > preference: max depart scan attempts = 4
Aug 18 17:01:00 pi0ute bash[203]: > preference: random advertisement expiration = 35
Aug 18 17:01:00 pi0ute bash[203]: > stopping other instances of 'monitor.sh'
Aug 18 17:01:01 pi0ute sudo[280]: root : TTY=unknown ; PWD=/home/pi/monitor ; USER=root ; COMMAND=/bin/systemctl stop presence
Aug 18 17:01:01 pi0ute sudo[280]: pam_unix(sudo:session): session opened for user root by (uid=0)
Aug 18 17:01:02 pi0ute sudo[280]: pam_unix(sudo:session): session closed for user root
Aug 18 17:01:02 pi0ute sudo[339]: root : TTY=unknown ; PWD=/home/pi/monitor ; USER=root ; COMMAND=/bin/hciconfig hci0 down
Aug 18 17:01:02 pi0ute sudo[339]: pam_unix(sudo:session): session opened for user root by (uid=0)
Aug 18 17:01:03 pi0ute bash[203]: Can't get device info: No such device
Aug 18 17:01:03 pi0ute sudo[339]: pam_unix(sudo:session): session closed for user root
Aug 18 17:01:03 pi0ute sudo[356]: root : TTY=unknown ; PWD=/home/pi/monitor ; USER=root ; COMMAND=/bin/rm main_pipe
Aug 18 17:01:03 pi0ute sudo[356]: pam_unix(sudo:session): session opened for user root by (uid=0)
Aug 18 17:01:03 pi0ute sudo[356]: pam_unix(sudo:session): session closed for user root
Aug 18 17:01:03 pi0ute sudo[371]: root : TTY=unknown ; PWD=/home/pi/monitor ; USER=root ; COMMAND=/bin/rm log_pipe
Aug 18 17:01:03 pi0ute sudo[371]: pam_unix(sudo:session): session opened for user root by (uid=0)
Aug 18 17:01:03 pi0ute sudo[371]: pam_unix(sudo:session): session closed for user root
Aug 18 17:01:04 pi0ute sudo[420]: root : TTY=unknown ; PWD=/home/pi/monitor ; USER=root ; COMMAND=/usr/bin/hcidump --raw
Aug 18 17:01:04 pi0ute sudo[417]: root : TTY=unknown ; PWD=/home/pi/monitor ; USER=root ; COMMAND=/usr/bin/timeout --signal SIGINT 30 hcitool lescan
Aug 18 17:01:04 pi0ute bash[203]: Error: Network is unreachable
Aug 18 17:01:04 pi0ute sudo[417]: pam_unix(sudo:session): session opened for user root by (uid=0)
Aug 18 17:01:04 pi0ute sudo[420]: pam_unix(sudo:session): session opened for user root by (uid=0)
Aug 18 17:01:04 pi0ute sudo[420]: pam_unix(sudo:session): session closed for user root
Aug 18 17:01:04 pi0ute sudo[417]: pam_unix(sudo:session): session closed for user root
Aug 18 17:01:09 pi0ute sudo[483]: root : TTY=unknown ; PWD=/home/pi/monitor ; USER=root ; COMMAND=/usr/bin/timeout --signal SIGINT 30 hcitool lescan
Aug 18 17:01:09 pi0ute sudo[483]: pam_unix(sudo:session): session opened for user root by (uid=0)
Aug 18 17:01:48 pi0ute sudo[483]: pam_unix(sudo:session): session closed for user root
Aug 18 17:01:54 pi0ute sudo[570]: root : TTY=unknown ; PWD=/home/pi/monitor ; USER=root ; COMMAND=/usr/bin/timeout --signal SIGINT 30 hcitool lescan
Aug 18 17:01:54 pi0ute sudo[570]: pam_unix(sudo:session): session opened for user root by (uid=0)
Aug 18 17:02:24 pi0ute sudo[570]: pam_unix(sudo:session): session closed for user root
Aug 18 17:02:29 pi0ute sudo[582]: root : TTY=unknown ; PWD=/home/pi/monitor ; USER=root ; COMMAND=/usr/bin/timeout --signal SIGINT 30 hcitool lescan
Aug 18 17:02:29 pi0ute sudo[582]: pam_unix(sudo:session): session opened for user root by (uid=0)
Aug 18 17:02:59 pi0ute sudo[582]: pam_unix(sudo:session): session closed for user root
I’m not too sure where to go from here. I’d be grateful for any suggestions?
EDIT - I deleted the script and pasted in a version from another Zero and now all is ok. Must’ve got corrupted somehow. This left here just in case it helps someone else.
1 Like
Hello @andrewjfreyer,
Well its me again. Thanks for the response. Ok now I have a reverse question. Is it possible to like reduce the sensitivity of the Bluetooth device?
Why I am asking is that instead of trying to figure out who is in the house, how about as a room sensor? And really not many of use have big houses, mine pretty tinny 
. So even one is an over kill
Apologies if this question has been asked before, but men 500+ responses, such a drag 
Thanks in anticipation 
1 Like
Less sensitive as in covering a smaller area? If you decrease repetitions and time intervals, you’ll effectively reduce the pi’s range.
I think he wants it for room detection. Which maybe you can achieve with combining motion sensor data (if you got motion sensors) from each room to make it semi accurate. So if John is near bluetooth tracker that’s closest to the bedroom (maybe signal strength?), combine that with the fact that motion sensor in bedroom got triggered 2 min ago, its high probability that John is in fact in the bedroom.
What kind of hypervisor do you use? KVM? If so, can you share what you used to share it with the VM?
I’m currently using VirtualBox, and the connection of peripherals to the VM is done through the USB option. I simply select the bluetooth device (identified by the vendor id and the device id)
and the VM OS recognized it as a bluetooth adapter:
I haven’t tried KVM, have you been able to compare it with Virtualbox or other hypervisors? If so, do you see any advantage?
Last time I used virtualbox, I did not find a reliable way to make it run in the background. My linux servers run without an X server and it is a must have that I can do all operations from the command line.
@Edzilla
I used the Virtualbox Headless command and later phpvirtualbox to have a GUI. Unfortunately phpvirtualbox is a little unmaintained and has to be patched manually to run with latest Vbox versions.
Some useful links: https://wiki.debian.org/VirtualBox
https://sourceforge.net/p/phpvirtualbox/wiki/Home/
I’ll open another thread with additional info if you need some more information, sorry everyone for the offtopic.
Meanwhile I have installed a second instance of presence script in an spare raspberry pi zero w I had lying around.
It’s working perfect, and now I can use any of the sensors to let HA determine if I’m home.
I’m also thinking of adding some sort of cpu and ram usage/uptime information over MQTT. Did anyone try this before?
1 Like
No, but it’s easy to add in to the mqtt file. Let me know what you come up with!