@PianSom I committed a few times today. x.66 is the most recent version.
@andrewjfreyer
I have updated to .66 - all seems fine, though the alpha distance function is a bit random. This, of course, may be in part due to the fact that I am running your script on a Pi that is doing other stuff, hence the time taken for a Bluetooth ping is not necessarily indicative of how long the response itself takes. With my phone stationary on the same desk as the Pi I am seeing everything from “very close” to “far”; I am seeing the same for my wife’s phone which is on the other side of the house right now That said, there is probably some rough correlation observable.
I am still seeing the behaviour I reported above - going to flight mode, watching the confidence decrease then turning off flight mode gives no device name for the first 100 response.
Two other comments:
- I am less convinced than you of the use of the “confidence” wind-down. I wonder if it would be cleaner just to post seen/not seen and let the home automation software interpret the result?
- Is it worth considering reducing the network strain by looping less frequently? Perhaps introducing a
sleeptime which depends on if anything has changed recently?
@piansom the confidence wind-down is tailored for the averaging sensor I describe above. If we switch from 100 to 0 the moment a device is not detected, we introduce dramatic high-frequency changes into all other logic down the pipeline. I added this after about two to three months of using the system with just 100 or 0 and the “wind-down” reduced false positives of “leaving.” That said, your mileage may vary. The number of false positives will depend on your setup, the location of your device, etc.
As far as the ‘sleep’ comment, you can control all of the various delays in the behavior_preferences file. If you would like to increase delays when ‘nothing has changed’ note that you will necessarily increase detecting a change in the future. The values I’ve chosen as ‘default’ work best for my use cases, but may not be appropriate for all people. Feel free to change the delays as you see fit.
Lastly, yea, I agree that the alpha feature of detecting distance isn’t working very accurately. There is a tradeoff here; to get true RSSI we will likely have to connect to each device, which will slow us down and will drain end-user device battery. I am likely to remove distance estimation, as it seems this is out of scope for the project.
EDIT - @PianSom, also I have updated to .67, which should address the blank name bug that you identified. I have also switched from distance abstraction to a straight ms scan duration time. If you want to use this as a proxy for actual distance, that’s just fine. I think the abstraction in the alpha feature was a bit too user and environment specific to warrant inclusion in the master branch.
I will update in a short while. Is there a recommended git command for pulling the latest version while leaving the configuration files untouched?
I agree with you on the RSSI likely being too costly. And I would happily turn off the scan duration feature - personally I don’t think it will add enough reliable info in my situation.
Thanks for pointing out the behavior_preferences features - which I had completely overlooked, sorry.
On the downstream logic - did you ever play with a HA Bayesian binary sensor? (I have, with rather mixed results.) My temptation, when I have full house coverage, is to use Appdaemon to write some logic which works for me (combining info from your code, my router wifi connections, and door activity). I will therefore probably ignore the confidence logic you have so carefully honed 
I have ordered another Pi Zero, which should arrive in the next few days. Although the supporting hardware I ordered from China will take a bit longer. My suspicion is that I will need 3 or 4 listeners to cover my house, but we will see…
Ha! This logic is, ostensibly, a percentage based on how many repetitions remain to be performed before a device is considered ‘away’. This is not particularly elegant or inspired, haha.
Yes, I have looked at the bayesian sensor, but I haven’t implemented it yet as a replacement for the simple max_min average. Likely a good idea to experiment with.
For the git command, you could stash, pull, and pop your changes back. Like this:
git stash
git pull
git stash pop
This would restore any files you modify.
Just updated. Hmmmm …
Well, I suppose it’s possible that the scan duration doesn’t change. Though what happened on that second line seems odd.
presence/owner/IanStudy/40:98:AD:xx:xx:xx {"confidence":"100","name":"BunnyHop","scan_duration_ms":"108704","timestamp":"Wed Apr 18 2018 17:34:14 GMT+0100 (BST)"}
presence/owner/IanStudy/40:98:AD:xx:xx:xx {"confidence":"0","name":"Unknown","scan_duration_ms":"108704","timestamp":"Wed Apr 18 2018 17:34:44 GMT+0100 (BST)"}
presence/owner/IanStudy/40:98:AD:xx:xx:xx {"confidence":"100","name":"BunnyHop","scan_duration_ms":"108704","timestamp":"Wed Apr 18 2018 17:35:21 GMT+0100 (BST)"}Hmm. Scan durations are not quite that consistent on my end. I’ll see what’s going on with that. I’m not seeing this behavior on mine and it shouldn’t really be possible based on the loops.
@PianSom - found the bug relating to duration. Thanks for pointing this out. Corrected. I have not been able to narrow down why your second line is failing.
1 Like
Maybe a dumb question, but does mosquito get installed to every Raspberry Pi Zero W you use in this project? If one has mosquito already in use for other things, does it still need to be installed on any of the Raspberry Pi Zero W’s used in this project?
Thanks.
The instructions on github are for a complete soup to nuts installation. If you already have mosquitto successfully installed, there’s no need to reinstall.
Thanks for replying. I really didn’t think so but wanted to clarify. Interesting project. I hope to try it out soon.
Any issues or gotchas running on a regular old Pi B instead of a zero? I have a few unused Pi Bs around doing nothing …
Thanks for this - looks really intriguing!
Tom
2 Likes
@andrewjfreyer - latest version working smoothly; no blank lines atm. I’m slightly curious what scan_duration_ms means if a device isn’t found, but that’s not really important. I’m seeing quite a wide (factor of 5) variety of times while seated still, so it seems to be of little value.
UPDATE - just saw the following sequence fro my wife’s iPhone:
presence/owner/IanStudy/D0:C5:xxxx{"confidence":"100","name":"IbbleDibble","scan_duration_ms":"2090","timestamp":"Wed Apr 18 2018 20:12:07 GMT+0100 (BST)"}
presence/owner/IanStudy/D0:C5:xxxx {"confidence":"80","name":"IbbleDibble","scan_duration_ms":"5148","timestamp":"Wed Apr 18 2018 20:13:18 GMT+0100 (BST)"}
presence/owner/IanStudy/D0:C5:xxxx {"confidence":"60","name":"IbbleDibble","scan_duration_ms":"5147","timestamp":"Wed Apr 18 2018 20:13:31 GMT+0100 (BST)"}
presence/owner/IanStudy/D0:C5:xxxx {"confidence":"40","name":"IbbleDibble","scan_duration_ms":"25034","timestamp":"Wed Apr 18 2018 20:14:03 GMT+0100 (BST)"}
presence/owner/IanStudy/D0:C5:xxxx {"confidence":"20","name":"IbbleDibble","scan_duration_ms":"22","timestamp":"Wed Apr 18 2018 20:14:10 GMT+0100 (BST)"}
presence/owner/IanStudy/D0:C5:xxxx {"confidence":"0","name":"IbbleDibble","scan_duration_ms":"21","timestamp":"Wed Apr 18 2018 20:14:17 GMT+0100 (BST)"}
presence/owner/IanStudy/D0:C5:xxxx {"confidence":"0","name":"Unknown","scan_duration_ms":"","timestamp":"Wed Apr 18 2018 20:14:24 GMT+0100 (BST)"}
(Don’t ask me, I don’t know why she calls her phone IbbleDibble)
@gumbo and @eBoon - I’m running the script on a Pi that I use as my Mosquitto broker (and Appdaemon machine, and pihole server) with no issues. I installed following the excellent instructions from Step 3, omitting Step 8. Obviously you need a Pi with Bluetooth!
1 Like
Awesome! - now I have a use for those Pis that are just sitting around gathering dust! Installing now…
@eBoon nope! I’m running this also on an original pi that has an adapter. The only difference for installation is to note whether you’re running jessie or wheezy.
1 Like
@PianSom The scan duration is the duration that takes for hcitool to return a scan. If it doesn’t find anything, it’ll timeout after about 5 seconds, default. That’s what you’re seeing with the 5000ms just after you turn bluetooth off on her phone.
In some cases, I’ve noticed the same behavior - that hcitool returns nearly immediately. I’d guess this is internal caching, but I’m not sure I care enough to investigate.
I agree that the time to scan appears to be - at best - only moderately interesting. It seems to be loosely correlated to distance in my testing, but not by much.
OK - initial install looks promising - seems to be working as advertised. I’m an AppDaemon user so going to work on using with AppDaemon apps.
One question (so far) - any way to add comments in the owner_devices or guest_devices file?
Thanks for this - it looks really good!!
Anything after a hash should be treated as a comment, but I haven’t tested.
I get this:
Apr 18 21:09:43 raspberrypi bash[1062]: Error: Invalid publish topic 'location/owner/kitchen/# Tom's iPhone', does it contain '+' or '#'?
This is with adding a comment above each MAC with the name of the phone.
Thanks!
Yeah, that’s probably because I’m reading these files in. I’ll look into comment support sometime soon.
1 Like
With these default values in behavior_presence:
delayBetweenOwnerScansWhenAway=7
delayBetweenOwnerScansWhenPresent=30
delayBetweenGuestScans=5
verifyByRepeatedlyQuerying=5
I was expecting to see an mqtt message every 30 seconds, but I only see a message about every 90 seconds. Is that the expected behavior?
location/owner/kitchen/xxx {"confidence":"100","name":"Tom's iPhone","scan_duration_ms":"412","timestamp":"Wed Apr 18 2018 23:04:51 GMT+0000 (UTC)"}
location/owner/kitchen/xxx {"confidence":"100","name":"Tom's iPhone","scan_duration_ms":"299","timestamp":"Wed Apr 18 2018 23:06:23 GMT+0000 (UTC)"}
location/owner/kitchen/xxx {"confidence":"100","name":"Tom's iPhone","scan_duration_ms":"2381","timestamp":"Wed Apr 18 2018 23:07:56 GMT+0000 (UTC)"}
location/owner/kitchen/xxx {"confidence":"100","name":"Tom's iPhone","scan_duration_ms":"1126","timestamp":"Wed Apr 18 2018 23:09:30 GMT+0000 (UTC)"}
location/owner/kitchen/xxx {"confidence":"100","name":"Tom's iPhone","scan_duration_ms":"727","timestamp":"Wed Apr 18 2018 23:11:04 GMT+0000 (UTC)"}
Thanks!