[monitor] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

so if my manufacturer is “Unknown” and might possibly change will my flag change?
where do i find my flag values?

you are going to want to do this if your device is sending unknown.

#ARRIVE TRIGGER FILTER(S)
PREF_PASS_FILTER_ADV_FLAGS_ARRIVE=".*"
PREF_PASS_FILTER_MANUFACTURER_ARRIVE=".*"

#ARRIVE TRIGGER NEGATIVE FILTER(S)
PREF_FAIL_FILTER_ADV_FLAGS_ARRIVE="NONE"
PREF_FAIL_FILTER_MANUFACTURER_ARRIVE="NONE"
1 Like

Yes, that’s what I’ve done. (That should probably be the default when this is merged with master).

My question was where does one find the flag value?
Or maybe I’m not seeing it because my phones aren’t randomly broadcasting, and none of this matters anyways.

1 Like

Yes, it will be. Beta is still in testing. :slight_smile:

I’m fairly sure I also have an Android phone that doesn’t do BLE advertisements. Mine is a OnePlus 5. I have however found a “workaround” that seems to be reliable. I have installed the app Beacon Simulator. If I create a beacon in the app and contiously have it running it seems I also get the ADV_RAND messages in the monitor.sh log and presence detection is working very reliable.

2 Likes

so you have to have that app running all the time?

i’ve a partial workaround for my phones not broadcasting. its partial since I only have the one pi running monitor currently, and it doesn’t adequately cover my house.

anyways, its basically a modified version of the automation sequence that @benjimatt posted here

  • when the door is opened I stop the depart sequence from running.
  • when my door is closed i check whether both phones are away (0 confidence)
    — if so then I run an arrive scan.
    — if not then i run the depart sequence from the linked post above. I don’t do the restart sequence (because I only have the one pi running monitor, so it can’t be out of sync with itself). instead at the end of the depart sequence I do an arrival scan.

the obvious problem is that one of the phones can be in the house but outside of the pi’s bluetooth range. so it comes back as not home. not a huge problem for me since i’m also tracking based on WiFi and a bunch of logic based stuff surrounding my doors and motion sensors. I might add some additional trigger for an arrival scan like triggering a certain motion sensor that is close to pi.

here’s my node red flow for the door open trigger depart/arrive scans.

[{"id":"200d9ab3.10c406","type":"server-state-changed","z":"46af5846.433258","name":"Front Door Closed","server":"37567a14.bdb636","entityidfilter":"binary_sensor.door_front","entityidfiltertype":"exact","outputinitially":false,"state_type":"str","haltifstate":"","halt_if_type":"str","halt_if_compare":"is","outputs":1,"x":110,"y":400,"wires":[["68fcd0e0.206c2"]]},{"id":"68fcd0e0.206c2","type":"rbe","z":"46af5846.433258","name":"","func":"rbe","gap":"","start":"","inout":"out","property":"payload","x":270,"y":400,"wires":[["1172a172.e3f3cf"]]},{"id":"1172a172.e3f3cf","type":"switch","z":"46af5846.433258","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"off","vt":"str"},{"t":"else"}],"checkall":"false","repair":false,"outputs":2,"x":310,"y":480,"wires":[["63924c3.0141db4"],["a47dbf0d.a0485"]]},{"id":"7d37fa8f.716e94","type":"mqtt out","z":"46af5846.433258","name":"","topic":"monitor/scan/depart","qos":"","retain":"","broker":"51a5b2b3.56d4ec","x":1180,"y":300,"wires":[]},{"id":"dfd41ded.0d2aa","type":"looptimer","z":"46af5846.433258","duration":"10","units":"Second","maxloops":"3","maxtimeout":"1","maxtimeoutunits":"Minute","name":"","x":960,"y":360,"wires":[["7d37fa8f.716e94"],["81071387.0d62c"]]},{"id":"fd0bd0bf.d3c16","type":"delay","z":"46af5846.433258","name":"","pauseType":"delay","timeout":"10","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":720,"y":360,"wires":[["dfd41ded.0d2aa"]]},{"id":"a47dbf0d.a0485","type":"change","z":"46af5846.433258","name":"Clear Loop","rules":[{"t":"set","p":"payload","pt":"msg","to":"stop","tot":"str"},{"t":"set","p":"reset","pt":"msg","to":"reset","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":490,"y":500,"wires":[["fd0bd0bf.d3c16","dfd41ded.0d2aa","81071387.0d62c"]]},{"id":"e63db97e.b1b6f8","type":"mqtt out","z":"46af5846.433258","name":"","topic":"monitor/scan/arrive","qos":"","retain":"","broker":"51a5b2b3.56d4ec","x":1330,"y":420,"wires":[]},{"id":"81071387.0d62c","type":"delay","z":"46af5846.433258","name":"","pauseType":"delay","timeout":"10","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":1160,"y":420,"wires":[["e63db97e.b1b6f8"]]},{"id":"63924c3.0141db4","type":"function","z":"46af5846.433258","name":"BT Monitor Values","func":"const haCtx = global.get('homeassistant');\nconst configCtx = haCtx.homeAssistant;\nconst jBTState = configCtx.states['sensor.rpibt_jphone'];\nif ( jBTState != \"0\" ) { return [null, msg]; }\nconst eBTState = configCtx.states['sensor.rpibt_ephone'];\nif ( eBTState != \"0\" ) { return [null, msg]; }\n\nreturn [msg, null];","outputs":2,"noerr":0,"x":510,"y":340,"wires":[["544f6fa7.8de78"],["fd0bd0bf.d3c16"]]},{"id":"544f6fa7.8de78","type":"mqtt out","z":"46af5846.433258","name":"","topic":"monitor/scan/arrive","qos":"","retain":"","broker":"51a5b2b3.56d4ec","x":750,"y":260,"wires":[]},{"id":"37567a14.bdb636","type":"server","z":"","name":"Home Assistant","legacy":false,"hassio":false,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open"},{"id":"51a5b2b3.56d4ec","type":"mqtt-broker","z":"","name":"","broker":"","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""}]
2 Likes

Yes, I have the app running at all times. I have three Pi Zero W covering my house using sample config provided by andrewjfreyer with one mqtt sensor per Pi and a min_max sensor. It’s been running reliable for me for a week now

1 Like

I’ve been using this for a week now and quite honestly, I’m still in awe :). The Nmap scanner which I still have running is grossly inaccurate by comparison.

Monitor is getting it right 99% of the time…that 1% is because my RPi is not able to see my whole house, but I’ll be solving that soon. One thing I do notice, my iphone 8 is measured at being consistent on the confidence scale more than the wife’s iphone6s…but monitor still handles even that :slight_smile:

My apple watch never shows up (not sure why, maybe because its permanently connected to my phone).

Just want to say thank you for this. Is there a beer fund?

1 Like

lol, no beer fund haha - I’m just glad this is working well for so many folks.

2 Likes

Wondering if anyone else running master build with 2 rPi Zero W is going offline? Once in a while my send pi will show offline and within HA it will say unknown. I will need to restart the pi or service in order for this to cone back online.

I tried using beta as well but i find it is not fast at detecting my devices. With pi1 i am using -tr and with my second pi upstairs i am using -tad.

What are other using? and what are others behave file look like that have more than 1 pi running monitor?

I cant think of any trigger i can use for our front door without a sensor, i do have a cam in the front and can use the motion for the triggers but i would get some false once in a while as well.

Thank you!

1 Like

Yes, this will happen on master if you are in a noisy BT environment. Suggest the beta!

Massive merge into master! Version 0.1.956 merges hundreds of bug fixes and optimizations into the master branch from beta. Please read the new readme and comments in this thread related to beta.

Instead of posting all the many, many changes and updates - I’ll ask everyone read the updated readme. Post questions here, and I’ll do my best to answer if they’re not already addressed in the readme’s FAQs.

GitHub Here

5 Likes

Thanks for all your work on this.

In your most recent update of the install walk through it looks step #9 got dropped (initial run with sudo bash monitor.sh)

Also, add me to the list of people with android phones that don’t appear to be sending random advertisements. My wife and I both have samsung galaxy s9s that aren’t detected using your suggested hcitool lescan and hcidump method.

I have paired my phone with both regular bluetooth devices and a BLE device (a 2017 nvidia shield remote), and the phone seems to be connected and working just fine with the bluetooth devices. Is it possible that hcitool isn’t detecting the advertisements? Or would it definitely be a problem with my phone?

#9 is fixed. Thanks for the note.

For the Samsung Galaxy devices, I think it’d be interesting to conduct a few tests off this thread. Can you DM me when it’s convenient?

Before I start, just wanted to thank @andrewjfreyer, awesome work sir! I’ve been watching this thread and decided to add a Pi Zero W to my collection so I can join in on the fun!

However, upon setting everything up it doesn’t appear to be finding any random advertisements from any of my Android phones, Galaxy S5, J2 Pro or Note 9.

Would be happy to assist debug in anyway I can.

Yes, almost all of the reports of no advertisement have been from Android phones. I’m going to borrow/buy a few for testing.

Thanks for the report!

1 Like

Awesome, pleased to hear you’re not forgetting about Android users :+1:. Will be keeping a close eye on this.

Hey that was a great find FYI !!!

I can simulate an apple beacon which also means I can now do this

#ARRIVE TRIGGER FILTER(S)
PREF_PASS_FILTER_ADV_FLAGS_ARRIVE=".*"
PREF_PASS_FILTER_MANUFACTURER_ARRIVE="Apple"

for all the android users, you may want to give it a try
Thanks again @rwilco
FYI dont forget to go into settings and turn on “Broadcast Resilience”

1 Like

I have found another app that might shed some light on the Android situation, nRF Connect. Under Device Information there’s a section for Bluetooth low energy that list the phone capabilities. The item “Periodic advertisement supported” on my phone says “NO”. I’d be interested to hear from others if this is indeed an indicator if the phone will work out of the box with @andrewjfreyers excellent monitor service.

Yeah my oneplus 3 says the same thing. I wonder how accurate this app is