[presence] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

Github Link

Make sure to check out monitor as well, as it is more advanced than presence


Introduction

Here’s the problem I faced: all presence detection options I tried didn’t work reliably for my family.

The various GPS-based options (e.g., OwnTracks, iOS App, HomeKit, etc.) periodically would stop working if an app on our phones crashed. Plus the minimum radius for ‘home’ or ‘away’ was often way larger than our house. I would register as ‘home’ when I was on a dog walk a few blocks away. Not great.

Similarly, the Wi-Fi options (e.g., nmap, ubiquiti, etc.) were neither fast nor reliable because we have iOS devices that disconnect to save battery. Plus, our ubiquiti APs do not report a disconnect for quite some time - in some cases, fifteen minutes would go by. Yes, I know we can configure the timeouts, but that was not workable either. Still too slow. Also, the phones would disconnect when charging at night. Bad.

Also, Bluetooth LE options (e.g., Happy Bubbles, built-in bluetooth presence sensor, Radius Networks Script, room-assisant, etc.) either required BTLE tags ($$$ + regular batter change), were limited to detection within bluetooth’s range, or became expensive to implement (five happy bubbles + btle tags = ~$180). Don’t get me wrong, I love the happy bubbles stuff and that project is awesome. But for us, it did not work reliably enough to justify the cost. Plus, the server spammed the mqtt broker and slowed Home Assistant down. Not good.

In sum, these were the problems I had when implementing the current presence solutions:

  • Too slow to respond (wifi)
  • Too unreliable (GPS, wifi, btle)
  • Too expensive (happy bubbles + btle tags)
  • Too big of a radius (GPS, wifi)
  • Too small of a radius (bluetooth, happy bubbles)
  • False positives and false negatives. Switching from home to away when we are at the house is not ok. Similarly, a false positive when we are not home is a huge problem. (GPS)

I wanted a cheaper solution that would meet these specs:

  • Fast to respond. Arrival should be recognized in no more than 5 seconds when I’m on the curb or in the garage.
  • Reliable without adding anything to my wife’s phone or keychain or purses. Should work without her caring or knowing that it’s working.
  • Cheap. Less than $50 would be awesome.
  • Tight radius around the house. When I’m at at the curb walking the dog, I should be marked as ‘away’.
  • Let me know when guests arrive without slowing down the search for my ‘owner’ devices
  • Respond to BTLE devices too, as a backup

Here’s my solution that I’ve been tinkering with for the past six months or so. Still working on it and tuning, but it works great so far. I use Raspberry Pi Zero Ws positioned around the house, each running a shell script that I wrote and that is available on github here. I have the four devices positioned throughout the house, each connected to a cheap power brick (anker or monoprice) via a monoprice usb cable. Each configuration is about $16 ($10 pi, $6 usb micro power supply). For four of these (which is way overkill for my house, probably), the total price is $64. That’s much more palatable to me than the $180 I spent on the five happy bubbles that are now sitting in my drawer, and it still is able to track BTLE devices! (as of version 0.4)

This reports when my wife and I are home or away, and reports when a registered ‘guest’ device arrives home. I may implement guest device ‘away’ in the future, but I decided that the feature was not quite necessary for my use case.

Summary

The script generates a JSON-formatted MQTT message that is reported to a broker whenever a specified bluetooth device responds to a name query. If the device responds, the JSON message includes the name of the device and a confidence of 100.

After a delay, another name query is sent and, if the device does not respond, a verification-of-absence loop begins that queries for the device (on a shorter interval) a set number of times. Each time the device does not respond, the confidence is reduced, eventually to 0. This value is averaged with the other sensors to give an aggregate confidence that can be used for automations. To eliminate false negatives, I also use a condition for “away” state that requires recent main floor door access. I’ll talk about that a bit later.

Also, a configuration file accessed by the script defines ‘owner devices’ and another defines ‘guest devices.’ The script only scans for guest devices when not scanning for owner devices; detection of owner devices is prioritized over detection of guest devices.

Topics are formatted like this:

 location/owner/pi_zero_location/00:00:00:00:00:00 
 location/guest/pi_zero_location/00:00:00:00:00:00

Messages are JSON formatted and contain name and confidence fields, in addition to other fields that might be interesting to end-users:

 { name : "Andrew's iPhone", confidence : 100, timestamp : "[javascript formatted timestamp]", scan_duration_ms : "0000"}
 { name : "", confidence : 0}

The timestamp is formatted according to what javascript expects, namely:

+%a %b %d %Y %H:%M:%S GMT%z (%Z)

When the script starts, the configuration file is analyzed and estimations of time for each event transition.

As an example, with two owner devices, with beacon detection off and default settings:

presence 0.x.xx  - Started. Performance predictions based on current settings:
  > Est. to verify all (2) owners as 'away' from all 'home': 60 seconds to 210 seconds.
  > Est. to verify one owner is 'away': 30 to 121 seconds.
  > Est. to recognize one owner is 'home': 0.15 seconds to 12 seconds.

Tweaking your settings will adjust these numbers. The script is set up to detect devices as quickly as possible, and taking time to verify that a device is actually gone.


Example Use with Home Assistant

The presence script can be used as an input to a number of mqtt sensors in Home Assistant.. Output from these sensors can be averaged to give a highly-accurate numerical occupancy confidence.

In order to detect presence in a home that has three floors and a garage, we might inclue one Raspberry Pi per floor. For average houses, a single well-placed sensor can probably work, but for more reliability at the edges of the house, more sensors are better.

- platform: mqtt
  state_topic: 'location/owner/first floor/00:00:00:00:00:00'
  value_template: '{{ value_json.confidence }}'
  unit_of_measurement: '%'
  name: 'Andrew First Floor'

- platform: mqtt
  state_topic: 'location/owner/second floor/00:00:00:00:00:00'
  value_template: '{{ value_json.confidence }}'
  unit_of_measurement: '%'
  name: 'Andrew Second Floor'

- platform: mqtt
  state_topic: 'location/owner/third floor/00:00:00:00:00:00'
  value_template: '{{ value_json.confidence }}'
  unit_of_measurement: '%'
  name: 'Andrew Third Floor'

- platform: mqtt
  state_topic: 'location/owner/garage/00:00:00:00:00:00'
  value_template: '{{ value_json.confidence }}'
  unit_of_measurement: '%'
  name: 'Andrew Garage'

These sensors can be combined/averaged using a min_max:

- platform: min_max
  name: "Andrew Home Occupancy Confidence"
  type: mean
  round_digits: 0
  entity_ids:
    - sensor.andrew_garage
    - sensor.andrew_third_floor
    - sensor.andrew_second_floor
    - sensor.andrew_first_floor

So, as a result of this combination, we use the entity sensor.andrew_home_occupancy_confidence in automations to control the state of an input_boolean that represents a very high confidence of a user being home or not.

As an example:

- alias: Andrew Occupancy 
  hide_entity: true
  trigger:
    - platform: numeric_state
      entity_id: sensor.andrew_home_occupancy_confidence
      above: 10
  action:
    - service: home-assistant.turn_on
      data:
        entity_id: input_boolean.andrew_occupancy

Another automation is used to set the away mode, this time triggered based on a value being above 10. I have also added a condition that a first floor door has been opened in the last 10 minutes. This prevents false exit events.

That’s it! The state of input_boolean.andrew_occupancy very reliably holds my occupancy state. I have the same setup for my wife’s iPhone as well. We’ve been using this setup for roughly six months, and it’s the solution I’m going to continue to use. It works excellently, despite how complicated it seems.

Truth be told, the automation above is not my primary automation to turn on the input_booleans. I use node-red and some smoothing and statistics nodes.


Example Use with Node-Red

Recently, I transitioned many of my automations to node-red. I used the hass.io add-on as my node-red server.

This is a portion of my automation used to set home and away:


Looks super complex and cumbersome, I know. To save you a bit of boring detail, I’ll go through the nodes from left to right:

  • Leftmost nodes subscribe to the mqtt topics for individual Raspberry Pi Zero Ws. The node on the bottom takes the average of the four (from the max_min sensor above). By combining all four raspberry pi’s with their average, I’ve effectively low-pass filtered the entire bunch right off the bat.

  • Next, each of the mqtt messages received are passed to a JSON formatter. Not strictly necessary, but a bit more convenient. After formatting as a JSON message, the payloads are passed into a function that recasts the string ‘confidence’ into a number. This is important for the next step. The function also adds a topic to the message. The topic is a plain-english name for each sensor. The function looks like this:

return {topic: 'Garage', payload: parseInt(msg.payload.confidence) || 0};
  • I recast the string to an integer so that these values can be smoothed with a smooth node. Right now, I average the value of the last five values received by this node.

  • After smoothing, the smoothed value is sent to an RBE node that will only pass the value if combined and twice-filtered/averaged confidence has changed.

  • After change-filtering, I pass the value to a function that asks whether the confidence is greater than a threshold. This function outputs true or false only. I’m still playing with the threshold, but right now, 5 seems to be working well since I have four sensors. As an example, if at least one sensor detects that I am home, it will output 100. Averaging this value against three other zero values returns 25. Averaging 25 (twice) with three zeros results in a value of 10 (25+25+0+0+0)/5. In other words, I mark as “away” only after most sensors don’t see me at all, and the last sensor to recognize my phone is only half-sure that I’m there.

  • After reducing to a boolean confidence, I pass the occupancy value to a rate limiter that passes only 1 message per every ten seconds. This prevents rapid changes from home to away when I’m right on the edge of the ‘occupied’ zone. Probably not strictly necessary, but since my node-red installation is running on the same box as home assistant, anything I can do to prevent node-red from working on parallel tasks improves performance.

  • After rate limiting, I pass the boolean confidence to a switch that bifurcates the path. If true, I set my occupancy input boolean to true.

  • On the other hand, fi the input boolean is false, I set the input boolean to false only if an exterior door on the main floor (including the garage lift) has been opened or closed in the last 10 minutes. This is a final check against false exit events.

Now, astute readers will note that I have two alternate paths shown. The top alternate path allows me to bypass the smoothing if one of the sensors switches from 0 to 100 (meaning that I have arrived home). This function looks like this:

var confidence =  parseInt(msg.payload);
if (confidence >= 5 ){
    return {payload: true};
}else{
    return null
}

This allows an ‘arrival’ event to be recognized slightly sooner. This alternate path is not strictly necessary.

Lastly, I also have an alternate path that notifies me if any of the raspberry pis go offline. ‘Offline’ is defined as not receiving any message for at least 10 minutes.

The entire flow is available by copying the following and importing to node-red:

[{"id":"dcd7a3e9.622e6","type":"inject","z":"247a6012.ad0588","name":"30s ","topic":"","payload":"","payloadType":"date","repeat":"30","crontab":"","once":false,"onceDelay":"","x":130,"y":440,"wires":[["ae864ca4.e676d8"]]},{"id":"723329a9.df4fb8","type":"function","z":"247a6012.ad0588","name":"> 5","func":"var confidence =  parseInt(msg.payload);\nreturn {payload: confidence > 5 };","outputs":1,"noerr":0,"x":1250,"y":330,"wires":[["33baaee6.5e5fca"]]},{"id":"a1928cf9.98ba08","type":"rbe","z":"247a6012.ad0588","name":"","func":"rbe","gap":"","start":"","inout":"out","property":"payload","x":1080,"y":330,"wires":[["723329a9.df4fb8","1a73a15d.55ade7"]]},{"id":"3b25dc0f.50915c","type":"api-call-service","z":"247a6012.ad0588","name":"Set Home","server":"58ba2a40.903c6c","service_domain":"homeassistant","service":"turn_on","data":"{\"entity_id\" : \"input_boolean.andrew_occupancy\"}","mergecontext":"","x":1770,"y":280,"wires":[[]]},{"id":"d68fd19.42fabb","type":"comment","z":"247a6012.ad0588","name":"ANDREW IS HOME OR AWAY","info":"","x":170,"y":160,"wires":[]},{"id":"d8da0e0c.544c78","type":"switch","z":"247a6012.ad0588","name":"Switch","property":"payload","propertyType":"msg","rules":[{"t":"true"},{"t":"false"}],"checkall":"true","repair":false,"outputs":2,"x":1590,"y":330,"wires":[["3b25dc0f.50915c"],["43fad02d.31854"]]},{"id":"3e0bf278.fe9c6e","type":"api-call-service","z":"247a6012.ad0588","name":"Set Away","server":"58ba2a40.903c6c","service_domain":"homeassistant","service":"turn_off","data":"{\"entity_id\" : \"input_boolean.andrew_occupancy\"}","mergecontext":"","x":2210,"y":380,"wires":[[]]},{"id":"5ce2a8c4.825378","type":"mqtt in","z":"247a6012.ad0588","name":"First Floor","topic":"location/owner/first floor/34:08:BC:15:24:F7","qos":"2","broker":"6e9ecea1.694bf8","x":120,"y":380,"wires":[["635652e4.54013c"]]},{"id":"635652e4.54013c","type":"json","z":"247a6012.ad0588","name":"Format","property":"payload","action":"","pretty":true,"x":320,"y":380,"wires":[["82b05d1f.66f7b8"]]},{"id":"82b05d1f.66f7b8","type":"function","z":"247a6012.ad0588","name":"Parse","func":"return {topic: 'First Floor', payload: parseInt(msg.payload.confidence) || 0};","outputs":1,"noerr":0,"x":530,"y":380,"wires":[["57cddb63.1cb484","43318e46.1cdbc8","90ce7582.c2eaf"]]},{"id":"ae864ca4.e676d8","type":"api-current-state","z":"247a6012.ad0588","name":"Occupancy","server":"58ba2a40.903c6c","halt_if":"","entity_id":"sensor.andrew_occupancy_confidence","x":310,"y":440,"wires":[["468f7799.a3085"]]},{"id":"468f7799.a3085","type":"function","z":"247a6012.ad0588","name":"Parse Confidence","func":"return {topic: 'timer', payload: parseInt(msg.payload)};","outputs":1,"noerr":0,"x":500,"y":440,"wires":[["57cddb63.1cb484","43318e46.1cdbc8"]]},{"id":"57cddb63.1cb484","type":"smooth","z":"247a6012.ad0588","name":"","property":"payload","action":"mean","count":"5","round":"0","mult":"single","x":920,"y":330,"wires":[["a1928cf9.98ba08","16deef63.23a421"]]},{"id":"16deef63.23a421","type":"debug","z":"247a6012.ad0588","name":"","active":false,"console":"false","complete":"false","x":1080,"y":390,"wires":[]},{"id":"43fad02d.31854","type":"api-current-state","z":"247a6012.ad0588","name":"Recent Door Access","server":"58ba2a40.903c6c","halt_if":"","entity_id":"input_boolean.1fl_recent_door_access","x":1810,"y":380,"wires":[["ed941d94.211df"]]},{"id":"ed941d94.211df","type":"switch","z":"247a6012.ad0588","name":"ON","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"on","vt":"str"},{"t":"eq","v":"off","vt":"str"}],"checkall":"true","outputs":2,"x":1990,"y":380,"wires":[["3e0bf278.fe9c6e"],[]]},{"id":"43318e46.1cdbc8","type":"function","z":"247a6012.ad0588","name":">= 5 || null","func":"var confidence =  parseInt(msg.payload);\nif (confidence >= 5 ){\n    return {payload: true};\n}else{\n    return null\n}","outputs":1,"noerr":0,"x":1060,"y":180,"wires":[["33baaee6.5e5fca"]]},{"id":"90ce7582.c2eaf","type":"interval-length","z":"247a6012.ad0588","format":"mills","bytopic":true,"minimum":"","maximum":"","window":"","timeout":false,"msgTimeout":"10","minimumunit":"msecs","maximumunit":"msecs","windowunit":"msecs","msgTimeoutUnit":"mins","startup":true,"msgField":"timeout","repeatTimeout":true,"name":"10m Timeout","x":1090,"y":680,"wires":[[],["640dbdc7.3ceeb4"]]},{"id":"f8c3a.085ee3c7","type":"api-call-service","z":"247a6012.ad0588","name":"Notify Andrew","server":"4c32fd70.c128cc","service_domain":"notify","service":"ios_andrews_iphone","data":"{}","mergecontext":"","x":1550,"y":680,"wires":[[]]},{"id":"640dbdc7.3ceeb4","type":"function","z":"247a6012.ad0588","name":"Alert! Notification","func":"var payload = msg.payload;\nvar msg = {payload : {data: { title : \"853 Elm: Notice\", message: \"Heartbeat from \" + msg.topic + \" not detected for 10 minutes. The server may be down.\", data : { push : {badge : 0}}}}};\n\nreturn msg;","outputs":1,"noerr":0,"x":1300,"y":680,"wires":[["f8c3a.085ee3c7"]]},{"id":"42067d42.2dbd04","type":"mqtt in","z":"247a6012.ad0588","name":"Second Floor","topic":"location/owner/second floor/34:08:BC:15:24:F7","qos":"2","broker":"6e9ecea1.694bf8","x":130,"y":330,"wires":[["ab3f1893.36425"]]},{"id":"b8497c97.b7384","type":"mqtt in","z":"247a6012.ad0588","name":"Third Floor","topic":"location/owner/third floor/34:08:BC:15:24:F7","qos":"2","broker":"6e9ecea1.694bf8","x":120,"y":280,"wires":[["3e8b4e94.d422a2"]]},{"id":"2d869c7a.84ee6c","type":"mqtt in","z":"247a6012.ad0588","name":"Garage","topic":"location/owner/garage/34:08:BC:15:24:F7","qos":"2","broker":"6e9ecea1.694bf8","x":110,"y":220,"wires":[["bbc4e61c.fd6ac8"]]},{"id":"ab3f1893.36425","type":"json","z":"247a6012.ad0588","name":"Format","property":"payload","action":"","pretty":true,"x":320,"y":330,"wires":[["3bc286a8.6c76aa"]]},{"id":"3bc286a8.6c76aa","type":"function","z":"247a6012.ad0588","name":"Parse","func":"return {topic: 'Second Floor', payload: parseInt(msg.payload.confidence) || 0};","outputs":1,"noerr":0,"x":530,"y":330,"wires":[["90ce7582.c2eaf","57cddb63.1cb484","43318e46.1cdbc8"]]},{"id":"3e8b4e94.d422a2","type":"json","z":"247a6012.ad0588","name":"Format","property":"payload","action":"","pretty":true,"x":320,"y":280,"wires":[["4588a594.1564c4"]]},{"id":"4588a594.1564c4","type":"function","z":"247a6012.ad0588","name":"Parse","func":"return {topic: 'Third Floor', payload: parseInt(msg.payload.confidence) || 0};","outputs":1,"noerr":0,"x":530,"y":280,"wires":[["90ce7582.c2eaf","57cddb63.1cb484","43318e46.1cdbc8"]]},{"id":"bbc4e61c.fd6ac8","type":"json","z":"247a6012.ad0588","name":"Format","property":"payload","action":"","pretty":true,"x":320,"y":220,"wires":[["bdf10878.08cde"]]},{"id":"bdf10878.08cde","type":"function","z":"247a6012.ad0588","name":"Parse","func":"return {topic: 'Garage', payload: parseInt(msg.payload.confidence) || 0};","outputs":1,"noerr":0,"x":530,"y":220,"wires":[["57cddb63.1cb484","90ce7582.c2eaf","43318e46.1cdbc8"]]},{"id":"1a73a15d.55ade7","type":"interval-length","z":"247a6012.ad0588","format":"mills","bytopic":false,"minimum":"","maximum":"","window":"","timeout":false,"msgTimeout":"7","minimumunit":"msecs","maximumunit":"msecs","windowunit":"msecs","msgTimeoutUnit":"mins","startup":true,"msgField":"timeout","repeatTimeout":false,"name":"","x":1460,"y":470,"wires":[[],["89b2dee8.2b689"]]},{"id":"89b2dee8.2b689","type":"function","z":"247a6012.ad0588","name":"FALLBACK TIMEOUT","func":"var confidence =  parseInt(msg.payload);\n\n//after the timeout, if the confidence is zero, set to AWAY\nif (confidence === 0 ){\n    return {payload : \"\", debug: \"Fallback\"};   \n}\n\nreturn null;","outputs":1,"noerr":0,"x":1720,"y":470,"wires":[["d2a06aed.f094c","3e0bf278.fe9c6e"]]},{"id":"d2a06aed.f094c","type":"debug","z":"247a6012.ad0588","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"debug","x":2190,"y":470,"wires":[]},{"id":"66d0cb36.e41b54","type":"comment","z":"247a6012.ad0588","name":"ALERT FOR FAILURE","info":"","x":1120,"y":620,"wires":[]},{"id":"33baaee6.5e5fca","type":"delay","z":"247a6012.ad0588","name":"1msg/10s","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"10","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"x":1430,"y":330,"wires":[["d8da0e0c.544c78"]]},{"id":"58ba2a40.903c6c","type":"server","z":"","name":"Home Assistant","url":"https://ADDRESS REDACTED:443","pass":"PASSWORD REDACTED"},{"id":"6e9ecea1.694bf8","type":"mqtt-broker","z":"","broker":"192.168.1.27","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"willTopic":"","willQos":"0","willRetain":"false","willPayload":"","birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":""},{"id":"4c32fd70.c128cc","type":"server","z":"","name":"Home Assistant","url":"https://ADDRESS REDACTED:443","pass":"PASSWORD REDACTED"}]

Example Use with AppDaemon

@PianSom has detailed instructions on using this with AppDaemon in this comment. Thanks, @PianSom!

102 Likes

Thanks for sharing

2 Likes

Very interesting - thanks.

I have one question: doesn’t this eat battery life on your devices?

And also one suggestion: isn’t on-going configuration slightly painful, with each pi needing to be accessed for a Mac change? Might it be better to set tracked Mac addresses (step 11) via eg retained MQTT messages?

EDIT - I just looked at the code and saw it was shell script, rather than the python I had envisaged. Which may make the suggestion rather more complex to implement. Also on line 20 of your script, shouldn’t the Base be set to home/pi/presence ?

1 Like

I love your approach, I’ve have spent about the same amount of effort in getting occupancy right. I think I’m going to take your setup and see if I can get it work in a Appdaemon setup instead of yaml automations and node-red.

4 Likes

No noticeable battery degradation on any of the devices.

As far as configuration is concerned, my wife and I only change phones (at most) once a year. Updating the configuration takes a moment, but not long. In the next few months, I do intend to update how these macs are shared between pi’s.

Thanks for the note on the script. Yes, you are correct for the instructions I’ve provided.


EDIT - @PianSom, for updating the software on each device, I have assigned them sequential ip addresses on my network and have placed SSH keys on each. This way, I can update all four by running the following command in my mac’s terminal:

for suffix in $(seq 88 91); do  ssh [email protected].$suffix "cd presence/; rm presence.sh; git pull; sudo reboot"; done

Feel free to substitute the final command with a service restart command if you prefer. One of the four devices is actually an original pi A with a bluetooth dongle. This is not updated to Jessie (still on Wheezy), so it still works with init.d services. Far easier just to reboot all devices, despite that it’s pretty lazy, haha.

3 Likes

I looked at your github and the instructions are very well written and dummy-proof. I applaud you for that.

Just started setting up a Pi zero last night for installing Room-assistant on and now wondering what the difference between the two is? If no difference I will try this one as it seems very clear how to proceed.

2 Likes

Pretty cool idea, I guess I’d have to learn to stop toggling my bluetooth off on my phone. My wife will also do the same thing especially if we are riding in the vehicle together so my phone will attach to the car.

2 Likes

Room-assistant is a fantastic piece of software that I also tried. In fact, it was the thing that convinced me to purchase the Pi Zero W. The two are directed to the same problem, but we solve the problem differently, despite that we both use bluetooth as the protocol.

More specifically, like Happy Bubbles, room-assistant relies on BTLE tags and does not play well with either iOS or android phones. In order to achieve a high confidence that a phone is actually present, we have to have confidence that the beacon our phone broadcasts will always be broadcasting. For iOS and Android, this requires an app to be running; neither OS (as best I can tell) natively broadcasts a BLTE beacon at regular intervals. I’d guess this is for privacy reasons.

The tags that I purchased ran out of battery about every six months. Not awesome. I tried two different brands with similar results.

Also, as I eluded to above, my wife doesn’t prefer to add technology to her keys, purses and other daily carries. Especially if I have to take her keys every six months to replace a battery, or to remind her to periodically launch an app on her phone.

My solution doesn’t require her to do anything. And, frankly, it didn’t require me to even access her phone. A (perhaps well-known) secret of iOS devices is that the Wi-Fi mac address and the bluetooth mac address are one hexidecimal digit separated from one another. After grabbing her wifi mac address (nmap query or from the router), I incremented the address by one and I had her bluetooth Mac. Address incrementing used to be a built-in feature of the presence script, but I scrapped it since I rarely change the configuration. Here’s what it looked like:

### pass wifi mac address with colons as $1
function incrementWiFiMacAddress () {
	if [ ! -z "$1" ]; then 

		#receive
		addr="$1"
		#trim to last two
		trim=${addr:15:2} #(echo "$addr" | tail -c 3)
		prefix=${addr:0:14}

		#math it
		mac_decimal=$(echo "obase=10;ibase=16; $trim" | bc ) # to convert to decimal
		mac_incremented=$(expr "$mac_decimal" + 1 ) # to add one 
		mac_hex_incremented=$(echo "obase=16;ibase=10; $mac_incremented" | bc ) # to convert to decimal

		#output variables
		bt_addr="$prefix:$(printf '%02x' 0x$mac_hex_incremented)"

		echo "$bt_addr"
	fi
}
2 Likes

That is the last thing that I can’t 100% account for… we both inadvertently and periodically turn off bluetooth or (in my case) accidentally turn on airplane mode from my watch.

In our home, we both receive notifications when one or both of us leaves or enters the house. So, usually, we know relatively quickly that bluetooth is turned off. As a backup, I have an automation that sends us another alert if GPS and Wifi both believe we are home for at least 5 minutes. This alert basically reads: “Turn your bluetooth back on, dumb-dumb.”

2 Likes

I have quite a large house with some pretty thick walls, and wonder how many Zero’s I’d need.

Can I ask what is the typical distance that you have found the Bluetooth pickup range to be irl?

Most of my sensors detect us no matter where we are and all report 100. The advantage here, however, is to prevent the false negatives when we go to far ends of the house. 100% accuracy in exchange for redundancy.

My range for each one is around 150ft. Reliability for a sensor tapers noticeably after 75 feet or so. Our three stories and detatched garage are covered very adequately with four sensors. The advantage of this system is that you can add as many sensors as you need for less than $20 a pop (including a power supply and a case!).

Start with one. It’s also a good idea to add the devices one at a time since the Pi Foundation limits purchase quantities at all retailers. First Pi on the same order costs $10, 2+ cost $14.99 a piece.

1 Like

Can this be used for room detection? If these Pis can provide some information about the signal strength/distance, we can use something similar for room detection.

3 Likes

First, each of the devices report to their own topic. So in a way, you can detect which devices a phone is nearby. But, as I noted above, most of my devices in my house are able to detect me without an issue; all report 100 at all times that I am home.

More granular distance estimation is feasible, yes. The most common technique is using RSSI as a proxy for distance. The problem is that RSSI measurements require a connection which, in turn for many devices (but not all), requires pairing to each device. Pairing is a hard pass for me. Also, constant connection and disconnection to various devices around the house slows the system (especially with more than one sensor) and reduces battery life on the phones. It also can interrupt bluetooth connections to other devices when you’re at home.

I have a few tests in the pipeline that would allow for more granular room detection, but for the time being, room-specific detection is not useful unless you have an exceptionally large house.

I was also wondering about room detection but now wondering why not just throw a pir sensor or similar type of sensor onto the pi’s gpio and call it a day? I have some pir sensors that are zwave and since they are battery powered they are kind of slow to react, but i think a powered pir sensor using mqtt might be relative quick as far as response time. And they are dirt cheap.
I guess if you arent moving around it might cause issues though

2 Likes

PIR sensors on the Pis are out of scope for my project, but that’s another huge reason to prefer pi boards over the Happy Bubbles boards. Broad expandability.

Excellent idea! I have my Unifi controller going which does work but of course has the delays. I was thinking of some other ways to detect that the device is on the network without waking the radio from deep sleep or power save mode. Think I could go with the two step approach like you’ve done as this doesn’t add any additional behavior modifications for the wife and no other devices with batteries. One less battery to keep up with is a win for me (I have a pet peeve for stationary battery power devices).

1 Like

I like this. I too have been very frustrated with presence detection. Zanzito combined with WiFi and an input Boolean is what I use presently. It is fairly reliable, but I may have to give your project a try. I already have multiple RPis around my house, plus I have a spare Pi Zero W on my workbench. My house isn’t huge, so I bet I could get away with 2-3 Pi sensors

Thanks for sharing!

1 Like

Hi

I just installed your script on a pi I use for various infrastructure - all worked. I am testing it with a device called “BunnyHop”. When I turn off then turn back on the BunnyHop device these are the mqtt postings I see (with address redacted)-

presence/owner/IanStudy/MACADDRESS {"confidence":"100","name":"BunnyHop"}
presence/owner/IanStudy/MACADDRESS {"confidence":"80","name":"BunnyHop"}
presence/owner/IanStudy/MACADDRESS {"confidence":"60","name":"BunnyHop"}
presence/owner/IanStudy/MACADDRESS {"confidence":"40","name":"BunnyHop"}
presence/owner/IanStudy/MACADDRESS {"confidence":"100","name":""}
presence/owner/IanStudy/MACADDRESS {"confidence":"100","name":"BunnyHop"}

ie the device name does not seem to be picked up for the first broadcast - thought you may be interested.

Also one other thought - it may be helpful for the time of the post be be included in MQTT posting.

Thanks for the note. I’ll investigate why the name didn’t appear. I’ll add a time stamp as well.

EDIT - @PianSom, I added a bit more intelligence and error checking to the script. The only way I was able to reproduce your error was to introduce a newline character into the configuration file. A javascript-formatted date is now passed in the ‘timestamp’ field. I have also added an experimental ‘distance’ feature.

Hey @andrewjfreyer

I just replaced with 0.3.65 and am getting the following

Apr 17 22:04:16 pi-hole systemd[1]: Started Presence service.
Apr 17 22:04:16 pi-hole bash[23943]: /home/pi/presence/presence.sh: line 175: syntax error near unexpected token `else'
Apr 17 22:04:16 pi-hole bash[23943]: /home/pi/presence/presence.sh: line 175: `        else'
Apr 17 22:04:16 pi-hole systemd[1]: presence.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Apr 17 22:04:16 pi-hole systemd[1]: presence.service: Unit entered failed state.
Apr 17 22:04:16 pi-hole systemd[1]: presence.service: Failed with result 'exit-code'.