[presence] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

automation
mqtt
Tags: #<Tag:0x00007f058fa7c560> #<Tag:0x00007f058fa7c010>

#70

Great looking project! I haven’t tried to set it up yet as I don’t have the equipment or time right now, but I’m following along with interest!

Out of curiosity, are you aware of FIND?

I stumbled across it the same day I found your post as I was doing a little research on presence detection. It sounds like they could potentially be very complimentary. FIND has a “passive tracking” mode that is vaguely similar to your strategy (from the standpoint that there’s no software installation or configuration requirements on the tracked devices) but it sounds like it has a much slower detection rate (at least in passive mode) than your solution due to it only listening for Wifi broadcasts as opposed actively polling for bluetooth devices the way you do. Where it shines though is it does statistical analysis of the responses allowing it to identify not just presence but specific room locations!

I know I’m just making noise at this point, but the thought of being able to combine the techniques in both your projects… :drooling_face: :smiley:

Anyway, just thought you’d enjoy looking at FIND if you weren’t already aware of it. Keep up the good work!


#71

Ha! far from it. :slight_smile:

I could probably put at least bluetooth into a passive mode with hcidump, but since the Pi zero’s only internet connection is Wi-Fi, we’d likely have to do a complicated technique of disconnecting from wifi regularly… not ideal for responsiveness…

I’ll see what I can find out. Neat find (so to speak)!


#72

I ended up unplugging my presence Pi yesterday when I noticed that it was bogging down my Harmony Hub (made the red status light come on). This morning, seeing that there was an update, I did a git pull and a reboot. I haven’t seen the issue yet with 0.4.11

One weird thing that did happen this morning. My wife (and her phone) are gone. Nowhere near the house. Presence briefly reported that she was present, with a confidence as high as 83% (it ramped up 16…33…50…66…83). Then it quickly dropped back to zero.

Edit - just did it again. I took a look at the MQTT traffic and presence was sending status for my phone on the topic for my wife’s phone. At least I think that’s the case - the “name” value is the bluetooth name of my phone. After a few seconds, the confidence for my wife’s phone dropped back to 0, but I’m still seeing status reports on her topic with the name of my phone.


#73

I have noticed the same thing on my end, and I’m working on finding and squashing this bug.


#74

Hey @gumbo - I installed a new pi zero with a fresh install using Andrew’s instructions. It worked with no problems at all.

I can only suggest that you do reboots (pull power) after each step and see where your problem starts.


#75

Using this package with Appdaemon

I am planning on using Andrew’s excellent script with Appdaemon to manage presence status. I have just made a start on this, and thought it may be worth sharing progress to see if anyone has better ideas.

First off, because AD isn’t MQTT-aware yet, I set up a simple automation in HA to listen for all appropriate MQTT posts and pass them to a HA event (note that the MQTT topic is one I have chosen in the script config file):

- id: 01_mqtt_event_presence
  trigger:
    - platform: mqtt
      topic: 'presence/owner/#'
  action:
    event: HASS_MQTT_PRESENCE
    event_data_template:
      topic: '{{ trigger.topic }}'
      payload: '{{ trigger.payload }}'

Then within AD I set up a simple Python script to listen for an event from HA, check whether we have 100% presence confidence from it and - for the moment - just log the results:

import appdaemon.plugins.hass.hassapi as hass
import json

class PersonPresent(hass.Hass):

  def initialize(self):
    self.listen_event(self.event_listener, 'HASS_MQTT_PRESENCE')

  def event_listener(self, event_name, data, *args, **kwargs):
    topic =   data['topic'] if data.get('topic') else {}
    if self.args["DeviceMAC"] in topic:
      payload = json.loads(data['payload']) if data.get('payload') else {}
      confidence = payload.get('confidence')
      if confidence == "100":
        self.log("{} spotted!".format(self.args["DeviceID"]))
      else:
        self.log("{} not found".format(self.args["DeviceID"]))

where the associated yaml file looks like

my_bt_presence:
  module: mqtt
  class: PersonPresent
  DeviceID: "My Phone"
  DeviceMAC: "xx:xx:xx:xx:xx:xx"

Advice on app design please
Advice on app design please
[monitor] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection
#76

I added to iPhones and an Android phone to the ownerdevices file. One iPhone and the Android are seen by Presence but the other iPhone not. Both are iPhone 5s, MAC is correct and BT is on. Any suggestions ?


#77

From the install instructions:

#select the option to keep the ‘bluez’ that is already installed

There is nothing to select - I can accept to install (leading to an error) or decline (which aborts the process). Any idea ?

Lars


#78

All good, tried again and works now.

Lars


#79

Older phones will likely require a pairing operation since they may not be BT4.0 compatible. That said… the 5S is 4.0 compatible.

Have you restarted the phone? Modern OS?


#80

@gumbo As a test, I reformatted one of my cards and tested my own instructions this morning. All worked from start to finish without modification of the instructions. Anything else you can tell us about what’s not happening or which instruction is failing?


#81

@ih8gates, I identified (what I believe to be) the culprit of this bug and posted an updated version. Please let me know if you continue to see it.


#82

Noticed a side-effect: when I run Presence my Harmony hub becomes unresponsive (led turns red). Besides that the Pi Zero responds very slow to keystrokes: press a key an it takes up to 10 seconds before the Pi responds.


#83

Installed it on a Pi 3, works fine!

Noticed the response lag re keystrokes here also.

Lars


#84

Hi,
firstly thanks a lot for sharing this.
I run this project on a dedicated Pi 0.

I just have one question is it possible to make it comatible with this component?


or this

Actually I just need the information that I’m at home or not :slight_smile:
BR


#85

The combined sensor I recommended (average) gives you just that! :slight_smile:


#86

Interesting. Any logs to report?


#87

The Harmony issue was mentioned earlier in this topic, the led turns red and it doesn’t respond to commands. The Pi itself becoming very slow in a SSH sessions was also mentioned before. I assume by logging you meant the output of Presence.sh ?

presence 0.4.11  - Started. Performance predictions based on current settings:

Est. to verify all (3) owners as ‘away’ from all ‘home’: 90 seconds to 300 seconds.
Est. to verify one owner is ‘away’: 30 to 122 seconds.
Est. to recognize one owner is ‘home’: 0.15 seconds to 13 seconds.
location/owner/Woonkamer/88:6B:6E:7F:–:-- { confidence : 0, name : , scan_duration_ms: 5055, timestamp : Mon Apr 23 2018 06:07:18 GMT+0000 (UTC)}
location/owner/Woonkamer/88:75:98:C6:–:-- { confidence : 100, name : Galaxy S8, scan_duration_ms: 1159, timestamp : Mon Apr 23 2018 06:07:19 GMT+0000 (UTC)}
location/owner/Woonkamer/F4:5C:89:D5:–:-- { confidence : 100, name : iPhone van --------, scan_duration_ms: 931, timestamp : Mon Apr 23 2018 06:07:20 GMT+0000 (UTC)}
DEBUG Scanning for 0 guest devices between owner scans, when at least one device is present.
DEBUG Appropriate Delay: 30
location/owner/Woonkamer/88:6B:6E:7F:–:-- { confidence : 0, name : , scan_duration_ms: 5060, timestamp : Mon Apr 23 2018 06:07:59 GMT+0000 (UTC)}
location/owner/Woonkamer/88:75:98:C6:–:-- { confidence : 100, name : Galaxy S8, scan_duration_ms: 1217, timestamp : Mon Apr 23 2018 06:08:00 GMT+0000 (UTC)}
location/owner/Woonkamer/F4:5C:89:D5:–:-- { confidence : 100, name : iPhone van --------, scan_duration_ms: 1463, timestamp : Mon Apr 23 2018 06:08:02 GMT+0000 (UTC)}
DEBUG Scanning for 0 guest devices between owner scans, when at least one device is present.
DEBUG Appropriate Delay: 30


#88

Hi @andrewjfreyer

I too have noticed a dramatic slowness in performance on a Pi Zero compared to a Pi B. I wonder if a reason may be a difference in implementation of timeout (single-core v multi-core, perhaps)?

If this is the issue, could this link help?


#89

Hey all, I couldnt see this mentioned but how does this get around the fact that android and iphones will have their mac address changed for privacy reasons?

Does it just look for a broadcasting name and assume that if its the same over multiple times then that is the same device regardless of mac addresses?