[presence] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

@eboon and I discussed the shell slowdowns via DM, but for the room I’ll note that I experience the same thing on my Zeros, but not my Raspi A.

I believe this is something weird going on with my own internal network - no reflections on Andrew’s project. I noticed this afternoon that while I was running, my entire network slowed down so much that AppDaemon was negatively affected and was lagging - once I stopped the script everything back to normal. So something’s going on, just have not yet figured it out. I had something similar affect the network when I was running HASS on a Pi - never did figure it out - just moved to a VM on my Synology NAS and things have been good ever since!

I really want to be able to run this - so I’m going to figure out what the heck is going on.

Tom

Ya my pi zero ssh connection slowed down noticeably once I started the service.

This seems to be working well for me so far but i tried to set a static ip and messed everything up. Not sure the proper way to do that via pi zero’s wifi. The method i tried via a google search messed up the connection and i couldnt get it working again and had to start all over.

First off, thanks @andrewjfreyer - this is a great project.

I finally got a chance to get my Pi Zero up and running yesterday. I’m noticing the same SSH slowdown that others have noted.

I’m also seeing that the confidence draw-down isn’t happening for me. My confidence values (for 2 phones) pretty much split between 0 and 100. So the chart in HA looks like:

image

(I’m running whatever version was current as of yesterday - 4/20 - morning.)

@ih8gates, the drawdowns would take place over the course of seconds. We would not be able to see them happening on a chart of this scale.

However, if you look to the right-hand side, you can see that the ‘pretty much’ zero values you’re seeing are roughly 66% and 33%. These are drawdowns that were corrected.

If I had to guess, you either have a large house, and/or your bluetooth devices were moving into an area subject to interference. Increasing the verifyByRepeatedlyQuerying value will add more error correction and should smooth your experience.

Else, you may have to add another Pi to cover the area of your home.

I do notice that some of my pis slow down, but, like @eboon, top shows that the processor is not particularly taxed. I think we’re looking at a shell issue. I’ve tried disabling dns for my shells, but that had little effect.

Increasing the delay time does help for me. Seems like the slowdowns correspond to when the BT hardware is active. Can anyone else confirm?

That makes sense. Also, since I’m still in tinker mode, I had the Pi in my office, which is not at all central in the house.

I think I might be seeing something similar to @eBoon with the slowdown. I’m editing my HA YAML over samba and sometimes the editor (notepad++) takes an unusually long time to save a change. When I stop the presence service, that goes away. I edit my HA config this way all the time and this is the first time I’ve noticed this. But I get that this isn’t necessarily helpful info…

I haven’t tested any other network speeds to see if it’s a whole-network issue.

edit: To clarify - I’m running presence on a dedicated Pi Zero, not on the same Pi as HA.

Yes, I’m seeing the same thing. I ignored it in the development of the script because my pis were only running this and I figured the zeros were just slow machines.

I think The bt hardware may be the culprit, but I’m still investigating. If you increase you delays substantially, do you still notice slowdowns? Mine seem to reduce, but it’s difficult to tell.

Updated to 0.4.10, corrects a number of bugs and adds an estimation of time for each event transition.

1 Like

Great looking project! I haven’t tried to set it up yet as I don’t have the equipment or time right now, but I’m following along with interest!

Out of curiosity, are you aware of FIND?

I stumbled across it the same day I found your post as I was doing a little research on presence detection. It sounds like they could potentially be very complimentary. FIND has a “passive tracking” mode that is vaguely similar to your strategy (from the standpoint that there’s no software installation or configuration requirements on the tracked devices) but it sounds like it has a much slower detection rate (at least in passive mode) than your solution due to it only listening for Wifi broadcasts as opposed actively polling for bluetooth devices the way you do. Where it shines though is it does statistical analysis of the responses allowing it to identify not just presence but specific room locations!

I know I’m just making noise at this point, but the thought of being able to combine the techniques in both your projects… :drooling_face: :smiley:

Anyway, just thought you’d enjoy looking at FIND if you weren’t already aware of it. Keep up the good work!

Ha! far from it. :slight_smile:

I could probably put at least bluetooth into a passive mode with hcidump, but since the Pi zero’s only internet connection is Wi-Fi, we’d likely have to do a complicated technique of disconnecting from wifi regularly… not ideal for responsiveness…

I’ll see what I can find out. Neat find (so to speak)!

I ended up unplugging my presence Pi yesterday when I noticed that it was bogging down my Harmony Hub (made the red status light come on). This morning, seeing that there was an update, I did a git pull and a reboot. I haven’t seen the issue yet with 0.4.11

One weird thing that did happen this morning. My wife (and her phone) are gone. Nowhere near the house. Presence briefly reported that she was present, with a confidence as high as 83% (it ramped up 16…33…50…66…83). Then it quickly dropped back to zero.

Edit - just did it again. I took a look at the MQTT traffic and presence was sending status for my phone on the topic for my wife’s phone. At least I think that’s the case - the “name” value is the bluetooth name of my phone. After a few seconds, the confidence for my wife’s phone dropped back to 0, but I’m still seeing status reports on her topic with the name of my phone.

I have noticed the same thing on my end, and I’m working on finding and squashing this bug.

1 Like

Hey @gumbo - I installed a new pi zero with a fresh install using Andrew’s instructions. It worked with no problems at all.

I can only suggest that you do reboots (pull power) after each step and see where your problem starts.

Using this package with Appdaemon

I am planning on using Andrew’s excellent script with Appdaemon to manage presence status. I have just made a start on this, and thought it may be worth sharing progress to see if anyone has better ideas.

First off, because AD isn’t MQTT-aware yet, I set up a simple automation in HA to listen for all appropriate MQTT posts and pass them to a HA event (note that the MQTT topic is one I have chosen in the script config file):

- id: 01_mqtt_event_presence
  trigger:
    - platform: mqtt
      topic: 'presence/owner/#'
  action:
    event: HASS_MQTT_PRESENCE
    event_data_template:
      topic: '{{ trigger.topic }}'
      payload: '{{ trigger.payload }}'

Then within AD I set up a simple Python script to listen for an event from HA, check whether we have 100% presence confidence from it and - for the moment - just log the results:

import appdaemon.plugins.hass.hassapi as hass
import json

class PersonPresent(hass.Hass):

  def initialize(self):
    self.listen_event(self.event_listener, 'HASS_MQTT_PRESENCE')

  def event_listener(self, event_name, data, *args, **kwargs):
    topic =   data['topic'] if data.get('topic') else {}
    if self.args["DeviceMAC"] in topic:
      payload = json.loads(data['payload']) if data.get('payload') else {}
      confidence = payload.get('confidence')
      if confidence == "100":
        self.log("{} spotted!".format(self.args["DeviceID"]))
      else:
        self.log("{} not found".format(self.args["DeviceID"]))

where the associated yaml file looks like

my_bt_presence:
  module: mqtt
  class: PersonPresent
  DeviceID: "My Phone"
  DeviceMAC: "xx:xx:xx:xx:xx:xx"
6 Likes

I added to iPhones and an Android phone to the ownerdevices file. One iPhone and the Android are seen by Presence but the other iPhone not. Both are iPhone 5s, MAC is correct and BT is on. Any suggestions ?

From the install instructions:

#select the option to keep the ‘bluez’ that is already installed

There is nothing to select - I can accept to install (leading to an error) or decline (which aborts the process). Any idea ?

Lars

All good, tried again and works now.

Lars

1 Like

Older phones will likely require a pairing operation since they may not be BT4.0 compatible. That said… the 5S is 4.0 compatible.

Have you restarted the phone? Modern OS?

@gumbo As a test, I reformatted one of my cards and tested my own instructions this morning. All worked from start to finish without modification of the instructions. Anything else you can tell us about what’s not happening or which instruction is failing?