[presence] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

I do, especially my Harmony Hub doesn’t like Presence.

1 Like

I was seeing the same here.

The only other place I notice slowness is when (as described earlier in this thread) trying to save a file to my Home Assistant Pi over Samba. But I’m not seeing that issue now. Possibly since upgrading to 0.4.13

Like @eBoon, I have no issue with Presence bogging down the dedicated Pi Zero.

I’ll report back if I see any issues, but things have been smooth for the last ~15 hours since upgrading.

1 Like

Rest assured, I’m trying to figure out why this is happening with @eBoon and you.

Some questions:

Is your Harmony hub physically nearby the raspberry pi?

Are you running a local DNS service? Like pihole?

Have you modify the behavior preferences file?

Are you experiencing slowdowns on your network generally, or only on the harmony hub?

Are you using local host names or IP addresses to communicate with your MQTT broker?

At least on the iOS side, the physical hardware address will not change. The Mac fuzzing I think you’re referring to relates LE beaconing/inq requests.

My Harmony is not near the Pi Zero, about 6 meters away but in the same room. I am indeed running a local PiHole DNS. I did not modify the behaviour preferences file. I did notice some network issues when scanning for device with the Fing app, it sometimes found only few of the devices on my network. I’m using an IpAddress to communicate with the MQTT broker.

Thanks for the effort, good to see something promising after the expensive Happy Bubbles disaster.

2 Likes

I went AWOL this weekend - back and testing this morning with the latest 0.4.13 version. Here’s a log snippet for what I’m seeing this morning:

Apr 23 09:47:21 raspberrypi bash[5361]: #033[1;35mpresence/owner/kitchen/CC:xx { confidence : 0, name : Tom's iPhone, scan_duration_ms: 5027, timestamp : Mon Apr 23 2018 09:47:21 GMT-0500 (CDT)} #033[0m
Apr 23 09:47:21 raspberrypi bash[5361]: #033[0;33mDEBUG Scan result: Sue’s iPhone #033[0m
Apr 23 09:47:21 raspberrypi bash[5361]: #033[1;35mpresence/owner/kitchen/78:xx { confidence : 100, name : Sue’s iPhone, scan_duration_ms: 525, timestamp : Mon Apr 23 2018 09:47:21 GMT-0500 (CDT)} #033[0m
Apr 23 09:47:24 raspberrypi bash[5361]: #033[0;33mDEBUG Scan result: Abby’s iPhone #033[0m
Apr 23 09:47:24 raspberrypi bash[5361]: #033[1;35mpresence/owner/kitchen/A8:xx { confidence : 100, name : Abby’s iPhone, scan_duration_ms: 2923, timestamp : Mon Apr 23 2018 09:47:24 GMT-0500 (CDT)} #033[0m
Apr 23 09:47:24 raspberrypi bash[5361]: #033[0;33mDEBUG Scanning for 2 guest devices between owner scans, when at least one device is present.#033[0m
Apr 23 09:47:29 raspberrypi bash[5361]: #033[0;33mDEBUG Scan result:  #033[0m
Apr 23 09:47:29 raspberrypi bash[5361]: #033[1;35mpresence/guest/kitchen/78:xxx { confidence : 0, name : , scan_duration_ms: 5024, timestamp : Mon Apr 23 2018 09:47:29 GMT-0500 (CDT)} #033[0m
Apr 23 09:47:54 raspberrypi bash[5361]: #033[0;33mDEBUG Appropriate Delay: 24#033[0m
Apr 23 09:48:19 raspberrypi bash[5361]: #033[0;33mDEBUG Scan result: Tom's iPhone #033[0m
Apr 23 09:48:19 raspberrypi bash[5361]: #033[1;35mpresence/owner/kitchen/CC:xx { confidence : 100, name : Tom's iPhone, scan_duration_ms: 1175, timestamp : Mon Apr 23 2018 09:48:19 GMT-0500 (CDT)} #033[0m

The first log entry at 09:47:21 shows my phone away - which is true. As soon as I saw that message I turned BT back on. The log (last line) shows detection of my phone again at 09:48:19. Almost a minute…slower than what I was seeing late last week.

I currently have 3 owners and 2 guests.

@eboon What do the calculated estimates on script launch show?

Apr 23 09:23:22 raspberrypi systemd[1]: Started Presence service.
Apr 23 09:23:22 raspberrypi bash[5361]: #033[0;32mpresence 0.4.13 #033[0m - Started. Performance predictions based on current settings:
Apr 23 09:23:22 raspberrypi bash[5361]:   > Est. to verify all (3) owners as 'away' from all 'home': 90 seconds to 300 seconds.
Apr 23 09:23:22 raspberrypi bash[5361]:   > Est. to verify one owner is 'away': 30 to 122 seconds.
Apr 23 09:23:22 raspberrypi bash[5361]:   > Est. to recognize one owner is 'home': 0.15 seconds to 13 seconds.
Apr 23 09:23:23 raspberrypi bash[5361]: #033[0;33mDEBUG Scan result: Tom's iPhone #033[0m

@piotr - PiHole seems to be one point of consistency from the users who have reported network slowdowns, although if you’e using an IP address, I’m not sure how PiHole could be causing the problem…

OK - let me just throw this out there - is it possible that the hcitool would be impacting other devices that have BT on that are not specifically specified in owner or guest file?

Yes, I suppose that is possible, although hcitool is only pinging specific addresses. Devices should reject those requests very quickly and without much power/time consumption.

You can test whether BT is the culprit on your network by disabling the presence script and running a command with a bogus MAC address like this:

hcitool name 00:00:00:00:00:00 

The command should run for about 5 seconds before returning either “Device not available” or “”, depending on your hcitool version.

It might be worthwhile to loop this so you can see if this is causing your network slowdown. For example:

while true; do 
hcitool name 00:00:00:00:00:00 
sleep 3
done
1 Like

And my Pi Zero is not using PiHole for DNS.

Thanks. If you have three owners, and two of them are home the script as currently written will still wait for the “delay when present” time between each set of scans. In your case, it seems like the delay is 30 seconds. After scanning for guest devices, which takes about 4 seconds, you have 26 seconds of dead time before the next owner scan interval begins.

The logs seem a bit strange though, since your second guest device is not showing up at all and there’s a 30 second lag between when the appropriate delay is calculated and implemented. This portion of the log is the most interesting bit to me.

Do you see the second device at all?

…same here. PiHole running - but not being used by my Pi running presence.

Thanks again for all your investigation and work to make this tool better - really think this will be a great tool to enhance presence detection for me.

yes - default values in behavior file - so 30 second delay between owner scans when present. Since I redacted the MACs, it’s hard to see, but actually both guests are being scanned - it’s just that it alternates between the two - so:

presence/guest/kitchen/A8:5B:xxx {"confidence":"0","name":"Unknown","scan_duration_ms":"5025","timestamp":"Mon Apr 23 2018 11:12:30 GMT-0500 (CDT)"}
presence/owner/kitchen/CC:08:xxx {"confidence":"100","name":"Tom's iPhone","scan_duration_ms":"1083","timestamp":"Mon Apr 23 2018 11:13:20 GMT-0500 (CDT)"}
presence/owner/kitchen/78:D7:xxx {"confidence":"83","name":"Sue’s iPhone","scan_duration_ms":"5027","timestamp":"Mon Apr 23 2018 11:13:33 GMT-0500 (CDT)"}
presence/owner/kitchen/78:D7:xxx {"confidence":"100","name":"Sue’s iPhone","scan_duration_ms":"4113","timestamp":"Mon Apr 23 2018 11:13:40 GMT-0500 (CDT)"}
presence/owner/kitchen/A8:5B:xxx {"confidence":"100","name":"Abby’s iPhone","scan_duration_ms":"1001","timestamp":"Mon Apr 23 2018 11:13:41 GMT-0500 (CDT)"}
presence/guest/kitchen/78:7E:xxx {"confidence":"0","name":"Unknown","scan_duration_ms":"5024","timestamp":"Mon Apr 23 2018 11:13:47 GMT-0500 (CDT)"}

guests looks like this (redacted):

A8:5B:xxx # J
78:7E:xxx # H

Hope that makes sense.

Gotcha. Looks like a delay calculation issue then. I’ll investigate and hopefully be able to isolate the guest device issue later tonight.

I’ve had no problems with false positives since updating to 0.4.13

Both my wife and I (the 2 phones I monitor) have left and returned with Presence correctly seeing both actions.

I’m not using PiHole. I was noticing issues with my Harmony Hub on Saturday. But I haven’t been seeing the slowdown issues since updating to 0.4.13

I managed to get it installed (at least I think I did) by skipping the command sudo rpi-update. It was odd. If I executed that command, the RPi Zero W would never reboot. I skipped it and proceeded without apparent incident. Just checked the service, and it is active.

Now I am not getting anything on the HA side but hopefully that will change after I post some more “dumb” questions.

1 Like

Good to hear it’s up and running! Odd that rpi-update command failed. Perhaps your board doesn’t like firmware updates.

Anyhow, does the quoted statement mean that you are not seeing any updates on your MQTT broker?