[presence] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

mcfrojd · January 10, 2019, 3:23pm

How do i format this code:

json_attributes:

id
name
rssi
version
manufacturer
type
timestamp
retained
confidence

Now when you need to use “json_attributes_topic:” instead.
The new one don’t seem to take a list.

nass623 · January 11, 2019, 2:40am

I finally got the monitor service up and running and I am able to get the confidence value in home assistant. The only thing I see that seems weird to me (I am new to home assistant) is that the pi zero connects and disconnects multiple times. Seems like the pi zero connects, sends the value and then disconnects. I see this over and over on the mosquitto mqtt log. Is this normal or should I see a single message once it connects? Thanks for working on this, I have tried owntracks and live360 but those use a lot of battery so this will be a much better solution for me.

nass623 · January 12, 2019, 2:57pm

Turns out that changing my password on the mqtt broker configuration in HA fixed the issue. Maybe the fact that i had a “#” as part of the password was messing up the configuration. Anyway, I was able to get it running and integrated with home assistant. So far I am very pleased so thanks again.

andrewjfreyer · January 13, 2019, 8:15pm

Yes, a pound sign would have been treated like a comment.

Thanks for the note!

andrewjfreyer · January 13, 2019, 8:17pm

Mqtt connection and disconnection is normal

PCRx · January 15, 2019, 7:08pm

I’ve got an existing RPi 3 that is running Plex Media Player. 99% of the time it sits idle. Would it be possible to run this on the same pi to make better use of the hardware or does it need to be a standalone setup?

andrewjfreyer · January 15, 2019, 9:02pm

Monitor/Presence only require exclusive access to the bluetooth hardware. Other than that, anything else can run on the same box.

marksev1 · January 16, 2019, 2:18pm

Any ideas on how good this would work on an Orange Pi Zero? Since its easier for me to get a hold of these versus the “original” Raspberry Pi Zero’s.

andrewjfreyer · January 16, 2019, 8:43pm

I don’t think the Orange Pi Zero has built in bluetooth, but this should work just fine with a dongle.

bartjam · January 20, 2019, 2:53pm

Great work Andrew.

Probably i am missing something very basic:

I have added 2 mac adresses in my owner devices like this:

00:00:00:00:00:01 # user1
00:00:00:00:00:02 # user2

I used mqtt.fx to test if the topics where being filled, no problems over there. So far so good.

Then i have added the following in my configuration.yaml

sensor:
platform: mqtt
state_topic: ‘location/office/00:00:00:00:00:01’
value_template: ‘{{ value_json.confidence }}’
unit_of_measurement: ‘%’
name: ‘User1’

sensor:
platform: mqtt
state_topic: ‘location/office/00:00:00:00:00:02’
value_template: ‘{{ value_json.confidence }}’
unit_of_measurement: ‘%’
name: ‘User2’

The problem now is that only user2 is being tracked, what am i missing?

andrewjfreyer · January 20, 2019, 6:10pm

Try running the script from the command line so you can see what’s going on. That might offer some insight into why user one is not being shown

paulmona · February 9, 2019, 3:59am

I had read in the original thread that Bluetooth Low Energy would work as a fall back. I have 4 Pi Zero W around my house, works really well! My wife is however prone to not bringing her phone with her, shes one of THOSE people.

Is there a way to use Bluetooth Low Energy? I grabbed a beacon from Amazon and plan to just throw it in her purse.

andrewjfreyer · February 9, 2019, 6:32am

Move over to monitor - it’s purpose built for that.

paulmona · February 9, 2019, 3:34pm

Doh. I wish I had seen that, thankfully setting up Presence wasn’t that time consuming.

billimek · February 15, 2019, 11:59am

@andrewjfreyer thank you so much for putting this out there and the ongoing active development. This is a really awesome solution to presence IMO.

Two questions:

Are you still using node-red w/this solution, or did that change with the switch to the monitor approach?

If you’re still using node-red, would you be willing to share an export of the flow you’re using? I know you describe in detail what the nodes are doing, but translating that to the same in node-red is still quite a feat (to me at least).

I’m running a setup similar to yours (5 nodes: 1 upstairs, 3 on the main level (including garage), 1 in basement). Right now they are all running with the default arguments. You mention using -tad, -tr, etc depending on which node it is. Is this an important configuration to consider? Should I expect things to be too chatty & cause interference if all 5 nodes are running w/defaults?

Thanks again for this project!

knutz · February 15, 2019, 1:46pm

@andrewjfreyer, Thanks for all your work on this, when i found this I thought “great this is just what I need”. However if you could add a couple of updates to the original post…

First it would have saved me a lot of time if it said that Monitor is a more feature rich development of this project and should be considered first.

Secondly, and this might be more contentious, that there is an issue on the Raspberry Pis using the inbuilt WiFi and Bluetooth concurrently. There is a long post here about the issue:
https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=140471

The general consensus seems to be add a WiFi or BT dongle if you need both working simultaneously.

My own experiences include days of trying to diagnose unreliable WiFi connectivity, the rejection of a Pi Zero W as faulty and much gnashing of teeth. Not only have I found that connectivity to the Pi Zero W is impaired by running both but it also creates issues between other devices on the same access point. Running an iperf3 test between two devices on 2.4 GHz drops to 0 data transferred when a 3rd device (pi) starts a Bluetooth scan when on WiFi. Turning off WiFi on the pi and using a USB Ethernet (wired) dongle and there is NO impact to the iperf3 test. So the Bluetooth scan does not seem to be adversely impacting 2.4GHz WiFi per se.
These tests have been with the latest releases of software/firmware and UniFi AP-AC-Pro Access Point on latest code.
It would be useful if some others experiencing performance issues could corroborate this behaviour.

If this has been covered in one of the 795 replies to this post (i only got through half of them) then it adds to the need for this to be an edit to the original post.

I write this to hopefully help others not criticise an excellent contribution to Hass.
Best Wishes

andrewjfreyer · February 15, 2019, 3:23pm

A while back I switched back to standard automations to change an input boolean. This was when my node-red server was less reliable. I’m now running a node-red container on a much more stable host, and I may switch back, but it works well.

Yes, the tdr/tad/tar/ta options are for “still in the house so we didn’t leave, but we’ll never arrive here” nodes. My third floor node is -tad, so it listens to the behavior of the other nodes only. First, second floor and garage are -tdr, which means they’re listening for arrival advertisements, but will only scan for depart when an mqtt message is sent to monitor/scan/depart.

My front door and my garage door trigger these depart alerts. I do use node red for this, see the monitor thread for more detail.

For five nodes, unless your house is giant, I would set most of your nodes to -tad.

andrewjfreyer · February 15, 2019, 3:25pm

Yes, it’s a known issue. However, mqtt messages sent from montor are so short, it doesn’t much matter or impact performance in my and other user’s testing.

But yes, monitor’s purpose is to reduce interference with 2.4GHz as much as possible by reducing the number of times that a bluetooth scan occurs.

knutz · February 15, 2019, 5:32pm

@andrewjfreyer, Thanks for your fast response acknowledging that it is a known problem with the Pi hardware/firmware.
You say that “it does not much matter or impact performance in your or other user testing” but there are 33 replies to this post mentioning interference and many others citing network performance issues. It also confusing that it is repeatedly referred to in this thread as “Interference” when correctly operating WiFi and BT devices do not have a problem co-existing on 2.4Mhz. I love my Pis but i do think we should call it out when there is a design or implementation problem rather than suggesting it is just the nature of two protocols sharing a frequency.
I applaud your herculean efforts to get round this problem in Monitor and will no doubt try it but I would still urge you to add notes to the original post pointing new Hass users like myself to Monitor and the Pi WiFi/BT problem.
Thanks

andrewjfreyer · February 15, 2019, 11:48pm

There isn’t one problem here, nor is there only one solution. Bluetooth and Wifi are both operating on 2.4GHz bands, and they are known to interfere with one another (quick google search can easily verify). Since Wi-Fi is a higher-throughput protocol, it is less noise tolerant.

Users in this thread have experienced problems with presence causing problems with other wifi systems, unrelated to raspberry pi. It is not just an issue related to Pi.

So, to sum:

• Yes, co-operation of wifi and bluetooth on Pi can cause problems with the pi. Probably could be designed better or probably could upgrade the chipsets to be more noise tolerant.

• Yes, noisy bluetooth devices can cause interference with 2.4GHz Wi-Fi networks in general.

The first is a small problem for users of presence or monitor. The second is a major problem for some users of presence.

That said, for $5 - $10, I’m not going to blame the raspberry pi foundation’s design choices or learning curves They’re providing a great service to the DIY community by getting any hardware out there at this price point.