[monitor] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

I know @andrewjfreyer is a clever guy, so yeah, if it had been that easy, then I am sure there would have been a flag/setting somewhere to do it :smiley:

I went through the pastbin log I had posted earlier, and added every single MAC from it to the ignore list, was 25 or so, so it should have covered anything that was static in the house causing it. I also changed the setting you suggested above, Not sure what it really changes though,

and since then, I havenā€™t had a tablet disconnect yet, but its early days. I still have all 5 mac address in the known file, I have been constantly pinging them from my windows machine, and they do obviously lag a little when it is doing an arrival scan. I havenā€™t setup a trigger yet on my door for departure scanā€™s yet, kinda wanted to and get the interference down to a minimum first, which is seems to be slowly getting there, before I setup the departure side.

My tablets are only connected via 2.4ghz, they are a Ā£30 no name android tablet with a IEEE 802.11b/g/n chipset only. I just use them as exampleā€™s of the interference, as itā€™s easy to screengrab the history of them. Unfortunately I also have Sonoffā€™s around the house that are 2.4ghz only I believe, the Xiaomi is also in my collection, and both of those setā€™s of devices also have problems with the interference.

If I could swap everything over to 5Ghz, it would solve a lot of problems.

Iā€™ll monitor how it goes from now, and see what happens.

I seem to be running into an issue with iBeacons.
I have 2 rpi 0Ws at each end of the house, one master and one slave running "monitor.sh -m -r -b -x " and ā€œmonitor.sh -m -b -x -tā€ respectively.
The issue Iā€™m having is that I can detect iBeacons, but after pulling the batteries/switching them off, I cannot get confidence in either instance of monitor to drop to zero. All iBeacon instances in the log report at 100 confidence, and I have yet to see any expire and set to 0. My script preferences are all at the default too.
Am I doing something wrong with the script arguments?

Thanks for the docker image. I have running on my libreelec device.

I do see a error and a warning in the log:

./monitor.sh: line 70: hciconfig: command not found

Warning: Recommended package ā€˜l2pingā€™ not found. Please consider installing ā€˜l2pingā€™.

Also very interesting in this, anyone have tried to monitor those fitness bands?

Rodon, I see the same behaviour with two Kontakt iBeacons. Only way Iā€™ve had them working is by adding them to the known_static_addresses file and pickup the MQTT by MAC address in HASS. Think the problem is that monitor.sh is reporting the 100% confidence on the iBeacon UUID address mqtt topic and everything else on MAC address mqtt topic.

@phdelodder

Iā€™ve just uploaded v.0.0.6 which should fix l2ping, hciconfig and xxd errors.

Thanks for the fast update!

Donā€™t know if you fixed this,but otherwise you should check if she got any device paired with her iPhone. I think I heard about the same thing happening, and it was caused by no paired device on iPhone, then it dosnā€™T send out messages

Thanks, no I never did find a solution. So you are saying that if she has never paired a device before, then it wonā€™t work properly? Can it be any device?

The concept on which monitor is built is brilliant. Kudos to @andrewjfreyer!

Iā€™m pretty new to Home Assistant (<10 days) and even newer to monitor (3 days). I may, therefore, be asking the dumbest question. Iā€™ve searched for the answer but so far without luck, so Iā€™ll ask anyway.

Iā€™ve been able to set up monitor so that it publishes cell phone presence via MQTT to Home Assistant. However, since Iā€™m building up my HA system, I restart it regularly and Iā€™ve noticed that the MQTT sensor in HA is ā€œunknownā€ after HA restart. It changes to ā€œpresentā€ or ā€œnot presentā€ if when the RPi detects a change (if the phone arrives or departs), but not until that happens. Iā€™ve tried to ā€œforceā€ detection by publishing ARRIVE or DEPART to MQTT to trigger monitor to scan, but it appears that monitor does not publish anything to MQTT until it detects a change (from ā€œpresentā€ to ā€œnot presentā€ or vice versa).

Restarting monitor.service after restarting HA does the trick, of course, but thatā€™s obviously cheatingā€¦

Iā€™m confident that thereā€™s a trick Iā€™m missing and would be grateful if someone could point me in the right direction.

Iā€™m a really noob too so donā€™t know if this would fix your problem, but maybe
@benjimattā€™s answer, running location/scan/restart above can help you on the way.

1 Like

not a trick just a setting.

Im not sure which one you need but I would try this

  monitor -a  report all known device scan results, not just changes
  monitor -x  retain mqtt status messages

so for you

sudo monitor.sh -a -x

In the past 24 hours the two devices I have running monitor have made 4,530 requests to api.macvendors.com. Is this really necessary? Do I have something configured incorrectly to be hammering this api so frequently? Thatā€™s an average of 3.14 requests every minute from the two devices.

Anyone else seeing this behavior?

2 Likes

so it looks like there is a section of @andrewjfreyer code that is called
ā€œdetermine_manufacturer ()ā€

	#IF CACHE DOES NOT EXIST, USE MACVENDORS.COM
	if [ -z "$manufacturer" ]; then 
		local remote_result=$(curl -sL https://api.macvendors.com/${address:0:8} | grep -vi "error")

You could probably ommit this in the code if you wanted. But if you want a quicker solution you could make a static route on your router that redirects requests going to https://api.macvendors.com* to 127.0.0.1. its up to you.

Why would you do that? The code does something that the author intended. What assessment have you made of stopping this behaviour?

i suppose it could be an issue for beacons ā€¦ if you read the monitor.sh file it looks like if it cannot get a manufacturer it will report it as unknown. This is happening when the message is published to the MQTT server. I mean personally it would affect any name scans that I am doing. This isnt something id ā€œrecommendā€ but for people that really want to control their bandwidth it doesnt seem like it will affect muchā€¦

Thank you, @fillilutten and @benjimatt. I knew it would be something easy.

I just tested a restart of HA and then published to location/scan/restart and that did the trick. I will investigate the effects of monitor -a and monitor -x as well.

1 Like

Just popped in to say thanks @andrewjfreyer. I had been monitoring (no pun intended) this thread for awhile but not being great in bash hadnā€™t tried using monitor yet. Your readme.md is pretty comprehensive though - and I was able to start tracking our cell phones very quickly. A few days of comparing state changes fand I should be able to (finally) uninstall GPSlogger from my phone!

1 Like

Iā€™m pulling my hair off, trying to figure out the usage of triggered scans, especially for arrival.
Iā€™ve read almost the entire thread, but was only able to find something about the use case for triggered departure.

Iā€™m getting the point of using the trigger for departure (-td), which has a number of advantages, less 2.4G activity, no fail drops and so on.

But I canā€™t figure out why I would use triggered scans for arrival scanning. Iā€™m having 2 RPI, one will be configured as the master, and one as the slave.

If I let the master to take action on announcement, but the slave to only run arrival scans on trigger, what will the case be if Iā€™m not visible for the slave when I enter my house, then the slave will never see that Iā€™m home?

Iā€™m just trying to figure out the best working case for me. Iā€™m living in a one floor house, with one RPI in each end of the house.

Iā€™m thinking about running both the slave and master with the option -td, and triggering departure by the doorsensors. But if any other options could be better Iā€™m all ears.

What in behavior preferences do I change to make my arrival speed up? PREF_ARRIVE_SCAN_INTERVAL=45? Would this make any change even if iā€™m running in -td mode? Donā€™t know what they refer to with ā€œperiodic scanningā€ Itā€™s fairly quick now but want to try and make it even quicker if possible.

And one more thing - Can you setup monitor so it only runs really quick arrival scans when known_devices are 0. In my case all the things I want to track are in there, since I dont run any generic beacons. And that would able me to run more arrival scans when not home, but when home having monitor not do anything + not cause any wifi interference and just waiting for mqtt departure?