[monitor] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

@andrewjfreyer edit:UPDATE (The below change fixed the issue)
could it possibly be the same issue as last time with the 3 (line 432)

	if [[ "$known_addr_stated" =~ .*[0-9A-Z]{3}.* ]]; then 
		#SET KNOWN ADDRESS
		known_addr=${known_addr_stated:1}

where as line 293 is

		if [[ "$known_addr_stated" =~ .*[0-9A-Fa-f]{3}.* ]]; then 
			#SET KNOWN ADDRESS
			known_addr=${known_addr_stated:1}

Great catch! Another dumb error on my part. So sorry!

Update: 0.1.666

Major Change Log:

  • README.md typo correction
  • Mac addresses are prepended with prior state information, and this was not properly stripped. Fixed.
  • Other minor bugfixes

Note the creation of the beta branch, if you’d like to test out features before they’re merged with the master branch.

2 Likes

thats for all of the work. took me forever to learn some of these git commands

I need to remember

git stash #couldnt do a pull because I modified monitor.sh
git pull
git stash drop #dropping the old change

1 Like

Same here. For what its worth, you can save one command by simply deleting modified files:

rm monitor.sh
git pull

i was literally going to just ask how can I revert the changes so I did a google search on reverting git changes and that gave me 8000 different ways lol. Thanks for the future help

1 Like

I’m not Git expert but I am getting there :smile:. This is a good tutorial. I find I need to reread it regularly.

In this simple case, the easiest way to reset the local repository back to how it was when you last updated it (i.e. remove all your changes) is to use -

git reset --hard

Note this destroys all changes, new files, the lot and leaves you with a local repository that looks just like it did when you did the last git pull (or the original git clone).

You still then need to pull down any changes so do a

git pull

HTH

1 Like

@andrewjfreyer, in 0.1.660 you changed the following example in the README from type: mean to type: max. What is the advantage of max over mean? I understand the difference between the two, but what is the real-world advantage (or problem you were trying to solve)?

- platform: min_max
  name: "Home Occupancy Confidence of 00:00:00:00:00:00"
    type: mean
  round_digits: 0
  entity_ids:
    - sensor.third_floor
    - sensor.second_floor
    - sensor.first_floor

Thanks!

1 Like

They’re functionally the same, but I think on consideration that max is a better implementation. Indeed, if we have three nodes in a home, only one of which can see your device, mean would output confidence of 33% whereas max would output confidence of 100%.

3 Likes

Previously I worked with room assistant to send mqtt messages from my ibeacon. This was not working for me 100% of the time and is heavy on my mqtt broker so I’m looking into ‘monitor.sh’ right now. I’ve set this up with a raspberry pi zero w but can’t seem to get it working with my ibeacon the way I like it. My phone is working and gives a declining confidence interval just as it should.

When I use ‘monitor.sh -b’ it gives the following mqtt output for my iBeacon every few minutes:
{"id":"E2C56DB5-DFFB-48D2-B060-D0F5A71096E0-0-0", "retained":"false", "version":"0.1.666", "confidence":"100","name":"Undiscoverable Device Name","timestamp":"Wed Sep 26 2018 21:00:57 GMT+0200 (CEST)","manufacturer":"Unknown","type":"APPLE_IBEACON","rssi":"-95","rssi":"-95","power":"-59"}

But also the info of another iBeacon that is near that is not mine. When I add the mac address to ‘known_beacon_addresses’ of the one mentioned above it does not publish any mqtt messages of my beacon (also tried using the UUID but that didn’t work either). It also does not give any declining confidence or even 0 when it’s away from my raspberry.

So story short, I can’t seem to get it working with my iBeacon and I hope someone who has done it before can give me an explanation of how they set it up.

I know the documentation is spotty (sorry), but for iBeacons, you don’t need to add to the known_beacons file. Subscribe to the topic $mqtt_topicpath/$mqtt_publisher_identity/E2C56DB5-DFFB-48D2-B060-D0F5A71096E0-0-0 for status of that iBeacon.

I’m failing to understand when you would use a depart or arrive mqtt scan. I’m monitoring for 4 phones, have contact sensors on my front / back door. How do I know the intent of a device when a door opens, how do I know whether to send a depart or arrive message to monitor?

Thanks so much for this project! Way better then WIFI presence detection :slight_smile:

Edit: Wouldn’t you want just a scan option to trigger because you don’t know if devices are coming or going?

1 Like

If you have multiple monitors some could run arrival and some depart checks at the same time?

With default settings, an arrival scan is triggered by the random, untraceable, advertisement packet transmitted from your phone. Depart scans are triggered by a timeout. Either or both are triggered also by the MQTT triggers you mentioned.

That’s somewhat confusing, I know. I hope it will be more clear as I develop the documentation further, but in answer to your question, I’ll try an analogy:

Your phone is constantly shouting into the Bluetooth environment, “I can connect! I can connect!“ so that everyone nearby can hear. But, your phone isn’t advertising its identity so no one who hears this shout knows who is shouting. A dark room with an unknown voice.

In response to hearing one of these shouts, monitor shouts back asking for each of the phones it believes are absent, “is Rob here? is Katie here? is Andrew here? is Jonathon here?”

In response, your phone (and any of the other three) shouts back, “Yes! Rob is here!” Ok. Great. monitor marks your phone as present.

After a period of time defind in your settings, monitor will ask again, “is Rob still here?”

In response, your phone (and any of the other three still present) shouts back, “Yes! Rob is here!”

In addition, in response to particular MQTT triggers, monitor will ask, “is Rob still here?” (only if your phone is currently marked as present) or “is Rob here?” (only if your phone is currently marked as absent).

Beacons behave a bit differently. Some perriodically shout “Gary is here!” whereas some should “I am Steve” so we have to treat beacons a bit differently. Most wont respond if we ask whether they’re there, so we just have to wait for them to identify themselves.

Random Advertisement = “I can connect!”

Arrival scan = “is Rob here?”

Depart scan = “is Rob still here?”

So, with default settings, arrival and depart scans will be triggered automatically. If you like, you can also trigger arrival scans or departure scans from MQTT.

If you set monitor to trigger arrive and depart scans only on MQTT, the script will ignore random advertisements entirely.

Personally, I have my system set to depart scan only when triggered by my exterior door sensors. This configuration speeds up detection of departure substantially.

Does this analogy makes sense?

7 Likes

Yes it does! Thank you for the detailed explanation :grinning: I am learning so much about bluetooth!

So when a exterior door sensor is opened, trigger a MQTT message depart scan even though someone could be arriving or departing. To speed up departure.

Is there a scenario where is makes sense to trigger a MQTT arrive scan message?

For example, I have an HomeAssistant alarm system I disarm when someone presence is detected. Could I speed up this process by sending a MQTT arrive scan message instead of a depart scan message when a exterior door sensor is opened?

Thinking this through as I type this, I could have automatons for both arrive and depart scan messages with conditions around them depending if people are already present or not. Prioritizing an arrive scan message if no one is home and prioritizing a depart scan message if anyone is home.

1 Like

What you are asking essentially isnt necessary. By default a scan is ran and once a random advertisement is seen it will do an arrive scan on your known devices.

So if the bluetooth is set to ON for your cell phone a random advertisement WILL be sent constantly. The RPI will see this and trigger a name scan that will pick up your phone. If detection is too slow for you that means you may want to get an extra RPI. To be honest each of my RPI’s find my device between 1-10 seconds.

1 Like

@andrewjfreyer question.

#MAX RETRY ATTEMPTS FOR DEPART
PREF_DEPART_SCAN_ATTEMPTS=3

Would this be the order of operations?

  • depart scan sent
  • name scan attempt 1.
  • name scan attempt 2.
  • name scan attempt 3.
  • Confidence 50
  • name scan attempt 1.
  • name scan attempt 2.
  • name scan attempt 3.
  • confidence 25

The formula for confidence drop is this:

100 / 2 ^ $repetition

So for:

PREF_DEPART_SCAN_ATTEMPTS=3
  • Depart scan — start —
  • name scan 1 (fails)
  • report confidence 50
  • name scan 2 (fails)
  • report confidence 25
  • name scan 3 (fails)
  • report confidence 12
    (wait for other devices to finish)
  • report confidence 0
  • Depart scan — end —

@benjimatt I’ve been toying with other formulas as well, so this may be slightly updated in the future. But to your comment, a confidence drop is reported for each repetition.

For example, I’ve been experimenting with:

100/($repetitions^(1/($repetitions-$repetition)))

And have been getting some good results and faster depart scans. The sequence of loss in confidence would follow:

Repetitions Sequence
1 0
2 50, 0
3 57, 33, 0
4 62, 50, 25, 0
5 66, 52, 44, 20, 0
6 69, 64, 55, 40, 17, 0
7 72, 67, 61, 52, 38, 14, 0

so I see you merged to the beta branch. Which repetition are you using? still 3?

1 Like