[monitor] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

Thanks for the quick reply.
Unfortunately it’s not working still.

When I start the service with -b only, it finds the tag MAC and UUID at 100 confidence, but when I take out the battery, the UUID slowly goes down to 0 but the MAC stays at 100.

1 Like

Also, when I start the service with the battery already out, the UUID starts at 0 but the MAC goes straight to 100.

1 Like

This part is normal, for now at least. Many beacons do not broadcast their UUID on every advertisement, so we have to wait until we can find an association between a MAC address and a UUID.

Thanks for the note. I’ll continue testing.

Just to clarify, the UUID works well. It starts at 0 when the tag is without battery or out of range. The MAC goes to 100 confidence no matter if the tag is present, out of rage, in an RF blocking bag or has it’s battery pulled when Monitor is started.

When watching the terminal window, moments after the tag is turned on, Monitor sees it’s MAC and UUID and reports; it just doesn’t ever expire the MAC when the tag is turned off. The UUID does slowly go down after some time.

I can also confirm that the Google Home Device Tracker that take’s the Tag’s MAC (but doesn’t work at all with the 2 iPhones we use here) is working correctly with this tag.

Also, this isn’t a big deal for me as I can easily use just the UUID; just figure you should know that the MAC isn’t working as expected with this tag.

Thanks again!

Appreciate that. It sounds like the MAC address broadcast by your beacon is a public address?

Just a couple things I wanted to give a heads up to. Personally I have stopped using the Availability topics. For ME anytime I would send a restart command the retained value from the MQTT server would not kick in. the sensor would say unavailable and eventually would get values. This messed up a lot of my automations. Essentially kicking them off twice.

Furthermore I have changed my depart script. I dont really want to restart everytime, just when the values arent matching. Here is what I am currently testing? (hopefully this code is correct. ) This is the scripts section.

mqtt_depart:
  sequence:
  - delay: 00:00:10
  - data_template:
      topic: location/scan/depart
    service: mqtt.publish
  - delay: 00:00:20
  - data_template:
      topic: location/scan/depart
    service: mqtt.publish
  - delay: 00:00:30
  - data_template:
      topic: location/scan/depart
    service: mqtt.publish
  - delay: 00:00:40
  - data_template:
      topic: location/scan/depart
    service: mqtt.publish
  - delay: 00:00:20
  - service_template: >
      {% if states.sensor.benji_bedroom.state | float != states.sensor.benji_living_room.state | float %}
        script.mqtt_restart
      {% elif states.sensor.bridget_bedroom.state | float != states.sensor.bridget_living_room.state | float  %}
        script.mqtt_restart
      {% else %}
        script.do_nothing
      {% endif %}

mqtt_restart:
  sequence:
  - data_template:
      topic: location/scan/restart
    service: mqtt.publish
  - delay: 00:00:40
  - service_template: >
      {% if states.sensor.benji_bedroom.state | float != states.sensor.benji_living_room.state | float %}
        script.mqtt_notify_fail
      {% elif states.sensor.bridget_bedroom.state | float != states.sensor.bridget_living_room.state | float  %}
        script.mqtt_notify_fail
      {% else %}
        script.do_nothing
      {% endif %}
      
do_nothing:
  sequence:
  - delay: 00:00:01
  
mqtt_notify_fail:
  sequence:
  - service: notify.benji
    data:
      title: Sensor Mismatch
      message: Your sensors are missmatched after a restart
2 Likes

Thanks for the continued updates to this useful script. I too agree that the HA availability topics are … in need of work.

I have a similar need and am exploring the idea of using both. Room-Assistant doesn’t do phone detection well, but Monitor seems to supplement that. I was debating about using a Bayesian sensor per room. Curious how you progressed with this.

To be honest I could not find an easy way to do that, and I am still looking for a simpler way.

Hey benjimatt,
I like it!

First off, I’m just getting my programming hands wet so I’m not sure if the following is correct but I’ll give it a shot…

I see 1 problem with your script… The reason for having multiple Pi Monitors is to get better coverage in our homes.
For example, in a situation with 2 Monitors and 2 Devices being monitored, Monitor A might see Device A as 100% confidence, while Monitor B Might see Device A at 0% confidence.

So, if the owner of Device B leaves the house while the owner of Device A stays home (while having different confidence percentages when it happens), the mqtt_restart script would fire.

I assume you are doing the restart & mismatch notification because you are running Monitor with the -trd tag and sometimes, after the 4 depart messages, one of your monitor instances will still show >0 confidence even though you’ve left?

Why not just have it restart if any of your devices show confidence that’s neither 0 nor 100?

What do you think of this instead?:

  - service_template: >
      {% if states.sensor.benji_bedroom.state | float in(range(1,100)) %}
        script.mqtt_restart
      {% elif states.sensor.benji_living_room.state | float in(range(1,100)) %}
        script.mqtt_restart
      {% elif states.sensor.bridget_bedroom.state | float in(range(1,100)) %}
        script.mqtt_restart
      {% elif states.sensor.bridget_living_room.state | float in(range(1,100)) %}
        script.mqtt_restart
      {% else %}
        script.do_nothing
      {% endif %}

It just checks if any of your sensors confidence level is between 0 & 100.
That should work no?
I know that it doesn’t account for if one of Device B’s 2 sensors (device still home) has a confidence between 0 & 100; but usually, confidence goes from 0 to 100 or back pretty fast.

Actually… Now that I wrote that all, I’m curious as to why you want to restart the script. Is it because of the -trd tag? And that it gets stuck at a confidence between 0 & 100 when you’re leaving sometimes?
If that’s the case, why not have more delays and repeats?

Anyways, let me know if the above code works for you.

2 Likes

Thanks for the suggestion, couple things, first off my python and jinja2 are pretty poor lol and I basically just started using templating to try to condense my scripts and automations. I basically only code in .Net or Powershell.

Realistically I use two Pi Zero’s for accuracy not range. 1 Pi can basically reach my entire apartment. The problem is that I don’t want some of my automations going off in case of the sensor either failing OR falling in value. This is actually the reason I am using –TRD. By default this script will do its own depart to see if the user has left the area or not (I forget the interval). This doesn’t work for me because my GF lets her devices die sometimes and she has no desire to change the way she lives lol.

I let Andrew know this and he added triggers for depart and arrive. So the only time my depart sequence triggers is if the device restarts OR I sent a depart trigger. For my house when the door opens I send a script.turn_off to the Mqtt_depart script and when the door closes the script starts. I did this part just in case im toward the end of the script and I walk back in the house. It cancels and then starts again.

You are correct about the Values, the issue is that ive been with this project for a long time and it has become better and better with each Beta release. There was a time where my values would get stuck and the only way to recover was with a restart. Andrew eventually implemented the location/scan/restart which allowed me to restart all nodes with one Mqtt topic!!

Ok back to the mqtt_depart script. Where I live is pretty close to the elevator and for the most part I wait maybe 60 seconds. Sometimes I wait much longer (this is the reason for so many delays). My door is really really close to the elevator so the Monitor instances still pick me up when I am at outside of the apartment. You must remember that these instances count down from 100. So if I am toward the end of the depart script and im further from one sensor than the other , one may have me at 100 whereas the other has me at 0 but I may still be at the elevator. Either way Yes if I want to simplify it I could technically go off of the Occupancy sensor that I have which is the MEAN of both sensors. Unfortunately I have had 2 Pi’s fail and it was probably my crappy sd cards (who knows). This makes my values stay at 50 percent and essentially never get to 100 for the Mean of both sensors. Maybe based on your suggestion I can change it to

  - service_template: >
      {% if states.sensor.benji_bedroom.state | float in(range(1,100)) or states.sensor.benji_living_room.state | float in(range(1,100)) %}
        script.mqtt_restart
      {% elif states.sensor.bridget_bedroom.state | float in(range(1,100)) or states.sensor.bridget_living_room.state | float in(range(1,100)) %}
        script.mqtt_restart
      {% else %}
        script.do_nothing
      {% endif %}

(actually I just made the change, that honestly makes more sense thanks) .

just in case you are curious (its nothing special and far from all my files but may give you an idea of how im doing everything)

Ah yes. As I was writing it, I was starting to realize that the time was the reason you did this.
I actually use a version of your script from reading this thread (I also am very close to the elevator in my building) so thank very much for that!

Question (or suggestion) though… why not add a few more depart & delay in your mqtt_depart script? Just to make sure that you’re out for sure? Then you can add the availability topics back in since you’ll be restarting less.

1 Like

test the availably topics and you will see why I dont want to use them anymore lol.

maybe if I change my automations a bit it would work.

right now I have a Benjis_home script and a benjis_not_home. When the availability topic is offline HASS will not look at the retained value on the MQTT server. No fault of andrew I think its HA that needs to change.

So I would restart and once the sensor came back it would activate the automation again :frowning:
its possible I was doing something wrong. But test it…maybe I just missed something

1 Like

Can this project detect my car’s Bluetooth?

1 Like

wow, i did it. read through every freaking post in this thread.

seems i’m in the group of people who’s phones (android, LG G7 ThinQ) don’t send out random BLE advertisements. I’ve tried pairing with a bluetooth speaker which didn’t work. My phone has also been paired with a FitBit and my truck. Next step is to try pairing with a BLE tracker, I have one for my dog that is out of batteries.

Since it was suggested in about 100 posts in this thread, I’ve switched to the beta branch. But now it seems that sending an mqtt message to monitor/scan/ARRIVE doesn’t do anything, and it worked before.

The behavior_preferences file in this beta is considerably different than before (and the documentation)

# ---------------------------
#
# BEHAVIOR PREFERENCES
#
# ---------------------------

#MAX RETRY ATTEMPTS FOR ARRIVAL
PREF_ARRIVAL_SCAN_ATTEMPTS=1

#MAX RETRY ATTEMPTS FOR DEPART
PREF_DEPART_SCAN_ATTEMPTS=2

#SECONDS UNTIL A BEACON IS CONSIDERED EXPIRED
PREF_BEACON_EXPIRATION=180

#MINIMUM TIME BEWTEEN THE SAME TYPE OF SCAN (ARRIVE SCAN, DEPART SCAN)
PREF_MINIMUM_TIME_BETWEEN_SCANS=15

#ARRIVE TRIGGER FILTER(S)
PREF_PASS_FILTER_ADV_FLAGS_ARRIVE="0x1a"
PREF_PASS_FILTER_MANUFACTURER_ARRIVE="Apple"

#ARRIVE TRIGGER NEGATIVE FILTER(S)
PREF_FAIL_FILTER_ADV_FLAGS_ARRIVE="NONE"
PREF_FAIL_FILTER_MANUFACTURER_ARRIVE="Google|Samsung"

I noticed the one post above about playing with the filters but I really didn’t understand it too well. Because I’m on an android should I be removing “Google” from the Pref_fail and move it to the pass?

Should I be adding in those previous behavior_pref fields that are documented?

edit: think I was having an issue with MQTT. Did a reboot of my pi and it’s responding to MQTT arrive message now. Still looking for feedback on my other questions about the behavior_pref file
and i’m getting this error in my logs, anything to be concerned with?

./support/btle: line 204: pdu_type: command not found

and another edit since my phone isn’t broadcasting randomly. I should try and trigger the arrive scan manually correct? The depart scan won’t do anything if the phone isn’t above 0 confidence correct?

A lot of cars turn their bluetooth radios off when the ignition is off - the answer will depend on your car.

You probably caught this version between updates that I pushed. I’d recommend updating again. Current beta version is 0.1.948 (~430 commits ahead of master).

For your testing purposes with non-Apple devices (docs will be updated on beta eventually… sorry), replace the PASS filters with .* and replace each FAIL filter with NONE or some other garbage non-null string.

Then run from the command line and watch how your phone presents itself as a random advertisement. Some phones will say “Google” while others say “unknown” or another chipset manufacturer. When in doubt on the filters, use .* to match anything.

ok updated

so if my phones are reporting manufacturer as “Unknown” and “Samsung Electronics CoLtd” I should add those to the pass regex as “Unknown|Samsung” ?

what’s adv_flags ?
edit: nevermind, i see you’ve updated the documentation for beta. i’ll read that first.

1 Like