[monitor] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

[Caswell1000]

Hi, thanks for a great product/service, I am still on presence and want to upgrade to monitor, should I wipe the SD card clean or should I just install on the same card, how do I deactivate presence in that case.

[casperse]

Something has changed during the last upgrade of HA…
I just cut power to the monitor and rebooted HA and all my devices still show 100%?

  - platform: mqtt
    state_topic: 'monitor/bt_garage/bt_henrik_iphone'
    value_template: '{{ value_json.confidence }}'
    unit_of_measurement: '%'
    name: 'bt_garage_henrik_iphone'

is it the mqtt % value?

[duceduc]

Anyone monitor their rpi zero W? I noticed my processing usage is pretty high. Only using the pi for presence detection, 16GB SD Card, and have 4 devices inputted in the known file.

[Eeeeeediot]

I am also getting a lot of disconnections - don’t remember seeing these before. Running 2021.2.2

1613236626: Client living_room_monitor1440 disconnected.
1613236626: New connection from 192.168.0.226 on port 1883.
1613236626: New connection from 192.168.0.226 on port 1883.
1613236626: New client connected from 192.168.0.226 as living_room_monitor1485 (p2, c1, k60, u'Stephen').
1613236626: New client connected from 192.168.0.226 as living_room_monitor1487 (p2, c1, k60, u'Stephen').
1613236627: Client living_room_monitor1485 disconnected.
1613236627: Client living_room_monitor1487 disconnected.
1613236627: New connection from 192.168.0.226 on port 1883.
1613236627: New connection from 192.168.0.226 on port 1883.
1613236627: New client connected from 192.168.0.226 as living_room_monitor1506 (p2, c1, k60, u'Stephen').
1613236627: New client connected from 192.168.0.226 as living_room_monitor1503 (p2, c1, k60, u'Stephen').
1613236627: Client living_room_monitor1506 disconnected.
1613236627: Client living_room_monitor1503 disconnected.
1613236627: New connection from 192.168.0.226 on port 1883.
1613236627: New client connected from 192.168.0.226 as living_room_monitor1532 (p2, c1, k60, u'Stephen').
1613236627: Client living_room_monitor1532 disconnected.
1613236627: New connection from 192.168.0.226 on port 1883.
1613236627: New client connected from 192.168.0.226 as living_room_monitor1547 (p2, c1, k60, u'Stephen').
1613236627: Client living_room_monitor1547 disconnected.
1613236627: New connection from 192.168.0.226 on port 1883.
1613236627: New client connected from 192.168.0.226 as living_room_monitor1556 (p2, c1, k60, u'Stephen').
1613236627: Client living_room_monitor1556 disconnected.
1613236627: New connection from 192.168.0.226 on port 1883.
1613236627: New client connected from 192.168.0.226 as living_room_monitor1559 (p2, c1, k60, u'Stephen').
1613236628: Client living_room_monitor1559 disconnected.
1613236629: New connection from 192.168.0.226 on port 1883.
1613236629: New client connected from 192.168.0.226 as living_room_monitor1651 (p2, c1, k60, u'Stephen').
1613236629: Client living_room_monitor1651 disconnected.
1613236629: New connection from 192.168.0.226 on port 1883.
1613236629: New client connected from 192.168.0.226 as living_room_monitor1652 (p2, c1, k60, u'Stephen').
1613236629: Client living_room_monitor1652 disconnected.
1613236631: New connection from 192.168.0.226 on port 1883.
1613236631: New client connected from 192.168.0.226 as living_room_monitor1771 (p2, c1, k60, u'Stephen').
1613236631: Client living_room_monitor1771 disconnected.
1613236631: New connection from 192.168.0.226 on port 1883.
1613236631: New client connected from 192.168.0.226 as living_room_monitor1773 (p2, c1, k60, u'Stephen').
1613236631: Client living_room_monitor1773 disconnected.
1613236632: New connection from 192.168.0.226 on port 1883.
1613236632: New client connected from 192.168.0.226 as living_room_monitor1891 (p2, c1, k60, u'Stephen').
1613236633: Client living_room_monitor1891 disconnected.
1613236633: New connection from 192.168.0.226 on port 1883.
1613236633: New client connected from 192.168.0.226 as living_room_monitor1916 (p2, c1, k60, u'Stephen').
1613236633: Client living_room_monitor1916 disconnected.
1613236633: New connection from 192.168.0.226 on port 1883.
1613236633: New client connected from 192.168.0.226 as living_room_monitor1932 (p2, c1, k60, u'Stephen').
1613236633: Client living_room_monitor1932 disconnected.
1613236633: New connection from 192.168.0.226 on port 1883.
1613236633: New client connected from 192.168.0.226 as living_room_monitor1942 (p2, c1, k60, u'Stephen').
1613236633: Client living_room_monitor1942 disconnected.
1613236633: New connection from 192.168.0.226 on port 1883.
1613236633: New client connected from 192.168.0.226 as living_room_monitor1969 (p2, c1, k60, u'Stephen').
1613236633: Client living_room_monitor1969 disconnected.
1613236634: New connection from 192.168.0.226 on port 1883.
1613236634: New client connected from 192.168.0.226 as living_room_monitor1980 (p2, c1, k60, u'Stephen').
1613236634: Client living_room_monitor1980 disconnected.
1613236634: New connection from 192.168.0.226 on port 1883.
1613236634: New client connected from 192.168.0.226 as living_room_monitor1990 (p2, c1, k60, u'Stephen').
1613236634: Client living_room_monitor1990 disconnected.
1613236634: New connection from 192.168.0.226 on port 1883.
1613236634: New client connected from 192.168.0.226 as living_room_monitor1992 (p2, c1, k60, u'Stephen').
1613236634: Client living_room_monitor1992 disconnected.
1613236637: New connection from 192.168.0.226 on port 1883.
1613236637: New client connected from 192.168.0.226 as living_room_monitor2188 (p2, c1, k60, u'Stephen').
1613236637: Client living_room_monitor2188 disconnected.
1613236637: New connection from 192.168.0.226 on port 1883.
1613236637: New client connected from 192.168.0.226 as living_room_monitor2196 (p2, c1, k60, u'Stephen').
1613236637: Client living_room_monitor2196 disconnected.
1613236638: New connection from 192.168.0.226 on port 1883.
1613236638: New client connected from 192.168.0.226 as living_room_monitor2237 (p2, c1, k60, u'Stephen').
1613236638: Client living_room_monitor2237 disconnected.
1613236638: New connection from 192.168.0.226 on port 1883.
1613236638: New client connected from 192.168.0.226 as living_room_monitor2238 (p2, c1, k60, u'Stephen').
1613236638: Client living_room_monitor2238 disconnected.
1613236639: New connection from 192.168.0.226 on port 1883.
1613236639: New client connected from 192.168.0.226 as living_room_monitor2322 (p2, c1, k60, u'Stephen').
1613236639: Client living_room_monitor2322 disconnected.
1613236639: New connection from 192.168.0.226 on port 1883.
1613236639: New client connected from 192.168.0.226 as living_room_monitor2324 (p2, c1, k60, u'Stephen').
1613236639: Client living_room_monitor2324 disconnected.
1613236640: New connection from 192.168.0.226 on port 1883.
1613236640: New client connected from 192.168.0.226 as living_room_monitor2367 (p2, c1, k60, u'Stephen').
1613236640: Client living_room_monitor2367 disconnected.
1613236664: New connection from 192.168.0.226 on port 1883.
1613236664: New client connected from 192.168.0.226 as living_room_monitor2498 (p2, c1, k60, u'Stephen').
1613236664: Client living_room_monitor2498 disconnected.
1613236683: New connection from 192.168.0.226 on port 1883.
1613236683: New client connected from 192.168.0.226 as living_room_monitor2924 (p2, c1, k60, u'Stephen').
1613236683: Client living_room_monitor2924 disconnected.
1613236683: New connection from 192.168.0.226 on port 1883.
1613236683: New client connected from 192.168.0.226 as living_room_monitor2945 (p2, c1, k60, u'Stephen').
1613236683: Client living_room_monitor2945 disconnected.
1613236685: New connection from 192.168.0.226 on port 1883.
1613236685: New client connected from 192.168.0.226 as living_room_monitor3199 (p2, c1, k60, u'Stephen').
1613236686: Client living_room_monitor3199 disconnected.
1613236686: New connection from 192.168.0.226 on port 1883.
1613236686: New client connected from 192.168.0.226 as living_room_monitor3201 (p2, c1, k60, u'Stephen').
1613236686: Client living_room_monitor3201 disconnected.
1613236694: New connection from 192.168.0.226 on port 1883.
1613236694: New client connected from 192.168.0.226 as living_room_monitor3532 (p2, c1, k60, u'Stephen').
1613236694: Client living_room_monitor3532 disconnected.

[phairplay]

Is this confirmed that the latest update breaks the MQTT connection?

[Tinkerer (aka DubhAd on GitHub)]

Latest update of what?

I’ve got no issues on HA 2021.2.3

[bachoo786]

Quick question I know it’s a long thread and this might be already answered but my question is can monitor work like room assistant? It’s only because I am using an android and room assistant doesn’t work with android yet I believe. Thanks.

[Tinkerer (aka DubhAd on GitHub)]

No, it’s not designed to tell you which room.

However, RA works with any Bluetooth device… it doesn’t not work with Android.

[phairplay]

the comment above mine and a few previously to that mention issue with connections after updating to the recent release.

[VilleL (Ville Lumikero)]

I’m having problems getting monitor to work after both my wife and I got ourselves new iPhones. I’ve replaced the old BT MAC addresses in known_static_addresses, but they aren’t detected at all. I get a blank when running hcitool name 00:MAC:00.

The addresses have been copied from Settings/General/About. I’ve also made sure to pair the new phones with other BT devices as has been advised in this thread.

Any ideas on how to troubleshoot?

[doerakje (Luca)]

It’s said that stupid questions do not exist, but I might have found one…

I have three pi’s: at the frontdoor, at the backdoor and at the second floor. All three of them are running fine, messages are visible as expected in MQTT explorer. So far so good, thanks to the well written install guide on https://github.com/andrewjfreyer/monitor.

I want my frontdoor Pi to monitor with the flags -tr -b and the other pi’s with the flags -tad, just like in the ‘example with home assistant’ on the page mentioned above.

Here’s the question, since I’m not a debian genius:
How or where do I set these flags? My goal is that whenever I reboot my pi monitor will automagically start with the desired flags. Have read quite a good part of this topic, but didn’t find the answer. Probably because it’s a stupid question…

[paddy0174 (Patrick)]

Funny enough, I checked the github page, as I was sure, the options are listed there, but I couldn’t find them.

Anyway, the script does show the commands, if you call it with option -h.

pi@pihallway:~/monitor $ sudo bash monitor.sh -h
> starting monitor.sh (v. 0.2.200)...


monitor.sh

Andrew J Freyer, 2018
https://github.com/andrewjfreyer/monitor

GNU General Public License

usage:

        monitor -h      show usage information
        monitor -R      redact private information from logs
        monitor -S      silent operation (no logging)
        monitor -c addr create connection to bluetooth device
        monitor -C      clean retained messages from MQTT broker

        monitor -V      print verbose/debug logging messages
        monitor -v      print version number
        monitor -d      restore to default behavior_preferences
        monitor -u      update 'monitor.service' to current command line settings
                        (excluding -u,-V, -F, and -d flags)

        monitor -r      repeatedly scan for arrival & departure of known devices
        monitor -s  report all mqtt messages to a single topic with
                                $mqtt_topicpath/$mqtt_publisher_identity  (defined in MQTT preferences file)

        monitor -f      format MQTT topics with only letters and numbers
        monitor -a      report all known device scan results, not just changes
        monitor -x      retain mqtt status messages
        monitor -b      report bluetooth beacon advertisements (e.g., generic beacons, ibeacons, and so on)
        monitor -t[adr] scan for known devices only on mqtt trigger messages:
                                a $mqtt_topicpath/scan/ARRIVE (defined in MQTT preferences file)
                                d $mqtt_topicpath/scan/DEPART (defined in MQTT preferences file)
                                r send ARRIVE or DEPART messages to trigger other devices to scan
        monitor -D [dir]        use alternative directory for configuration files

As you can see, there is an option to save the used parameters, -u specifically.

So, what you want to do is as follows:

• log in with SSH to the corresponding Pi
• go into the monitor folder (cd monitor/)
• call monitor.sh with the parameters/options you want and add the -u option, like so

sudo bash monitor.sh -tad -u

monitor.sh will give you a confirmation message, that monitor.service has been updated. Repeat for all Pis you have running.

You can always change that by calling the script again, choosing different options.

[doerakje (Luca)]

Thank God the question wasn’t that stupid. I checked monitor -h, but missed the monitor -u flag.
Thanks!

[Djal]

Hi, how can i force an arrival scan from HA. becasue actually it could take up to 4mn to detetc presence (iPhone or BLE devices) ?

run monitor on PI3

Thxs

[paddy0174 (Patrick)]

@paddy0174

monitor -t[adr] scan for known devices only on mqtt trigger messages:
      a $mqtt_topicpath/scan/ARRIVE (defined in MQTT preferences file)
      d $mqtt_topicpath/scan/DEPART (defined in MQTT preferences file)
      r send ARRIVE or DEPART messages to trigger other devices to scan

Try with these options.

EDIT: Or here in the documentation: GitHub - andrewjfreyer/monitor: Distributed advertisement-based BTLE presence detection reported via mqtt

[Mikefila]

I made a nodered flow that is similar to Odianosen25 home presence app if anyone is interested the details are in this thread.

Hands free access for existing bell/buzzer and intercom systems. Using bluetooth, monitor.sh, and a modified home presense app in node red

I wanted to add hands free access my existing old school bell/buzzer system. I also wanted something that didn’t require additional apps or access to HA in general. Andrew J Frey’s monitor app only requires adding a users bluetooth mac. Access generally takes 7-8 seconds. I mirrored Odianosen25 home presence app except the device tracker as this application is a little different. I am using 2 nodes, each ~50’/15m one beyond and one off to the side and above the entry point. The idea is to locat…

[duceduc]

Anyone have added the Tile Mate and does it work?

[poudenes]

Tried it. But HA didn’t pick up the tile soon enough for me to make people presence at home.
When its dark outside and we came home it took 1-3 minutes before we went on in HA as HOME.

I use Geofency and Locative with a radius around my house to become HOME or AWAY.
Work for 3 years now without any problem. When we come home in the dark, the lights are already on

[bustinthejus]

Is anyone having issues with Monitor oscillating between 0, 45, and 90 for about 5 minutes after a departure before it finally settles on 0? It detects arrival pretty consistently, but for some reason it has problems with the departure. I’m just using it with the -r flag, and it seems to register “0” pretty quickly, but it takes a while to settle. I’m thinking about picking up a door sensor to trigger arrival and departure scans, but if it’s fluctuating even when I’m no where within range, I’m not sure if that would help.

[JohnWH (John W)]

First of all many thanks to Andrew J Freyer for creating such a brilliant solution for presence detection. As a newbie to HA, and after many disappointments with GPS tracking of iPhones, I bought a Raspberry Pi zero and have been absolutely delighted with the reliability of [monitor].

In order to reduce the time to detect changes (mainly when leaving home) I would like to set up a trigger to run a ‘depart’ scan. Using MQTT Explorer I can successfully run, and see immediate results, when I publish commands such as monitor/KNOWN DEVICE STATES or monitor/scan/restart however publishing either a monitor/scan/arrive or monitor/scan/depart command doesn’t seem to be working.

One other (again probably a newbie) question is can the output from [monitor] be ‘converted’ into device_tracker format to allow it to be selected as one of the options for user tracking in HA?