[monitor] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

Default is four repetitions for depart still.

Thank you for awesome script, have tried ESP32, room,etc but nothing worked.
Been using ā€œmonitorā€ for a few days and is working great for iphone, some old motorola and samsung phone.
Having trouble with NUT mini. I tried put MAC in known_static_addresses and known_beacon_addresses but getting confidence 0 all the time.

Do you have some ideas how to set up for NUT?

If I try like this:
sudo bash monitor.sh -b -g

0.1.667 11:41:03 am [CMD-RSSI] FE:BC:D2:06:69:1F Undiscoverable Device Name RAND RSSI: -40 dBm (Moderate Motion Departing, changed -40)
0.1.667 11:41:03 am [CMD-RAND] FE:BC:D2:06:69:1F ADV_IND -40 dBm
0.1.667 11:41:04 am [CMD-SCAN] (No. 1) fe:bc:d2:06:69:1f arrival?
0.1.667 11:41:09 am monitor/pizerow/fe:bc:d2:06:69:1f
{
retain: false
version : 0.1.667
address : fe:bc:d2:06:69:1f
confidence : 0
name : nut
timestamp : Sun Sep 30 2018 11:41:09 GMT+0000 (UTC)
manufacturer : Unknown
type : KNOWN_MAC
}
0.1.667 11:41:10 am [CMD-NAME] fe:bc:d2:06:69:1f nut Unknown

Any ideas?

Thank you

1 Like

Try running with -g Without putting the MAC address in either the known addresses or known beacons file.

How can then define this in HA if there is no MAC to put in state topic under mqtt?

With -g option this are only logs about NUT:

0.1.667 05:03:45 pm [CMD-RSSI] FE:BC:D2:06:69:1F Undiscoverable Device Name RAND RSSI: -46 dBm (Slow/No Motion, changed 16)
0.1.667 05:04:14 pm [CMD-RSSI] FE:BC:D2:06:69:1F Undiscoverable Device Name RAND RSSI: -76 dBm (Moderate Motion Departing, changed -30)
0.1.667 05:04:19 pm [CMD-RSSI] FE:BC:D2:06:69:1F Undiscoverable Device Name RAND RSSI: -61 dBm (Slow/No Motion, changed 15)

Maybe NUT isnā€™t compatible and should try some other trackers? Do you have some working tags that you can recomend?

Thanx

1 Like

Try the mac address in lowercase.

1 Like

They are in lover-case (in known_static_addresses file).
Only in monitor logs MAC are uper-case.

Regards

2 Likes

Did that status option remove this capability? Iā€™m not seeing location/kitchen/status any moreā€¦

1 Like

Status is still being sent, but if you have default settings, itā€™ll post to monitor/first floor/status

Are there options or flags to run the monitor service to track only static addresses and not waste cycles or scans on beacon type devices?

1 Like

With default settings, monitor will only track static devices (cell phones, laptops, etc.), but this does use ā€˜cyclesā€™ (as it were) to listen to the bluetooth environment.

Andrew, absolutely a great script, but I have some difficulties with my Kontakt iBeacons. By adding -b I correctly get a mqtt message as below, which I can pickup in HASS on ā€˜location/bryggers/F7826DA6-4FA2-4E98-8024-BC5B71E0893E-15557-20097ā€™ ā€¦ But, when I remove the battery from the iBeacon to simulate it leaving home, I never get more updates on ā€˜location/bryggers/F7826DA6-4FA2-4E98-8024-BC5B71E0893E-15557-20097ā€™.

Iā€™ve tried playing around with -a and -g as well, but cannot spot a different result. On a sidenote it seems like updates on its Bluetooth mac happens if I add it to the known_static_devices file.

Any clue ? :slight_smile: THANKS!

0.1.667 05:34:36 pm location/bryggers/F7826DA6-4FA2-4E98-8024-BC5B71E0893E-15557-20097
{
retain: false
version : 0.1.667
address : F7826DA6-4FA2-4E98-8024-BC5B71E0893E-15557-20097
confidence : 100
name : Kontakt
timestamp : Tue Oct 02 2018 17:34:35 GMT+0200 (CEST)
manufacturer : Unknown
type : APPLE_IBEACON
rssi : -58
power : -77
}

@andrewjfreyer

im on the beta build right now and I get an error that makes Monitor restart

ā€œOct 02 21:16:53 blue0 bash[5382]: /home/pi/monitor/monitor.sh: line 871: 4A: value too great for base (error token is ā€œ4Aā€)
Oct 02 21:16:53 blue0 bash[5382]: 0.1.669 09:16:53 pm [CHECK-DEL] 4A:76:08:F5:24:2E expired after 62 seconds
Oct 02 21:17:03 blue0 systemd[1]: monitor.service: Service hold-off time over, scheduling restart.
Oct 02 21:17:03 blue0 systemd[1]: Stopped Monitor Service.
Oct 02 21:17:03 blue0 systemd[1]: Started Monitor Service.
Oct 02 21:17:03 blue0 bash[13206]: Starting monitor.sh (v. 0.1.669)ā€¦ā€

Im assuming this is because of the Log redaction you made

			line=$(echo "$line" | sed "s/\([0-9A-Fa-f]\{2\}:\)\{5\}/ [REDACTED]:/g;s/\([0-9A-F-a-f]\{36\}\)/ [REDACTED]/g" )

but even after changing to

			line=$(echo "$line" | sed "s/\([0-9A-Fa-f]\{2\}:\)\{5\}/ [REDACTED]:/g;s/\([0-9A-Fa-f]\{36\}\)/ [REDACTED]/g" )

I still get the same error. any ideas?

Is the following error an issue with bluetooth out of interest?

Oct 03 13:08:29 prespi bash[17509]: 0.1.667 01:08:29 pm [ERROR]        Correcting HCI error: Invalid device: Network is down
Oct 03 13:08:44 prespi bash[17509]: Can't init device hci0: Connection timed out (110)
Oct 03 13:08:49 prespi bash[17509]: [97B blob data]

This is on a Raspberry Pi Zero W that was freshly installed with 2018-06-27-raspbian-stretch-lite two days ago and is running 0.1.667. Nothing else is installed on the Pi.

According to the logs (journalctl -u monitor.service -f) the last entry was at 04:53 this morning with no obvious errors ([CHECK-DEL] was the last entry). I restarted with sudo systemctl restart monitor.service and it gave errors 1 min later.

Oct 03 04:52:25 prespi bash[15847]: 0.1.667 04:52:25 am [CMD-INFO]        **** Completed departure scan. ****
Oct 03 04:52:25 prespi bash[15847]: 0.1.667 04:52:25 am [CMD-NAME]        00:00:00:00:00:00 Me  Apple, Inc.
Oct 03 04:52:32 prespi bash[15847]: 0.1.667 04:52:32 am [CHECK-DEL]        00:00:00:00:00:00 expired after 146 (0 bias) seconds
Oct 03 04:53:01 prespi bash[15847]: 0.1.667 04:53:01 am [CHECK-DEL]        00:00:00:00:00:00 expired after 176 (0 bias) seconds
Oct 03 04:53:02 prespi bash[15847]: 0.1.667 04:53:02 am [CHECK-DEL]        00:00:00:00:00:00 expired after 179 (0 bias) seconds
Oct 03 13:08:13 prespi systemd[1]: Stopping Monitor Service...
Oct 03 13:08:13 prespi bash[15847]: /home/pi/monitor/monitor.sh: line 51: 15918 Terminated              log_listener
Oct 03 13:08:13 prespi bash[15847]: /home/pi/monitor/monitor.sh: line 51: 15919 Terminated              btle_scanner
Oct 03 13:08:13 prespi bash[15847]: /home/pi/monitor/monitor.sh: line 51: 15920 Terminated              btle_listener
Oct 03 13:08:13 prespi bash[15847]: /home/pi/monitor/monitor.sh: line 51: 15921 Terminated              mqtt_listener
Oct 03 13:08:13 prespi bash[15847]: /home/pi/monitor/monitor.sh: line 51: 15922 Terminated              periodic_trigger
Oct 03 13:08:13 prespi bash[15847]: /home/pi/monitor/monitor.sh: line 51: 15926 Terminated              refresh_databases
Oct 03 13:08:13 prespi bash[15847]: Exited.
Oct 03 13:08:13 prespi systemd[1]: Stopped Monitor Service.
Oct 03 13:08:13 prespi systemd[1]: Started Monitor Service.
Oct 03 13:08:13 prespi bash[17509]: Starting monitor.sh (v. 0.1.667)...
Oct 03 13:08:14 prespi bash[17509]: > preference: delay between scans = 3
Oct 03 13:08:14 prespi bash[17509]: > preference: periodic arrive/depart check interval = 15
Oct 03 13:08:14 prespi bash[17509]: > preference: periodic arrive interval = 45
Oct 03 13:08:14 prespi bash[17509]: > preference: periodic depart interval = 90
Oct 03 13:08:14 prespi bash[17509]: > preference: database refresh interval = 35
Oct 03 13:08:14 prespi bash[17509]: > preference: max arrival scan attempts = 2
Oct 03 13:08:14 prespi bash[17509]: > preference: max depart scan attempts = 4
Oct 03 13:08:14 prespi bash[17509]: > preference: random advertisement expiration = 45
Oct 03 13:08:14 prespi bash[17509]: > preference: beacon rssi change required for reporting = 10
Oct 03 13:08:14 prespi bash[17509]: > preference: bluetooth environmental report frequency = 300
Oct 03 13:08:14 prespi bash[17509]: > preference: forced departure check interval = 240
Oct 03 13:08:14 prespi bash[17509]: > preference: interval until beacon is considered expired = 145
Oct 03 13:08:14 prespi bash[17509]: > preference: trigger a departure scan at other nodes below [x] confidence = 25
Oct 03 13:08:14 prespi bash[17509]: > preference: preferred HCI device = hci0
Oct 03 13:08:14 prespi bash[17509]: > preference: minimum time between the same type of scan = 15
Oct 03 13:08:14 prespi bash[17509]: > preference: mqtt scan start/end reporting = false
Oct 03 13:08:14 prespi bash[17509]: > mqtt trigger: monitor/scan/ARRIVE
Oct 03 13:08:14 prespi bash[17509]: > mqtt trigger: monitor/scan/DEPART
Oct 03 13:08:28 prespi bash[17509]: Can't init device hci0: Connection timed out (110)
Oct 03 13:08:29 prespi bash[17509]: > known device: 00:00:00:00:00:00 will publish to: monitor/study/00:00:00:00:00:00
Oct 03 13:08:29 prespi bash[17509]: > known device: 00:00:00:00:00:00 will publish to: monitor/study/00:00:00:00:00:00
Oct 03 13:08:29 prespi bash[17509]: 0.1.667 01:08:29 pm [ERROR]        Correcting HCI error: Invalid device: Network is down
Oct 03 13:08:44 prespi bash[17509]: Can't init device hci0: Connection timed out (110)
Oct 03 13:08:49 prespi bash[17509]: [97B blob data]
Oct 03 13:09:05 prespi bash[17509]: Can't init device hci0: Connection timed out (110)
Oct 03 13:09:10 prespi bash[17509]: [97B blob data]
Oct 03 13:09:25 prespi bash[17509]: Can't init device hci0: Connection timed out (110)
Oct 03 13:09:30 prespi bash[17509]: [97B blob data]
Oct 03 13:09:46 prespi bash[17509]: Can't init device hci0: Connection timed out (110)

Iā€™m working on the same problem as well the past few days. My phone which is under known_beacons is working well and counting down but my apple_ibeacon only gives a confidence 100 message every few minutes when it is near, it is never going decreasing in confidence. Itā€™s reporting using ā€˜monitor.sh -bā€™. The beacon is set to report every second with a ble message. Changing ā€˜PREF_BEACON_EXPIRATION=145ā€™ to another number does not help.

I have some intrest in this project, Iā€™m wondering how can the

sensor.home_occupancy_confidence

Be convert or result in a device tracker that I then would use with a fritzbox device tracker and gpslogger device tracker combined in a binary_sensor.bayesian that would then result in a grouped device tracker?

Or am I overthinking it?

I had a problem similar to a few others in this thread (and the previous thread) where an iPhone would not respond to BT name queries. The script would generate 0 confidence messages even when the iPhone was right next to the RPi.

In my house, we have two iPhone 7s and one android being ā€˜monitoredā€™ by 2 RPi zero W. The Android and one of the iPhones were recognized by the script and the system works without fail. The other iPhone never generated anything other than 0 confidence. I triple checked the mac address, turned BT on/off, rebooted the phone, deleted and reentered the device in known_static_addresses, watched the log thru the command line and monitored the MQTT traffic but it never was recognized as anything other than ā€˜0ā€™ and only generated an initial MQTT msg as the level never changed regardless of the phoneā€™s location.

I tried to ping the iPhone thru hcitool name but it didnā€™t respond. It wouldnā€™t appear as ā€˜availableā€™ to other BT devices until the BT settings screen on the iPhone was open. Once the BT settings screen was closed, the device again became invisible and unresponsive to name queries.

This has been tormenting me for the better part of the past week and I was ready to scrap the system (much of my planned automation based on Monitor just wouldnā€™t work if one person in the house was unaccounted forā€¦). The invisible iPhone is fairly new and as such had not yet been paired with any BT accessory. The other iPhone (same exact model) and the Android had been paired with various devices previously. This seemed to me the only difference between the two iPhones. I paired the iphone to a Bluetooth speaker and instantly the phone was visible and responsive in Monitor. I unpaired the speaker and the phone remained visible. I donā€™t understand the black magic behind the BT protocols of the iPhone but this seemed to fix this problem for me.

Just thought I would share my experience in case someone else was facing the same problem.

6 Likes

THANK YOU!!
I have been puling my hair out with same problem after wipping old iphone 5.
Paired to speakers and confidence instantly 100%.

1 Like

That is remarkably interesting and thorough feedback. Thank you for finding this out!

@andrewjfreyer
This is happening on the new build which forces a reboot :zipper_mouth_face:

Oct 05 00:14:37 blue0 bash[490]: 0.1.670 12:14:37 am [CMD-RSSI]        71:0F:30:8A:37:0B Undiscoverable Device Name RAND RSSI: -59 dBm (Slow/No Motion, changed 11) 
Oct 05 00:14:41 blue0 bash[490]: /home/pi/monitor/monitor.sh: line 871: 7E: value too great for base (error token is "7E")
Oct 05 00:14:41 blue0 bash[490]: 0.1.670 12:14:41 am [CHECK-DEL]        7E:E5:68:59:AA:53 expired after 75 seconds 
Oct 05 00:14:51 blue0 systemd[1]: monitor.service: Service hold-off time over, scheduling restart.
Oct 05 00:14:51 blue0 systemd[1]: Stopped Monitor Service.
Oct 05 00:14:51 blue0 systemd[1]: Started Monitor Service.
Oct 05 00:14:51 blue0 bash[5679]: Starting monitor.sh (v. 0.1.670)...
Oct 05 00:14:51 blue0 bash[5679]: > trigger mode: report out MQTT arrive/depart scan triggers to other devices
1 Like

@benjimatt let me investigate.