[monitor] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

@andrewjfreyer, thank you very much for your clear explanation. Now I understand this requires more horsepower at each node, and yeah I’d need at least 3-4 to cover my property. That isn’t grossly expensive using pi zeros. LOL, at first I was thinking maybe an esp8266 or something could be used for the distributed parts.

Just something to be aware of, it’s still extremely difficult to get multiple pi zeros. I’m trying to order at least two more and I’ve come to learn that the Pi Foundation has not been interested in ramping up production to meet the demands over the past three years because it would cannibalize sales of the larger ones. This has resulted in resellers being perpetually sold out, won’t sell you more than one (ever), or they’re only available at a hugely inflated cost in a kit with stuff you don’t want.

Sorry if I missed it, but is there any kind of hardware compatibility table? I’m wondering if this works on a Pi 3 B+ or an older one with a separate USB bluetooth dongle, or something like a Banana Pi M2 zero.

2 Likes

Wow, thanks for the info… DOH!!! I should have stocked up on zeroes when I had the chance.

What about esp32… it has btle built in?

[edit: NM… I just saw all the posts above about issues with esp32 working with monitor.]

I run one node on a Pi3, works fine there.

I run another on an Orange Pi Zero LTS. Armbian doesn’t support the on board Bluetooth, but I bought a BT dongle, and that works just fine. Indeed, pretty much any Linux system will work - if all else fails just add a supported BT dongle.

1 Like

Its working now. OMG, I did spend couples of days. Thank you, do appreciate most.

1 Like

Hi, referring to my post number 1738, I did the test without username and password. The system response to mqtt subscribe with terminal, superb. While I do setup user id and password in Mosquitto broker, my monitor.sh gives fatal errors as follows: -

===================================================

              *** BETA/DEV BRANCH ***

===================================================
> starting monitor.sh (v. 0.2.200)...
> removing web request caches
> 88:75:98:FB:E9:E8 confidence topic: monitor/Dir_Room/avi (has not previously connected to hci0)
> E0:AA:96:07:D9:59 confidence topic: monitor/Dir_Room/rama (has not previously connected to hci0)
> FD:51:A1:48:4C:AF confidence topic: monitor/Dir_Room/james (has not previously connected to hci0)
> A8:87:B3:9D:D8:13 confidence topic: monitor/Dir_Room/dilan (has not previously connected to hci0)
[+] 0.2.200 19-01-2020 09:15:09 am [CMD-MQTT]   warning: issue connecting to mqtt server (lookup error). enqueuing command to try again after a delay
[+] 0.2.200 19-01-2020 09:15:12 am [CMD-NAME]   88:75:98:FB:E9:E8 Avi  Samsung Electronics CoLtd
[+] 0.2.200 19-01-2020 09:15:12 am [CMD-MQTT]   warning: issue connecting to mqtt server (lookup error). enqueuing command to try again after a delay
[+] 0.2.200 19-01-2020 09:15:27 am [CMD-ERRO]   fatal mqtt error - messages may not be delivered as intended (issue connecting to mqtt server (lookup error). enqueuing commnd to try again after a delay / 4)
[+] 0.2.200 19-01-2020 09:15:28 am [CMD-ERRO]   fatal mqtt error - messages may not be delivered as intended (issue connecting to mqtt server (lookup error). enqueuing commnd to try again after a delay / 4)
[+] 0.2.200 19-01-2020 09:15:31 am [CMD-ERRO]   fatal mqtt error - messages may not be delivered as intended (issue connecting to mqtt server (lookup error). enqueuing commnd to try again after a delay / 4)
[+] 0.2.200 19-01-2020 09:15:32 am [CMD-MQTT]   warning: issue connecting to mqtt server (lookup error). enqueuing command to try again after a delay
[+] 0.2.200 19-01-2020 09:15:33 am [CMD-NAME]   E0:AA:96:07:D9:59 Tramanah  Samsung Electronics CoLtd
[+] 0.2.200 19-01-2020 09:15:33 am [CMD-MQTT]   warning: issue connecting to mqtt server (lookup error). enqueuing command to try again after a delay
[+] 0.2.200 19-01-2020 09:15:33 am [CMD-MQTT]   warning: issue connecting to mqtt server (lookup error). enqueuing command to try again after a delay
[+] 0.2.200 19-01-2020 09:15:40 am [CMD-ERRO] fatal mqtt error - messages may not be delivered as intended (issue connecting to mqtt server (lookup error). enqueuing commnd to try again after a delay / 4)

The mqtt reference setting are : -

# ---------------------------
# MOSQUITTO PREFERENCES
# ---------------------------

# IP ADDRESS OR HOSTNAME OF MQTT BROKER
mqtt_address=192.168.0.155

# MQTT BROKER USERNAME
mqtt_user=marat

# MQTT BROKER PASSWORD
mqtt_password=marat@123

# MQTT PUBLISH TOPIC ROOT
mqtt_topicpath=monitor

# PUBLISHER IDENTITY
mqtt_publisher_identity='Dir_Room'

# MQTT PORT
mqtt_port='1883'

# MQTT CERTIFICATE FILE
mqtt_certificate_path=''

#MQTT VERSION (EXAMPLE: 'mqttv311')
mqtt_version='mqttv311'

The password setup in the broker is as follows: -

sudo mosquitto_passwd -c /etc/mosquitto/passwd marat
the file is created and requested for a password twice.

sudo nano /etc/mosquitto/conf.d/default.conf

On empty file and did paste the following script

allow_anonymous false
password_file /etc/mosquitto/passwd

sudo systemctl restart mosquitto

I did open two terminal and tested by subscribing and pusblishing with password and its works. While with monitor its through fatal error.

:cry:

Literally 9 posts above yours is the announcement of a hassio addon.

1 Like

Might be an issue with the special character in your password. Try a password with no special chaarcters.

Cheers buddy. Its works without special character. :pray:t5:

It took me a while, too, to understand, how monitor is working. You do not need it as a hass.io addon. :slight_smile:

If you’re running hass.io, you need other Pis, and there you should run monitor.sh. Look at monitor.sh as a client program, that reports to your mqtt on your main hass.io (or HA) instance. It doesn’t make sense to use it on your main instance, as you would block all other BT traffic, for example for Xiaomi/Mija temperature displays.

A normal setup would look like this:

  • HA or hass.io on a “bigger” Pi like 3b+ or 4
  • one or more Pi Zero(s) where you would run monitor.sh on. These you place around your flat or house.
  • The Pis with monitor.sh report back to your main Pi via mqtt.

This is awesome! Testing your addon for a few days now, works great!

Only thing I can’t seem to understand is how to trigger a name call upon MQTT message. In MQTT explorer I see my messages to monitor/scan/arrive are published, but the script doesn’t seem to do a name call upon any of my MQTT messages. Is there a specific payload I need to use for those messages? I tried to change to various different topics (now at monitor/keuken/ my phones alias is ‘iphonerik’) but trying altered versions for topics sadly did not help. Any idea on what I might be missing here?

Thanks for your feedback.

The addon is just a shell for the original script. I do not understand your question yet and, as a result, I can’t understand if you have difficulties with setting up an add-on or setting up a script.

Perhaps it will help if you send me the full text of the addon configs.

Hi, is there a way to scan for known beacons via MQTT trigger messages? I have a beacon (Xiaomi Band 2) that I would like to scan for on-demand. Ultimately, the goal is to have a motion sensor in HA trigger a beacon scan in monitor.sh via MQTT to see if the band is nearby.

I can get the band’s RSSI value (and everything else about it) to display on the screen when doing periodic scans like monitor.sh -barV. My topic path is monitor.

However, when doing monitor.sh -tad and publishing monitor/scan/ARRIVE or monitor/scan/DEPART, the band never shows up. On the monitor/pi/miband2 topic, the confidence is always 0 and under monitor/pi/miband2/rssi, the value is always -99.

Since beacons don’t respond to name requests, I don’t know if my band will respond to MQTT triggers via the -tad options the same way my phone does (which works perfectly).

Thanks for your efforts, it turns out my previous problem is resolved by changing topic_root to something that is not ‘monitor’.

When I published ‘monitor/scan/restart’ the script did not return any name scanning. After some testing with changing the main topic of my monitor instance and publishing ‘monitor/scan/restart’ which did result in a restart, from that moment on, I’m now also able to successfully start the arrive and depart sequence.

The main problem might have been that I had two monitor instances with identical ‘topic_root’ set. They did have a different ‘publisher’.

1 Like

Hi good day, currently I am running at setting -t (trigger mode), if I didn’t trigger the mode for quite period of time, I am getting syntax error:

./support/btle: line 310: [[: -: syntax error: operand expected (error token is “-”)
monitor.sh: line 1773: -: syntax error: operand expected (error token is “-”)
Terminated

> starting monitor.sh (v. 0.2.200)...
> removing web request caches
> 88:75:98:FB:E9:E8 confidence topic: DRRM/Dir_Room/avi (has not previously connected to hci0)
> E0:AA:96:07:D9:59 confidence topic: DRRM/Dir_Room/rama (has not previously connected to hci0)
> A8:87:B3:9D:D8:13 confidence topic: DRRM/Dir_Room/dilan (has not previously connected to hci0)
> known beacon: FD:51:A1:48:4C:AF publishes to: DRRM/Dir_Room/james
[+] 0.2.200 22-01-2020 07:16:58 pm [CMD-NAME]   88:75:98:FB:E9:E8 Avi  Samsung Electronics CoLtd
[+] 0.2.200 22-01-2020 07:17:01 pm [CMD-MQTT]   DRRM/Dir_Room/avi { ... confidence : 100 ... }
[+] 0.2.200 22-01-2020 07:17:17 pm [CMD-MQTT]   DRRM/Dir_Room/rama { ... confidence : 0 ... }
[+] 0.2.200 22-01-2020 07:17:21 pm [CMD-NAME]   E0:AA:96:07:D9:59 Rama  Samsung Electronics CoLtd
[+] 0.2.200 22-01-2020 07:17:38 pm [CMD-MQTT]   DRRM/Dir_Room/dilan { ... confidence : 0 ... }
[+] 0.2.200 22-01-2020 07:17:41 pm [CMD-NAME]   A8:87:B3:9D:D8:13 Dilan  Samsung Electronics CoLtd
[+] 0.2.200 22-01-2020 07:17:54 pm [CMD-NAME]   E0:AA:96:07:D9:59 Rama  Samsung Electronics CoLtd
[+] 0.2.200 22-01-2020 07:18:05 pm [CMD-RSSI]   PUBL AA:BB:CC:92:23:45 RSSI: -91 dBm (initial reading | 109 dBm)
[+] 0.2.200 22-01-2020 07:19:26 pm [CMD-NAME]   E0:AA:96:07:D9:59 Rama  Samsung Electronics CoLtd
[+] 0.2.200 22-01-2020 07:19:26 pm [CMD-NAME]   A8:87:B3:9D:D8:13 Dilan  Samsung Electronics CoLtd
.
.
.
type[+] 0.2.200 23-01-2020 04:27:13 am [CMD-NAME]   A8:87:B3:9D:D8:13 Dilan  Samsung Electronics CoLtd
[+] 0.2.200 23-01-2020 04:29:55 am [DEL-RAND]   RAND 14:AB:3A:D3:0F:FE expired after 188 seconds
[+] 0.2.200 23-01-2020 04:35:35 am [CMD-NAME]   88:75:98:FB:E9:E8 Avi  Samsung Electronics CoLtd
[+] 0.2.200 23-01-2020 04:35:35 am [CMD-NAME]   E0:AA:96:07:D9:59 Rama  Samsung Electronics CoLtd
[+] 0.2.200 23-01-2020 04:35:36 am [CMD-NAME]   A8:87:B3:9D:D8:13 Dilan  Samsung Electronics CoLtd
[+] 0.2.200 23-01-2020 04:38:17 am [DEL-RAND]   RAND 35:22:CC:F6:C4:09 expired after 189 seconds
[+] 0.2.200 23-01-2020 04:42:06 am [CMD-NAME]   88:75:98:FB:E9:E8 Avi  Samsung Electronics CoLtd
[+] 0.2.200 23-01-2020 04:42:06 am [CMD-NAME]   E0:AA:96:07:D9:59 Rama  Samsung Electronics CoLtd
[+] 0.2.200 23-01-2020 04:42:06 am [CMD-NAME]   A8:87:B3:9D:D8:13 Dilan  Samsung Electronics CoLtd
./support/btle: line 310: [[: -: syntax error: operand expected (error token is "-")
monitor.sh: line 1773: -: syntax error: operand expected (error token is "-")
Terminated

Thanks, I’ll look into this. Can you also make a bug report on GitHub?

I used to have HA running standalone on the RPi with monitor alongside and it worked flawlessly.

But I moved to hassio instead, and I’ve been running the add-on version of monitor for about a week now and it stops working after about 3hrs. Nothing in the logs unfortunately. So I just issue a restart every hr and it works perfectly now.

Somewhere I read it might be the BT on the RPi is getting more than one task executing against it and locks it out. No idea where to start researching that and I don’t really mind the restarts since it does so within seconds and works just fine for my needs.

I’ll probably start investing in some pi zeroes and place monitor on those around the house. I’ve a need to detect when in garage, etc, and pi zero seems the way to go.

Hi! I’m enjoying getting up to speed on this project and I’ve been using the monitor tool on standalone RPi Zero W for about a week. I am still trying to understand how the BTLE announcements and related traffic in my house are showing up. I have manufacturer filters set up just to pick Apple devices on arrival and I am not tracking any beacons (no “-b” option).
In my service logs (not verbose) after start up and an initial scan and couple of minutes of settling down I see an overwhelming number of [DEL-RAND] lines in the log making it difficult to see what’s happening through this noise. See small sample below.

Is this normal? Can these messages be limited to verbose logging only?

Jan 25 18:45:23 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:23 pm [DEL-RAND]        RAND 2C:5C:6F:FB:63:52 expired after 182 seconds
Jan 25 18:45:21 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:21 pm [DEL-RAND]        RAND 3B:73:29:F0:4C:52 expired after 181 seconds
Jan 25 18:45:21 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:21 pm [DEL-RAND]        RAND 0A:95:29:B4:F2:E2 expired after 181 seconds
Jan 25 18:45:21 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:21 pm [DEL-RAND]        RAND 1E:B7:55:9A:A6:B2 expired after 187 seconds
Jan 25 18:45:21 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:21 pm [DEL-RAND]        RAND 25:AA:99:BD:62:67 expired after 188 seconds
Jan 25 18:45:21 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:21 pm [DEL-RAND]        RAND 81:E0:49:01:27:06 expired after 197 seconds
Jan 25 18:45:21 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:21 pm [DEL-RAND]        RAND 05:2E:52:92:2F:B8 expired after 193 seconds
Jan 25 18:45:21 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:21 pm [DEL-RAND]        RAND 1A:60:2C:B0:5C:4A expired after 192 seconds
Jan 25 18:45:21 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:21 pm [DEL-RAND]        RAND 09:C1:A1:B5:03:19 expired after 194 seconds
Jan 25 18:45:20 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:20 pm [DEL-RAND]        RAND 3D:C9:EE:FF:B7:E4 expired after 187 seconds
Jan 25 18:45:20 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:20 pm [DEL-RAND]        RAND 31:1D:EA:85:71:CD expired after 184 seconds
Jan 25 18:45:20 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:20 pm [DEL-RAND]        RAND 17:ED:90:9C:A7:32 expired after 181 seconds
Jan 25 18:45:20 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:20 pm [DEL-RAND]        RAND 04:24:D5:92:10:2B expired after 195 seconds
Jan 25 18:45:02 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:02 pm [DEL-RAND]        RAND 32:53:02:F2:AF:1D expired after 182 seconds
Jan 25 18:45:02 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:02 pm [DEL-RAND]        RAND 29:A4:6D:87:E6:E2 expired after 181 seconds
Jan 25 18:45:00 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:00 pm [DEL-RAND]        RAND 6C:26:D9:03:BB:49 expired after 190 seconds
Jan 25 18:45:00 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:45:00 pm [DEL-RAND]        RAND 2A:77:65:39:FE:8C expired after 199 seconds
Jan 25 18:44:41 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:44:41 pm [DEL-RAND]        RAND 4B:A0:EA:84:7B:51 expired after 187 seconds
Jan 25 18:44:41 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:44:41 pm [DEL-RAND]        RAND 09:4C:FC:8C:69:23 expired after 194 seconds
Jan 25 18:44:41 raspberrypi bash[26309]: [+] 0.2.200 25-01-2020 06:44:41 pm [DEL-RAND]        RAND 28:76:D7:BE:92:97 expired after 193 seconds

I’ve been running the hassio add-on for the past few days on an Intel NUC, its working well but I also get the problem where is shuts down occasionally.

The range only covers the first floor and it doesn’t reach the bedrooms (I will eventually put in a RPi to cover the second floor to get full coverage of the house).

I’ve a sensor setup to show when the add-on is running or not and it seems that the script shuts down when there is no BT activity (known or random devices from neighbors) detected for awhile. I’ve done no real testing to confirm this but the times it is shutting down match us being in bed or out of the house.

Not much of an issue as I’ve set the external doors/motion detectors to start the add-on so presence is detected quickly enough for my needs, but interesting to see the issue appears over both NUC and RPi .

Add-on version 0.3.6 released. I think this release will be the first stable.

3 Likes