[monitor] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

When I try to include a monitor sensor in the person: component I get :

Invalid config for [person]: Entity ID 'sensor.walt_bt' does not belong to domain 'device_tracker' for dictionary value @ data['person'][0]['device_trackers']. Got ['sensor.walt_bt', 'device_tracker.walt_owntracks']. (See /config/configuration.yaml, line 113). Please check the docs at https://home-assistant.io/components/person/

Anyone else got this kind of thing happening?

The error message clearly states that you can only add entities from the domain device_tracker to the person entity. If you read the docs referenced in the error message you’ll see this as well.

Sorry, this is true, I phrased my question badly, I wondered if anyone had modified HA or had a workaround for this problem. Currently installing the composite platform.

This topic discusses your issue, if you have not seen it yet:

So I have been watching the script in verbose mode and when it stops responding I get the following message ./support/btle: line 430: packet_pipe: No such file or directory. Has anyone seen or experienced this? Only a power cycle will bring the script back up

I wrote an addon for Hass.io to conveniently work with this monitor script. The addon is currently in beta testing.
If you want to install it:
Go to “ADD-ON STORE” and add the https://github.com/Limych/hassio-addons to “Add new repository by URL”.
After that Limych’s add-ons should show under the title: Limych’s Hass.io Add-ons. And you will be able to install Bluetooth Presence Monitor.

7 Likes

Did this ever get resolved for you? I’m having the identical issue on my Apple Watch 5.

No, I’m afraid not.
I have since then stoped using this plugin and only use my iPhone with geofence as presence detection.

Did you get this fixed? If I had to take a guess in the dark, did you edit the known_static_addresses file on a Windows computer and move it to the Pi? If so you may need to run dos2unix to fix the new line characters.

Hey yall. I’m back! kinda

Hey all, I really love and appreicate the number of people who have helped new users troubleshoot in my absence over the past eight months or so. As many of you have heard through scattered comments on github, my wife and I had our first kid in March - thus, my radio silence. Now that my son is a bit older, I can get back on the wagon and help a bit more!

Please feel free to message here or on GitHub and I’ll do my best to help out.

What a great community. Thanks for everything!

20 Likes

Congratulations on surviving this far into parenthood. I’d claim that it gets easier from here, but it just gets different :stuck_out_tongue:

3 Likes

I am seeing regular problems with both versions 0.2.198 and 0.2.197, on two completely different systems with different BT hardware running in a VM and on bare metal. The “hcitool name” fails to return the name for present devices with probability 50%. What I see in the logs around these failed name scans is that:::
when two “hcitool name” commands are executed immediately one after another then the result of name command is empty even for a present device.

Does this make sense? If so, what do you think is going on?

Here is a section of the log. The affected device is person1_phone. Other named (flowercare1 and other_device) devices were not present.
The lines with ‘-----’ is something i added to print the pid.

   Jan 12 21:16:58 media bash[9639]: ------ hcitool name XX:XX:FB:09:71:C0, pid 9639                                                                                                              
Jan 12 21:16:59 media bash[9639]: XX:XX:FB:09:71:C0 --> Galaxy S8+, duration 1, pid 9639                                                                                                       
Jan 12 21:16:59 media bash[9639]: [+] 0.2.198 12-01-2020 09:16:59 PM [CMD-NAME]        XX:XX:FB:09:71:C0 Galaxy S8+  Samsung Electronics CoLtd                                                 
Jan 12 21:16:59 media bash[9639]: [+] 0.2.198 12-01-2020 09:16:59 PM [CMD-MQTT]        monitor/house/person1_phone { ... confidence : 100 ... }                                                
....                                                                                                                                                                                           
Jan 12 21:17:34 media bash[9639]: ------ hcitool name XX:XX:FB:09:71:C0, pid 9639                                                                                                              
Jan 12 21:17:34 media bash[9639]: ------ hcitool name XX:XX:67:02:72:ED, pid 9639                                                                                                              
Jan 12 21:17:34 media bash[9639]: XX:XX:FB:09:71:C0 --> , duration 0, pid 9639                                                                                                                 
Jan 12 21:17:34 media bash[9639]: XX:XX:67:02:72:ED --> , duration 0, pid 9639                                                                                                                 
Jan 12 21:17:34 media bash[9639]: [+] 0.2.198 12-01-2020 09:17:34 PM [CMD-MQTT]        monitor/house/person1_phone { ... confidence : 90 ... }                                                 
Jan 12 21:17:34 media bash[9639]: [+] 0.2.198 12-01-2020 09:17:34 PM [CMD-MQTT]        monitor/house/other_device { ... confidence : 0 ... }                                                   
Jan 12 21:17:37 media bash[9639]: ------ hcitool name XX:XX:FB:09:71:C0, pid 9639                                                                                                              
Jan 12 21:17:37 media bash[9639]: ------ hcitool name XX:XX:FF:5F:E7:64, pid 9639                                                                                                              
Jan 12 21:17:37 media bash[9639]: XX:XX:FB:09:71:C0 --> , duration 0, pid 9639                                                                                                                 
Jan 12 21:17:37 media bash[9639]: XX:XX:FF:5F:E7:64 --> , duration 0, pid 9639                                                                                                                 
Jan 12 21:17:37 media bash[9639]: [+] 0.2.198 12-01-2020 09:17:37 PM [CMD-MQTT]        monitor/house/flowercare1 { ... confidence : 0 ... }                                                    
Jan 12 21:17:37 media bash[9639]: [+] 0.2.198 12-01-2020 09:17:37 PM [CMD-MQTT]        monitor/house/person1_phone { ... confidence : 45 ... }                                      
  

Some more details:

  1. If I run “hcitool name” in a loop in a shell, the device never fails to respond.
  2. There are 10 known static addresses configured.
  3. If I remove all other devices from known_static_addresses besides person1_phone, everything works fine.
  4. Periodic scan is on.
  5. Beacon mode is on, but I do not think it matters for my issue.

My prefs:

PREF_ARRIVAL_SCAN_ATTEMPTS=1
PREF_DEPART_SCAN_ATTEMPTS=2
PREF_BEACON_EXPIRATION=240

PREF_MINIMUM_TIME_BETWEEN_SCANS=15

PREF_PASS_FILTER_ADV_FLAGS_ARRIVE=".*"
PREF_PASS_FILTER_MANUFACTURER_ARRIVE=".*"
PREF_FAIL_FILTER_ADV_FLAGS_ARRIVE="NONE" 
PREF_FAIL_FILTER_MANUFACTURER_ARRIVE="NONE"

PREF_DEVICE_TRACKER_REPORT=false
PREF_DEVICE_TRACKER_HOME_STRING='home' 
PREF_DEVICE_TRACKER_AWAY_STRING='not_home'
PREF_DEVICE_TRACKER_TOPIC_BRANCH='device_tracker'

PREF_BEACON_MODE=true
PREF_SHOULD_RETAIN=false

PREF_RSSI_IGNORE_BELOW=-100
PREF_MQTT_REPORT_SCAN_MESSAGES=true
PREF_REPORT_ALL_MODE=true
PREF_TRIGGER_MODE_ARRIVE=true
PREF_TRIGGER_MODE_DEPART=false

PREF_PERIODIC_MODE=true

I am not sure what i am missing. I am trying to track a iBeacon. When start the monitor with -b , it detects the ibeacon with 100% confidence and the beacon moved away it was able to detect until the confidence level went to 0. Later when the iBeacon came with the range, there was no messages in the log and monitor was not able to detect the presence of iBeacon. Monitor is running in a pi. Below is my preference config
#MAX RETRY ATTEMPTS FOR ARRIVAL
PREF_ARRIVAL_SCAN_ATTEMPTS=1

#MAX RETRY ATTEMPTS FOR DEPART
PREF_DEPART_SCAN_ATTEMPTS=2

#SECONDS UNTIL A BEACON IS CONSIDERED EXPIRED
PREF_BEACON_EXPIRATION=240

#MINIMUM TIME BEWTEEN THE SAME TYPE OF SCAN (ARRIVE SCAN, DEPART SCAN)
PREF_MINIMUM_TIME_BETWEEN_SCANS=15

#ARRIVE TRIGGER FILTER(S)
PREF_PASS_FILTER_ADV_FLAGS_ARRIVE=“0x1a”
PREF_PASS_FILTER_MANUFACTURER_ARRIVE=“Nest|LG|Apple|Amazfit”

#ARRIVE TRIGGER NEGATIVE FILTER(S)
PREF_FAIL_FILTER_ADV_FLAGS_ARRIVE=“NONE”
PREF_FAIL_FILTER_MANUFACTURER_ARRIVE=“Nest|LG|Apple|Amazfit”

Below are the log messages related to the iBeacon

[-] 0.2.200 14-01-2020 02:53:57 pm [CMD-MQTT] monitor/Entrance/vishnu
{
“id”:“18:04:ED:51:91:62”,
“confidence”:“100”,
“name”:“vishnu”,
“manufacturer”:“unknown”,
“type”:“GENERIC_BEACON_PUBLIC”,
“report_delay”:“0”,
“observed_interval”:“0”,
“rssi”:"-84",
“flags”:“0x06”,
“movement”:“moderate depart”,
“retained”:“false”,
“timestamp”:“Tue Jan 14 2020 14:53:57 GMT+0000 (GMT)”,
“version”:“0.2.200”
}

[-] 0.2.200 14-01-2020 02:57:45 pm [CMD-MQTT] monitor/Entrance/vishnu
{
“id”:“18:04:ED:51:91:62”,
“confidence”:“0”,
“last_seen”:“1579013702”,
“retained”:“false”,
“timestamp”:“Tue Jan 14 2020 14:57:45 GMT+0000 (GMT)”,
“version”:“0.2.200”
}

Thanks for the report. I’m reviewing your logs and seeing whether I can duplicate this behavior. Thanks!

This sounds interesting but I run hassos… hope to see this as a hassio add-on or integration at some point.

That’s a bit out of scope for this project. The point here is to positionally distribute bluetooth detection nodes throughout a home on cheap hardware (pi zero w) to get a comprehensive (and somewhat redundant) presence detection system. Having only a single node is fine, but doesn’t leverage monitor to its fullest benefit.

If you live in a small apartment that would be adequately covered by a single node, and you don’t use bluetooth for anything else on your hassio box, you can search the forums, github, and this thread for other users who have wrapped monitor into a docker container.

3 Likes

Hey, thanks for the response. Well, I installed dos2unix and ran the command. It failed. It seems to have something to do with permissions, and if it wasn’t already obvious, I’m totally out of my depth here.

The documentation I read for dos2unix seemed to suggest that running the “file” command tells you the format of the file. Mine is reporting as “ASCII”, but I don’t know if that is correct or not.

pi@pi1-monitor:~/monitor $ dos2unix known_static_addresses
dos2unix: Failed to change the owner and group of temporary output file ./d2utmpMakYQ9: Operation not permitted
dos2unix: problems converting file known_static_addresses
pi@pi1-monitor:~/monitor $ file known_static_addresses
known_static_addresses: ASCII text
pi@pi1-monitor:~/monitor $ 

If you’re having difficulty with files, just delete them from the monitor directory and run monitor again. It’ll recreate all files not present. You can edit those files with nano from the command line.

1 Like

Hi, I am new for this project, I did the installation as per in instruction, and the setup is attached. I am unable to view the MQTT subscribe message in the terminal itself.

# ---------------------------
# MOSQUITTO PREFERENCES
# ---------------------------

# IP ADDRESS OR HOSTNAME OF MQTT BROKER
mqtt_address=192.168.0.155

# MQTT BROKER USERNAME
mqtt_user=marat

# MQTT BROKER PASSWORD
mqtt_password=marat@123

# MQTT PUBLISH TOPIC ROOT
mqtt_topicpath=monitor

# PUBLISHER IDENTITY
mqtt_publisher_identity='Dir_Room'

# MQTT PORT
mqtt_port='1883'

# MQTT CERTIFICATE FILE
mqtt_certificate_path=''

#MQTT VERSION (EXAMPLE: 'mqttv311')
mqtt_version='1.6.8'

# ---------------------------
# BEHAVIOR PREFERENCES
# ---------------------------

#MAX RETRY ATTEMPTS FOR ARRIVAL
PREF_ARRIVAL_SCAN_ATTEMPTS=1

#MAX RETRY ATTEMPTS FOR DEPART
PREF_DEPART_SCAN_ATTEMPTS=2

#SECONDS UNTIL A BEACON IS CONSIDERED EXPIRED
PREF_BEACON_EXPIRATION=240

#MINIMUM TIME BETWEEN THE SAME TYPE OF SCAN (ARRIVE SCAN, DEPART SCAN)
PREF_MINIMUM_TIME_BETWEEN_SCANS=15

#ARRIVE TRIGGER FILTER(S)
PREF_PASS_FILTER_ADV_FLAGS_ARRIVE=".*"
PREF_PASS_FILTER_MANUFACTURER_ARRIVE=".*"

#ARRIVE TRIGGER NEGATIVE FILTER(S)
PREF_FAIL_FILTER_ADV_FLAGS_ARRIVE="NONE"
PREF_FAIL_FILTER_MANUFACTURER_ARRIVE="NONE"

In raspberry pi I did execute the following code:-

pi@raspberrypi:~/monitor $ sudo bash monitor.sh -r

===================================================

              *** BETA/DEV BRANCH ***

===================================================
> starting monitor.sh (v. 0.2.200)...
> warning: periodic scan mode may cause interference with 2.4GHz networks if run on a Raspberry Pi
> 88:75:98:FB:E9:E8 confidence topic: monitor/Dir_Room/avi (has not previously connected to hci0)
> E0:AA:96:07:D9:59 confidence topic: monitor/Dir_Room/rama (has not previously connected to hci0)
[+] 0.2.200 17-01-2020 04:04:29 pm [CMD-INFO]   cycling bluetooth hardware on hci0. please wait a moment...
[+] 0.2.200 17-01-2020 04:04:30 pm [CMD-MQTT]   monitor/Dir_Room/avi { ... confidence : 0 ... }
[+] 0.2.200 17-01-2020 04:04:31 pm [CMD-NAME]   88:75:98:FB:E9:E8 Avi  Samsung Electronics CoLtd
[+] 0.2.200 17-01-2020 04:04:31 pm [CMD-MQTT]   monitor/Dir_Room/avi { ... confidence : 0 ... }
[+] 0.2.200 17-01-2020 04:04:31 pm [CMD-NAME]   88:75:98:FB:E9:E8 Avi  Samsung Electronics CoLtd
[+] 0.2.200 17-01-2020 04:04:39 pm [CMD-MQTT]   monitor/Dir_Room/rama { ... confidence : 100 ... }
[+] 0.2.200 17-01-2020 04:04:39 pm [CMD-NAME]   E0:AA:96:07:D9:59 Tramanah  Samsung Electronics CoLtd
[+] 0.2.200 17-01-2020 04:04:40 pm [CMD-NAME]   E0:AA:96:07:D9:59 Tramanah  Samsung Electronics CoLtd
[+] 0.2.200 17-01-2020 04:04:40 pm [CMD-MQTT]   monitor/Dir_Room/rama { ... confidence : 100 ... }
[+] 0.2.200 17-01-2020 04:05:27 pm [CMD-INFO]   cycling bluetooth hardware on hci0. please wait a moment...
[+] 0.2.200 17-01-2020 04:05:28 pm [CMD-MQTT]   monitor/Dir_Room/rama { ... confidence : 90 ... }
[+] 0.2.200 17-01-2020 04:05:33 pm [CMD-MQTT]   monitor/Dir_Room/rama { ... confidence : 45 ... }
[+] 0.2.200 17-01-2020 04:05:37 pm [CMD-MQTT]   monitor/Dir_Room/rama { ... confidence : 0 ... }
[+] 0.2.200 17-01-2020 04:05:37 pm [CMD-NAME]   88:75:98:FB:E9:E8 Avi  Samsung Electronics CoLtd
[+] 0.2.200 17-01-2020 04:05:38 pm [CMD-MQTT]   monitor/Dir_Room/avi { ... confidence : 100 ... }
[+] 0.2.200 17-01-2020 04:05:38 pm [CMD-NAME]   E0:AA:96:07:D9:59 Rama  Samsung Electronics CoLtd

I did open another terminal to subscribe mosquittio events.
pi@raspberrypi:~ $ mosquitto_sub -d -t monitor/# -u marat -P marat@123

Client mosq-zEPyaFvkvsu5P2rGuv sending CONNECT
Client mosq-zEPyaFvkvsu5P2rGuv received CONNACK (0)
Client mosq-zEPyaFvkvsu5P2rGuv sending SUBSCRIBE (Mid: 1, Topic: monitor/#, QoS: 0, Options: 0x00)
Client mosq-zEPyaFvkvsu5P2rGuv received SUBACK
Subscribed (mid: 1): 0
Client mosq-zEPyaFvkvsu5P2rGuv sending PINGREQ
Client mosq-zEPyaFvkvsu5P2rGuv received PINGRESP
Client mosq-zEPyaFvkvsu5P2rGuv sending PINGREQ
Client mosq-zEPyaFvkvsu5P2rGuv received PINGRESP

Do I miss anything in my settings? I am trying pass few days and no luck…

Change the protocol version in your mqtt_preferences to mqttv311 and restart monitor. Took me a few hours to find that one with my own setup… :wink: