[presence] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

@andrewjfreyer - thanks - I will pull again and report back if I am still seeing dups.

Did you have any thoughts about (1) the LWT post and (2) retaining the messages if -c is used?

The former would be helpful in determining online/offline status of a node more reliably than waiting for 20 mins (especially now!). The latter is more of a nice-to-have while development is taking place

EDIT - the node which reported the duplicate above needed updating; the the two I have did not

1 Like

@andrewjfreyer Thanks for this awesome project. I got it up and running in no time and so far it feels to me as if just one pi will work in my home (but lets see!). Something I found when doing (the usual) repeated startups is that the input boolean won’t necessarily be correct after startup.

My reading around the numeric_state trigger is that it ONLY gets triggered when the value crosses the threshold. ie. if you set your input_boolean to “off” and the first value you get back is a confidence of 100, you don’t set it to “on” (since the confidence doesn’t cross from below 10 to above 10).

My solution was to set it on startup:

- id: hass_startup_set_occupancy
  hide_entity: true
  trigger:
    - platform: homeassistant
      event: start
  condition:
    condition: and
    conditions:
      - condition: numeric_state
        entity_id: sensor.chris_occupancy_confidence
        above: 10
  action:
    - service: input_boolean.turn_on
      entity_id: input_boolean.chris_occupancy
3 Likes

Hello.
Just started on this project. Got my Rpi3B as client, installed as instructed, using a remote MQTT broker. But before this is an issue, I get some errors I don’t understand. When I try
sudo systemctl status presence.service, I get the following status:

● presence.service - Presence service
   Loaded: loaded (/etc/systemd/system/presence.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2018-04-25 20:08:30 UTC; 17min ago
 Main PID: 286 (bash)
   CGroup: /system.slice/presence.service
           ├─ 286 /bin/bash /home/pi/presence/presence.sh &
           └─2267 sleep 30

Apr 25 20:08:30 raspberrypi systemd[1]: Started Presence service.
Apr 25 20:08:31 raspberrypi bash[286]: Error: Network is unreachable
Apr 25 20:08:31 raspberrypi bash[286]: Error: Network is unreachable
Apr 25 20:08:31 raspberrypi bash[286]: Error: Network is unreachable
Apr 25 20:08:31 raspberrypi bash[286]: Error: Network is unreachable

BT is working it seems. sudo service bluetooth status gives:

bluetooth.service - Bluetooth service
   Loaded: loaded (/lib/systemd/system/bluetooth.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2018-04-25 20:19:48 UTC; 10min ago
     Docs: man:bluetoothd(8)
 Main PID: 1623 (bluetoothd)
   Status: "Running"
   CGroup: /system.slice/bluetooth.service
           └─1623 /usr/lib/bluetooth/bluetoothd

Wlan is disabled, running wired. Any hints are greatly appreciated!

This is an error from mosquitto - has that installed properly? Is your broker online?

I think so, I did as instructed on the page, but in mqtt_preferences I point to my MQTT broker running on another computer. My HA configuration uses the same IP,user and pw without any complaints. Do I need a separate user for this config?

I tried to point mqtt to the local RPI installation, but with the same error. I defined user and pw for MQTT using mosquitto_passwd -c passwordfile <username>, is this not the correct way?

I use Hassio, so my Mosquitto broker setup process was different, but a tip: I find that an MQTT client is handy for debugging. I use MQTTLens (a Chrome app). I can subscribe to the expected topic on my broker and see what’s going on.

1 Like

You shouldn’t need a separate MQTT user (although you can have one if you like), just make sure that all the fields required in the mqtt_preferences file for presence are populated.

Update: Got it working. Used my existing MQTT broker with existing user and pw. When I restarted the service: sudo systemctl restart presence.service, then suddenly everything worked.

Another question I have (that already might been answered) is from my HA-log, I can read the following (cleaned it a little bit:

Received message on location/owner/home/40:98:XX:XX:XX:XX: b'{"confidence":"100","name":"iPhone","scan_duration_ms":"990","timestamp":"Wed Apr 25 2018 21:23:33 GMT+0000 (UTC)"}'
Received message on location/owner/home/40:98:XX:XX:XX:XX: b'{"confidence":"83","name":"iPhone","scan_duration_ms":"5022","timestamp":"Wed Apr 25 2018 21:24:27 GMT+0000 (UTC)"}'
Received message on location/owner/home/40:98:XX:XX:XX:XX: b'{"confidence":"66","name":"iPhone","scan_duration_ms":"5022","timestamp":"Wed Apr 25 2018 21:24:36 GMT+0000 (UTC)"}'
Received message on location/owner/home/40:98:XX:XX:XX:XX: b'{"confidence":"50","name":"iPhone","scan_duration_ms":"5022","timestamp":"Wed Apr 25 2018 21:24:44 GMT+0000 (UTC)"}'
Received message on location/owner/home/40:98:XX:XX:XX:XX: b'{"confidence":"33","name":"iPhone","scan_duration_ms":"5022","timestamp":"Wed Apr 25 2018 21:24:52 GMT+0000 (UTC)"}'
Received message on location/owner/home/40:98:XX:XX:XX:XX: b'{"confidence":"16","name":"iPhone","scan_duration_ms":"5022","timestamp":"Wed Apr 25 2018 21:25:00 GMT+0000 (UTC)"}'
Received message on location/owner/home/40:98:XX:XX:XX:XX: b'{"confidence":"0","name":"iPhone","scan_duration_ms":"5022","timestamp":"Wed Apr 25 2018 21:25:08 GMT+0000 (UTC)"}'

When confidence was “100”, I turned off BT on my phone and watched the confidence level going down step by step. Is this normal behaviour? When turned on again, it goes straight to “100”. Not a big deal but just wondering :slight_smile: But so far this seems to be the most brilliant way to track my presence. Great work!

Great to hear you got it working!

Yes, that is default behavior. The confidence drops from 100% the first time that a device is not recognized, and a sequence of scans are performed to verify that the device is not present. If you would prefer to show only 100 or 0 confidence, use the -b flag when launching the script.

Just to check, with the -b flag set does the script show 0 confidence immediately a scan fails, or does it show 100 until the verification loop completes and only then 0?

1 Like

Very interesting Andrew, I love the project! Thanks for sharing your good work.

1 Like

I decided to try installing on a RPi 3. All was going well (including the sudo rpi-update) until I completed step 6.

The command sudo service bluetooth status gave the following output:

bluetooth.service - Bluetooth service
Loaded: loaded (/lib/systemd/system/bluetooth.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2018-04-26 01:53:44 EDT; 27s ago
Docs: man:bluetoothd(8)
Main PID: 13626 (bluetoothd)
Status: “Running”
CGroup: /system.slice/bluetooth.service
└─13626 /usr/lib/bluetooth/bluetoothd

Apr 26 01:53:44 raspberrypi systemd[1]: Starting Bluetooth service…
Apr 26 01:53:44 raspberrypi bluetoothd[13626]: Bluetooth daemon 5.43
Apr 26 01:53:44 raspberrypi systemd[1]: Started Bluetooth service.
Apr 26 01:53:44 raspberrypi bluetoothd[13626]: Starting SDP server
Apr 26 01:53:44 raspberrypi bluetoothd[13626]: Bluetooth management interface 1.14 initialized

Step 7 calls for sudo reboot.

I decided to run the same command again upon reboot sudo service bluetooth status
and got the following:

bluetooth.service - Bluetooth service
Loaded: loaded (/lib/systemd/system/bluetooth.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2018-04-26 01:55:11 EDT; 2min 12s ago
Docs: man:bluetoothd(8)
Main PID: 422 (bluetoothd)
Status: “Running”
CGroup: /system.slice/bluetooth.service
└─422 /usr/lib/bluetooth/bluetoothd

Apr 26 01:55:11 raspberrypi systemd[1]: Starting Bluetooth service…
Apr 26 01:55:11 raspberrypi bluetoothd[422]: Bluetooth daemon 5.43
Apr 26 01:55:11 raspberrypi systemd[1]: Started Bluetooth service.
Apr 26 01:55:11 raspberrypi bluetoothd[422]: Starting SDP server
Apr 26 01:55:11 raspberrypi bluetoothd[422]: Bluetooth management interface 1.14 initialized
Apr 26 01:55:11 raspberrypi bluetoothd[422]: Failed to obtain handles for “Service Changed” characteristic
Apr 26 01:55:11 raspberrypi bluetoothd[422]: Sap driver initialization failed.
Apr 26 01:55:11 raspberrypi bluetoothd[422]: sap-server: Operation not permitted (1)

This does not look good.

Any suggestions as to what I should do to correct this?

Thanks.

It looks fine. The last three red lines are not to be worried about:

Apr 26 01:55:11 raspberrypi bluetoothd[422]: Failed to obtain handles for “Service Changed” characteristic
Apr 26 01:55:11 raspberrypi bluetoothd[422]: Sap driver initialization failed.
Apr 26 01:55:11 raspberrypi bluetoothd[422]: sap-server: Operation not permitted (1)

You can find information on Google if you want to remove this sap-error. I have the same log entry on my Pi and everything is working fine.

1 Like

It will wait until the verification loop completes. If you want to diable verification, modify behavior preferences. This is not recommended. You’ll receive a number of false exit events.

1 Like

Yeah, I guess my feeling, philosophically, is that the Zero’s shouldn’t be deciding whether someone is home or not. I think my preference is they just post what they see (or don’t see) and let something with a bit more power/information decide the interpretation.

@PianSom the verification is to prevent false hiccups, not really to decide whether someone is home or not. Including the verifications reduced false negatives substantially without much effort, so that’s why it’s enabled by default.

Regarding the lagging network connectivity: when I run a continuous ping at the RPi the script runs on, the typical behavior is this:

64 bytes from 192.168.178.59: icmp_seq=6 ttl=64 time=11.971 ms
64 bytes from 192.168.178.59: icmp_seq=7 ttl=64 time=16.149 ms
64 bytes from 192.168.178.59: icmp_seq=8 ttl=64 time=17.386 ms
Request timeout for icmp_seq 9
Request timeout for icmp_seq 10
Request timeout for icmp_seq 11
Request timeout for icmp_seq 12
Request timeout for icmp_seq 13
64 bytes from 192.168.178.59: icmp_seq=13 ttl=64 time=1136.231 ms
64 bytes from 192.168.178.59: icmp_seq=14 ttl=64 time=133.708 ms
64 bytes from 192.168.178.59: icmp_seq=15 ttl=64 time=11.374 ms
64 bytes from 192.168.178.59: icmp_seq=16 ttl=64 time=13.963 ms
Request timeout for icmp_seq 18
Request timeout for icmp_seq 19
Request timeout for icmp_seq 20
Request timeout for icmp_seq 21
64 bytes from 192.168.178.59: icmp_seq=22 ttl=64 time=111.126 ms
64 bytes from 192.168.178.59: icmp_seq=23 ttl=64 time=11.844 ms
64 bytes from 192.168.178.59: icmp_seq=24 ttl=64 time=18.976 ms
Request timeout for icmp_seq 25
Request timeout for icmp_seq 26
Request timeout for icmp_seq 27
Request timeout for icmp_seq 28
Request timeout for icmp_seq 29
64 bytes from 192.168.178.59: icmp_seq=28 ttl=64 time=2185.118 ms
64 bytes from 192.168.178.59: icmp_seq=29 ttl=64 time=1182.628 ms
64 bytes from 192.168.178.59: icmp_seq=30 ttl=64 time=179.364 ms
64 bytes from 192.168.178.59: icmp_seq=31 ttl=64 time=24.616 ms
64 bytes from 192.168.178.59: icmp_seq=32 ttl=64 time=10.638 ms

Looks like the BT scans somehow interfere with the wifi connection. Probably a limitation of the cheap hardware. Maybe something to be aware of.

Lars

Thanks for sharing this project. Works instantly compared to ibeacons that my rpi zero w was running previously.

1 Like

That looks exactly like my pings. What router do you have?
For reference, I’m using an Airport Express (gen3) extending an Airport Express (gen2).
Changing channels doesnt do anything.