[presence] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

Caswell1000 · July 3, 2018, 9:45pm

I had the wrong assumption that the wifi MAC is the same as the BT MAC, but re-reading the thread made me realise that they are not one, so now I have 100 registering on the jfreyer scale and I am at peace, thanks a lot for your hard work. and I did not have to look for the .sh file…

location/owner/garage/E8:4E:84:XX:72:XX{"confidence":"100","name":"CN2014","scan_duration_ms":"981","timestamp":"Tue Jul 03 2018 21:40:31 GMT+0000 (UTC)"}
location/owner/garage/14:9D:XX:81:25:XX{"confidence":"100","name":"HUAWEI P10 CN","scan_duration_ms":"5037","timestamp":"Tue Jul 03 2018 21:40:37 GMT+0000 (UTC)"}

adifoto · July 4, 2018, 12:42am

Ha Ha we making the same mistakes, not installing MQTT (i am glad i so your message) and forgetting that the one i sett was not BT MAC.
Now all works fine

THANKS SO MUCH for all that contributed and help with this project.
Finally my lights will stay one when i am home and my HA will not think that i left home

timkeizer · July 5, 2018, 7:13am

@NKF
Did you get it to work with your phones? I currently have an iPhone 5, working fine. iPhone 8, only visible when in the bluetooth scanning menu…

Anybody else having issues with the phone not being visible on BLE?

@andrewjfreyer What you have made is a great share!

eric3616191 · July 5, 2018, 12:01pm

good，Thanks for sharing, thank you

timkeizer · July 5, 2018, 7:47pm

uhm… and then I paired the iPhone 8 with a carkit, and after it … suddenly was detected when I got home.

andrewjfreyer · July 6, 2018, 4:58am

Hey all - check out my github for an alpha of the rewrite. The project is named, for the time being, monitor. Any thoughts or reports would be greatly appreciated. I have been testing for a very long time, and it works for my family with the same performance, if not a bit faster, while scanning about 75% less often than presence (substantially mitigating wifi interference issues).

Please let me know what y’all think.

timkeizer · July 6, 2018, 6:47am

What a nice surprise. Will be installing it asap, as I still have some interference. I’ll be sure to share on my experiences.

jpm · July 6, 2018, 7:42am

If you pushed it, then i already have it, since i made a script that checks for new versions and restarts the service.

Not particularly about this versions but I had some problems, already reported by many other, like the wi-fi interference (which made me make the “restart script”, but know my most commum problem is reporting of “name”: “unknown”, which happens about 7 times out of 10 and makes me get a lot of in/out of the house HA task running.

I also have a device (IOS 11.3.x) that takes about 16min to get the out of the house, which not sure if this is connected is my 1st device on the owner file, the rest is almost instantaneous.

I also notice that the devices that are further on the house, so far I have two instances, so less used is mare unresponsive and “loses” the connection more often that the one where we are connected most of the time.

To end and also Its a requisite so Please disregard if you have no interest in it, but I was wondering if it would be possible for the script to have a “scan” feature, this is because I have a company that cleans my house, most of the times they send the same people, but some times, they send others, and I notice that some clean very well and others…not so well, and I would like to “know” who is/was in the house so that I can specific that I dont want “this, this and that” and want, “this, this and that”

PS: I also connected my pie3 to the network to try to avoid the wifi interference.

andrewjfreyer · July 6, 2018, 2:37pm

Hi @jpm - The script I reference above is different from the presence script that is the subject of this thread. The monitor script is currently in alpha.

No, it is not possible to scan for arbitrary devices and know who owns those devices. You will either have to know the Bluetooth MAC address of each housekeeper’s device

leranp · July 6, 2018, 2:58pm

hii,
i am trying to get this working ,
but i have an errors

pi@raspberrypi:~/presence $ sudo systemctl status presence.service
● presence.service - Presence service
   Loaded: loaded (/etc/systemd/system/presence.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-07-06 14:53:30 UTC; 4min 16s ago
 Main PID: 6454 (bash)
   CGroup: /system.slice/presence.service
           ├─6454 /bin/bash /home/pi/presence/presence.sh &
           └─6838 sleep 6

Jul 06 14:56:44 raspberrypi bash[6454]: Connection Refused: not authorised.
Jul 06 14:56:44 raspberrypi bash[6454]: Error: The connection was refused.

andrewjfreyer · July 6, 2018, 3:08pm

Looks like a broker authentication issue. Have you verified your username and password?

leranp · July 6, 2018, 3:16pm

yes
i have also tried to do pub with commend and it worked,
my broker is in different raspberry

i have changed the file permission to root and now i have different error
pi@raspberrypi:~ $ sudo systemctl status presence.service
● presence.service - Presence service
Loaded: loaded (/etc/systemd/system/presence.service; enabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Fri 2018-07-06 15:25:36 UTC; 2s ago
Process: 4904 ExecStart=/bin/bash /home/pi/presence/presence.sh & (code=exited, status=1/FAILURE)
Main PID: 4904 (code=exited, status=1/FAILURE)

Jul 06 15:25:36 raspberrypi systemd[1]: presence.service: Failed with result ‘exit-code’.

andrewjfreyer · July 6, 2018, 4:38pm

I’d suggest running manually with the debugging flag on to see what’s happening.

leranp · July 6, 2018, 9:04pm

this is the result
pi@raspberrypi:~/presence $ bash presence.sh -d
presence 0.4.28 - Started. Performance predictions based on current settings:

Est. to verify all (1) owners as ‘away’ from all ‘home’: 30 seconds to 120 seconds.
Est. to verify one owner is ‘away’: 30 to 120 seconds.
Est. to recognize one owner is ‘home’: 0.15 seconds to 11 seconds.
Connection Refused: not authorised.
Error: The connection was refused.
DEBUG MSG: Scan result:
/owner/raspberrypi_zero/C1:0D:24:84:D5:FB { confidence : 0, name : Unknown, scan_duration_ms: 5059, timestamp : Fri Jul 06 2018 21:03:13 GMT+0000 (UTC)}
Connection Refused: not authorised.
Error: The connection was refused.
DEBUG MSG: Scanning for 0 guest devices between owner scans (no owner device is present).
DEBUG MSG: Remainder of delay before next owner scan: 6
DEBUG MSG: Scan result:

timkeizer · July 6, 2018, 9:38pm

Nice! i just got it working . Needed to install BC and Bluez_hcidump, before autostart got working (so tried manual start with -d). Now it seems to work like a charm. Will monitor coming days to see if interference is lower. Thanks again Andrew

andrewjfreyer · July 6, 2018, 11:12pm

Thanks. I know it’s probably a pain to get up and running since I haven’t written any docs for that yet. Appreciate the testing!

andrewjfreyer · July 6, 2018, 11:12pm

Make sure to run the script as root. That’s needed for a number of bluetooth services.

xbmcnut · July 7, 2018, 7:06am

For those wanting to get ‘presence’ to fully integrate into HA so you have device_tracker.xxx, here is what I did to get the UI shown below in both native and lovelace.

N.B All of my ‘presence’ items are in one ‘package’ which keeps it real tidy.

  - alias: Pete Occupancy
    hide_entity: true
    trigger:
      - platform: numeric_state
        entity_id: sensor.pete_home_occupancy_confidence
        above: 10
    action:
      - service: homeassistant.turn_on
        data:
          entity_id: input_boolean.pete_occupancy
      - service: mqtt.publish
        data:
          topic: 'location/pete'
          payload: 'home'

  - alias: Pete Away
    hide_entity: true
    trigger:
      - platform: numeric_state
        entity_id: sensor.pete_home_occupancy_confidence
        below: 10
    action:
      - service: homeassistant.turn_off
        data:
          entity_id: input_boolean.pete_occupancy
      - service: mqtt.publish
        data:
          topic: 'location/pete'
          payload: 'not_home'

I have of course also defined a device_tracker below and then tweaked that device tracker in the known_devices.yaml adding a name and picture.

device_tracker:
  - platform: mqtt
    devices:
      pete_mia1_pizero: 'location/pete'

leranp · July 7, 2018, 8:18am

i did all the installation again in raspberry pi zero and it doesn’t work ,
i am sure it is the mosquito broker, but when pub\sub to the broker manually it is o.k.
do you have any idea for solve this issue?

NKF · July 9, 2018, 1:03pm

Sorry, no time to test it. Maybe later this week.

I was thinking, is it a possibility that my (bluetooth) smart watch connection is interfering?