[presence] Reliable, Multi-User, Distributed Bluetooth Occupancy/Presence Detection

This problem likely came about from a permissions issue. I would guess that you had previously created/run the script that created behavior preferences as root and then subsequently the script was run with user permissions. When the script tried to add the necessary options, it would fail.

Errors like this will always be correctable by removing behavior preferences and re-running the script.

Thanks for the report! I’ll add some more error checking tonight.

It was a fresh card, just off the boat from China, so the first run was when the Zero rebooted after I plugged it in, and so was started by the service, which of course has

User=root
ExecStart=/bin/bash /home/pi/presence/presence.sh &
WorkingDirectory=/home/pi/presence

And yes, root doesn’t have write permission for ~pi/presence.

Not the biggest bug in the world :slight_smile:

Hi @andrewjfreyer - have you thought about implementing a LWT to help determine Online/Offline status?

wrong thread …

Progress is being made on my attempt at getting this to work. I ran sudo systemctl status presence.service as @jnvd3b suggested and it found the phone. It also had the following line:

Apr 24 20:28:23 raspberrypi bash[369]: /home/pi/presence/presence.sh: line 279: -h: command not found

For reference, line 279 is:

$MQTTPubPath -h “$mqtt_address” -u “$mqtt_user” -P “$mqtt_password” -t “$mqtt_topicpath$1” -m “{“confidence”:”$2",“name”:"$name",“scan_duration_ms”:"$4",“timestamp”:"$stamp"}"

Any suggestions on how to fix this error?

Thanks.

This means that you don’t have mosquitto installed properly. What is the output of the command:


which mosquitto_pub

Running which mosquitto_pub returns nothing.

Would the fact that I have mosquitto running on a separate RPi factor into this? I have a few of the “Bruh multisensors” and multiple lights running via MQTT. They have been basically working fine for months.

Thanks.

Per the instructions on github, you need mosquitto installed for this script to work as well. We’re not using these pis as a broker, but as a publisher.

Updated to 0.4.25 which includes a host of new features and options, my favorite of which is triggering. By invoking the script with the -t flag, a scan operation will only occur in response to receiving an MQTT message posted to /$mqtt_topicpath/scan.

If -t1 is used, then a message trigger will be required for every scan.

If -t2 is used, a message trigger will be required only when at least one owner is home. Otherwise, periodic scanning will be used. Connect the trigger to a door sensor (e.g., front door, garage door, etc.), and the system will begin scanning for an owner’s departure when the door is used. This will dramatically reduce the number of scan operations performed when at least one owner is home.

For example, in node-red:

Other options include:

-d for printing debug messages

-c for only messaging changes in state of a particular owner or guest device

-b binary messaging only - only print confidences of 100 or 0.

So, if we ran 0.4.25 with the following options:

bash presence.sh -c -b -t2

Then the script would automatically scan (according to behavior_preferences) when all owners are away and will not message the broker at all (since nothing changes when all owners are away). Once an owner arrives home, scanning switches from periodic to triggered. The trigger can be coupled to the front door so that when a second owner comes home and uses the front door, a scan is initiated to detect the user. Only the new user’s status change would be reported.

In addition, the message of the triggering mqtt message can include a duration parameter that will define how long scanning should repeat after it is triggered. The default is 120 seconds.

I know this is a lot; please let me know if anyone experiences issues or does not understand one or more of the new options.

3 Likes

Okay. I must have misunderstood your reply to my first post in this thread about installing mosquitto on mulitple RPi’s. When you responded:

I thought you meant I did not have to install mosquitto on the RPi’s that have your project period.

My mistake.

Ah, yes. My fault for the confusion. I mean, “if you have mosquitto installed on this pi then you do not need to reinstall”

I updated to 0.4.25. I did not change behavior_preferences I did not add any flags.

Got the following log errors:

Apr 25 07:02:53 pi0ute systemd[1]: Started Presence service.
Apr 25 07:02:53 pi0ute bash[9192]: cat: /home/pi/presence/guest_devices: No such file or directory
Apr 25 07:02:53 pi0ute bash[9192]: presence 0.4.25 - WARNING:
Apr 25 07:02:53 pi0ute bash[9192]: Name scan timeout is relatively low at 0. New bluetooth
Apr 25 07:02:53 pi0ute bash[9192]: devices may take more time than this to be discovered.
Apr 25 07:02:53 pi0ute bash[9192]: presence 0.4.25 - WARNING:$
Apr 25 07:02:53 pi0ute bash[9192]: Delay between owner scans when away is relatively
Apr 25 07:02:53 pi0ute bash[9192]: low at 0. This may slow down the server because the BT hardware
Apr 25 07:02:53 pi0ute bash[9192]: will be actively scanning more frequently. Consider increasing
Apr 25 07:02:53 pi0ute bash[9192]: this value. The greater this value, the more time it will take
Apr 25 07:02:53 pi0ute bash[9192]: to recognize when a device has arrived.
Apr 25 07:02:53 pi0ute bash[9192]: presence 0.4.25 - WARNING:
Apr 25 07:02:53 pi0ute bash[9192]: Delay between owner scans when present is relatively
Apr 25 07:02:53 pi0ute bash[9192]: low at 0. This may slow down the server because the BT hardware
Apr 25 07:02:53 pi0ute bash[9192]: will be actively scanning more frequently. Consider increasing
Apr 25 07:02:53 pi0ute bash[9192]: this value. The greater this value, the more time it will take
Apr 25 07:02:53 pi0ute bash[9192]: to recognize that a devices has left.
Apr 25 07:02:53 pi0ute bash[9192]: presence 0.4.25 - WARNING:
Apr 25 07:02:53 pi0ute bash[9192]: Verification loop (i.e., verification_of_away_loop_size) is relatively
Apr 25 07:02:53 pi0ute bash[9192]: low at 0. This can increase the risk of false exit events.
Apr 25 07:02:53 pi0ute bash[9192]: The greater this value, the lower the probability of false exit events.
Apr 25 07:02:53 pi0ute bash[9192]: presence 0.4.25 - WARNING:
Apr 25 07:02:53 pi0ute bash[9192]: Verification loop delay is relatively short or
Apr 25 07:02:53 pi0ute bash[9192]: low at . This can increase the risk of false exit events.
Apr 25 07:02:53 pi0ute bash[9192]: The greater this value, the lower the probability of
Apr 25 07:02:53 pi0ute bash[9192]: false exit events.
Apr 25 07:02:53 pi0ute bash[9192]: sleep: invalid time interval ‘’
Apr 25 07:02:53 pi0ute bash[9192]: Try 'sleep --help' for more information.
Apr 25 07:02:53 pi0ute bash[9192]: sleep: invalid time interval ‘’
Apr 25 07:02:53 pi0ute bash[9192]: Try 'sleep --help' for more information.
Apr 25 07:02:58 pi0ute bash[9192]: sleep: invalid time interval ‘’
Apr 25 07:02:58 pi0ute bash[9192]: Try 'sleep --help' for more information.

with the latter repeated.

Replacing my behavior_preferences with the following version solved the problem:

name_scan_timeout=5
delay_between_owner_scans_away=6
delay_between_owner_scans_present=30
verification_of_away_loop_size=6
verification_of_away_loop_delay=3
beacon_scan_interval=5
beacon_scan_enabled=0
1 Like

The behavior preferences file changed format and needs to be updated. There is some crude backward compatibility built in, but the best solution is to delete the file and start fresh. The format you copied above is correct for anyone who would prefer to copy paste.

1 Like

Nice work! I will have a play today.

Did you have any thoughts about my suggestion of implementing LWT above?

Also, a couple of suggestions/thoughts on the changes:

  • I like the -t2 option, but there are two circumstances that I can imagine where it could cause an issue. Firstly, if a door sensor fails (mine do occasionally due to a z-wave error, or a battery failure). And secondly, I tend to have my back door open for a few hours in the Summer, and my family may come and go. Because of these type of behaviours, I wonder if it may be sensible to have a low frequency (every 5 mins?) scan, rather than just an on-request one? (Of course, this can be taken care of elsewhere in the control process, rather than in the script.)
  • I like the -c flag to stop the large amounts of MQTT posting. Might it be sensible to make the default behaviour that the MQTT message is posted with the retain flag if -c is specified?

EDIT - the -c flag is not quite working as expected. I am sporadically getting repeat messages. I have not been able to determine the circumstances as yet. For example, with -b -c

presence/owner/pi0ute/[redacted_mac] {"confidence":"100","name":"BunnyHop","scan_duration_ms":"2097","timestamp":"Wed Apr 25 2018 09:12:54 GMT+0100 (BST)"}
presence/owner/pi0ute/[redacted_mac] {"confidence":"100","name":"BunnyHop","scan_duration_ms":"4947","timestamp":"Wed Apr 25 2018 09:13:48 GMT+0100 (BST)"}
1 Like

@andrewjfreyer Excellent project, good work. I have 2 questions, apologies if you have addressed already but I couldn’t find them addressed.

  1. Have you considered sharing this as a Hassio addon? I don’t know how straightforward that would be as you require access to the bluetooth hardware. I’m interested to help out if this is of interest.

  2. Have you experimented with the minimum number of pi’s required to satisfy yourself of adequate presence detection? My suggestion is you could combine bluetooth presence sensing with the other techniques you listed (geofence etc) and use the bayesian sensor to calculate the likelihood of presence.

Cheers

Thank you very much for the updates!
I’ll run it asap.

1 Like

Hi, thanks for this great concept. I’ve got it running on a a Pi W, and subscribing to the broker works great.

But here’s the thing. I already use MQTT for another sensor, and my research says HA can only connect to one broker… but your example suggested multiple pis for each level without mentioning this. Am I missing something?

[EDIT]
I think I get it now - I can use my existing broker - I just point each of the pis to that broker using the mqtt_preferences file?

1 Like

Yes - that is correct!

1 Like

@PianSom regarding number one, I included a periodic timer in node-red to do exactly this.

I’ll see if I can figure out why you’re seeing duplicate messages. I did push a small update to fix one bug last night; perhaps you pulled the version with the bug?

1 Like

I think I’ll keep this out of the add-on space for the time being since this is meant to be run - at least partially - on devices separate from HA.

Re the second point, I think I’ll leave that to the user for the time being. Yes, many sensors can be combined in different ways. For my use case, an average has made a lot of sense. I’ll investiage and draft instructions for stats sensors sometime later. Thanks!

1 Like