I have been using @andrewjfreyer’s excellent new beta of his monitor script to watch for bluetooth presence. I integrated his previous script in a basic proof-of-concept Appdaemon app a few months ago (see here), and am now looking to move to the next level and am seeking the advice of those more wise and experienced than I.
In brief, the shell script runs on a number of (actually 3, at present) Pi Zeros scattered about my house. When a specified device is spotted a Pi it posts an MQTT message (“100”). If the Pi subsequently fails to spot the device next time it looks it will publish, over time, a decreasing sequence (“50”, … “6”, "0) for that device.
I am thinking the best way to proceed is to have an app keep a device-specific global variable of the sum of the last posting from each of the [3] Pi’s. (This will require storage of the previous state for the device, as well as the sum.) And then if the sum changes state to/from zero I will flip a “device present” input_boolean.
(I intend to further develop the app to incorporate other information - network presence, etc. once this first stage is working.)
My questions are
does this seem like a sensible approach to the problem?
how does one declare and store persistent variables in AD?
i am probably to tired at the moment to give a decent reaction to this, because i read your posting 3 times and i dont seem to get it in my head but
this i can give an answer too
depending on how persistent it should be you can go different ways.
you can safe variables to a db
you can safe variables to a text file (which i have been doing for a long time now)
you can use set_state to save variables to a sensor in HA.
the last one is the one i use most. because it makes variables visible and you can listen to the state changes. with closing from AD, HA or starting AD or HA up you can set the value or save sensor values in general. making that it keeps state even with restarting.
i have lots of AD variables stored in HA as sensors. but i also am a bit freaky about logs. i just love to be able to see the state everything is in.
you can create an app with priority 1. that will initialise before any other app. in that app in the initialise you can use something like this:
for sensor,settings in self.args["sensors"].items():
if not self.entity_exists("sensor." + sensor):
sensorstate = self.get_saved_value(sensor,settings["value"])
self.set_state("sensor." + sensor, state = sensorstate, attributes = {"friendly_name": settings["friendly_name"]})
i use that to make sure that the sensor gets the last state again before anything else happens. the get_saved_value is a function i created myself, everything else is default AD
2) you can use get_state and set_state also for custom attributes. i use that also to view state history, by setting the state from a certain time
3) you can use set_state in app A, and a listen_state in app B to transport values from 1 app to another. for example i use self.set_state(“sensor.notify_message”, state=“some message”) in all kind of apps and then i use a notify app like this:
def initialize(self):
if not self.entity_exists("sensor.notify_message"):
self.set_state("sensor.notify_message",state=" ")
self.listen_state(self.prepare_notify,"sensor.notify_message")
def prepare_notify(self, entity, attribute, old, new, kwargs):
if new == " ":
return
#do some stuff with the message, for example find out if it should repeat
self.call_service("notify/pushetta",message=kwargs["message"])
# you can place any kind of notify here. TTS send it to a notify service, or let alexa speak the text
you can also use the listen_state for those specific sensors to store the value from that moment (in a txt file or a db) which you can use to retrieve it in the prio 1 app.
i hope i have given you some pointers that you can use.
Well I don’t use the script, but I am considering integrating it once I got some free time. But if I wanted to do the above, I will use @ReneTode’s sensors approach but in a different way.
To start with I am a bit lazy, so will prefer the App creating and populating the sensors itself. So I begin
First I will use one of the mqtt plugins created, and listen to the topic location/#. When I receive any thing like say location/owner/living_room/00:00:00:00:00:00, in which I am expecting a payload like { confidence : 100, name : Andrew’s iPhone, scan_duration_ms: 500, timestamp : Sat Apr 21 2018 11:52:04 GMT-0600 (MDT)}. I will check if a sensor like this exist sensor.living_room_andrews_iphone, which is basically a strip of the payload and location. If it doesn’t I use HA’a mqtt discovery to create it, and have HA deal with updating it itself; or on the other hand use the self.set_state() function, anyone you prefer. By doing the above, the system automatically creates sensors for every phone registered and I need not touch the sensor.yaml in HA. This in my opinion is extremely flexible.
Next create listen states in another app, and then check the state value which is the confidence. As you creat each sensor, you can store them in a file as suggested above. I use python shelve to store my data. So when AD restarts, it checks for those sensors and repopulates their values and listen function within itself. Their present states can be stored in a dictionary within an app, and each time one if updated, you calculate what you need based on previous values.
This way in my opinion, gives so much flexibility with relatively little code written. I haven’t done it, as I said i don’t use it. But if I was the one, that’s what I would do. Lazy but it should work.
Hi @Odianosen25 - no, I’m afraid life got in the way and I haven’t had a chance to make any real progress on this yet.
In fact, I am not actually convinced that @andrewjfreyer’s script publishing declining probabilities is the right approach. My thinking was to ignore anything other than 100% confidence (ie a BT presence is confirmed) and move the logic of whether a non-100% confidence implies absence to the python code. One of the reasons for this is that I would like to include other-sensors (in particular a network connection) in my presence determination. (So, for example, if my phone connects to wifi then I am present, regardless of what the script confidence is. But if all of the confidence publications are non-100 and an external door has recently opened then I am absent, regardless of whether my router thinks I am wifi-connected.)
I’d be very interested to hear if you manage to make any progress - and should have time for some testing if you need it.
Wow that’s some complexity for presence detection @PianSom , but you make a valid point. As I just setup the script yesterday and going via what it published, though I was home it sometimes registered me as 83%. So the ability to combine all these in code, seems valid to be honest.
@PianSom so I don’t hijack the other thread anymore I have had really really good results with @andrewjfreyer script. So much so, I don’t even bother with the unifi tracker anymore for anything. Running just the script on the zero’s with nothing else on them, and don’t seem to have any major wifi interference. Have it flipping a input boolean as to if I am home or not after a 5 minute grace period, and this is the last week matched up to the unifi. It picks up when I go to the shop round the corner which the unifi can’t.
So, I have made a start on coding. I abandoned my original idea of keeping track of @andrewjfreyer’s probability of presence, and decided that it was easier to just look at 100% certainty but to have a time delay before changing the presence boolean.
What happens is -
the monitor script runs on three Pi Zeros, and posts to MQTT in a vanilla fashion
I use unifi2mqtt to post to MQTT whether a phone device is present on the network
in HA, MQTT subscriptions listen for changes in the above postings and flip a sensor Seen/Not seen as appropriate (NB Seen for monitor means 100% certainty)
the code below changes a presence boolean if, for a given person, (a) any sensor is showing Seen then the presence boolean is set to TRUE (and any timer previously set up for the next condition is cancelled), or (b) if all sensors are showing Not seen then a timer is set for n minutes, after which the boolean is changed to FALSE
With apologies in advance for my very basic Python coding (I can just feel the stinging @ReneTode rebuke now! ) here is where I am:
class person_presence(hass.Hass):
def initialize(self):
self.log("Hello from person_presence", level='DEBUG')
self.listen_state(self.change_boolean, self.args["network_device"])
self.listen_state(self.change_boolean, self.args["bluetooth_device_kitchen"])
self.listen_state(self.change_boolean, self.args["bluetooth_device_utility"])
self.listen_state(self.change_boolean, self.args["bluetooth_device_upstairs"])
def change_boolean(self, entity, attribute, old, new, kwargs):
self.log("Potential change of state for {}".format(self.args["person_boolean"]))
if self.get_state(self.args["network_device"]) == self.args["home_state"] or \
self.get_state(self.args["bluetooth_device_kitchen"]) == self.args["home_state"] or \
self.get_state(self.args["bluetooth_device_utility"]) == self.args["home_state"] or \
self.get_state(self.args["bluetooth_device_upstairs"]) == self.args["home_state"]:
self.person_present(self)
elif self.get_state(self.args["network_device"]) != self.args["home_state"] and \
self.get_state(self.args["bluetooth_device_kitchen"]) != self.args["home_state"] and \
self.get_state(self.args["bluetooth_device_utility"]) != self.args["home_state"] and \
self.get_state(self.args["bluetooth_device_upstairs"]) != self.args["home_state"]:
self.person_poss_absent(self)
def person_present(self, kwargs):
self.log("Present: Immediately updating {} from {} to on, and cancelling Poss Absent".format(self.args["person_boolean"], self.get_state(self.args["person_boolean"]) ), level='DEBUG')
self.turn_on(self.args["person_boolean"])
try:
self.log('Going to cancel a Poss Absent action for {}'.format(self.args["person_boolean"]), level='DEBUG')
self.cancel_timer(self.handle)
except AttributeError:
self.log('Tried to cancel a Poss Absent action for {}, but none existed'.format(self.args["person_boolean"]), level='DEBUG')
def person_poss_absent(self, kwargs):
self.log("Possibly Absent: Updating {} to from {} to off in {} mins".format(self.args["person_boolean"], self.get_state(self.args["person_boolean"]), self.args["away_delay"] ), level='DEBUG')
timer=60*self.args["away_delay"]
self.handle=self.run_in(self.person_absent, timer)
def person_absent (self, kwargs):
self.log("Absent: Turning off {} now".format(self.args["person_boolean"]), level='DEBUG')
self.turn_off(self.args["person_boolean"])
I need to spend some more time tuning monitor and working out whether to use non-default settings to trigger bluetooth scans more efficiently (eg when another sensor suggests a change in presence is imminent; I have got some cheap IBeacons to put in the car, which I am hoping will have enough range to spot me before I get home)
the Unifi router presence is not very reliable. I think there are a couple of branches of the base code worth exploring
I would love to do away with the HA MQTT interface and just use AD for MQTT. One day …
Never knew about this, why use it if just sticking to the presence script alone?
Good start, will look into it and make some mods of my own and post later. Haven’t had the time to work on this side of things.
But it seems your code works off the back of the user setting up different things in HA and AppD. And if there are multiple ppl in the home, it just escalates more and to me that is a drag
Lets see how it goes, and I will endeavour to send update on this.
Oh, really? Tell me more. I can’t see anything in the AD docs about having a MQTT interface. I’d much prefer to do all my listening and processing in AD and then just use MQTT as an output for HA to pick up.
I agree that the the whole thing is a bit held together by string (with multiple sensors all over the place) and I would much rather have something simpler and easier to configure. Having AD directly subscribing/publishing with MQTT would be fantastic.
[EDIT - unless you meant using this type of thing??]
Because for me wifi sometimes has longer range than Bluetooth, so especially if I am approaching on foot I will connect by wifi before monitor spots me. (The downside is that the router often takes longer to realise I am gone, so when I refine my [lack of] presence algorithm from a simple AND I will take that into account.)
Since it takes longer, might just use it for only when it registers u home and the others are still giving a presence of less than 100% and ignore the other.
And all I have to do to do something when a presence/owner/pi0ups/XX:XX:XX:XX:XX:XX is
1 - declare a MQTT section in appdeamon.yaml which includes the topics I am interested (eg presence/#) in, and
2 - in an initilalize have a self.listen_event(self.some_function,'presence/owner/pi0ups/XX:XX:XX:XX:XX:XX')
And then I can publish at will using self.mqtt_send(topic, payload, qos=0, retain=false, kwargs)
First you need to move the plugin files into your plugin directory. normally one should be able to just put custom plugins into a custom_dir folder in the appdaemon conf directory, but I haven’t been able to get that to work, so still trying to figure that out.
So what I do is to load mine into my directory which is /home/pi/srv/appdaemon/lib/python3.5/site-packages/appdaemon/plugins if using hassbian. Still waiting on @aimc or @ReneTode to show me otherwise. Till then well this is all I know.
Your listen function is more like this: self.listen_event(self.some_function,<event name as specified in appdaemon.yaml('MQTT' default)>, topic = 'presence/owner/pi0ups/XX:XX:XX:XX:XX:XX')
Below is a sample I use for my TTS at home
import appdaemon.plugins.mqtt.mqttapi as mqtt
import json
class TTSApp(mqtt.Mqtt):
def initialize(self):
self.set_namespace('mqtt')
self.speed = self.args['speed']
self.pitch = self.args['pitch']
self.allowed_sites = self.app_config["snips_data"]["sites"]
for site in self.allowed_sites:
self.listen_event(self.tts_send_message, 'MQTT', topic = 'ha/tts/{}/say'.format(site.lower().replace(' ', '_')))
def tts_send_message(self, event_name, data, kwargs):
topic = data['topic']
text = "<pitch level=\'{}\'><speed level=\'{}\'>{}</speed></pitch>".format(self.pitch, self.speed, data['payload'])
self.log(data['payload'], level='INFO')
siteId = topic.split('/')[2]
payload = json.dumps({"siteId":siteId, "text":text})
self.mqtt_send('hermes/tts/say', payload, namespace = 'snipsmqtt')
please dont apologise for simple coding. thats the best there is
and dont forget, i love helping people improve but i am not even close to being an expert (but in the land of the blinds oneeye is king)
i would probably write the app the same when i start writing it.
then i would start optimising for readability.
i would place the devices in a list in the yaml and then use this:
def initialize(self):
for device in self.args["devices"]:
self.listen_state(self.change_boolean, device)
self.log("Hello from person_presence", level='DEBUG')
in change_boolean then you could use
Person_present = False
for device in self.arg["devices"]:
if self.get_state(device) == self.args["home_state"]:
Person_present = True
if Person_present:
self.person_present(self)
else:
self.person_poss_absent(self)
the big advantage is that you now can easy add or delete devices in the yaml without changing the code.
which also would make the code more sharable.
about the MQTT part: i havent played around with it enough to be able to get into it.
i have a working mqtt app, but not based on the pluging, but just with:
import paho.mqtt.client as mqtt
import paho.mqtt.publish as publish
import paho.mqtt.subscribe as subscribe
i also have the plugin (same way as @Odianosen25 ) and an app based on that but i created it a while ago and dont know for sure if it is working
Thanks for the input. The plugin works really great as my system is like 60% MQTT especially considering I use Snips and Zigbee2mqtt.
It makes it easier for me to manage things, and I connect to two different brokers at the same time. That’s why I have different “namespaces”, snipsmqtt and mqtt. And I am in the process of adding a third cloud based on.
Thanks @ReneTode - much more elegant and efficient. I knew your insight would be valuable. (In spite of what @Odianosen25 thinks )
I hadn’t realised that the MQTT stuff was in a custom component. I try to avoid these, as it’s yet another thing to make sure continues working over time. My current method - cumbersome though it is - at least is based on core functionality. Hopefully AD will natively include MQTT functionality at some stage, because - like @Odianosen25 - virtually all of my IoT devices are MQTT-based .
I have had really really good results with 
) here is where I am:
I am so telling him you said that 
)