Home (Monitor) Presence App V2

rationi_meae · August 3, 2020, 10:32pm

To pair I tried this:

cd monitor
sudo systemctl stop monitor
sudo bash monitor.sh -c <bluetooth mac address>

And I got this:

> starting monitor.sh (v. 0.2.200)...
> removing web request caches
> removing previous pairings to <bluetooth mac address>
> creating connection to <bluetooth mac address>...
> witihin 5 seconds, please set <bluetooth mac address> to discoverable...
> error: connection to <bluetooth mac address> failed

The bluetooth on the galaxy and iphone are turned on.

For the binary_sensor.monitor_galaxy (and _iphone), I only see “nearest_monitor: unknown.”

So near yet so far. What else am I missing? Thank you.

Odianosen25 · August 4, 2020, 2:16pm

Until you are able to pair, I won’t be able to assist I am afraid. What’s going on is it’s just not seeing your Bluetooth.

Regards

rationi_meae · August 4, 2020, 10:03pm

I finally got it to pair.

sudo bash monitor.sh -c <bluetooth mac address>

> starting monitor.sh (v. 0.2.200)...
> removing previous pairings to <bluetooth mac address>
> creating connection to <bluetooth mac address>...
> witihin 5 seconds, please set <bluetooth mac address> to discoverable...
> connection created to <bluetooth mac address>

Now, in monitor README.md, it has a section for “Home Assistant Example” that shows some configuration of the HA yaml files. If my understanding is correct, I do not need to do those because Monitor-App actually takes care of those behind the scenes. Am I correct?

Thank you.

Odianosen25 · August 5, 2020, 9:01am

@rationi_meae,

Yes you are correct, that’s the entire point of the app. If you have further questions, do mention me in your questions, so I can send a faster reply. For some reasons, I am not getting any notifications of a question being asked, and why the late replies.

Regards

skynet01 · August 10, 2020, 8:22am

@Odianosen25 thank you for this amazing add on. I’ve been juggling monitor installs manually for about a year and wish I found this sooner. I got a few questions:

I noticed in the AddDaemon under Entities > Mqtt there are entities that start with monitor.xxx like “monitor.monitor_state” but they are not showing up in HA. Is that correct? Any way to see from HA how many nodes are online?
Anyway to rename the binary_sensor entities that are created? Or at the very least rename the friendly name?
Can the gateway sensor be a cover or only binary sensors are supported? like (cover.garage_door) it has different states like closed, opening, open etc…
How do you do Service calls? Is that something that’s done from HA using a script? or it’s done from AppDaemon
Is there an option to turn off this feature?: "If any location doesn’t respond after a set time system_timeout , it sets all entities generated from that location to 0" this results incorrectly marking people away and my alarm arming while users are still home. It would make more sense to check if the gateway sensors were triggered beforehand, or throw a flag that some node is down. Also if the user walks to the node that’s still online it won’t do a scan since the gateway sensor wasn’t triggered…

Thank you for your hard work

Odianosen25 · August 10, 2020, 12:56pm

Hello @skynet01,

Yes as those in AD are for AD. I built it really for myself as I like using AD’s internal entities more, so allowed users to decide which they will prefer to use

Yes. I left it as MQTT data in JSON, not sure why I didn’t just create entities for them, but you can create your if you like, or put in a feature request on GIT for it to be added in next release.

I didn’t consider that, will look into it in next release. Might be an easy fix, and what extent do you mean? So just hit me a request on Git also

For now I left it as binary_sensors only. I am sure in HA, you can develop binary sensors based on the covers, so easy to add. But it shouldn’t be an issue. Still send me a request on Git, and kindly let me know the different states covers have. So when adding it, I know what I am doing, since I got none

For now, its an internal AD stuff only, and I primarily use it in other AD apps that need to instruct monitor to do stuffs.

Nope didn’t allow for that. As if it is disabled, it will lead to false readings in the event the node is down, and you wouldn’t want that either.

Hmmm is your monitor system set to scan depart only on trigger? that should solve it.

it does all this, as its set to only scan when the gateway is opened. The developer added the only scan at trigger for this exact reason, but you need to run your monitor nodes with this option.

True. if you got motion sensors, you can add them and it will make it more reliable.

Thanks you, but its work not from me alone, by others also. And just put in the requests on Git, and I will try look into them when next I look into the script. Till then I'll be back

skynet01 · August 10, 2020, 11:56pm

Thank you for awesome breakdown of the answers to the questions. I created a few git issues recoomendations as per your request!

Eeeeeediot · August 22, 2020, 4:41pm

Hello @Odianosen25.

I run Hass.io/HassOS on a Rpi4 and I have installed AppDaemon 4 with the HASS and MQTT plugins and the app through HACS. I have also set up the config file as per below and there doesn’t seem to be any errors but nothing seems to be tracked either. The 3 sensors (everyone_home etc…) were created but all show off/away. Should there not be entities created?

Also, how do you run the appdeamon app with the extra arguments using Hassio/HassOS?

appdaemon.yaml

secrets: /config/secrets.yaml
appdaemon:
  latitude: xxx
  longitude: xxx
  elevation: xxx
  time_zone: Europe/Amsterdam

  exclude_dirs: 
    - to_install

  plugins:
    HASS:
      type: hass
    MQTT:
      type: mqtt
      namespace: mqtt
      client_host: 192.168.0.xxx
      client_user: xxxxxx
      client_password: xxxxxxxxx
      client_id: appdaemon

http:
  url: http://127.0.0.1:5050
admin:
api:
hadashboard:

home_presence_app.yaml

#################################################################################################################################################################################################
# Monitor App
#################################################################################################################################################################################################
home_presence_app:
  module: home_presence_app
  class: HomePresenceApp
  plugin: 
    - HASS
    - MQTT

  monitor_topic: presence
  mqtt_event: MQTT
  user_device_domain: device_tracker
  everyone_not_home: everyone_not_home
  everyone_home: everyone_home
  somebody_is_home: somebody_is_home

  depart_check_time: 30
  depart_scans: 3
  minimum_confidence: 60
  not_home_timeout: 15
  system_check: 30
  system_timeout: 60

  home_gateway_sensors:
    - binary_sensor.hall_frontdoor

  # reboot the all nodes at 12 midnight on Mondays and Thursdays
  scheduled_restart:
    time: 00:00:01
    days:
      - mon
      - thu
    location: all

  # other location configuration options
    #location: living_room, kitchen

    #location:
    # - living_room
    # - kitchen

  # home_motion_sensors:
  #   - binary_sensor.living_room_motion_sensor_occupancy
  #   - binary_sensor.kitchen_motion_sensor_occupancy
  #   - binary_sensor.hallway_motion_sensor_occupancy
    
  log_level: DEBUG
  known_devices:
    - xx:xx:xx:xx:xx:xx Stephen's Phone
    # - xx:xx:xx:xx:xx:xx April's iPhone
  
  # known_beacons:
  #   - xx:xx:xx:xx:xx:xx Odianosen's Car Keys
  
  # remote_monitors:
  #   disable: False
  #   kitchen:
  #     auto_reboot_when_offline: True
  #     host: !secret kitchen_monitor_host
  #     username: !secret kitchen_monitor_username
  #     password: !secret kitchen_monitor_password
    
  #   living_room:
  #     host: 192.168.1.xxx
  #     username: !secret living_room_monitor_username
  #     password: !secret living_room_monitor_password
  #     reboot_command: sudo /sbin/reboot now
  #     auto_reboot_when_offline: True
  #     time: 02:00:01

Appdeamon log

[17:44:06] INFO: Starting AppDaemon...
2020-08-22 18:44:09.839844 INFO AppDaemon: AppDaemon Version 4.0.3 starting
2020-08-22 18:44:09.841310 INFO AppDaemon: Python version is 3.8.2
2020-08-22 18:44:09.842438 INFO AppDaemon: Configuration read from: /config/appdaemon/appdaemon.yaml
2020-08-22 18:44:09.843669 INFO AppDaemon: Added log: AppDaemon
2020-08-22 18:44:09.844620 INFO AppDaemon: Added log: Error
2020-08-22 18:44:09.845674 INFO AppDaemon: Added log: Access
2020-08-22 18:44:09.846907 INFO AppDaemon: Added log: Diag
2020-08-22 18:44:09.912738 INFO AppDaemon: Loading Plugin HASS using class HassPlugin from module hassplugin
2020-08-22 18:44:09.980812 INFO HASS: HASS Plugin Initializing
2020-08-22 18:44:09.981660 INFO HASS: HASS Plugin initialization complete
2020-08-22 18:44:09.982447 INFO AppDaemon: Loading Plugin MQTT using class MqttPlugin from module mqttplugin
2020-08-22 18:44:10.016068 INFO MQTT: MQTT Plugin Initializing
2020-08-22 18:44:10.016884 INFO MQTT: Using 'appdaemon/status' as Will Topic
2020-08-22 18:44:10.017544 INFO MQTT: Using 'appdaemon/status' as Birth Topic
2020-08-22 18:44:10.019715 INFO AppDaemon: Initializing HTTP
2020-08-22 18:44:10.020881 INFO AppDaemon: Using 'ws' for event stream
2020-08-22 18:44:10.060268 INFO AppDaemon: Starting API
2020-08-22 18:44:10.071170 INFO AppDaemon: Starting Admin Interface
2020-08-22 18:44:10.072096 INFO AppDaemon: Starting Dashboards
2020-08-22 18:44:10.124135 INFO HASS: Connected to Home Assistant 0.114.3
2020-08-22 18:44:10.268553 INFO AppDaemon: App 'hello_world' added
2020-08-22 18:44:10.271661 INFO AppDaemon: App 'Recoder1' added
2020-08-22 18:44:10.279781 INFO AppDaemon: App 'home_presence_app' added
2020-08-22 18:44:10.284179 INFO AppDaemon: Found 3 total apps
2020-08-22 18:44:10.287412 INFO AppDaemon: Starting Apps with 3 workers and 3 pins
2020-08-22 18:44:10.296220 INFO AppDaemon: Running on port 5050
2020-08-22 18:44:10.325627 INFO HASS: Evaluating startup conditions
2020-08-22 18:44:10.421243 INFO AppDaemon: Got initial state from namespace default
2020-08-22 18:44:11.197314 INFO MQTT: Connected to Broker at URL 192.168.0.200:1883
2020-08-22 18:44:11.235129 INFO AppDaemon: Got initial state from namespace mqtt
2020-08-22 18:44:11.236378 INFO MQTT: MQTT Plugin initialization complete
2020-08-22 18:44:12.355214 INFO AppDaemon: Scheduler running in realtime
2020-08-22 18:44:12.366232 INFO AppDaemon: Adding /config/appdaemon/apps to module import path
2020-08-22 18:44:12.367500 INFO AppDaemon: Adding /config/appdaemon/apps/Monitor-App to module import path
2020-08-22 18:44:12.374881 INFO AppDaemon: Loading App Module: /config/appdaemon/apps/hello.py
2020-08-22 18:44:12.394323 INFO AppDaemon: Loading App Module: /config/appdaemon/apps/recoder.py
2020-08-22 18:44:12.405441 INFO AppDaemon: Loading App Module: /config/appdaemon/apps/Monitor-App/home_presence_app.py
2020-08-22 18:44:12.411578 INFO AppDaemon: Initializing app hello_world using class HelloWorld from module hello
2020-08-22 18:44:12.414985 INFO AppDaemon: Initializing app Recoder1 using class Recoder from module recoder
2020-08-22 18:44:12.420228 INFO AppDaemon: Initializing app home_presence_app using class HomePresenceApp from module home_presence_app
2020-08-22 18:44:12.623133 INFO hello_world: Hello from AppDaemon
2020-08-22 18:44:12.626715 INFO hello_world: You are now ready to run Apps!
2020-08-22 18:44:12.634770 DEBUG Recoder1: Calling listen_state for Recoder1
2020-08-22 18:44:12.653630 DEBUG home_presence_app: set state: presence.monitor_state, {'state': 'idle', 'attributes': {'locations': [], 'version': '2.3.4', 'nodes': 0, 'online_nodes': [], 'offline_nodes': [], 'friendly_name': 'Monitor System State'}, 'replace': True}
2020-08-22 18:44:12.655160 WARNING home_presence_app: home_presence_app: Entity presence.monitor_state not found in namespace mqtt
2020-08-22 18:44:12.657322 INFO AppDaemon: home_presence_app: Entity presence.monitor_state created in namespace: mqtt
2020-08-22 18:44:12.662180 DEBUG home_presence_app: Calling listen_state for home_presence_app
2020-08-22 18:44:12.675804 INFO home_presence_app: Previous message repeated 1 times
2020-08-22 18:44:12.677269 DEBUG home_presence_app: Creating Binary Sensor for everyone_not_home
2020-08-22 18:44:12.679659 DEBUG home_presence_app: set state: binary_sensor.everyone_not_home, {'state': 'off', 'attributes': {'friendly_name': 'Everyone Not Home', 'device_class': 'presence'}}
2020-08-22 18:44:12.681620 WARNING home_presence_app: home_presence_app: Entity binary_sensor.everyone_not_home not found in namespace default
2020-08-22 18:44:12.683680 INFO AppDaemon: home_presence_app: Entity binary_sensor.everyone_not_home created in namespace: default
2020-08-22 18:44:12.729914 DEBUG home_presence_app: Creating Binary Sensor for everyone_home
2020-08-22 18:44:12.731844 DEBUG home_presence_app: set state: binary_sensor.everyone_home, {'state': 'off', 'attributes': {'friendly_name': 'Everyone Home', 'device_class': 'presence'}}
2020-08-22 18:44:12.733585 WARNING home_presence_app: home_presence_app: Entity binary_sensor.everyone_home not found in namespace default
2020-08-22 18:44:12.736622 INFO AppDaemon: home_presence_app: Entity binary_sensor.everyone_home created in namespace: default
2020-08-22 18:44:12.783668 DEBUG home_presence_app: Creating Binary Sensor for somebody_is_home
2020-08-22 18:44:12.785993 DEBUG home_presence_app: set state: binary_sensor.somebody_is_home, {'state': 'off', 'attributes': {'friendly_name': 'Somebody Is Home', 'device_class': 'presence'}}
2020-08-22 18:44:12.787954 WARNING home_presence_app: home_presence_app: Entity binary_sensor.somebody_is_home not found in namespace default
2020-08-22 18:44:12.790503 INFO AppDaemon: home_presence_app: Entity binary_sensor.somebody_is_home created in namespace: default
2020-08-22 18:44:12.825303 DEBUG home_presence_app: Calling listen_state for home_presence_app
2020-08-22 18:44:12.830715 INFO home_presence_app: Setting up Monitor auto reboot
2020-08-22 18:44:12.839430 DEBUG home_presence_app: Registering run_every starting 2020-08-23 00:00:01+02:00 in 86400s intervals for home_presence_app
2020-08-22 18:44:12.845090 DEBUG home_presence_app: Registering run_every starting 2020-08-22 18:44:13.843423+02:00 in 30s intervals for home_presence_app
2020-08-22 18:44:12.848197 DEBUG home_presence_app: Calling listen_event for home_presence_app
2020-08-22 18:44:12.851550 DEBUG home_presence_app: Listening on MQTT Topic presence
2020-08-22 18:44:12.853026 DEBUG home_presence_app: Calling listen_event for home_presence_app
2020-08-22 18:44:12.855027 DEBUG home_presence_app: Registering run_in in 0 seconds for home_presence_app
2020-08-22 18:44:12.857906 DEBUG home_presence_app: register_service: presence/remove_known_device, {}
2020-08-22 18:44:12.858669 DEBUG home_presence_app: register_service: presence/run_arrive_scan, {}
2020-08-22 18:44:12.859254 DEBUG home_presence_app: register_service: presence/run_depart_scan, {}
2020-08-22 18:44:12.859816 DEBUG home_presence_app: register_service: presence/run_rssi_scan, {}
2020-08-22 18:44:12.860338 DEBUG home_presence_app: register_service: presence/run_node_command, {}
2020-08-22 18:44:12.860919 DEBUG home_presence_app: register_service: presence/restart_device, {}
2020-08-22 18:44:12.861496 DEBUG home_presence_app: register_service: presence/reload_device_state, {}
2020-08-22 18:44:12.862274 DEBUG home_presence_app: register_service: presence/load_known_devices, {}
2020-08-22 18:44:12.862828 DEBUG home_presence_app: register_service: presence/clear_location_entities, {}
2020-08-22 18:44:12.863401 DEBUG home_presence_app: register_service: presence/clean_devices, {}
2020-08-22 18:44:12.871166 INFO AppDaemon: App initialization complete
2020-08-22 18:44:13.021752 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'presence/state', 'payload': '{"locations": [], "version": "2.3.4", "nodes": 0, "online_nodes": [], "offline_nodes": [], "last_changed": "2020-08-22T18:44:12+02:00", "state": "idle"}'}
2020-08-22 18:44:13.040597 DEBUG home_presence_app: Registering run_in in 45 seconds for home_presence_app
2020-08-22 18:44:13.044900 DEBUG home_presence_app: Registering run_in in 105 seconds for home_presence_app
2020-08-22 18:44:14.016151 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'presence/echo', 'payload': ''}
2020-08-22 18:44:44.024518 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'presence/echo', 'payload': ''}
2020-08-22 18:44:58.020588 DEBUG home_presence_app: Registering run_in in 0 seconds for home_presence_app
2020-08-22 18:44:58.042164 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'presence/setup/ADD STATIC DEVICE', 'payload': "50:50:A4:00:EB:89 Stephen's Phone"}
2020-08-22 18:45:14.015052 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'presence/echo', 'payload': ''}
2020-08-22 18:45:44.018754 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'presence/echo', 'payload': ''}
2020-08-22 18:45:58.020054 DEBUG home_presence_app: Registering run_in in 0 seconds for home_presence_app
2020-08-22 18:45:58.045791 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'presence/KNOWN DEVICE STATES',

Odianosen25 · August 22, 2020, 6:21pm

Hello @Eeeeeediot,

The setting “monitor_topic”, do comment it out and it should fix your issue most likely.

Regards

Eeeeeediot · August 22, 2020, 8:52pm

The problem was that I didn’t have the monitor script running anywhere. I downloaded the add-on and it now works.

Nowhere in the documentation does it make it clear that the appdaemon app doesn’t actually include the monitor program - it took me ages to work that out!

Eldudemeister · August 23, 2020, 7:37am

Hi @Odianosen25 looking at implementing this. I’d like to try room detection as I have a few pi zeros. Does it scan periodically to update the nearest pi sensor or would a scan have to be triggered manually? I am using monitor standalone but really want to try your app out As it looks great! Thanks!

Odianosen25 · August 23, 2020, 9:50am

Lols,

Well it’s assumed the user would have installed it . But glad working now

Odianosen25 · August 23, 2020, 9:55am

Hello @Eldudemeister,

If you add motion sensors to the config, it will use the motion sensors to scan and update the RSSI values and recalculate the nearest monitor. This way it can be used for rudimentary room presence system.

You have to take not, it doesn’t scan immediately on motion, as it’s expected the user will have to settle into a room first. Can read it here https://github.com/Odianosen25/Monitor-App#rssi-tracking

Eldudemeister · August 23, 2020, 10:18am

Thanks, makes sense after reading that, have some spare motion sensors so will help with reliability

mefistofelis · August 23, 2020, 2:19pm

Sorry for being a total noob, but how do you get the script running???

paddy0174 · August 23, 2020, 3:46pm

You install monitor.sh from here on all Pis you want to run the monitoring on (installation instructions are on that github page). Afterwards you install the Presence App V2 from this thread on your HA installation.

The idea is to have nodes (different Pis, eg. in your garage, next to your front door, on the second floor) running monitor.sh, that report to the Presence App V2 on your HA machine. With the app you get all the sensors and everything necessary out-of-the-box.

Eeeeeediot · August 23, 2020, 5:12pm

There is an add-on if you are on HASSIO. The link is in the original repo

mefistofelis · August 23, 2020, 7:03pm

Thx, just a final question: These Pis must be dedicated to run monitor, or can they also have a secondary version of HA installed and running. My primary HA is Hassio in VM on a NUC.

Eeeeeediot · August 23, 2020, 9:45pm

I spoke too soon…

I am using the addon and your appdeamon program on the same RPi4 with HomeAssistantOS. None of my known_devices or known_beacons get loaded into the monitor script. I have not set them in the addon config as your docs says that the appdemon program should load them into all nodes but its not happening.

I also seem to have this mqtt message in the broker - not sure if its related.

home_presence_app.yaml

home_presence_app:
  module: home_presence_app
  class: HomePresenceApp
  home_gateway_sensors:
    - binary_sensor.hall_frontdoor
    
  known_devices:
    - 50:50:A4:00:EB:89 stephen_phone
    # - 14:87:6A:BD:75:82 april_phone
  
  # known_beacons:
  #   - DC:0D:30:01:02:BE white_ibeacon
  #   - FC:58:FA:53:F9:6A black_itag

  user_device_domain: device_tracker

EDIT: and this keeps happening too. The addon seems fine and continues to frunction.

2020-08-23 23:39:45.041266 INFO home_presence_app: Processing System Unavailable for Hassio
2020-08-23 23:39:45.085334 WARNING home_presence_app: Node at Hassio is Offline, will need to be checked
2020-08-23 23:42:15.042888 INFO home_presence_app: Processing System Unavailable for Hassio
2020-08-23 23:42:15.089198 WARNING home_presence_app: Node at Hassio is Offline, will need to be checked
2020-08-23 23:43:16.022878 INFO home_presence_app: Processing System Unavailable for Hassio
2020-08-23 23:43:16.049824 WARNING home_presence_app: Node at Hassio is Offline, will need to be checked

paddy0174 · August 23, 2020, 10:07pm

The Pis don’t need to be dedicated. monitor.sh is a very small script, that can run on any Linux.

BUT, you don’t need to have a second instance from Home Assistant. Maybe my use case gives a little example:
I have a Pi Zero WH on my guest toilet, as it is right next to the front door of our apartment. A PI 3b+ in our garage, and another Pi 3b+ in our living room. Every Pi has monitor.sh running. The Pi Zero is “dedicated” as I can’t find anything useful to run on this one. Both other Pis do a lot more than just monitor.sh.
My Home Assistant instance is running on a Pi4 with 4GB. On this machine I don’t need monitor.sh, as this machine is strategically in the middle of the apartment. There I’d run the Presence App V2.

For a useful presence detection, you run your nodes near to entry points like the garage or the front door or the terrace door. These doors trigger a scan, if they are opened/closed, as this is the moment you want to know, if someone is leaving or coming. The movement inside the apartment is nothing I would want to know.

That’s why I don’t need (or find it useful) to run monitor.sh on the machine, I’m running HA on. Normally people don’t come or leave through windows, so I need to know, if something changes at the doors. Then I can run some automations and a useful presence detection.

I hope this was a little explanatory, if not, just ask.