MQTT Unifi WiFi client device tracker as part of minimal-HA

Goal

Create a device tracker for specific WiFi clients on my Unifi APs using only HA’s MQTT integration; replace unifi_direct for device tracking and presence detection.

Background

I use a WiFi client tracker for presence detection for household members, who almost always have their mobile phones when they leave the house. This allows for automations like turning down the heat when nobody is home. I had previously used HA’s Ubiquiti Unifi AP (unifi_direct) platform to track devices on my WiFi.

I had a few issues with unifi_direct though and noticed there is an open issue related to the requirement to move Unifi specific parsing to PyPI. It seems improvements are stuck there. I decided to create my own device tracker with the following goals:

  1. MQTT integration is the only required HA dependency.
  2. Use SSH key auth instead of password to connect to Unifi AP to retrieve clients.
  3. Improve error handling; unifi_direct was causing noise in HA logs.
  4. One less HA component dependency to keep my HA installation simple and minimal.
  5. Have fun with another DIY project!

Here is the PyPI link:
unifi-tracker · PyPI
Here is the Github link:
idatum/unifi_tracker: Track Unifi AP WiFi client comings and goings (github.com)

I run the follow service under docker, along with HA and Mosquitto (and other services):

It’s a simple service using the unifi_tracker module:

  1. Periodically return client info from all APs.
  2. Union all client MACs from all APs.
  3. Do a diff between the previous and current set of MAC addresses.
  4. Publish diff to MQTT (Mosquitto) for processing in HA using the MQTT service.

A few key notes:

MQTT messages are retained so HA can restart and pick up current presence state. The topic has the MAC address, and the payload is “home”. Here is an example HA device_tracker.yaml (included in configuration.yaml):

- platform: mqtt
  consider_home: 60
  devices:
    tracked_phone: "device_tracker/unifi_tracker/xx:xx:xx:xx:xx:xx"
  qos: 1
  source_type: router

Note consider_home: to have HA correctly honor that setting, deleting the retained message is required instead of publishing “not_home” in the payload. To delete a retained MQTT message, you publish a retained topic with no payload. Here’s the code in device_tracker.py:

            last_clients, added, deleted = unifiTracker.scan_aps(ap_hosts=AP_hosts, last_mac_clients=last_clients)
          for mac in added:
              publish_state(topic=f'{Topic_base}/{mac}', state='home', retain=True)
          for mac in deleted:
              # Empty payload
              publish_state(topic=f'{Topic_base}/{mac}', state=None, retain=True)

Notice the empty (None) state payload for a deleted device. In HA for this example, the presence for the associated Person will change to away after about a minute.

If any AP fails to return output from mca-dump, the entire diff will fail. Note that clients can roam, switching from one AP to another. I only have a couple APs, but if you have many, the probability of failing to do a diff increases. I get 1 or 2 failures per hour from any of the my APs (it simply doesn’t return any results – no clue why).

I use multiprocessing.Pool to retrieve output in parallel from the APs.

There are 2 environment variables for MQTT credentials:

MQTT_USERNAME
MQTT_PASSWORD

The Unifi AP SSH username (using SSH key auth) is also an environment variable:

UNIFI_SSH_USERNAME

In summary: device_tracker.py drives the main processing and handles MQTT, unifi_tracker.py handles SSH and client diff with APs.

Summary

Works fine generally, basically like the existing HA unifi_direct, but allows me to more freely innovate and be less dependent on another component for running HA for my home automation.

I tried both the built-in unifi_direct integration and also the patched version (as custom_component), but still the device tracking fails with my U6-PRO (6.0.11) after 1-2 days with error “Failed to decode response from AP”.
You mentioned that your using it with Docker, so would you mind sharing the Dockerfile, or maybe you already built an image with that.
I will give it a try and let you know how it works.

Thanks for sharing your project.

The firmware version for my UAP-AC_PRO is 5.43.56, and unfortunately, I never tested on your version.

I uploaded an example Dockerfile to my github repo:
unifi_tracker/Dockerfile at main · idatum/unifi_tracker (github.com)

Couple things you’ll need to run:

  1. Environment variables for various arguments (e.g. UNIFI_SSH_USERNAME).
  2. A shared volume pointing to a private SSH key that corresponds to the public key you set for your devices via Unifi under System/Application Configuration/Device SSH Authentication.

Here is an example docker run command line:

docker run \
           -e UNIFI_AP_IP_LIST=$UNIFI_AP_IP_LIST \
           -e UNIFI_SSH_USERNAME=$UNIFI_SSH_USERNAME \
           -e MQTT_USERNAME=$MQTT_USERNAME \
           -e MQTT_HOSTNAME=$MQTT_HOSTNAME \
           -e MQTT_TOPIC=$MQTT_TOPIC \
           -e LOGGING_LEVEL=$LOGGING_LEVEL \
           -e SCAN_DELAY_SECS=$SCAN_DELAY_SECS \
           -e DEVICE_LOGGER_NAME=$DEVICE_LOGGER_NAME \
           -e MQTT_PASSWORD=$MQTT_PASSWORD \
           -v /home/admin/.ssh:/root/.ssh \
           --rm \
           --name unifitracker \
            unfitracker

Thanks for trying!

Everything works fine so far, let’s see how it goes for the next days.

I found one small issue when the container is stopped for maintenance and one(or more) tracked devices leave home. When the container restarts, the devices are not marked as away, as old topic(s) values are retained.

Indeed, your approach is far more flexible and can be integrated in other scenarios without HA.

Thanks, I’ll try to repro that issue.

So far, after 3 weeks everything is rock solid. My devices tracking is working perfect with U6-Pro which in the mean time I updated to latest firmware version 6.0.14.
Thanks @joelp.

1 Like

Thanks @yellowonblack, appreciate you giving this a try!

I’ve also upgraded my UAP-AC-Pro devices to the 6.0.14 firmware version.

Hey guys.

Want to give this a try to ditch a 24/7 controller running.
I have an odroid n2+ with hass os running.
How can i run the phyton service on my system with no direct access to docker?

thanks

This solution relies on having an MQTT broker service running as well as the HA MQTT integration.

You can, for example, run the python service as a unix daemon (e.g. a systemd unit).

1 Like

If added a couple options (in v0.0.4) to help make the state change to away more deterministic:

  • Option --sshTimeout to explicitly set SSH connect timeout in seconds (float).

  • Option --maxIdleTime to set AP client idle time threshold in seconds (int). Use --maxIdleTime to check the idletime field of sta_table and filter on clients that are below the given threshold. There are cases where I’ve seen a tracked client go out of range of the last connected AP yet still shows as connected for several minutes. I’ve been able to repro this case when a mobile phone goes out of range quickly (e.g. driving away in a car). If the mobile phone slowly goes out of range (e.g. carried while walking away), it correctly shows as disconnected. Setting this threshold ensures the status changes in a more deterministic amount of time.

What is the recommended --maxIdleTime? Can be those new options set as env variable in docker?

I tested this using 300 seconds (5 minutes). This seems like a reasonable balance between false positives and having the state change sooner. In my testing I noticed 20 minutes or more before the AP noticed the device was far out of range. 5 minutes is probably long enough for a firmware update and reboot, generally. But I never see this delay issue when explicitly turning off WiFi on my device --it’s only when the device goes out of range.

One other note: A couple of Fire tablets I have while in sleep mode take longer than Windows laptops (for example) before hitting the AP (phoning home?), so it can be more than 5 minutes before it reconnects back. So Fire tablets are an example where you’re going to get false positives using 5 minutes.

I use a .env file to set environment variables with docker-compose. There is the equivalent with straight docker, I think -e option for run? I use a Shell script to run device_tracker.py and the -maxIdleTime option is set to the passed in environment value set in .env.

The core component used for the MQTT device_tracker solution is under .../core/homeassistant/components/mqtt/device_tracker/.

Your log entry is generated from the device_tracker compoenent in .../core/homeassistant/components/device_tracker/legacy.py, method async_device_tracker_scan.

I don’t have this addon component in use. I believe these components do not have any dependencies; hence I don’t think your log entry is related to MQTT device tracking.

Disclaimer: I’m not an HA developer, so definitely defer to experts here :slight_smile:

EDIT: meant to say “component” instead of “addon”. My point here was that I couldn’t find any relationship between these components, and I don’t use any of the services listed for the legacy component.

1 Like

Sorry, posted in wrong thread! Deleted my OT posts.

Whew, okay thanks :slight_smile: