Continuing My Unifi Presence Detection Woes

[finity]

If anyone has seen my other posts about my experiences with the Unifi device trackers I think I need to post an another update on where I am on this.

So far I have to say that the unifi device tracker integration is pretty useless. At least in my experience so far.

Since my initial posts in other threads I have so far tried a total of four different ways of installing the unifi controller software - two different Docker containers outside of HA (jacobalberty & linuxserver.io) and I even grabbed an old spare Pi 3b+ and installed hassio just so I could try the unifi controller add-on. Lastly I even went as far as to buy a (very expensive) Unifi Cloud-key gen2.

So far none of these methods will reliably allow me to track my several phones.

here are three screen shots of different phones that are being tracked:

the first two are iPhones. The last is an Android phone.

In the first pic you can see that the iPhone randomly shows as home for exactly 5 minutes then goes to away again for a random period of time. Unfortunately, even tho the phone shows as home on those occasions my daughter wasn’t home at all yesterday…not at all…

On the second pic my wife’s phone shows as away for most of the day. Again, unfortunately, she was actually home for the vast majority of the time yesterday

Lastly, the third pic is my phone which shows that I was away all day except for those few times (that are the same exact five minute intervals as my daughters & wife’s phones) but I was actually home for the first 12 hours of that time frame and I was away for the entire second half of that time frame - ironically the only few times that it shows I was home I actually wasn’t.

I’ve checked the debug logs in my HA for the integration but there is nothing there to give any indication as to what is wrong.

While the unifi equipment is much better than my old stuff for network performance & reliability it sucks for presence detection.

I don’t hold out any hope of ever getting this resolved but I figured I would at least post here again about my experiences and provide a warning to anyone that might be interested in trying this. If you do try it I hope you fare better than I have.

[TazUk]

While the unifi equipment is much better than my old stuff for network performance & reliability it sucks for presence detection.

That’s not been my experience at all - I have two sites using two installs using the Unifi controller for presence detection with a variety of iOS, Android, Windows, Linux clients. One site uses the controller installed on the (Ubuntu) venv HA host, one site references the Gen2+ controller. I’ve not encountered this type of problem in the nearly two years that I’ve been running HA and Unifi.

I can see from your other thread that you are aware that iOS clients may sleep. Android clients may also sleep (dependent on client settings), and the most recent OSs for both platforms may default to using random MAC addresses rather than the device MACs when connecting to wireless networks (unless the network was configured before the client was updated to that version).

You asked how best to troubleshoot in your other thread, however you haven’t mentioned in this thread (or that) whether you’ve compared the controller’s client list (notably the MAC addresses) with the device_tracker attributes detailed in Developer Tools > States.

Forgive me if you’ve already tried this and I’ve missed it, but it would be a reasonable starting point to check that when your phone is connected, the MAC shown for it in the Unifi controller client list matches a device_tracker mac attribute that is shown as ‘home’ in HA developer states.

[finity]

Thanks for the reply…

Yeah, I know that iPhones can sleep and drop off the network occasionally and if that was the extent of the problems I could live with that but that’s not even close to what is happening.

As far as my Android phones as far as I know they don’t sleep/drop off the network. At least they didn’t when I was running my old ASUS AC1900 router for presence detection. And as far as I can remember I never really had that many times that my wife’s iPhone dropped off either. That router worked pretty well in that regard.

And even if the phones occasionally dropped off the network when they were actually home or if the MAC addresses would randomize that wouldn’t explain why my phone - that is never not connected to wifi and that I use quite a bit when home for browsing/HA so I know it’s connected - would never show home until I’m actually not home and then it periodically shows that it’s connected for random 5 minute blocks.

Or that my daughter’s iPhone shows home for those same random five minute intervals even tho she hasn’t been home and connected to the router in the last two days

Right now my wife’s iPhone is showing connected to the router (it has an active IP address) and it responded when I just now pinged it. Even tho the Unifi presence detection shows it as away right now. I also double checked the MAC address of her phone and it is the same in Unifi as it is in HA.

That’s the thing…none of it makes any sense. Phones show connected when they aren’t physically home and show as away even tho they’re actively connected and transferring data.

Maybe I should try to set up some other devices other than phones that I know are always connected (both wired and wireless) to see if they also show erratic behavior.

I’m also open to any other suggestion to try to figure this out.

[TazUk]

Right now my wife’s iPhone is showing connected to the router (it has an active IP address) and it responded when I just now pinged it. Even tho the Unifi presence detection shows it as away right now. I also double checked the MAC address of her phone and it is the same in Unifi as it is in HA.

So with her phone connected to the router, are there any device trackers showing home in the Unifi integration?

Re randomized MAC - I’ve just read that this is randomised on first connection to a new network and then remains set to the same random MAC for that network which is not how I thought it worked - apologies for the incorrect info.

EDIT:
A further random question / thought (I’m compiling code at work so thoughts are a bit random right now):
Are you authenticating to Unifi from HA with a read-only administrator level login?

[TazUk]

For reference, some config info from my 2 sites:

• Unifi controller version v5.12.35
• HA version 0.103.6
• HA platform Ubuntu 18.04, Python 3.7.3 venv
• Authentication via read-only local administrator Unifi login.

Both sites have two AC-AP-Pros, one also has USG + two US-8-60W switches.

Unifi configured via Integrations, Options:

• Enable newly added entities enabled (default)
• Track network clients enabled
• Track wired devices enabled (on 1 of 2 sites)
• Track network devices enabled
• Consider Home: 300 on 1 site, 120 on other.
  No SSID filters / additional options in configuration.yaml.

[finity]

@TazUk

So with her phone connected to the router, are there any device trackers showing home in the Unifi integration?

at 04:13 when I pinged her phone and it responded, my wife’s phone was showing as away. there were no phones showing as home at that moment in time.

My wife’s phone status changed to away at 02:13 but I was able to ping it at 04:13. Right now I can’t ping her phone. But it shows up in the active client list in the controller.

My son’s Android phone was showing as home up until 04:07 then it changed to away and it still shows as away. I just pinged his phone and it didn’t respond. His phone does not show up in the active client list right now.

Right now there are only two device trackers that should be home but both are away. The other two that should be away are also away so those are now correct at the moment.

@TazUk

Are you authenticating to Unifi from HA with a read-only administrator level login?

I’ve only got one user set up in the controller right now (as an admin) and that’s the authentication I’m using in the integration. I’m pretty sure that it’s not read-only since it’s the only one.

Should I be using a read only user?

as far as my config:

• Unifi controller version v5.12.35
• HA version 0.103.0
• HA platform Debian, non-hassio in Docker

One AP-AC-LR, one USG, one US-8-60W switch, one cloud key gen2

And I still have my old ASUS routers (AC68U AC1900) as AP’s only

Unifi configured via Integrations, Options:

• Enable newly added entities enabled (default)
• Track network clients enabled
• Track wired devices enabled
• Track network devices enabled
• Consider Home: 300
  No SSID filters / additional options in configuration.yaml.

So at least the configuration options as listed above are generally the same

[TazUk]

@finity

Right now there are only two device trackers that should be home but both are away. The other two that should be away are also away so those are now correct at the moment.

So in the HA user interface under Configuration > Integrations > Unifi you only have 4 device trackers limited (plus the other Unifi devices)?
And in Developer Tools > States you don’t have any additional (unifi) device_trackers with “source_type: router”?

@finity

I’ve only got one user set up in the controller right now (as an admin) and that’s the authentication I’m using in the integration. I’m pretty sure that it’s not read-only since it’s the only one.

Should I be using a read only user?

After 20 years in software development and 5 years in systems administration I tend to err on the side of caution, so use dedicated accounts for integrations and operate them with the minimum required permissions.
When I originally first setup my HA - Unifi integration I read some guidance indicating that it needed to be administrator level, and opted for a read-only account with a ridiculously strong password. This worked correctly from the off so I never saw a need to change it.
I have read of some issues which seemed to be resolved with this approach - given the issues you’re facing it seems reasonable to:

1. Use a separate account
2. Start with a read-only admin - you can always elevate it later if required.

NB: IIRC the revised integration offers / will offer the ability to toggle POE ports on switches if required - this isn’t something I’d expect to work with a read-only account, but given I have no intention of exposing network management functions to HA this isn’t an issue for me.

[finity]

@TazUk

So in the HA user interface under Configuration > Integrations > Unifi you only have 4 device trackers limited (plus the other Unifi devices)?

under the configuration->integrations->unifi controller (default) I have around 60 devices. Most of them show as “un-named device”. Many of them are named but aren’t included in the list of device trackers. I have them disabled. there are only 4 devices that are also in the device trackers list in the states page.

@TazUk

And in Developer Tools > States you don’t have any additional (unifi) device_trackers with “source_type: router”?

no. I only have the unifi device trackers that I have left as enabled in the entities page from the unifi integration.

@TazUk

given the issues you’re facing it seems reasonable to:

1. Use a separate account
2. Start with a read-only admin

I’ll give these suggestions a try.

Do you know if there’s a way to change the user and password in the integration without completely removing and re-adding the existing integration? I’m not looking forward to going thru and re-setting up my unifi integration for the 5th time. That’s a bunch of tedious work.

[TazUk]

I have around 60 devices. Most of them show as “un-named device”
Out of curiosity, are the ‘un-named devices’ containing _rx and _tx sensors?

Do you know if there’s a way to change the user and password in the integration without completely removing and re-adding the existing integration?

This isn’t what you want to hear, but no I’m afraid I don’t know of any way of replacing the credentials in an existing integration.
I would be inclined to go scorched earth, and (at the risk of stating the obvious) do the following:

1. Remove the Unifi Integration
2. Restart HA (you may need to comment out device_trackers if in automations etc to do so).
3. Check all unifi entities removed completely (from persons etc.)
4. Stop HA.
5. Check Unifi Controller credentials for separate account.
6. Check client names setup as required (so they can pass through to the device trackers on creation
7. Restart HA
8. Setup Unifi integration again using read-only admin credentials.

It is a fair amount of work without any guarantee of success I’m afraid.

I’ve also had a quick read through the current Github Unifi issues list but can’t see anything that would provide these problems.

[code-in-progress (Bill - I write code.)]

I have noticed some odd home/not_home wonkiness using the Unifi integration. I have almost the exact same setup as you do and ended up removing the HA integration and switching it over to NodeRed (which has a much better Unifi integration, imho). I have a flow that runs every minute and checks for 3 devices (my phone, my wife’s phone and my son’s phone) and calls device_tracker.see on HA. Using it this way, it’s been MUCH more reliable and I’ve not had any other issues.

[finity]

@TazUk

I may just do that…again.

Also do you know of any settings I could check in the controller software that might be causing the issues I’m having?

I could start from scratch there but I don’t know what I should do differently (if anything) and might end up making the same potential mistake again and ending up in the same place.

@code-in-progress

I’ve never used Node-Red but if all else fails I guess I could end up going that route.

I may ask you for a primer on getting the NodeRed unifi integration setup if the above doesn’t work. If you don’t mind…I’ll let you know how I get along.

[TazUk]

@finity

Also do you know of any settings I could check in the controller software that might be causing the issues I’m having?

I think probably one of the main benefits of breaking out authentication is that you would be able to confirm that HA was accessing the controller successfully by viewing the login events in the controller.
Other than that, both my sites are ‘Default’ with the standard 8443 port (there may still be issues with MQTT conflicts on this port depending on what you have configured) and I cannot think of anything else in particular.

[code-in-progress (Bill - I write code.)]

@finity

I may ask you for a primer on getting the NodeRed unifi integration setup if the above doesn’t work. If you don’t mind…I’ll let you know how I get along.

Absolutely! Once you get it setup, it’s a rabbit hole; Be warned But, the amount of things that you can do with NR is amazing.

[TazUk]

Just seen this post concerning Unifi which (if you are comfortable hacking around with config files) could be another route to modify the credentials.
Usual caveats apply.

[finity]

I just changed the core.config_entries file to a new read-only admin user I just created in the controller. So we’ll see how that goes now.

And as an FYI, once you log in with a different user it re-creates all of the entities again so I still needed to go thru the list and disable everything I didn’t want to track.

But this time I left a few extra things that I know for a fact (as much as is possible) that will always be connected (all of my unifi network devices and a couple of PC servers that are always on) so I can see if the non-phone wired devices show the same behavior.

EDIT:

I also enabled debug logging for the aiounifi component.

Any suggestions as to what I might be looking for in that log to inform me of any wonkiness?

[finity]

Thanks!

I think…

[finity]

@code-in-progress

I have almost the exact same setup as you do

For some reason this statement popped back into my head…

Do you also happen to have any other non-unifi network gear in your set up? I mentioned above I have a couple of ASUS routers I’m using as AP’s only. I’m wondering if it’s possible that there could be some undesired interaction between those and the Unifi stuff?

[finity]

After looking a bit more I have put one more piece if data into this mystery.

Generally (but not always…) if a phone is marked as away in HA, if I then restart HA, the phones will be marked as home for exactly five minutes. Which is the setting I have in the integration for the “time in seconds from last seen until considered away”.

So for some reason HA is somehow misinterpreting the restart and the subsequent reconnection to the unifi controller as the devices “being seen” and then marking them away 300 seconds later.

It seems as tho it only happens when the phone is away. Which makes sense I guess considering that if the phone is actually seen as home then it won’t transition to not seen.

However there are still a couple of anomalies -

1. My phone never goes to home even tho it is home and connected (but since it is always marked as away and I restart HA it is then seen as home for the aforementioned 5 minutes)

2. my daughters phone only exhibited the “seen as home” behavior this evening but it was actually away all night last night and this morning and it never was erroneously seen as home even tho I restarted HA five times during that same time frame.

So I’ve narrowed down the major issues to two -

1. figure out why HA switches devices that are away to home after a restart for the “consider home” time.

2. Figure out why my phone never shows as home (except after HA restart) even tho it is connected to the router.

I set up a ping sensor for my phone and it shows the accurate state of my phone so I know the router knows it’s connected correctly.

[TazUk]

The seen at home on restart issue was a bug in 0.103.4 / 0.103.5 fixed in 0.103.6.

I’ve never had to debug my Unifi setup with the logging so I’m afraid I have no suggestions for that.

[finity]

@TazUk

was a bug in 0.103.4 / 0.103.5

I’m on 103.0 so it must have started earlier than that.

I’ll update and see if that part goes away.