FortiOS device_tracker

I can talk to a few things here if this thread is still lively. Sorry, I just joined so that’s why so late - and just started to use HASS, but I happen to know FortiOS well and can possibly answer some of these questions as well as show how I’ve got my FortiGate finding what I want it to (or actually I’m filtering the results that I want - think Tablets, phones, computers, etc…basically not MAC addresses just willy-nilly and such). In any case, if there’s anyone interested we can go back and forth on this. Let me know if this is still worthwhile and I’ll jump in and help. And then beg you to help me how to figure out why Apple TV and media stuff is CRAZY WRONG in here.

I had this awesome device tracker working perfectly, but now it broke when I upgraded to FortiOS 7 :roll_eyes:.

Error:

Logger: homeassistant
Source: components/fortios/device_tracker.py:69
First occurred: April 8, 2021, 8:32:37 PM (5478 occurrences)
Last logged: 2:48:04 PM

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/device_tracker/legacy.py", line 334, in async_device_tracker_scan
    found_devices = await scanner.async_scan_devices()
  File "/usr/src/homeassistant/homeassistant/components/device_tracker/legacy.py", line 801, in async_scan_devices
    return await self.hass.async_add_executor_job(self.scan_devices)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/fortios/device_tracker.py", line 75, in scan_devices
    self.update()
  File "/usr/src/homeassistant/homeassistant/components/fortios/device_tracker.py", line 69, in update
    for client in clients_json["results"]:
KeyError: 'results'

Looks like the API key is still authenticating and the connection is there but there is a problem phasing the results. I’m kinda out of my league here.

The reason is that the API endpoint “/api/v2/monitor/user/device/select” has been removed in 7.0. Check this out, go to the GitHub of the creator of the FortiOS integration - to get exactly where I want you to go it is here (which is the device_tracker.py file - the heart of the issue): “core/device_tracker.py at 6e911ba19f9bb9b34cfd2981786540a6d485d4f9 · home-assistant/core · GitHub

Notice line 63…see it? They’re using another module so you see where they are saying: clients_json = self._fgt.monitor(“user/device/select”, “”) right there? That’s just an abstracted way of them calling that endpoint I said had been removed above.

Now - looking at the release notes of 7.0 I see: Removed monitor APIs - GET api/v2/monitor/user/device

Thus broken. I can actually tell you how to see this on your Gate (you’d get a 404). Now the good part. I can help you fix this if you want. Please go to that GitHub repo and post the Issue and see if they’ll fix it or if someone else is going to have to and they’ll merge it etc… however, this is what you can do in the meantime. Remember, if you do this, it will get overwritten next time you update possibly, so remember what you do until they fix it.

Go to that file within your Home Assistant - I use Core, so and mine is in the location “/srv/homeassistant/lib/python3.8/site-packages/homeassistant/components/fortios”. Yours will be different clearly, but you can find it easily by looking for something close to this on where your home assistant is calling its modules from. Now, change into that directory and look what you see there, device_tracker.py. VIM that thing up (or Nano or whatever you use) and modify that line 63 and make it say:

clients_json = self._fgt.monitor(“user/device/query”, “”)

Where it currently says:

clients_json = self._fgt.monitor(“user/device/select”, “”)

And you should start seeing data again…I haven’t tested, so if you still get some errors, let me know and I’ll test and tell you what else you can simply change to bring it back to life!

1 Like

@huberj001 This response is extremely well written and well detailed. Thank you so much for your analysis and instructions!

Thanks very much. Let me know if you decide to fix it and if it works or not - if it doesn’t, I’ll troubleshoot for you and actually test this time and get you what you need!

@huberj001, Its been 2 weeks since this has stopped working and I’m still nowhere with it. I switched over to Home Assistant Blue and apparently there’s no way (that I’ve found) to edit my own source code. How hard would it be for me to copy the integration to my own Github and create my own private HACS add-on? I can then remove the (broken) Core integration and link my own FortiOS add-on for the time being until other (more qualified) people fix the source code.

Hmmmm…okay - that stinks. Let me hit the writer of this code and see if he’ll just let me do a pull on it and merge my change. I’ll just add an API call to basically get system info and if the version comes back with a 7 in the major field then I’ll make a different API call. It’s a simple hack really, and I’d do it TOTALLY differently, but…this will solve it for you I think and will be fast. Let me see what he says and if he’ll pull it. Heck he may just fix it for us. Standby.

Not really sure if I’m on the right track here, but I found a way to create a custom_component based off of this integration and made the URI change that you recommended. The custom_component overrides the core component so my configuration is the same.
I have it available here: GitHub - disforw/fortios7: Fixes the core FortiOS integration for Fortigate firmware 7.0 and added added it via HACS.
I am now getting this warning over and over which eventually errors out with the unclosed connection.

/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.1.1'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  warnings.warn(
/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.1.1'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  warnings.warn(
2021-05-09 10:11:57 ERROR (Recorder) [homeassistant] Error doing job: Unclosed connection

I have come up with a temporary fix for device tracking in the new FortiOS7 if anyone is interested.
I created a custom component called “fortios7” and wrapped it as an HACS compatible repository (the repo URL is below). I say this is a temporary fix because in its current development, it only queries devices connected to a FortiAP. Personally, I only track cell phones, so I have no need to take this further, but if anyone needs it developed more, let me know.

HACS Repo URL: GitHub - disforw/fortios7: Fixes the core FortiOS integration for Fortigate firmware 7.0

I’ve left HA upgrades a little behind, and recently upgraded from 2021.12 to 2022.4…
Found out the hard way the FortiOS device tracker changed with 2022.2, and broke compatibility with my 100D. It can’t upgrade to 6.4.
So I went into the code and modified it locally to bring 6.2 compatibility back. I think I could have used the previous version, but this way, I understand how it works and I hope I’ll be able to maintain compatibility with my hardware.

I see 26 FortiOS integration users (analytics), if any amongst us has 6.2 limited device, I can provide the modified version.
Basically, you need to retrieve information with user/device/select path, whilst 6.4+ use user/device/query. Then the structure to get hostname is a bit different too.

All devices on Wifi are detected as not_home, no matter what.

In my configuration.yaml:

- platform: fortios
  host: "dedacted_ip:port"
  token: !secret fortios_token
  new_device_defaults:
    track_new_devices: false
  consider_home: 240

Excerpt from my known_devices.yaml:

juanplus_wehlan:
  name: JuanPlus (WehLAN)
  mac: REDACTED
  icon: mdi:cellphone
  picture:
  track: true

How it shows up in my Developer Settings:

FortiOS Admin Profile Settings:

Please help! It is really frustrating that it just doesnt work for me.
FortiOS v6.4.9

My YAML configuration is slightly different, and I just discovered it doesn’t match the online help !

  - platform: fortios
    host: !secret fg_host
    token: !secret fg_token
    track_new_devices: True
    consider_home: 180

Only difference in known_devices, I don’t have a value for “icon”.
“super_admin_readonly” group for API access has same permissions as you have.

Sorry not sure I can help there !
Wifi is provided by Forti or you use standalone access points ?

As this thread has a history of ‘being dormant for nearly a year’, I felt the need to reply / chime in.
I just installed HA (on Proxmox, tteck script). Device tracker has issues (for me, see below).
This HA either doesn’t use the code in location as @huberj001 states (“/srv/homeassistant/lib/python3.8/site-packages/homeassistant/components/fortios”), but I can simulate the API call as described (user/device/query vs user/device/select), beautiful find i must say!
So, indeed my FortiOS version is 7.0, but as said the code is not here for me to adjust…
And the GitHub by @disforw with the fix (for other versions of HA) is gone…

Welp! :slight_smile:

Does somebody have some suggestion?

posted some updates on github seems a rewrite is on the way ?