sorry for the strange question, but I went from pfSense to OPNsense a year ago and was looking for an integration like this.
Since the two projects are “cousins”, and OPNsense has XmlRpc, I was wondering if it would be feasible to adapt your integration to OPNsense, what’s your opinion?
I actually tried to confgure it for OPNsense but it can’t even login, I checked the xmlrpc.php code and it expects basic authentication, I expected issues on the xmlrpc calls, not on the login.
A year ago or so I looked into opnsense and using the same api to replicate another project of mine (GitHub - travisghansen/kubernetes-pfsense-controller: Integrate Kubernetes and pfSense) with the hope it would require only minor changes. Upon further research I discovered that the xmlrpc code of opnsense has been severely altered and gutted and the project team didn’t seem keen on bringing it back. In short it won’t work with opnsense
Good luck deciding which to run with! If you end up using the integration shout out any feature suggestions you may have!
thanks a lot for getting back to me, I appreciate it.
Can I ask you a little bit of details on what you found out regarding the code? Have you contacted the team privately or on their forum?
In case I decide to give it a shot and contact them, If I had to ask them what would be needed (in order to support your integration with minor changes obviously), what would that be in detail?
I’ve just reviewed some of the code and it looks like it may not be as crazy as I remember (especially if a simple pkg was made to reintroduce some functions the opnsense team may not want).
I’m really only using 3 xmlrpc methods, however 1 of them exec_php essentially allows me to invoke any other php code I want. I’m guessing it wouldn’t be too difficult to re-add those that have been removed.
I brought this up in the irc channel and the team essentially wanted the rest api to be used as I recall.
Having said that, things have changed enough that it would make sense to fork the project rather than try to unify the code base. Let’s open a discussion up on github to further explore the matter.
@nickh66 I’m going to work on the integration a little bit today and your feedback would help guide my efforts. How is the test going with the next branch code?
Actually i have not noticed any difference with the next branch. However, it seems to me that the ‘home’ or ‘away’ state tracking seems to be working fine. Especially for the iphone i’m testing with me (my wife’s phone). Interestingly & surprisingly to me the tracking of my android phone seems somewhat different, but still working i think.
This image is of the recent history of my android phone. You can see that the arp cache often gets down to close to zero but actually the state never goes to ‘not_home’. The second image shows that the state often becomes ‘unavailable’, but not ‘not_home’.
@nickh66 ok that’s pretty strange honestly. What datapoint exactly is being used for the arp timeout? Basically any device being tracked should never get below 1200 - <poll interval>.
The status going to unavailable isn’t super troublesome, it just means the integration failed to properly make the api call(s) to pfSense for whatever reason.
I’ve just pushed a pretty massive update which includes this logic among other things. Note that when you update however you must configure the integration and explicitly select which devices you want to track now (entities are no longer being created for all devices, only those explicitly added).
A bunch of services have been added to the integration as well including the ability halt/reboot the firewall, create ‘notices’ (little things that show in the bell icon of the pfSense UI) and clear them, and start/stop/restart pfSense daemons, etc.
I essentially consider the integration feature complete at this point (not that I’m not open to suggestions, just that I don’t have anything further on list atm).
The entity attribute i am using is ‘expires’. Perhaps i had the values set too low.
In any case I upgraded to the newest version today & i must say i really like the changes you made. Especially the selection list for choosing entities to for device tracking & the fact this list now includes the hostname for easy identification.
The only feedback i would give for the device tracking feature is this.
Keeping the text ‘pfsense’ as part of the entity id (like in the previous versions) would be helpful for differentiating the same entities discovered by mulitple trackers.
More of a thought exercise than direct feedback. When the entitiy is created in HA is it possible to use the device hostname or FQDN & not the MAC as the unique identifier inside HA. The reason being that these days for ios & android the default networking behaviour is to use a random MAC. So i guess in that situation there could eventually be multiple entities. Unless off course your integration already takes care of that & simply updates the associated attributes.
By the way, the updated version now showing much more accurate data as can be seen in the screen snip.
Yes! That graph looks much better (I’m wondering if the relevant code hasn’t actually been active for you during your last tests). You should see much better detection. I’m interested to know what the graph ends up looking like for an android device during an extended period of no use (ie: when asleep).
Regarding the randomized macs, to my knowledge (with iOS it’s this way, not entirely sure about android) the mac doesn’t change when you’re on the same network. The little blurb about it even says Using a private address helps reduce tracking of your iPhone **across different Wi-Fi networks**. I assume the android feature is similar. In other words the mac is the same for the same ssid but different across ssids is my understanding. The mac shouldn’t float around on you on your home network in other words.
Noted about the pfSense in the name. It’s such a personal preference and I’ve already had several folks express differing points of view. I’ll think about it a bit more however and see where it lands.
Excellent detail about private mac. I wasn’t aware of that. So now I will go & revisit some past threads i have read about phone presence detection. Could be just my misunderstanding but I got the impression that there may be a general misunderstanding out there about these phone features & their potential impact to HA device tracking. What you are saying makes total sense to me.
For the entity_id name, i wonder how much coding would be involved to parameterise that as an option for the user to choose how they prefer to represent that name? I’m trying to find time to learn Python but i’m so far unable to contribute any code. Could it be possible to have a check box in the UI that allows for something like adding either ‘pfsense’ or the users chosen ‘router’ name that is entered during the initial configuration?
Either way i love this integration & to me it seems a better option than nmap, since as i understand things nmap is scanning the entire network. I don’t have any evidence for this but i feel that directly querying my router is more efficient that a network broadcast.
Yeah, nmap isn’t great. It also doesn’t help in (perhaps unlikely) scenarios when HA is running on a separate network from where the device is installed…or said differently one benefit of the pfSense device tracker is it can track devices on non HA networks. When pfSense is queried to gather the arp entries I have an option to include hostname or not…I made the decision to enable that flag in this integration but it does add additional time to the request (which also contributed to the decision to have a distinct polling interval just for the device_tracker feature). I think on a reasonably sized network with a reasonable scan interval it will perform just fine. With the powerful ability to control the arp table and reset the entry on the scan interval for the selected devices this integration provides a relatively compelling option for tracking
I’ve re-added the fw/integration name to the entity naming for device_tracker entities to align with the rest and will commit shortly (at this point please switch back to the main branch FYI). That will likely be the last adjustment on the matter. They can be easily renamed manually as desired.
If you discover something different regarding the floating mac addresses do share here.
Is it possible when you install this integration to get only the information, just no possibility to turn on or off the rules etc. ? This is needed to do nothing from HA or change the settings of my router, just no strange things.
this integration is awesome ! Thank you so much !
Some feature interesting to add :
scan interval by feature (example, i’d like to refresh every 5 to 10s the current bandwith used, but number of device on the network can be refreshed every minute, update available every day)
