I made a pfSense integration which includes many sensors for statistics etc, switches to enable/disable firewall/nat rules and turn services on/off and device_tracker integration using the arp table.
Coold this has been a long time wish.
but, what is this?
" Configuration is managed entirely from the UI using
config_flow semantics."? idk what config_flow is
EDIT: found it, it means add it in “integrations”
Yeah thanks for bringing that up! I’ll clean up the README a bit more with better detail shortly. Any feedback you have is appreciated.
Have to first manually add a HACS repository before the integration will be available to install.
On the pfSense side i don’t seem to have any privilge named ‘system - ha node sync’ but i do have ‘webcfg: xmlrpc - interface stats’ and ‘webcfg: xmlrpc - library’. I havn’t experimated with those privileges yet - wanted to check first. I want to be extra cautious when messing with my firewall
On the latest pfSense it’s the very first item in the list.
yes quite right. doh! i eventually found it.
It took me a minute to find it myself actually! Sneaky little thing.
After i successfully configured the integration (it took a couple of attempts - i had to uncheck SSL verify) I found some exposed credentials in the HA logs. It looks like in some circumstances the used pfsense credentials will be written to the logs. Could be a problem.
Thanks for pointing that out! Although you may want to remove the graphic and change the creds now since they are visible in the last line of the logs. Do you know which scenario produced that error? I’ll see if I can ensuree the data is clensed (or not log at all) during failure scenarios.
I can’t say which specific scenario caused that error because it took me 4 or 5 attempts to get the correct combination of config options before it worked. Each time i tried i know for sure that the pfsense user/password was correct but it was failing if i used http instead of https & also failed if i tried ssl verification.
Also, i have since enable the device tracker option but i’m not seeing any devices discoverd in HA. Is there something else that needs to be done for that to function.
device_tracker for the moment creates the entities in disabled state by default (due to the sheer potential volume and that fact that generally folks are wanting to track very specific devices). Expand disabled entities and look for the entity which corresponds to the mac address you care to observe (it should be relatively obvious which it is) and enable it.
Once I figure out a better way to configure it in the UI I’ll probably make it so you enter the macs you want to track and then I’ll default the entities created from those mac to enabled.
Regarding the error and logs I’ll try to clean that all up shortly and push a new release that makes sure to redact sensitive info before logging.
It seems the device_tracker feature is failing, according to the logs generated. No disabled entities being created in my case.
On a positive note, thanks for working on this integration. The amount of data available is super helpful.
Integration: pfSense (documentation, issues)
First occurred: 11:44:50 (156 occurrences)
Last logged: 14:22:21
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File “/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py”, line 134, in _handle_refresh_interval
await self._async_refresh(log_failures=True, scheduled=True)
File “/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py”, line 265, in _async_refresh
File “/config/custom_components/pfsense/init.py”, line 334, in process_entities
entities = self.process_entities_callback(self.hass, self.config_entry)
File “/config/custom_components/pfsense/device_tracker.py”, line 39, in process_entities_callback
entry_mac = entry.get(“mac-address”).lower()
AttributeError: ‘NoneType’ object has no attribute ‘lower’
I’m unclear why an arp entry would be returned by pfsense without a mac address but I’ve just committed a change that will treat that scenario more robustly (along with fixes for logging creds etc). Please pull down the latest and try again. Thanks!
Thanks Travis. Integration updated & device_tracker functioning now
Regarding the device_tracker feature & the default pfSense ARP cache timeout, i had an idea/thought that i will look further into in the next few days.
The idea being to create a template sensor based on the ‘expires’ attribute. In my initial testing of charting the values for ‘expires’ over time for an android phone & an iphone, it appears that when the phones are connected to my wifi the ‘expires’ (arp cache timeout) value seems to never fall below 900 (when checking with a 30 second interval).
So this template sensor could be a good way to track accurate device presense, rather than modifying the pfSense arp settings for all devices.
I’ll see how this goes over the next few days as my wife & i move in & out of the house.
Cool! Let me know how it goes. I have been thinking about introducing a sort of refresh timeout parameter that would clear the arp entry for specific devices after the given threshold.
For example, the default arp window is 20 minutes. The default scan interval is 60 seconds. So every 60 seconds we check the arp table looking for entries. So lets say the hypothetical new value is 300 seconds, during the 60 second cycle I would compare the expires value against the 20 minutes and if it expires in less than 15 minutes (20 minutes minus 300 seconds) I would clear the arp entry for that specific address. If within the next 60 second window the device has had no activity (with pfsense) it would appear as offline. Essentially replacing the global timeout but specifically for the devices being ‘watched’.
I’ve also considered adding dhcp data to the mix. Looking for a dhcp entry with the matching mac address and seeing if that shows offline. Of course not all devices use dhcp so that would be a best effort basis.
Honestly I’m not sure how much value the feature holds over nmap if hass is running on the same network as pfsense. If you’re communicating with a remote instance of pfsense then nmap is not a viable replacement however.
@nickh66 did you ever get a chance to try that out? I’ve put a prototype in place that clears the arp entry of the individual devices being monitored on the poll interval after returning a response about being present or not. The end result seems pretty solid. Essentially you set the poll interval to something like 5 minutes, and by clearing the arp entry every poll interval you effectively enforce the device is never overly stale (roughly 2x the interval is the longest hypothetical time the device could be considered online when actually gone).
If this sounds like a sane approach I’ll add a boolean option to the integration which will allow turning it on/off.
So far the sensor is not working out how i had hoped. It seems, even though my phone is inside my wifi network all day & mostly sitting on my wireless charger at my desk, the arp cache seems to time out randomly. Causing unexpected ‘unavailable’ or ‘away’ status. It’s weird. I included the status history graph of my android 12 device for reference.
There are certainly some periods, like today between 0613am-0810am and 1840pm-1915pm where i was out of the house taking a walk, so that’s expected but at other times, like 2150pm when i was sitting on the couch watching TV.
I don’t pretend to fully understand why that might happen but i’m keen to explore the changes you made.
I just pushed the changes to the
next branch. Can you switch branches and give it a try? In that branch the behavior is hard-coded and requires no changes to options etc.
Okay, just removed the integration & hacs custom repository, then added everything again via next branch.
I’ll watch how things progress over the weekend.