pfSense stat monitor

It took a bit of work to put this together, so I thought I’d share it with the community. pfSense does not have a native API for pulling/pushing information, but with a little work you can get it to send stats to HA and you can also send commands to your pfSense router if you so desire. Be careful, this API is quite powerful. Mind your security.

How to set up:

  1. Install Faux API for pfSense (PFFA). Follow the commands listed on the github page: GitHub - ndejong/pfsense_fauxapi: REST based API interface for pfSense 2.3.x and 2.4.x to facilitate devops

The UI will load Faux API here:
fauxapi

  1. Update /etc/fauxapi/credentials.ini file after install to add a new user. By default, Faux API has no user… there are two examples, but they are hardcoded not to work for security reasons. @ndejong offers some bash scripts to create your keys, but I had trouble with them for some reason. You don’t have to use them, you just have to make sure to follow the rules:

    The API key + API secret values that you will need to create in /etc/fauxapi/credentials.ini have 
    the following rules:
    
    * <apikey_value> and <apisecret_value> may have alphanumeric chars ONLY!
    * <apikey_value> MUST start with the prefix PFFA (pfSense Faux API)
    * <apikey_value> MUST be >= 12 chars AND <= 40 chars in total length
    * <apisecret_value> MUST be >= 40 chars AND <= 128 chars in length
    * you must not use the sample key/secret in the credentials.ini since they are hard coded to fail.
    

    [PFFAexample01] # <apikey_value>
    secret = abcdefghijklmnopqrstuvwxyz0123456789abcd # <apisecret_value>
    permit = alias_, config_, gateway_, rule_, send_, system_, function_* # authorization
    comment = anything you want can go here. it will be visible in the Faux API in pfSense

You can edit the file using vi in the ssh shell or from the pfSense UI: Diagnostics > Edit File > ‘enter the path _/etc/fauxapi/credentials.ini_

  1. Set up authorization. This is a really nice feature with Faux API, you can control what the user can actually do and only give permission to that. I was only interested in getting stat information so this is what I used so that my user didn’t have access to do anything but get stats from the router:

    [PFFAexample01]
    secret = abcdefghijklmnopqrstuvwxyz0123456789abcd
    permit = system_stats
    comment = example key PFFAexample01 - hardcoded to be inoperative

  2. GET the JSON formatted information. The Faux API uses a timestamp and a nonce for authorization. Not a simple username and password like most APIs. So you can’t use the JSON Sensor. You’ll have to utilize the provided python library from @ndejong. The file you need is here: pfsense_fauxapi_client_python/PfsenseFauxapi/PfsenseFauxapi.py at master · ndejong/pfsense_fauxapi_client_python · GitHub
    Download that file and place it in your /config directory. Then you need a python script to use the library. I used a modified version of this: pfsense_fauxapi_client_python/examples/function-iterate.py at master · ndejong/pfsense_fauxapi_client_python · GitHub

import os, sys, json

sys.path.append(os.path.abspath(os.path.join(os.path.curdir, '../client-libs/python')))     # hack to make this work in-place
from PfsenseFauxapi import PfsenseFauxapi


# check args exist
if(len(sys.argv) < 4):
    print()
    print('usage: ' + sys.argv[0] + ' <host> <apikey> <apisecret>')
    print()
    print('pipe JSON output through jq for easy pretty print output:-')
    print(' $ ' + sys.argv[0] + ' <host> <apikey> <apisecret> | jq .')
    print()
    sys.exit(1)

# config
fauxapi_host=sys.argv[1]
fauxapi_apikey=sys.argv[2]
fauxapi_apisecret=sys.argv[3]

PfsenseFauxapi= PfsenseFauxapi(fauxapi_host, fauxapi_apikey, fauxapi_apisecret, debug=False)


# system_stats
# =============================================================================
print(json.dumps(
    PfsenseFauxapi.system_stats())
)
open('/config/python_scripts/pfSense_stats.json', 'w').close()  # clear data
with open('/config/pfSense_stats.json', 'a') as stat_file:
    print(json.dumps(
    PfsenseFauxapi.system_stats()), file=stat_file)

As you can see. I got rid of all the calls I wasn’t interested in and just kept the system_stats call. I also am dumping the JSON data into a file in my config directory (another directory could also be used if desired.

  1. Set up sensors in config.yaml.
sensor:
   #this will call the python script and store the cpu temp information in a sensor. the script will export 
   #the rest of the data into a file. we cant dump all of the data into 1 sensor since there is a 255 character 
   #limit for the sensor.
 - platform: command_line
    command: "python3 /config/pffa_get_system_stats.py <_host_> <apikey_value> <apisecret_value>"
    name: pfSense_CPU_temp
    value_template: '{{ value_json["data"]["stats"]["temp"] }}'
    unit_of_measurement : 'C'
    
  - platform: file
    file_path: /config/pfSense_stats.json
    name: pfSense_uptime
    value_template: '{{ value_json["data"]["stats"]["uptime"] }}'
  
  - platform: file
    file_path: /config/pfSense_stats.json
    name: pfSense_mem
    value_template: '{{ value_json["data"]["stats"]["mem"] }}'
    unit_of_measurement : '%'
    
  - platform: file
    file_path: /config/pfSense_stats.json
    name: pfSense_cpu
    value_template: '{{ ( ( ((value_json["data"]["stats"]["cpu"].split("|")[0] | float) / (value_json["data"]["stats"]["cpu"].split("|")[1] | float)) - 1.0 ) * 100.0 ) | round(1) }}'
    unit_of_measurement : '%'

  - platform: file
    file_path: /config/pfSense_stats.json
    name: pfSense_mbufpercent
    value_template: '{{ value_json["data"]["stats"]["mbufpercent"] }}'
    unit_of_measurement : '%'
  • Make sure your quote charaters are correct. Sometimes they copy over wrong.
  • Replace <host> with the internal IP of your pfSense router (ie. 192.168.1.1).
  • Replace <apikey_value> with your api_key (ie PFFArandomalphanumeric902)
  • Replace <apisecret_value> with your api_secret (ie abcdefghijklmnopqrstuvwxyz1234567890abcde)
  1. Add a group with your new sensors. They should update every minute.
pfSense Monitor:
  view: no
  entities:
    - sensor.pfSense_cpu
    - sensor.pfsense_cpu_temp
    - sensor.pfsense_mbufpercent
    - sensor.pfsense_mem
    - sensor.pfsense_uptime

I hope this helps someone. You could also use the other calls to perform actions based on HA events. ie, if HA loses internet, you could reboot the router or pull the gateway status… GitHub - ndejong/pfsense_fauxapi: REST based API interface for pfSense 2.3.x and 2.4.x to facilitate devops

16 Likes

Nice work!!

Hey.
I tried to install the package in pfsense but a get a ssl error when I run fetch https://raw.githubusercontent.com/ndejong/pfsense_fauxapi/master/package/pfSense-pkg-FauxAPI-1.3_1.txz in ssh.

Hrm. Just tried mine, did a copy and paste and it downloaded no problem. How are you attempting this? SSH? Or from the web GUI (Diagnostics>Command Prompt)?

Also tried it on another FreeBSD box… worked no problem there too.

Worked like a charm to install and configure the api itself on pfsense. I’m confused on where all the other files come from and are supposed to be places. Configuring sensors is also OK I think, but I get this error:

File "/home/.homeassistant/config/pfstats.py", line 4, in <module>
Jan 27 16:47:11 ProLiant hass[6482]:     from fauxapi_lib import FauxapiLib
Jan 27 16:47:11 ProLiant hass[6482]: ImportError: No module named 'fauxapi_lib'
Jan 27 16:47:11 ProLiant hass[6482]: /homehomeassistant/lib/python3.5/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-us
Jan 27 16:47:11 ProLiant hass[6482]:   InsecureRequestWarning)
Jan 27 16:47:11 ProLiant hass[6482]: /home/homeassistant/lib/python3.5/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-us
Jan 27 16:47:11 ProLiant hass[6482]:   InsecureRequestWarning)
Jan 27 16:47:11 ProLiant hass[6482]: /home/homeassistant/lib/python3.5/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-us
Jan 27 16:47:11 ProLiant hass[6482]:   InsecureRequestWarning)
Jan 27 16:47:11 ProLiant hass[6482]: 2019-01-27 16:47:11 ERROR (Thread-3) [homeassistant.components.sensor.command_line] Command failed: python3 /home/.homeassistant/config/pfstats.py serverIP apikey api_scret_value
lines 1-18/18 (END)

Ofc ip/api/value is another (just for this post).
created /home/homeassistant/config and I’m not really sure if it’s supposed to be like that, also created /home/homeassistant/client-libs/python which I placed fauxapi_lib.py (unchanged file).
pfstats.py:

import os, sys, json

sys.path.append(os.path.abspath(os.path.join(os.path.curdir, '../client-libs/python')))     # hack to make this work in-place
from fauxapi_lib import FauxapiLib


# check args exist
if(len(sys.argv) < 4):
    print()
    print('usage: ' + sys.argv[0] + ' <host> <apikey> <apisecret>')
    print()
    print('pipe JSON output through jq for easy pretty print output:-')
    print(' $ ' + sys.argv[0] + ' <host> <apikey> <apisecret> | jq .')
    print()
    sys.exit(1)

# config
fauxapi_host=sys.argv[1]
fauxapi_apikey=sys.argv[2]
fauxapi_apisecret=sys.argv[3]

FauxapiLib = FauxapiLib(fauxapi_host, fauxapi_apikey, fauxapi_apisecret, debug=False)


# system_stats
# =============================================================================
print(json.dumps(
    FauxapiLib.system_stats())
)
open('/home/.homeassistant/config/pfSense_stats.json', 'w').close()  # clear data
with open('/home/.homeassistant/config/pfSense_stats.json', 'a') as stat_file:
    print(json.dumps(
    FauxapiLib.system_stats()), file=stat_file)

Not sure why it can’t find the library, or where should it be placed, am I going about this wrong?

The insecure requests warning that you are getting is because your using SSL (https) to connect to a url that has an self signed certificate. This is normal and expected. Pfsense by default uses a self-signed certificate for the encrypted connection. You are being warned because your machine isn’t sure that the information passed can be trusted, but since this is your machine you know that you can trust it. I can’t remember doing anything to prevent this warning. Maybe the library takes care of this? I can’t remember.

How are you running HA? In my example, I am using hassio on a raspberry pi. It looks like your running things a bit differently and it looks like your python script is not finding the library file. Are they in the same directory? Maybe there are permission issues on the library file?

What does your HA config entry look like for the sensor?

I worked on that for several days, didn’t get it to work.
It’s running esxi/debian/container. Oh well I tried it; didn’t work.

Cheers!

I could not get the python script to work. I’m using hassio on ubuntu 18.04 on top unraid. What I ended doing was to write a bash script to accomplish the same thing. It can be found here. Thank you for the hard work.

I Can not seem to get it right. Got the Error loading /home/hass/homeassistant/configuration.yaml: mapping values are not allowed here in "/home/hass/homeassistant/sensor.yaml", line 234, column 13
Line 234 is the command: "python3…

This is the copy paste from the sensor code above:

  - platform: command_line
     command: "python3 /config/pffa_get_system_stats.py 192.168.1.1 PFFArandomalphanumeric902 abcdefghijklmnopqrstuvwxyz1234567890abcde"
     name: pfSense_CPU_temp
     value_template: '{{ value_json["data"]["stats"]["temp"] }}'
     unit_of_measurement : 'C'
    
   - platform: file
     file_path: /config/pfSense_stats.json
     name: pfSense_uptime
     value_template: '{{ value_json["data"]["stats"]["uptime"] }}'
  
   - platform: file
     file_path: /config/pfSense_stats.json
     name: pfSense_mem
     value_template: '{{ value_json["data"]["stats"]["mem"] }}'
     unit_of_measurement : '%'
    
   - platform: file
     file_path: /config/pfSense_stats.json
     name: pfSense_cpu
     value_template: '{{ ( ( ((value_json["data"]["stats"]["cpu"].split("|")[0] | float) / (value_json["data"]["stats"]["cpu"].split("|")[1] | float)) - 1.0 ) * 100.0 ) | round(1) }}'
     unit_of_measurement : '%'

   - platform: file
     file_path: /config/pfSense_stats.json
     name: pfSense_mbufpercent
     value_template: '{{ value_json["data"]["stats"]["mbufpercent"] }}'
     unit_of_measurement : '%'

Mapping value errors usually mean that your spacing is incorrect in your yaml. Yaml is very sensitive to spacing organization. From what you put in your code snippet, it looks like your missing 1 space before your first line

- platform: command_line

Got that fixed. Now I got this.I’m missing a file?

Command failed: python3 /config/pffa_get_system_stats.py 192.168.1.1

The sensors loaded but no data. Maybe a firewall issue?

image

Hrm… What version of HASS are you running? I’m using HASSIO. If you put the pffa_get_system_stats.py in your /config directory, you need to make sure that the file structure is the same when running the python command: Meaning that your ‘config’ directory is in your root directory… Does this make sense. In windows, this would look like c:/config. If you are running on raspbian or something the file structure is different.

Also, I noticed in the code that you pasted you had the default apikey_value and apisecret_values pasted in there. You can’t use the defaults, they are hardcoded to fail out for security reasons. You may have just put those in there as examples to share back here in the forum, but if you’re actually using them you need to change them to something else.

I do not have that file. Where is it? Sorry I’m new to this.

Ya I just pasted the default keys and secret. Did not want to show mine.

Cool.

Python script is in the second link, but you need the python library as well.

Sorry, I forgot that I changed the name of mine because I deleted everything but the system stats. I didn’t want anyother information.

Here is my simplified version of python-lib-iterate.py. With this, you obviously won’t be able to POST anything to pfSense. Just read the system stats. You can also limit this through permissions on the pfSense PFFA side.

#!/usr/bin/python3
# 
# Copyright 2017 Nicholas de Jong  <contact[at]nicholasdejong.com>
# 
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# 
#     http://www.apache.org/licenses/LICENSE-2.0
# 
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# 

import os, sys, json

sys.path.append(os.path.abspath(os.path.join(os.path.curdir, '../client-libs/python')))     # hack to make this work in-place
from fauxapi_lib import FauxapiLib


# check args exist
if(len(sys.argv) < 4):
    print()
    print('usage: ' + sys.argv[0] + ' <host> <apikey> <apisecret>')
    print()
    print('pipe JSON output through jq for easy pretty print output:-')
    print(' $ ' + sys.argv[0] + ' <host> <apikey> <apisecret> | jq .')
    print()
    sys.exit(1)

# config
fauxapi_host=sys.argv[1]
fauxapi_apikey=sys.argv[2]
fauxapi_apisecret=sys.argv[3]

FauxapiLib = FauxapiLib(fauxapi_host, fauxapi_apikey, fauxapi_apisecret, debug=False)


# system_stats
# =============================================================================
print(json.dumps(
    FauxapiLib.system_stats())
)
open('/config/python_scripts/pfSense_stats.json', 'w').close()  # clear data
with open('/config/python_scripts/pfSense_stats.json', 'a') as stat_file:
    print(json.dumps(
    FauxapiLib.system_stats()), file=stat_file)

Hi this is great thanks. Got most of it running but for some reason can only get it to display the CPU temp, the rest state “entity not available” but if I look at the json file I see it is loading the stats for the the other sensors.

for ref I have the files saved as follows:

/config/packages/components/pfSense
fauxapi_lib.py
pffa_get_system_stats.py

/config/packages/components/pfSense/python_scripts
pfSense_stats.json

and the sensors on a lovelace card as follows:

    - type: glance
      title: pfSense Router
      entities:
        - entity: sensor.pfSense_cpu
          name: pfSense CPU
        - entity: sensor.pfsense_cpu_temp
          name: pfSense CPU Temp
        - entity: sensor.pfsense_mbufpercent
          name: pfSense mbu %
        - entity: sensor.pfsense_mem
          name: pfSense Mem
        - entity: sensor.pfsense_uptime
          name: pfSense Uptime

If I remove it all from the card under unused entities it is only showing the CPU temp.

Any chance anybody has an idea what could be the issue?

Sounds like your sensors aren’t setup correctly. If the json file looks right, take another look at your sensor config in your configuration.yaml.

1 Like

Oh and did you look for them in your https://[IP]:8123/dev-state ? It sounds like you just tried loading them into Lovelace. You could have just had a typo or something and lovelace doesn’t see them.

1 Like

Thanks for the response.

they weren’t there in the dev-states, but rechecked the logs and issue was the dir wasn’t whitelisted.

All working now cheers.

1 Like