FreeNAS Stat Monitor

Koen · September 25, 2019, 2:43pm

Hi,

I have some issues with the ssh commands that are send. If I try those in cmd, than I have to seperately fill-in my password and than I get the expected “up” answer. HA is failing to do this the way you mention above. Got a way to resolve this issue?

Cheers,
Koen

Petrica · September 26, 2019, 9:44am

You can either use SSHPASS to send the password in plain text (not recommended) or add ssh keys from HA’s host to FreeNAS to be able to poll the data over ssh.

Downside of the above choices is that you still need at least one successful ssh login from HA to FreeNAS (and when doing upgrades or some other changes to HA host you need to redo the process). Or, you can skip this altogether and use Netdata sensor by enabling Netdata service on FreeNAS (just change bind to 0.0.0.0 instead of 127.0.0.1 in /etc/local/netdata/netdata.conf as is doesn’t publish the port by default - on FreeNAS that is ).

And, there’s more if you rather like the Netdata graphs to show in HA, you can use the iframe panel.

Limych · September 27, 2019, 8:25pm

My current FreeNAS HA monitor:

Configs:

github.com

Limych/HomeAssistantConfiguration/blob/master/lovelace/system/nas_monitor_card.yaml

type: vertical-stack
cards:
  - type: custom:vertical-style-card
    style: |
      ha-card {
        background: "{{ states.sensor.nas_pool_pool1_health.attributes.card_bg_color }}";
      }
    cards:
      - type: entities
        title: "NAS сервер"
        show_header_toggle: false
        entities:
          - entity: sensor.nas_last_boot
            name: "Last boot"
      # - type: markdown
      #   content: |
      #     [[ sensor.nas_alerts ]]

  - type: horizontal-stack
    cards:

This file has been truncated. show original

github.com

Limych/HomeAssistantConfiguration/blob/master/packages/system/nas_monitor.yaml

homeassistant:
  customize_glob:
    sensor.*_percent:
      unit_of_measurement: '%'
    #
    sensor.nas_cpu_*:
      icon: mdi:memory
    #
    sensor.nas_memory_*:
      icon: mdi:memory
    #
    sensor.nas_disk_*:
      icon: mdi:harddisk
    sensor.nas_disk_ada0*:
      friendly_name: "ada0"
    sensor.nas_disk_ada1*:
      friendly_name: "ada1"
    sensor.nas_disk_ada2*:
      friendly_name: "ada2"
    sensor.nas_disk_ada3*:

This file has been truncated. show original

And statistic pusher:

github.com

Limych/HomeAssistantConfiguration/blob/master/external_addons/ha_host_monitor.py

#!/usr/bin/env python

#  Copyright (c) 2020, Andrey "Limych" Khrolenok <[email protected]>
#  Creative Commons Attribution-NonCommercial-ShareAlike License v4.0
#  (see COPYING, LICENSE or https://creativecommons.org/licenses/by-nc-sa/4.0/)

# An actual version of this script you always can download from
# https://github.com/Limych/HomeAssistantConfiguration/tree/master/external_addons

import argparse
import datetime
import json
import multiprocessing
import os
import re
import socket
import subprocess

import psutil
import requests

This file has been truncated. show original

Dingin · October 4, 2019, 11:41pm

there is no need for password if you use SSHkey, the authentication is done in automatic by the public and privet keys between HA and the server

https://www.digitalocean.com/community/tutorials/how-to-configure-ssh-key-based-authentication-on-a-linux-server

DrBlokmeister · October 7, 2019, 2:47pm

Wow! You built your own addon! Is it possible to use that as-is? If so, how? Do you have a guide to follow? Thanks!

Limych · October 7, 2019, 9:43pm

Yes, you can use pusher script as-is at any system (Unix, Linux, Windows, OSx…). Just copy it to host you want to monitor and add Cron job to periodically run pusher script with proper parameters. That’s all.

For parameters description just manual run script without parameters.

I’m run it every 1 minute at my NAS.

DrBlokmeister · October 8, 2019, 11:19am

Thanks man! I’m running the code unmodified now on my NAS, and it gives a good output when running verbose mode. However, I don’t see the values showing up in the HA sensors I created. I see you’re reading the values via MQTT. I’m using Mosquitto. Do I need to point the script to the Mosquitto port, rather than the Hass main instance? Or is there something else I’m missing?

Limych · October 8, 2019, 7:37pm

The script does not access the MQTT-server directly. It uses the Home Assistant API to send data to the current MQTT-server. So the script doesn’t care what your server is.

If the data does not pass, perhaps the reason is some kind of failure in accessing the API. For example, to access the API, you need to specify the correct token, which is created through the HA interface. https://developers.home-assistant.io/docs/en/external_api_rest.html

You can obtain a token (“Long-Lived Access Token”) by logging into the frontend using a web browser, and going to your profile http://IP_ADDRESS:8123/profile .

DrBlokmeister · October 17, 2019, 6:28pm

Thanks for the response. I have already created an access token and used that in the config. Can it have to do with the fact that I use duckdns, and I try to send the data to https://myalias.duckdns.org/8123?

Limych · October 17, 2019, 6:46pm

I have not tested this variant, but I believe that there should be no difference.
In verbose mode, are there any errors from the server? If not, then, in theory, the API command work correctly …

DrBlokmeister · October 17, 2019, 6:58pm

Thanks for the quick response!

The response I get from running your script is below. Note that I have redacted my duckdns alias and API keys.

BlokmeisterNAS# python /mnt/Storage2/Other/Scripts/ha_host_monitor.py -c -v -s https://alias.duckdns.org:8123 API_KEY_REDACTED
Base URL: https://alias.duckdns.org:8123/api/services/mqtt/publish
SSL certificate check: False
blokmeisternas_8_8_8_8_state: {"last_boot": "2019-10-07T16:11:29", "cpu_load_1m": "0.57", "cpu_load_5m": "0.45", "cpu_load_1                                                         5m": "0.40", "cpu_temperature": 34.0, "cpu_stat": {"user": 2.0, "nice": 0.0, "system": 3.3, "idle": 94.7, "irq": 0.0}, "memo                                                         ry_stat": {"total": 16804630528, "available": 5493555200, "percent": 67.3, "used": 10382729216, "free": 405114880, "active":                                                          457871360, "inactive": 5088440320, "buffers": 0, "cached": 0, "shared": 240570368, "wired": 9924857856}, "swap_stat": {"tot                                                         al": 1572768, "used": 0, "free": 1572768, "percent": 0.0, "sin": 0, "sout": 5914}, "disks_stat": {"/": {"total": 11944650803                                                         2, "used": 2804934656, "free": 116641573376, "percent": 2.3}, "/mnt/NewStorage": {"total": 27400687971840, "used": 193297414                                                         28736, "free": 8070946543104, "percent": 70.5}, "/mnt/Storage2": {"total": 3140122140672, "used": 2482703224832, "free": 657                                                         418915840, "percent": 79.1}, "/mnt/Storage2/Jails": {"total": 657419005952, "used": 90112, "free": 657418915840, "percent":                                                          0.0}, "/mnt/Storage2/iocage": {"total": 657423183872, "used": 4268032, "free": 657418915840, "percent": 0.0}, "/mnt/Storage2                                                         /iocage/download": {"total": 657419005952, "used": 90112, "free": 657418915840, "percent": 0.0}, "/mnt/Storage2/iocage/downl                                                         oad/11.2-RELEASE": {"total": 657704017920, "used": 285102080, "free": 657418915840, "percent": 0.0}, "/mnt/Storage2/iocage/i                                                         mages": {"total": 657419005952, "used": 90112, "free": 657418915840, "percent": 0.0}, "/mnt/Storage2/iocage/jails": {"total"                                                         : 657419005952, "used": 90112, "free": 657418915840, "percent": 0.0}, "/mnt/Storage2/iocage/jails/asigra": {"total": 6574191                                                         77984, "used": 262144, "free": 657418915840, "percent": 0.0}, "/mnt/Storage2/iocage/jails/asigra/root": {"total": 6582881935                                                         36, "used": 869277696, "free": 657418915840, "percent": 0.1}, "/mnt/Storage2/iocage/jails/transmission": {"total": 657419104                                                         256, "used": 188416, "free": 657418915840, "percent": 0.0}, "/mnt/Storage2/iocage/jails/transmission/root": {"total": 658129                                                         502208, "used": 710586368, "free": 657418915840, "percent": 0.1}, "/mnt/Storage2/iocage/log": {"total": 657419014144, "used"                                                         : 98304, "free": 657418915840, "percent": 0.0}, "/mnt/Storage2/iocage/releases": {"total": 657419005952, "used": 90112, "fre                                                         e": 657418915840, "percent": 0.0}, "/mnt/Storage2/iocage/releases/11.2-RELEASE": {"total": 657419005952, "used": 90112, "fre                                                         e": 657418915840, "percent": 0.0}, "/mnt/Storage2/iocage/releases/11.2-RELEASE/root": {"total": 658305945600, "used": 887029                                                         760, "free": 657418915840, "percent": 0.1}, "/mnt/Storage2/iocage/templates": {"total": 657419005952, "used": 90112, "free":                                                          657418915840, "percent": 0.0}, "/var/db/system": {"total": 8070947020800, "used": 477696, "free": 8070946543104, "percent":                                                          0.0}, "/var/db/system/cores": {"total": 8070963302400, "used": 16759296, "free": 8070946543104, "percent": 0.0}, "/var/db/s                                                         ystem/samba4": {"total": 8070947341824, "used": 798720, "free": 8070946543104, "percent": 0.0}, "/var/db/system/syslog-3bc72                                                         4bbb18d493585697320a07060e7": {"total": 8070946686976, "used": 143872, "free": 8070946543104, "percent": 0.0}, "/var/db/syst                                                         em/rrd-3bc724bbb18d493585697320a07060e7": {"total": 8070946686976, "used": 143872, "free": 8070946543104, "percent": 0.0}, "                                                         /var/db/system/configs-3bc724bbb18d493585697320a07060e7": {"total": 8071023985152, "used": 77442048, "free": 8070946543104,                                                          "percent": 0.0}, "/var/db/system/webui": {"total": 8070946686976, "used": 143872, "free": 8070946543104, "percent": 0.0}, "/                                                         mnt/Storage2/iocage/jails/asigra/root/proc": {"total": 4096, "used": 4096, "free": 0, "percent": 100.0}}, "disks_temperature                                                         ": {"ada0": "31", "ada1": "31", "ada2": "30", "ada3": "30", "ada4": "30", "ada5": "30", "ada6": "32", "ada7": "31"}, "pools_                                                         stat": {"NewStorage": {"total": 35734127902720, "used": 24513953800192, "free": 11220174102528, "percent": 68.6, "health": "                                                         ONLINE"}, "Storage2": {"total": 3985729650688, "used": 2978206695424, "free": 1007522955264, "percent": 74.7, "health": "ONL                                                         INE"}}, "alerts": "[WARNING] New feature flags are available for volume NewStorage. Refer to the \"Upgrading a ZFS Pool\" su                                                         bsection in the User Guide \"Installing and Upgrading\" chapter and \"Upgrading\" section for more instructions.\n[WARNING]                                                          New feature flags are available for volume Storage2. Refer to the \"Upgrading a ZFS Pool\" subsection in the User Guide \"In                                                         stalling and Upgrading\" chapter and \"Upgrading\" section for more instructions.\n[CRITICAL] Pool jailusb state is UNKNOWN:                                                          "}

Limych · October 17, 2019, 7:51pm

I do not see error messages. It turns out that the command is sent correctly.

Try manually through the developer panel to call the mqtt.publish service with parameters
‘topic’: ‘sensor/blokmeisternas_8_8_8_8_state’,
‘payload’: ‘some data’,
‘retain’: True,
and see if this value gets into the MQTT server.

Then try sending the POST request with similar test data through the REST client to https://alias.duckdns.org:8123/api/services/mqtt/publish
Headers:
‘Authorization’: "Bearer " + your_token,
‘Content-Type’: ‘application/json’,

DrBlokmeister · October 18, 2019, 5:11pm

I feel like such an idiot. I just blindly copy-pasted your script, including the state topics. I corrected the state_topic to sensor/blokmeisternas_8_8_8_8_state and everything worked. Thank you so much for all your help man!

madrose · February 21, 2020, 8:40am

@Limych Thanks for the great add on for FreeNAS. I am using it and it work perfectly. I just wanted to ask regarding the templates.

as you can see the templates shows as a text for the entity and will not translate to color or anything else. How is this supposed to be used?

appreciate the help.

Schmandre · July 12, 2020, 5:50pm

Hello there,

first thank you for the integration. Unfortunately I got some errors while trying to call the api. Password and url was changed.

2020-07-12 19:42:18 WARNING (SyncWorker_15) [homeassistant.components.rest.sensor] REST result could not be parsed as JSON

Any ideas how I can debug this?

Greetings
Schmandre

gremblin · July 13, 2020, 8:15pm

try getting the raw response by performing a cURL command on your computer. See what the data looks like. If memory serves, if you dont have any warnings from the FN_alert_message, the response is empty and you’ll get this error.

Schmandre · July 13, 2020, 8:48pm

I get this error for every sensor. I’ll try to perform the curl command manually and get back to you with my results tomorrow.

Shogun1978 · July 20, 2020, 4:06pm

I was running FreeNAS in a VM for testing and now upgraded to TrueNAS 12 beta. It seems like the “old” api v1.0 is no longer available. There is now a v2.0 api. I already played around with it, but it seems like there are no “pct used” values any more. Version flag also changed. So be aware if you upgrade. Might that they will add v1.0 api in the TrueNAS final release for compatibility, what I really hope for.

gremblin · July 22, 2020, 4:47pm

@Shogun1978 Thanks for the heads up. I haven’t played around with the beta yet…

@Limych’s solution should still work with TruNAS though. It doesn’t rely on FreeNAS’s API. It’s pretty slick. FreeNAS Stat Monitor

Shogun1978 · July 22, 2020, 8:39pm

@gremblin: you have installed HA in a FreeNAS jail? Or as add-on? So you can grab the values locally?