IPMI Sensors

Tags: #<Tag:0x00007f32656364a8> #<Tag:0x00007f32656363b8> #<Tag:0x00007f32656362f0>

I have a Supermicro server and wanted to monitor the IPMI sensors within HA for future projects (UPS shutdown scripts, cooling automation, and general temperature/energy monitoring, etc).

After going down a rabbit hole attempting to use the SSH & Web Terminal Add-On to get IPMItools installed inside HA, I quickly determined that this was a futile effort. There were a few community add-ons like EvilMarty’s IPMI add-on here, but they weren’t quite what I was looking for (and I couldn’t find it in the Add-on Store any more.)

MQTT to the rescue!

This guide assumes you already have:

  1. A working instance of HA (obviously!)
  2. The MQTT and File Editor add-ons installed and running in HA
  3. A way to edit your YAML file, probably easiest through the “File Editor” extension.
  4. A motherboard with IPMI outputs that you want to bring into HA. (My guide is specifically based on a Supermicro board, but any IPMI equipped board that can be read using the Linux ipmitool command should work. You’ll just have to edit the commands to fit your specific data.)
  5. A separate container or machine to run a MQTT client with a Debian based Linux OS installed. I used a virtualized Ubuntu container installed in Proxmox, but you could probably spin up a Docker container or even a bare-metal machine like a Raspberry Pi. Either way, you just need a Debian based Linux distro installed (Debian, Ubuntu, etc). This guide assumes you are logged in via local user “mqtt” with sudo access, but you can adjust this guide to your local username on that machine.

Shoutout to sasukebinbin as a big inspiration. This thread was the starting point for my guide.

Other info I used:
MQTT Sensor Configuration Parameters
Templating to process incoming data
ipmitool Info
mosquitto_pub Info
Cut command
Awk comamnd

On to the guide:
Track down your IPMI IP address and login credentials.
I believe Supermicro boards default to the IP 192.168.1.99 and ADMIN/ADMIN for the login and password. (If you haven’t already, now is a good time to change all of these for security.)
You’ll also need your MQTT credentials. These can be found by going to Supervisor --> Mosquitto broker --> Configuration
Screenshot from 2021-02-10 07-28-11

Spin up your container or machine of choice and make sure you have an updated Linux distro running. If you’re handy with Docker, you can install it on the same machine that you have your HA instance running on.

Install ipmitool and mosquitto-clients:

sudo apt install ipmitool mosquitto-clients -y

Now test your IPMI credentials and determine what information you would like to bring into HA, by running the following command:

ipmitool -I lanplus -H <IPMI IP ADDRESS> -U <IPMI USERNAME> -P <IPMI PASSWORD> sdr elist full

You should get something like the following, showing the readout from your various motherboard sensors and their corresponding IPMI data. Mine follows the format of:
Sensor | Address | Status | Sensor Data Record (SDR) Entity | Sensor Value

CPU Temp         | 01h | ok  |  3.1 | 36 degrees C
PCH Temp         | 0Ah | ok  |  7.3 | 49 degrees C
System Temp      | 0Bh | ok  |  7.1 | 28 degrees C
Peripheral Temp  | 0Ch | ok  |  7.2 | 39 degrees C
VcpuVRM Temp     | 10h | ok  |  8.1 | 38 degrees C
VmemABVRM Temp   | 12h | ok  |  8.2 | 32 degrees C
VmemCDVRM Temp   | 13h | ok  |  8.3 | 27 degrees C
FAN1             | 41h | ok  | 29.1 | 4100 RPM
FAN2             | 42h | ns  | 29.2 | No Reading
FAN3             | 43h | ok  | 29.3 | 2400 RPM
FAN4             | 44h | ok  | 29.4 | 2400 RPM
FAN5             | 45h | ok  | 29.5 | 2400 RPM
FANA             | 47h | ns  | 29.7 | No Reading
FANB             | 48h | ok  | 29.8 | 2500 RPM
FANC             | 49h | ok  | 29.9 | 2600 RPM

(You may have more or fewer entries, with different SDRs, depending on your motherboard. I’ve removed many of the less useful entries from my list above.)

Take note of which sensors you want to bring into HA, and their corresponding sdr numbers. In my case, I was interested in the following temperature and fan speed readings:

Name         | SDR Entity
CPU Temp     | 3.1
PCH Temp     | 7.3
System Temp  | 7.1
VcpuVRM Temp | 8.1
FAN1         | 29.1
FAN3         | 29.3
FAN4         | 29.4
FAN5         | 29.5
FANB         | 29.8
FANC         | 29.9

This command will return the sensor output from a single SDR:

ipmitool -I lanplus -H <IPMI IP ADDRESS> -U <IPMI USERNAME> -P <IPMI PASSWORD> sdr entity <ENTITY NUMBER>

For example, if you wanted the “CPU Temp” from my output, you would enter the following:

ipmitool -I lanplus -H <IPMI IP ADDRESS> -U <IPMI USERNAME> -P <IPMI PASSWORD> sdr entity 3.1

In my case, this returns:

CPU Temp | 01h | ok | 3.1 | 38 degrees C

Great! But we’re only interested in the actual temperature number, not the rest of the text.
You can concatenate the output using the cut command:

ipmitool -I lanplus -H <IPMI IP ADDRESS> -U <IPMI USERNAME> -P <IPMI PASSWORD> sdr entity 3.1|cut -d '|' -f5|awk '{print $1}'

Which returns:

36

The piped cut and awk commands first ‘cut’ the data out of the fifth column after the ‘|’ character, and then returns the first field within that last column, leaving you with just the numerical value.


CPU Temp         | 01h | ok  |  3.1 | 36 degrees C

becomes

 36 degrees C

which finally turns into

36

It’s difficult to tell, but there is a leading space after the first cut, but the awk command takes care of removing that if it is present.

Similarly, you can get the FAN3 speed with the following command:

ipmitool -I lanplus -H <IPMI IP ADDRESS> -U <IPMI USERNAME> -P <IPMI PASSWORD> sdr entity 29.3|cut -d '|' -f5|awk '{print $1}'

Which returns:

2400

Ok, now that you know how to parse the ipmitool information, it’s time to wrap it in a shell script:
Create a script file with the following command:

nano /home/mqtt/ipmi-mqtt.sh

This uses nano as the text editor, and assumes you are logged in as local user ‘mqtt’. You can adjust as necessary, replacing nano with vim or your editor of choice, and replacing ‘mqtt’ with your local user for this machine. The user doesn’t affect anything with HA, just where your script is located on this machine.

Now paste the following text, adjusting for your credentials:

#!/bin/bash
SHELL=/bin/sh PATH=/bin:/sbin:/usr/bin:/usr/sbin
MQTT_IP='<MQTT SERVER/HA IP ADDRESS>'
MQTT_USER='<MQTT USERNAME>'
MQTT_PW="<MQTT PASSWORD>"
IPMI_IP='<IPMI IP ADDRESS>'
IPMI_USER='<IPMI USERNAME>'
IPMI_PW='<IPMI PASSWORD>'
NUMBER_OF_SENSORS='10'
CUT_COLUMN='5'
TOPIC[1]='server_cpu_temp'
TOPIC[2]='server_pch_temp'
TOPIC[3]='server_system_temp'
TOPIC[4]='server_vrm_temp'
TOPIC[5]='server_cpu_speed'
TOPIC[6]='server_front_fan1_speed'
TOPIC[7]='server_front_fan2_speed'
TOPIC[8]='server_front_fan3_speed'
TOPIC[9]='server_rear_fan1_speed'
TOPIC[10]='server_rear_fan2_speed'
SDR[1]='3.1'
SDR[2]='7.3'
SDR[3]='7.1'
SDR[4]='8.1'
SDR[5]='29.1'
SDR[6]='29.3'
SDR[7]='29.4'
SDR[8]='29.5'
SDR[9]='29.8'
SDR[10]='29.9'

for i in $(eval echo "{1..$NUMBER_OF_SENSORS}")
do
   PAYLOAD=$(ipmitool -I lanplus -H "$IPMI_IP" -U "$IPMI_USER" -P "$IPMI_PW" sdr entity "${SDR[$i]}"|cut -d '|' -f$CUT_COLUMN|awk '{print $1}')
   mosquitto_pub -r -t "${TOPIC[$i]}" -m "$PAYLOAD" -h "$MQTT_IP" -u "$MQTT_USER" -P "$MQTT_PW"
done

You can adjust the TOPIC names to whatever you want, and the SDR entities and cut values as necessary. Add/delete any additional/unused TOPIC and SDR pairs as needed to match your IPMI sensors, and adjust the NUMBER_OF_SENSORS variable to match. In my case, I’m importing the CPU, PCH, System, and VRM temperatures, as well as all of the fan speeds.
Most of the variables should be fairly self explanatory, but the CUT_COLUMN variable tells the script to look for the first field in the fifth column.

Once you’re done editing, you should have a shell script file with the appropriate headings, and a list of numbered TOPIC entries and corresponding matching PAYLOAD entries.
Exit and save. If you’re new to nano, “Ctrl+X” to exit, “Y” to save, and “Enter” to confirm filename.

Now make sure the file is executable by running

chmod +x /home/mqtt/ipmi-mqtt.sh

And adjust the permissions so only the owner can read and execute:

chmod 500 /home/mqtt/ipmi-mqtt.sh

Note, once you edit the permissions, you will need root level access to make any future edits, and will need to run the edit command with sudo privileges for your user:

sudo nano /home/mqtt/ipmi-mqtt.sh


Now create a cronjob that will run this script once a minute and update your sensors:

sudo crontab -e

If this is your first time editing the crontab, choose “1” to edit with nano, and enter this line at the bottom of the file:

*/1 * * * * cd /home/mqtt && ./ipmi-mqtt.sh>>ipmi-crontab.log

This will run your newly created ipmi-mqtt.sh script once every minute, sending your TOPIC and PAYLOAD pairs to the MQTT server to ingest in HA.

Again, “Ctrl+X” to Exit, “Y” to confirm changes, and “Enter” to confirm filename.


If you want higher resolution than one minute intervals, you could alternately run an infinitely looping shell script, use a cron scheduler with finer time intervals (like mcron), or set up your script as a more proper SystemD service.


Now that your newly configured MQTT client is sending the sensor payloads to HA, go back to HA and check the Mosquitto broker add-on page to confirm that everything is working correctly. Instead of going to configuration, this time click on the Log tab. After a minute or so, you should see a new connection from the client you just configured. If everything is working correctly, it’s time to set up sensors to use your new data:
Go to the File Editor and open your configuration.yaml file.
Under your sensor heading, add the following mqtt sensors:

#Server IPMI Sensors
  - platform: mqtt
    name: "CPU Temp"
    state_topic: "server_cpu_temp"
    unit_of_measurement: '°C'
  - platform: mqtt
    name: "PCH Temp"
    state_topic: "server_pch_temp"
    unit_of_measurement: '°C'
  - platform: mqtt
    name: "System Temp"
    state_topic: "server_system_temp"
    unit_of_measurement: '°C'
  - platform: mqtt
    name: "VRM Temp"
    state_topic: "server_vrm_temp"
    unit_of_measurement: '°C'
  - platform: mqtt
    name: "CPU Fan Speed"
    state_topic: "server_cpu_speed"
    value_template: '{{ value | multiply(0.001) | round(1) }}'
    unit_of_measurement: 'kRPM'
  - platform: mqtt
    name: "Front Fan1 Speed"
    state_topic: "server_front_fan1_speed"
    value_template: '{{ value | multiply(0.001) | round(1) }}'
    unit_of_measurement: 'kRPM'
  - platform: mqtt
    name: "Front Fan2 Speed"
    state_topic: "server_front_fan2_speed"
    value_template: '{{ value | multiply(0.001) | round(1) }}'
    unit_of_measurement: 'kRPM'
  - platform: mqtt
    name: "Front Fan3 Speed"
    state_topic: "server_front_fan3_speed"
    value_template: '{{ value | multiply(0.001) | round(1) }}'
    unit_of_measurement: 'kRPM'
  - platform: mqtt
    name: "Rear Fan1 Speed"
    state_topic: "server_rear_fan1_speed"
    value_template: '{{ value | multiply(0.001) | round(1) }}'
    unit_of_measurement: 'kRPM'
  - platform: mqtt
    name: "Rear Fan2 Speed"
    state_topic: "server_rear_fan2_speed"
    value_template: '{{ value | multiply(0.001) | round(1) }}'
    unit_of_measurement: 'kRPM'

Change the names to whatever you want, but make sure the state_topic values match what you entered as TOPICs in your shell script file.
Since my server fans run at well over 1000 RPM, I used the value_template to effectively divide by 1000 to ingest the values as kRPMs, ie 2.4 kRPM instead of the raw value of 2400 RPM. It’s absolutely not necessary, but it keeps my Lovelace cleaner. If you don’t need to divide by 1000, you can just omit that entire line and use the raw value as passed from the MQTT client. Note: there is no native divide function, so you will need to multiply by 0.001 to have an expression equivalent to diving by 1000.

Once your yaml file is configured to your liking, save, exit, and restart HA.

When it comes back online, check your log files to make sure the MQTT client is connecting, and there are no issues with your yaml data. If everything went smoothly, you should now have a new sensor for each line you set up in the yaml file.

Now you can use these sensors in your automation scripts, or just throw in your Lovelace dashboard for monitoring: