MQTT sensor, add attributes while remaining in autodiscovery

Olivier1974 · June 4, 2023, 11:06am

Hello there,

I’ve some MQTT sensor that are discovered automatically.
Config sent:

{
  "name": "Devolo BBox",
  "state_topic": "homeassistant/sensor/BBox/state",
  "unique_id": "BBox",
  "device": {
    "identifiers": [
      "30:D3:2D:12:CA:F9"
    ],
    "name": "BBox",
    "model": "dLAN 550 duo+",
    "manufacturer": "Devolo"
  }
}

Updating the satus with

mosquitto_pub -h <host> -p 1883 -u <mqtt_username> -P <mqtt_password> -t homeassistant/sensor/BBox/state -m on

is working fine.

But now I’d like to add some “attributes” to the sensor, like “TX” “RX” (as it is a Devolo plug)
Is it possible to do it while remaining in autodiscovery?
I’d like to avoid the mqtt sensor entry in a yaml file.

123 · June 4, 2023, 1:08pm

I believe it’s possible but you will have to publish the complete discovery configuration plus the additional two attributes. In other words, you can’t just publish the additional attributes, you have to publish the entity’s complete configuration.

The question becomes: What will happen after you restart the physical device (Devolo)? It will publish its original discovery configuration which doesn’t contain your two additional attributes (and so they’ll disappear from sensor.bbox).

Olivier1974 · June 4, 2023, 1:59pm

I’m the one that created the shell script gathering the PLC info and publish the topic.
It will run every X minutes, to have an history of the connectivity as well as the RX/TX values.
It contains the publishing of the config as well as the state.

    json_payload=$(jq -n --arg sensor_name "$sensor_name" --arg status "$status" --arg type "$type" --arg friendly_name "$friendly_name" '{"name": "Devolo '$friendly_name'", "state_topic": "homeassistant/sensor/'$sensor_name'/state", "unique_id": "'$sensor_name'", "device": {"identifiers": ["'$mac'"], "name": "'$friendly_name'", "model": "dLAN 550 duo+", "manufacturer": "Devolo"}}')

    # Publish the MQTT Discovery configuration to the appropriate topic
    mosquitto_pub -h $host -p $port -u $mqtt_username -P $mqtt_password -t homeassistant/sensor/$sensor_name/config -m "$json_payload"

    # Publish the sensor state to the MQTT state topic
    mosquitto_pub -h $host -p $port -u $mqtt_username -P $mqtt_password -t homeassistant/sensor/$sensor_name/state -m "$status"

The devolo network plugs (not wifi) are not providing any information as such. I’m using this

to get as much info as I can and inform home assistant.

So now, I have to figure out how to send the proper config to create the complete config, right?

123 · June 4, 2023, 2:31pm

In that case, it should be fairly easy for you to add two attributes to the existing discovery configuration.

Olivier1974 · June 4, 2023, 5:04pm

This is where I am, reading again and again the discovery-payload of this page

And still trying to understand what I should put in my config payload. I thought that this was a good idea but it is not, it is not working, no attribute.

{
  "name": "Devolo BBox",
  "state_topic": "homeassistant/sensor/BBox/state",
  "unique_id": "BBox",
  "device": {
    "identifiers": [
      "30:D3:2D:12:CA:F9"
    ],
    "name": "BBox",
    "model": "dLAN 550 duo+",
    "manufacturer": "Devolo",
  },
  "attributes": {
    "rx": 0,
    "tx": 0
  }
}

I’m a noob in mqtt, sorry.

EDIT:

{
  "name": "Devolo BBox",
  "state_topic": "homeassistant/sensor/BBox/state",
  "unique_id": "BBox",
  "device": {
    "identifiers": [
      "30:D3:2D:12:CA:F9"
    ],
    "name": "BBox",
    "model": "dLAN 550 duo+",
    "manufacturer": "Devolo"
  },
  "json_attributes_topic": "homeassistant/sensor/BBox/attributes",
  "json_attributes_template": "{{ value_json.data.value | tojson }}"
}

seems more like it should be.
And

mosquitto_pub -h $host -p $port -u $mqtt_username -P $mqtt_password -t homeassistant/sensor/BBox/attributes -m '{ "rx": 287, "tx": 265 }'

is understood

Great ! Thx @123

monty_burns_007 · July 25, 2023, 10:51am

Hi Olivier,

I’m looking for the same solution:
A tool or HA addon which get’s info from PLC (TPlink: Qualcomm Atheros chipset).
I’ve found also the openplc github and tools.

If I run the plcstat exe file on my windows PC I get :

C:\SharedVM\openplc>plcstat -t -m -i 3
P/L NET TEI ------ MAC ------ ------ BDA ------ TX RX CHIPSET FIRMWARE
LOC STA 015 18:A6:F7:57:B6:AA 38:F3:AB:39:40:BC n/a n/a QCA7500 MAC-QCA7500-2.13.0.30-01-20221117-CS
REM STA 006 18:A6:F7:A2:3D:74 E8:DB:84:89:30:C2 063 041 QCA7500 MAC-QCA7500-2.13.0.30-01-20221117-CS
REM CCO 007 84:16:F9:F3:C4:2D 2C:A5:9C:EC:E8:8F 059 053 QCA7500 MAC-QCA7550-2.13.0.30-01-20221117-CS
REM STA 013 84:16:F9:F3:BF:28 7C:64:56:F7:BC:A9 193 290 QCA7500 MAC-QCA7550-2.13.0.30-01-20221117-CS
NID 71:2F:7B:69:26:4A:07 SNID 001
STA TEI 015 MAC 18:A6:F7:57:B6:AA BDA 38:F3:AB:39:40:BC
STA TEI 006 MAC 18:A6:F7:A2:3D:74 BDA E8:DB:84:89:30:C2 TX 063 RX 041
CCO TEI 007 MAC 84:16:F9:F3:C4:2D BDA 2C:A5:9C:EC:E8:8F TX 059 RX 053
STA TEI 013 MAC 84:16:F9:F3:BF:28 BDA 7C:64:56:F7:BC:A9 TX 193 RX 290

So the TX and RX is available there. But how to get this (historically) in HA ?
Even more HA runs on Python, so I can’t use the windows or linux build binary file.
Can you share your plc script please ? Or how to use openplc in combination with MQTT publishing ?

thx in advance

Olivier1974 · July 25, 2023, 3:52pm

I can share it with you but it has evolved.
Please be aware that for some reason it is only updating the value of one plug, not the others, maybe a question of delay or string format, you know ', " all mixed. I’m still searching for the reason.

#!/bin/bash

# Check if the interface parameter is provided
if [ $# -lt 1 ]; then
    echo "Usage: $0 <interface>"
    exit 1
fi

interface="$1"
p=$(dirname $0)
# Look for PLC devices
plcstat_output=$(/usr/local/bin/plcstat -t -i "$interface" | tail -n +2)

# Run plcstat to update devolo.network file
while IFS= read -r line; do
    mac=$(echo "$line" | awk '{print $4}')
    type=$(echo "$line" | awk '{print $1}')
    plctool_output=$(/usr/local/bin/plctool -I -i "$interface" "$mac")

    # Extracting friendly_name from plctool output
    friendly_name=$(echo "$plctool_output" | awk '/USR/ {print $2}')

    # Check if friendly_name is empty or missing
    if [ -z "$friendly_name" ]; then
        friendly_name="$mac"  # Use MAC as friendly_name
    fi

    # Append unique MAC addresses (excluding the first line) to the devolo.network file
    # Create a temporary file
    tmp_file=$(mktemp)

    # Exclude the line containing the specific value and write the result to the temporary file
    grep -v -E "^[[:space:]]*$" devolo.network | grep -v "$mac" > "$tmp_file"

    # Overwrite the original file with the modified content
    mv "$tmp_file" "$p/devolo.network"

    # Append unique MAC addresses (excluding the first line) to the devolo.network file
    echo "$mac $friendly_name $type" >> "$p/devolo.network"

done <<< "$plcstat_output"

# Run the plcstat command and process the output
while IFS=" " read -r mac friendly_name type; do
    plctool_output=$(/usr/local/bin/plctool -I -i "$interface" "$mac" 2>/dev/null)

    # Extracting type, tx, and rx values from plcstat output
    line=$(echo "$plcstat_output" | awk -v mac="$mac" '$0 ~ mac { print $0 }')

    if [ -z "$line" ]; then
        # Plug is offline
        status="off"
    else
        # Plug is online
        status="on"
        read -r type _ _ _ _ tx rx _ <<< "$line"
    fi

    # Generate sensor name by removing unauthorized characters from friendly_name
    sensor_name=$(echo "$friendly_name" | tr -cd '[:alnum:]-_')

    # Create the JSON payload for MQTT Discovery configuration
    json_payload=$(jq -n --arg sensor_name "$sensor_name" --arg status "$status" --arg type "$type" --arg friendly_name "$friendly_name" \
       '{"name": "Devolo '$friendly_name'",
         "state_topic": "homeassistant/sensor/'$sensor_name'/state",
         "unique_id": "'$sensor_name'",
         "device": {"identifiers": ["'$mac'"],
                    "name": "'$friendly_name'",
                    "model": "dLAN 550 duo+",
                    "manufacturer": "Devolo"
                   },
         "json_attributes_topic": "homeassistant/sensor/'$sensor_name'/attributes",
         "json_attributes_template": "{{ value_json.data.value | tojson }}"
       }')

    # Publish the MQTT Discovery configuration to the appropriate topic
    mosquitto_pub -h localhost -u $mqtt_username -P $mqtt_password -t homeassistant/sensor/$sensor_name/config -m "$json_payload"

    # Publish the sensor state to the MQTT state topic
    mosquitto_pub -h localhost -u $mqtt_username -P $mqtt_password -t homeassistant/sensor/$sensor_name/state -m "$status"

    # Publish the sensor attributes to the MQTT attribute topic
    json_attribute="{ \"mac\": \"$mac\", \"type\": \"$type\""

    # Check if tx is a numeric value
    if [[ $tx =~ ^[0-9]+$ ]]; then
        json_attribute="$json_attribute, \"tx\": $tx"
    fi

    # Check if rx is a numeric value
    if [[ $rx =~ ^[0-9]+$ ]]; then
        json_attribute="$json_attribute, \"rx\": $rx"
    fi

    json_attribute="$json_attribute }"
    mosquitto_pub -h localhost -u $mqtt_username -P $mqtt_password -t homeassistant/sensor/$sensor_name/attributes -m "$json_attribute"

done < "$p/devolo.network"

You’ll see that model and manufacturer are hardcoded because all my plugs are the same.
Result looks like this in HA

Edit: If I run every command one by one, it is working, in the script it is only partially understood by mosquitto or HA.

monty_burns_007 · July 26, 2023, 5:06pm

thanks already for your script implementation.

Where do you run this bash script ?
On Home Assistant itself or another Linux PC/Pi ?
I can run the script for ex. under my /config/shell/ folder where I put the .sh file. (triggered by a HA script)
mosquitto_pub can be run from HA shell.

But where do you put the binaries then of plctool ?

How do I ciompile these to binary from source on the HA host ? (I currently only have windows EXE binaries op openplc tools)
Where to put these on the HA host.?

I run HA (OS) on a ESXi host.

If I type which mosquitto_pub then it says /usr/bin/mosquitto_pub
But I think /usr/bin is readonly and overwritten each time HA is updated.
I can put it in the /config/ branch which is custom and won’t be overwritten.

Or do we have to make a seperate docker to store these open-plc-utils ?

edit: I’ve found a solution already myself.
I’ve found windows and Linux 64bit binary files on: https://docs.insys-icom.de/pages/en_pl_config_slac.html
this contains “plctool”

I copied the Linux 64 bit binaries to a directory on my HA confg : /config/shell/
I mades these files executable with chmod 755

if I go into HA shell and now try:

shell: ./plctool -I -i enp2s1 18:A6:F7:57:B6:AA
PIB 0-0 21384 bytes
MAC 18:A6:F7:57:B6:AA
DAK CA:DF:8D:C3:76:AA:97:5C:4C:18:92:D7:96:E6:04:D4
NMK 4A:68:F8:36:92:AC:36:75:AC:C9:43:EF:CE:73:91:27
NID 71:2F:7B:69:26:4A:07
Security level 0
NET Qualcomm Atheros Enabled Network
MFG Qualcomm Atheros HomePlug AV Device
USR tpver_801191_180709_915
CCo Auto
MDU N/A

using the -m parameter I can show the network info viewed from the specifiek source MAC adress/PLC.

for ex/
shell: ./plctool -m -i enp2s1 18:A6:F7:57:B6:AA
enp2s1 18:A6:F7:57:B6:AA Fetch Network Information
enp2s1 18:A6:F7:57:B6:AA Found 1 Network(s)

source address = 18:A6:F7:57:B6:AA

    network->NID = 71:2F:7B:69:26:4A:07
    network->SNID = 1
    network->TEI = 1
    network->ROLE = 0x00 (STA)
    network->CCO_DA = 18:A6:F7:A2:3D:74
    network->CCO_TEI = 6
    network->STATIONS = 2

            station->MAC = 18:A6:F7:A2:3D:74
            station->TEI = 6
            station->BDA = 24:A1:60:37:86:58
            station->AvgPHYDR_TX = 056 mbps Primary
            station->AvgPHYDR_RX = 020 mbps Primary

            station->MAC = 84:16:F9:F3:BF:28
            station->TEI = 13
            station->BDA = 7C:64:56:F7:BC:A9
            station->AvgPHYDR_TX = 216 mbps Alternate
            station->AvgPHYDR_RX = 296 mbps Alternate

here is also TX and RX visible

So now I can adapt your bash script for use with “plctool” instead of “plcstat_output” since the last one is not contained in the download package above.

thx again for your script, this will help me a lot for the bash script to process all regexped info and publish it with mqtt

Olivier1974 · July 26, 2023, 6:15pm

It is running on the RPi that appears to be where I have HA, could be otherwise.
But this must (I think) run on a machine connected to the PLC (not sure about that)