REST Sensor: Import HTML and parse (not JSON, I think)?

Configuration
1

Hi,

I have a cheap Chinese “managed” PoE switch and want to integrate some functionalities into HA.

I have figured out that I can authenticate by sending a password hash(?) as a Cookie header in my GET/PUT command, like so:

curl --header "Cookie: admin=xxxxxxxxxxxxxx" http://192.168.178.5/pse_port.cgi

In the above case, I get this page which is what I am most interested in:

<html>

<head>
<title>PoE</title>
<meta http-equiv=refresh content=10>
<link rel="stylesheet" type="text/css" href="style.css">
</head>

<body>
<center>

<fieldset>
<legend>PSE Port Settings</legend>
<form method="post" action="pse_port.cgi">
  <table border="1">
    <tr>
      <th class=MidSize>Port </th>
      <th>State</th>
    </tr>
    <tr>
      <td align="center">
        <select name="portid" multiple size="6" >
          <option value="0">Port 1
          <option value="1">Port 2
          <option value="2">Port 3
          <option value="3">Port 4
          <option value="4">Port 5
          <option value="5">Port 6
          <option value="6">Port 7
          <option value="7">Port 8
      </td>
      <td align="center">
        <select name="state" class=MidSize>
          <option value="0">Disable
          <option value="1">Enable
        </select>
      </td>
    </tr>
  </table>
  <br style="line-height:50%">
  <input type="submit" name="submit" value="Apply">
  <input type="hidden" name="cmd" value="poe">
</form>
<hr>
<br>
<table border="1">
  <tr>
    <th width="90">Port</th>
    <th width="90">State</th>
    <th width="90">Power On/Off</th>
    <th width="90">Type</th>
    <th width="90">Power(w)</th>
    <th width="90">Voltage(v)</th>
    <th width="90">Current(ma)</th>
  </tr>
  <tr>
    <td>Port 1</td>
    <td>Enable</td>
    <td>Off</td>
<td>-</td><td>-</td><td>-</td><td>-</td>  </tr>
  <tr>
    <td>Port 2</td>
    <td>Enable</td>
    <td>On</td>
<td>Class4</td><td>5.967</td><td>51</td> <td>117</td>  </tr>
  <tr>
    <td>Port 3</td>
    <td>Enable</td>
    <td>On</td>
<td>Class4</td><td>5.916</td><td>51</td> <td>116</td>  </tr>
  <tr>
    <td>Port 4</td>
    <td>Enable</td>
    <td>Off</td>
<td>-</td><td>-</td><td>-</td><td>-</td>  </tr>
  <tr>
    <td>Port 5</td>
    <td>Enable</td>
    <td>Off</td>
<td>-</td><td>-</td><td>-</td><td>-</td>  </tr>
  <tr>
    <td>Port 6</td>
    <td>Enable</td>
    <td>Off</td>
<td>-</td><td>-</td><td>-</td><td>-</td>  </tr>
  <tr>
    <td>Port 7</td>
    <td>Enable</td>
    <td>Off</td>
<td>-</td><td>-</td><td>-</td><td>-</td>  </tr>
  <tr>
    <td>Port 8</td>
    <td>Disable</td>
    <td>Off</td>
<td>-</td><td>-</td><td>-</td><td>-</td>  </tr>
</table>
<br>
</fieldset>
</center>
</body>
</html>

Now I have two issues. First, the result has too many characters for a sensor. I understand that one way around this is to import it into an attribute, but I am unsure how. Currently, in my configuration.yaml I have:

sensor:
  - platform: rest
    name: poestatus
    resource: http://192.168.178.5/pse_port.cgi
#    scan_interval: 60
    headers:
      Cookie: admin=xxxxxxxxxxxxxx

Then, I would love to parse the text. I am interested in the tables:

<tr>
    <td>Port 2</td>
    <td>Enable</td>
    <td>On</td>
<td>Class4</td><td>5.967</td><td>51</td> <td>117</td>  </tr>

It is supposed to show:

Port #
State
Power On/Off
Type
Power(w)
Voltage(v)
Current(ma)

Is is possible to extract these into sensors of their own using, e.g. RegEx?

2

What you basically are doing , is accessing your device webinterface, which you could also do with a markdown/html card

cap
cap631×584 14.9 KB

Above is your html, and it’s underlying features, when calling the .CGI

Have you looked at the “scrape” integration ?
https://www.home-assistant.io/integrations/scrape/ .

3

Hi JC, and thanks for weighing in. The markdown/html card only loads the page like an iframe, right? That would not allow to e.g. add the power values to my energy tracking, I think.

Scrape put me on the right path, I think. Thanks! Right now, I’m trying this:

scrape:
  resource: http://192.168.178.5/pse_port.cgi
  headers:
    Cookie: admin=xxxxxxxxxxxxxxx
  sensor:
    - name: "Switch 1 Port 2 Power"
      select: "body > center > fieldset > table tr:nth-child(3) > td:nth-child(5)"
      unit_of_measurement: W
      state_class: measurement
      device_class: power
4

Right, i figured that was not what you were looking for

I have never tried the Scrape my self, been close a few times, but , some ideas “come and goes” like a Bus/Train departure table :laughing: ( and before i make “the jump”, there is another “route” which is more suitable/enjoyable )

This is Particular true when it comes to HA, one never knows what comes next month :slight_smile:

5

Have in mind it “only” show the actual Value, when reading “scan_interval”
You could add this sensor to a graph thou
How ever you still “only” have a power-value ( Not Energy, kWh, total increasing )

You can search in here for “conversion methods etc” , you can’t use an utility-meter as it only updates 15min (min) , so you have to create another template-sensor which reads the power-sensor ( I.e using “Integral” in /Settings/Helpers )

But still, you can’t really get near to the correct Usage, as you then have to scrape every seconds ( or so ) ( Or i.e every 5 seconds and multiply it with 5 ) … and ignore the rest of the discrepancies

Are you sure there is no “integrations” in HA, for your “Managed PoE Switch” ?
I mean it has an web-interface just type the url “without” pse_port.cgi in your browser, maybe there’s more than pse_port.cgi ?
And i.e Tuya has support for alot various cheap Chinese IP devices

6

I hadn’t thought that far. Was thinking I could just poll power figures every five minutes, calculate the average and use that to calculate a very rough approximation of consumption. In the end, I use PoE for access points, cameras and an SLZB Zigbee/Thread adapter. I guess power consumption does not fluctuate too wildly from the hourly average.

More interestingly, I can now also read the port status and (via REST command) toggle the PoE status of individual ports, which could be used in automations (e.g. cut off power from PoE cameras in some situation to make sure they are off).

The switch is a Mokerlink POE-2G08110GSM. Cheap, simple and very low power consumption. I couldn’t find any reference to an integration, and it does not even support SNMP.

7

ok, so basically type in the ip-number in a browser and you should get the “full” webinterface, could just be the .cgi you are using, but you should get to a login by only typing the ip-number

And yes scrape seem to be the only way for this peice, but it’s great if you can use a i.e input-select/toggle switch to change the “status” i.e turn of/on the port

8

You need a energy-sensor (state_class: total_increasing), to expose energy , power ( state class: measurement) is not that , but “integral Riemann-sum” in # Helpers, will fix that.

So what/how ever you collect and calculate into a Power-sensor that has to be concerted into an energy-sensor

well, if your the devices Consume the same at any giving time, you are right … however POE is not only (potentially) Powering the Device in the other end, with what ever it Needs(at a given time, it also use energy to shuffle your high-res TV/Video signals through the Port “forth and back” TP-ports is a 2 direction transport channel ( And Cables of variating length & quality) , where as your ha, might just use a few Watts at few Amps ( and maybe 3 times that when i.e updating " Downloading" etc.), with it’s 2-3m cable

So my short answer to your “assumption” is … guess again :grin: