How to scrape data from an xml?

Configuration
,
1

So I’ve just got an inverter for my solar production and I can read the data from and XML that the inverter provides. I wanted to add one of these values to HA for monitoring my energy consumption but I don’t really know how to make a sensor with the rest integration. I’m confused with the json_attributes_path, json_attributes and value_template…
The link is very normal to visit the xml: http://10.0.0.13/measurements.xml
Below you can see the XML I see when I visit the webpage.

<root>
<Device Name="StecaGrid 3611..." ... IpAddress="10.0.0.13" DateTime="2023-11-22T15:09:30" MilliSeconds="120">
<Measurements>
<Measurement Value="234.6" Unit="V" Type="AC_Voltage"/>
<Measurement Value="1.462" Unit="A" Type="AC_Current"/>
<Measurement Value="345.2" Unit="W" Type="AC_Power"/>
<Measurement Value="342.9" Unit="W" Type="AC_Power_fast"/>
<Measurement Value="50.000" Unit="Hz" Type="AC_Frequency"/>
<Measurement Value="190.3" Unit="V" Type="DC_Voltage1"/>
<Measurement Value="192.5" Unit="V" Type="DC_Voltage2"/>
<Measurement Value="0.964" Unit="A" Type="DC_Current1"/>
<Measurement Value="0.995" Unit="A" Type="DC_Current2"/>
<Measurement Value="345.7" Unit="V" Type="LINK_Voltage"/>
<Measurement Unit="W" Type="GridPower"/>
<Measurement Unit="W" Type="GridConsumedPower"/>
<Measurement Unit="W" Type="GridInjectedPower"/>
<Measurement Unit="W" Type="OwnConsumedPower"/>
<Measurement Value="100.0" Unit="%" Type="Derating"/>
</Measurements>
</Device>
</root>

I want to make a sensor that reads the AC_Power but I don’t know how to setup the configuration for it. This is what I already have in rest.yaml:

- resource: http://10.0.0.13/measurements.xml
  scan_interval: 10
  sensor:
  headers:
  - content_type: "text/xml"
    name: "AC Power"
    unit_of_measurement: "W"
    json_attributes_path: "x.Device.Measurements[2]"
    json_attributes:
      - Value
    value_template: "{{ value_json.Value }}"

I’m stuck on this so any help is welcome, I’m fairly new to configuring sensors. I also saw multiscrape but I’m not sure if it’s compatible with xml.

2 
- resource: http://10.0.0.13/measurements.xml
  scan_interval: 10
  sensor:
    name: "AC Power"
    unit_of_measurement: "W"
    json_attributes_path: $.root.Device.Measurements.Measurement[2]
    json_attributes:
      - @Value
    value_template: "{{ value_json.root.Device.Measurements.Measurement[2]['@Value'] }}"
3

Thanks a lot!
Had to remove the @ but it finally works, now I can start adding it to my production :slight_smile:

4

just make sure you add state_class and device_class if you plan to use the energy tab

5

So now I have this:

- resource: http://10.0.0.13/measurements.xml
  scan_interval: 2
  sensor:
    name: "AC Power"
    unit_of_measurement: "W"
    icon: 'mdi:solar-power'
    json_attributes_path: $.root.Device.Measurements.Measurement[2]
    json_attributes:
      - Value
    value_template: "{{ value_json.root.Device.Measurements.Measurement[2]['@Value'] }}"
    device_class: energy
    state_class: measurement

it won’t show up in de energy dashboard, don’t know what I’m doing wrong…

6

refresh the page and it should be available.

Just make sure the state is a number, if not, you might need to remove the @ in the value_template

7

How do i make sure it’s a number because when I remove the @ in value_template the sensor just doesn’t appear, even in Developer tools → states

8

just look at the state of the entity, if it’s a number then it’s a number

9

It’s showing a number but still won’t show up in the solar production

image
image1057×474 27.6 KB

10

energy units are not W. Energy is going to be Wh or kWh

11

It finally showed up and added it, unfortunately the sun is gone now but the Wh did it.
Huge thanks!

12

Hi,

First of all my apologies, to hijack this topic.
This is my first post over here and I’m struggling with yaml.
My goal is to get input from a steca.
I have made an stecagrid.yaml (also included it in configurations.yaml) and copied the solution of MichaelNeys.
Developer tools showed me a configuration warning:

  • Integration error: stecagrid - Integration ‘stecagrid’ not found.

All right,
lets copy it into sensors.yaml.
Then developer tools showed me:

  • Invalid config for ‘sensor’ at sensors.yaml, line 1: required key ‘platform’ not provided.

Could anyone be so kind to tell me what I’m doing wrong?
Many thanks.

I’m sure that a solution will help me to create much more sensors😁

Probably good to tell, is the fact I’m running
Home Assistant Core Update 2024.2.2

13

Hi,

Are you able to see the data of the inverter when visiting its IP with /measurements.xml behind it? (eg. http://10.0.0.13/measurements.xml)

In my config file I have rest: !include rest.yaml and this is inside my rest.yaml file:

- resource: http://10.0.0.13/measurements.xml
  scan_interval: 2
  sensor:
    - name: "Solar AC Voltage"
      unique_id: solar_ac_voltage
      unit_of_measurement: "V"
      json_attributes_path: $.root.Device.Measurements.Measurement[0]
      json_attributes:
        - Value
      value_template: "{{ value_json.root.Device.Measurements.Measurement[0]['@Value'] }}"
      device_class: energy
      state_class: measurement
    - name: "Solar AC Current"
      unique_id: solar_ac_current
      unit_of_measurement: "A"
      json_attributes_path: $.root.Device.Measurements.Measurement[1]
      json_attributes:
        - Value
      value_template: "{{ value_json.root.Device.Measurements.Measurement[1]['@Value'] }}"
      device_class: energy
      state_class: measurement
    - name: "Solar AC Power"
      unique_id: solar_ac_power
      unit_of_measurement: "W"
      icon: 'mdi:solar-power'
      json_attributes_path: $.root.Device.Measurements.Measurement[2]
      json_attributes:
        - Value
      value_template: "{{ value_json.root.Device.Measurements.Measurement[2]['@Value'] }}"
      device_class: energy
      state_class: measurement
    - name: "Solar AC Power fast"
      unique_id: solar_ac_power_fast
      unit_of_measurement: "W"
      icon: 'mdi:solar-power'
      json_attributes_path: $.root.Device.Measurements.Measurement[3]
      json_attributes:
        - Value
      value_template: "{{ value_json.root.Device.Measurements.Measurement[3]['@Value'] }}"
      device_class: energy
      state_class: measurement
    - name: "Solar AC Frequency"
      unique_id: solar_ac_frequency
      unit_of_measurement: "Hz"
      json_attributes_path: $.root.Device.Measurements.Measurement[4]
      json_attributes:
        - Value
      value_template: "{{ value_json.root.Device.Measurements.Measurement[4]['@Value'] }}"
      device_class: energy
      state_class: measurement
    - name: "Solar DC Voltage1"
      unique_id: solar_dc_voltage1
      unit_of_measurement: "V"
      json_attributes_path: $.root.Device.Measurements.Measurement[5]
      json_attributes:
        - Value
      value_template: "{{ value_json.root.Device.Measurements.Measurement[5]['@Value'] }}"
      device_class: energy
      state_class: measurement
    - name: "Solar DC Voltage2"
      unique_id: solar_dc_voltage2
      unit_of_measurement: "V"
      json_attributes_path: $.root.Device.Measurements.Measurement[6]
      json_attributes:
        - Value
      value_template: "{{ value_json.root.Device.Measurements.Measurement[6]['@Value'] }}"
      device_class: energy
      state_class: measurement
    - name: "Solar DC Current1"
      unique_id: solar_dc_current1
      unit_of_measurement: "A"
      json_attributes_path: $.root.Device.Measurements.Measurement[7]
      json_attributes:
        - Value
      value_template: "{{ value_json.root.Device.Measurements.Measurement[7]['@Value'] }}"
      device_class: energy
      state_class: measurement
    - name: "Solar DC Current2"
      unique_id: solar_dc_current2
      unit_of_measurement: "A"
      json_attributes_path: $.root.Device.Measurements.Measurement[8]
      json_attributes:
        - Value
      value_template: "{{ value_json.root.Device.Measurements.Measurement[8]['@Value'] }}"
      device_class: energy
      state_class: measurement
    - name: "LINK Voltage"
      unique_id: solar_link_voltage
      unit_of_measurement: "V"
      json_attributes_path: $.root.Device.Measurements.Measurement[9]
      json_attributes:
        - Value
      value_template: "{{ value_json.root.Device.Measurements.Measurement[9]['@Value'] }}"
      device_class: energy
      state_class: measurement
    - name: "Steca Omvormer Derating"
      unique_id: omvormer_derating
      unit_of_measurement: "%"
      icon: 'mdi:solar-panel'
      json_attributes_path: $.root.Device.Measurements.Measurement[14]
      json_attributes:
        - Value
      value_template: "{{ value_json.root.Device.Measurements.Measurement[14]['@Value'] }}"
      device_class: energy
      state_class: measurement

It’s possible the json_attrivutes_path will be different for you if you have a different model. You can see my XML in my first post on this thread.

My scan_interval is very frequently but you can edit this if you want.
I was struggling with this a very long time too because when I begun with Home-Assistant I didn’t work with rest and hadn’t much coding experience. Well next year I’ll be studying Informatics at uni just because I one day decided to start with this simple things lol.

Make some changes to the rest.yaml so it fits for your sensors. Like to hear when it doesn’t work for you!

1 Like
14

Do you have any suggestions on how I could make this value into a Wh for the production in HA because now It’s just W and the values are wrong with what I use… Plus the energy dashboard also needed a total increasing or am I wrong?

15

use the reiman sum integration to turn it into Wh or kWh

16

Saw that somewhere else to but it’s still weird, should I maybe use left or right?

17

use whatever works best for you

1 Like
19

Hi,

Back again. I have now a sensor which will give me the output of my device. Unfortunatly the sensor has to be reloaded every morning, since the output is 0 during the night. To automate such a thing I have created a sensor (ESPHome) with a photoresistor which will trigger the reloading when the lux output reaches a chosen value for over 5 minutes.
So far so good.
To see the output of the day, I create a helper with a Riemann sum, but thats not very reliable. The xml output will also not return the total output of a day.
I have now done some research and have seen that the total output of the day can be seen in the following link
http://steca.ip/gen.yield.day.chart.js (there is also a month, a year and a total :wink: )

Is it possible to scrape the right info to create some sensors or charts or something like that?

My coding skills are minimal, so I don’t have a clue…

20

The link you provided did not work for me. I just use the data from the XML and register it every 2 seconds, it wil not be exactly right but 2 seconds is already pretty pricise and create a Riemannsum with a helper.

It’s a very clever solution with the lux-sensor although I don’t understand why it wouldn’t work after the value was 0. I personally don’t know how HA handles it but I think it maybe could have something to do with device class of the sensor?

21

Hi Michael

Been very busy so excuse me for the late reply. I don’t know either why my sensor isn’t starting after 0 output, but that work around works :slight_smile:
Perhaps we have different inverters i’ve got a StecaGrid 5503
http://10.0.0.xxx/gen.yield.day.chart.js is working for me and creates:

var chartData =
{
"labels":
[
"00:00", "", "", "", "", "",
"01:00", "", "", "", "", "",
"02:00", "", "", "", "", "",
"03:00", "", "", "", "", "",
"04:00", "", "", "", "", "",
"05:00", "", "", "", "", "",
"06:00", "", "", "", "", "",
"07:00", "", "", "", "", "",
"08:00", "", "", "", "", "",
"09:00", "", "", "", "", "",
"10:00", "", "", "", "", "",
"11:00", "", "", "", "", "",
"12:00", "", "", "", "", "",
"13:00", "", "", "", "", "",
"14:00", "", "", "", "", "",
"15:00", "", "", "", "", "",
"16:00", "", "", "", "", "",
"17:00", "", "", "", "", "",
"18:00", "", "", "", "", "",
"19:00", "", "", "", "", "",
"20:00", "", "", "", "", "",
"21:00", "", "", "", "", "",
"22:00", "", "", "", "", "",
"23:00", "", "", "", "", ""
],
"datasets":
[
{
"strokeColor": style_strokecolor.color,
"data": [
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,12,
36,30,42,66,60,66,102,186,162,210,258,282,384,558,534,
408,384,306,600,618,750,1086,1530,1500,1716,1788,1428,2298,2478,1008,
732,456,444,444,438,468,576,900,732,630,582,732,930,726,684,
876,1170,1218,1134,624,426,378,462,606,480,396,330,312,300,354,
288,252,366,300,222,216,174,222,186,132,102,42,18,6,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0]
}
]
}
var max = 5750;
var steps = 23;
var input = document.getElementById("inputId");
input.setAttribute("min",   "2024-03-12");
input.setAttribute("max",   "2024-04-11");
input.setAttribute("value", "2024-04-11");
document.getElementById("labelValueId").innerHTML = "   7.030kWh 11.04.2024";
document.getElementById("buttonPrevId").disabled  = false;
document.getElementById("buttonNextId").disabled  = true;
var myLine = new Chart(document.getElementById("canvasId")
.getContext("2d"))
.Line(chartData,
{
bla bla

so scraping blabla.inner HTML would be nice…