Need some help – something is just not clicking for me. I have WELSERVER in the house monitoring the Heat/AC temperatures. It will make a XML file that is updated with the values every minute or so. I am trying to use multiscrape to get the values into HA, but I can’t wrap my head around the select setting.
I get a “Could not find a tag for given selector” error with this configuration:
I just don’t understand the select portion of the configuration. All the examples that I find don’t explain how they know what to put where; I feel that there is something simple that escapes me.
This is from my configuration.yaml:
- name: WEL scraper
resource: http://10.10.0.44:5150/data.xml
scan_interval: 300
sensor:
- unique_id: 1st_supply_temp
name: Supply Temp for 1st floor
select: "#folder0 > div.opened > div:nth-child(8) > span > span:nth-child(2) > span.html-attribute-value"
``````````````````````````````````````````````````````````````````````````````````
This is the page from that url:
````````````````````````````````````````````````````````````````````````````````
<devices>
<device>
<name>date</name>
<value>02/22/2023</value>
</device>
<device>
<name>time</name>
<value>15:53:57</value>
</device>
<device>
<name>Basemnt_Temp</name>
<value>73.174995</value>
</device>
<device>
<name>1st_Supply_Air</name>
<value>74.074989</value>
</device>
<device>
<name>1st_Return_Air</name>
<value>73.849998</value>
</device>
<device>
<name>Attic_Temp</name>
<value>80.150001</value>
</device>
<device>
<name>2nd_Supply_Air</name>
<value>54.049999</value>
</device>
<device>
<name>2nd_Return_Air</name>
<value>70.924995</value>
</device>
<device>
<name>deltaT1</name>
<value>-0.218506</value>
</device>
<device>
<name>deltaT2</name>
<value>16.861397</value>
</device>
</devices>
Troon
(Troon)
February 23, 2023, 2:31pm
2
Welcome .
Scrape is not the right answer if you have XML available. I suspect you were trying to work out a select
statement based off your browser’s HTML rendering of the XML file, which is doomed to failure.
Use a REST sensor instead.
Here’s your XML nicely formatted:
<?xml version="1.0"?>
<devices>
<device>
<name>date</name>
<value>02/22/2023</value>
</device>
<device>
<name>time</name>
<value>15:53:57</value>
</device>
<device>
<name>Basemnt_Temp</name>
<value>73.174995</value>
</device>
<device>
<name>1st_Supply_Air</name>
<value>74.074989</value>
</device>
<device>
<name>1st_Return_Air</name>
<value>73.849998</value>
</device>
<device>
<name>Attic_Temp</name>
<value>80.150001</value>
</device>
<device>
<name>2nd_Supply_Air</name>
<value>54.049999</value>
</device>
<device>
<name>2nd_Return_Air</name>
<value>70.924995</value>
</device>
<device>
<name>deltaT1</name>
<value>-0.218506</value>
</device>
<device>
<name>deltaT2</name>
<value>16.861397</value>
</device>
</devices>
…and here’s how to pull out Attic_Temp
and Basemnt_Temp
— the others should be obvious (but do ask if not):
rest:
- resource: http://10.10.0.44:5150/data.xml
scan_interval: 300
sensor:
- name: Attic temperature
unit_of_measurement: '°F'
device_class: temperature
value_template: >
{{ value_json['devices']['device']
|selectattr('name','eq','Attic_Temp')
|map(attribute='value')
|first }}
- name: Basement temperature
unit_of_measurement: '°F'
device_class: temperature
value_template: >
{{ value_json['devices']['device']
|selectattr('name','eq','Basemnt_Temp')
|map(attribute='value')
|first }}
1 Like
Awesome! That did exactly what I was looking for!!
My last programming for Clarion for DOS. Where do you learn the magic of these commands used inside the configuration file?
Troon
(Troon)
February 24, 2023, 7:18am
4
Most of the work is understanding the data structures (dictionaries, lists, strings etc) and how to manipulate them. There’s a very good template editor within HA, under Developer Tools / Template — and that links to a couple of useful references on the Jinja2 template language.
The docs are very good too, although they can take a bit of navigating and it helps if you know what you’re looking for.
In this case, your resource was providing an XML document. I knew that the REST sensor can read XML, converting it to JSON (ref ). Your XML was being turned into this:
{
"devices": {
"device": [
{
"name": "date",
"value": "02/22/2023"
},
{
"name": "time",
"value": "15:53:57"
},
{
"name": "Basemnt_Temp",
"value": "73.174995"
},
{
"name": "1st_Supply_Air",
"value": "74.074989"
},
{
"name": "1st_Return_Air",
"value": "73.849998"
},
{
"name": "Attic_Temp",
"value": "80.150001"
},
{
"name": "2nd_Supply_Air",
"value": "54.049999"
},
{
"name": "2nd_Return_Air",
"value": "70.924995"
},
{
"name": "deltaT1",
"value": "-0.218506"
},
{
"name": "deltaT2",
"value": "16.861397"
}
]
}
}
You can paste that into the template editor and play with it:
1 Like
Thank you. That’s helpful.
1 Like
kaykor
(Randy)
March 19, 2024, 2:54pm
6
Like to make a clarification for welserver. There is two xml’s available and this example show the file name as data.xml (not working) when in fact it is wel.xml. Working.
And a big thank you for the simple example and clarifications.
data.xml
<Devices>
<Device Name="Date" Value="03/19/2024"/>
<Device Name="Time" Value="09:05:44"/>
<Device Name="server" Value="62.824993"/>
<Device Name="util_room" Value="86.787498"/>
<Device Name="3_supply" Value="93.762489"/>
<Device Name="3_return" Value="87.912498"/>
<Device Name="2_supply" Value="158.674972"/>
<Device Name="2_return" Value="141.800003"/>
<Device Name="1_supply" Value="155.187484"/>
<Device Name="1_return" Value="77.562492"/>
<Device Name="back_return_A4" Value="132.124984"/>
<Device Name="back_supply_A4" Value="97.362495"/>
<Device Name="back_supply_B4" Value="84.987495"/>
<Device Name="back_return_B4" Value="88.587501"/>
<Device Name="dh_boiler_sup" Value="160.587478"/>
<Device Name="dh_code_supply" Value="132.124984"/>
<Device Name="dh_cold_supply" Value="55.399997"/>
<Device Name="dh_recirc" Value="129.537490"/>
<Device Name="dh_boiler_ret" Value="153.162490"/>
<Device Name="dh_tank_out" Value="138.987503"/>
<Device Name="dh_tank_in" Value="57.312496"/>
<Device Name="sys_supply" Value="136.062484"/>
<Device Name="sys_return" Value="102.537498"/>
<Device Name="apt_7_zone_1" Value="0.000000"/>
<Device Name="apt_7_zone_2" Value="1.000000"/>
<Device Name="apt_7_zone_3" Value="0.000000"/>
<Device Name="apt_7_zone_4" Value="0.000000"/>
<Device Name="zone_control_5" Value="0.000000"/>
<Device Name="zone_control_6" Value="0.000000"/>
<Device Name="zone_control_7" Value="0.000000"/>
<Device Name="zone_control_8" Value="0.000000"/>
<Device Name="ac_drain_water" Value="65.412498"/>
<Device Name="basement_supply" Value="87.462501"/>
<Device Name="basement_return" Value="89.374992"/>
<Device Name="ac_air_return" Value="62.037494"/>
<Device Name="ac_air_supply" Value="62.712497"/>
<Device Name="apt_frt_hw" Value="81.275001"/>
<Device Name="apt_frt_ac_cold" Value="50.224994"/>
<Device Name="ht-mani-pos-7" Value="64.287498"/>
<Device Name="ht-mani-pos-6" Value="65.299995"/>
<Device Name="ht-mani-pos-5" Value="65.974998"/>
<Device Name="ht-mani-pos-4" Value="66.424995"/>
<Device Name="ht-mani-pos-3" Value="67.212501"/>
<Device Name="ht-mani-pos-2" Value="68.787498"/>
<Device Name="ht-mani-pos-1" Value="70.025001"/>
</Devices>
wel.xml
<devices>
<device>
<name>date</name>
<value>03/19/2024</value>
</device>
<device>
<name>time</name>
<value>09:47:45</value>
</device>
<device>
<name>server</name>
<value>62.374996</value>
</device>
<device>
<name>util_room</name>
<value>84.424995</value>
</device>
<device>
<name>3_supply</name>
<value>105.462486</value>
</device>
<device>
<name>3_return</name>
<value>89.599998</value>
</device>
<device>
<name>2_supply</name>
<value>126.162483</value>
</device>
<device>
<name>2_return</name>
<value>103.549995</value>
</device>
<device>
<name>1_supply</name>
<value>141.012496</value>
</device>
<device>
<name>1_return</name>
<value>72.499992</value>
</device>
<device>
<name>back_return_A4</name>
<value>113.449989</value>
</device>
<device>
<name>back_supply_A4</name>
<value>110.974998</value>
</device>
<device>
<name>back_supply_B4</name>
<value>82.849998</value>
</device>
<device>
<name>back_return_B4</name>
<value>88.699989</value>
</device>
<device>
<name>dh_boiler_sup</name>
<value>132.349990</value>
</device>
<device>
<name>dh_code_supply</name>
<value>121.999992</value>
</device>
<device>
<name>dh_cold_supply</name>
<value>64.737495</value>
</device>
<device>
<name>dh_recirc</name>
<value>108.949989</value>
</device>
<device>
<name>dh_boiler_ret</name>
<value>131.899978</value>
</device>
<device>
<name>dh_tank_out</name>
<value>134.599990</value>
</device>
<device>
<name>dh_tank_in</name>
<value>80.937492</value>
</device>
<device>
<name>sys_supply</name>
<value>113.337486</value>
</device>
<device>
<name>sys_return</name>
<value>110.299995</value>
</device>
<device>
<name>apt_7_zone_1</name>
<value>0.000000</value>
</device>
<device>
<name>apt_7_zone_2</name>
<value>1.000000</value>
</device>
<device>
<name>apt_7_zone_3</name>
<value>0.000000</value>
</device>
<device>
<name>apt_7_zone_4</name>
<value>0.000000</value>
</device>
<device>
<name>zone_control_5</name>
<value>0.000000</value>
</device>
<device>
<name>zone_control_6</name>
<value>0.000000</value>
</device>
<device>
<name>zone_control_7</name>
<value>0.000000</value>
</device>
<device>
<name>zone_control_8</name>
<value>0.000000</value>
</device>
<device>
<name>ac_drain_water</name>
<value>67.212501</value>
</device>
<device>
<name>basement_supply</name>
<value>109.962486</value>
</device>
<device>
<name>basement_return</name>
<value>109.062492</value>
</device>
<device>
<name>ac_air_return</name>
<value>61.587497</value>
</device>
<device>
<name>ac_air_supply</name>
<value>62.599994</value>
</device>
<device>
<name>apt_frt_hw</name>
<value>75.650001</value>
</device>
<device>
<name>apt_frt_ac_cold</name>
<value>56.637500</value>
</device>
<device>
<name>ht-mani-pos-7</name>
<value>62.149997</value>
</device>
<device>
<name>ht-mani-pos-6</name>
<value>62.487499</value>
</device>
<device>
<name>ht-mani-pos-5</name>
<value>62.712497</value>
</device>
<device>
<name>ht-mani-pos-4</name>
<value>62.937496</value>
</device>
<device>
<name>ht-mani-pos-3</name>
<value>63.274997</value>
</device>
<device>
<name>ht-mani-pos-2</name>
<value>64.062492</value>
</device>
<device>
<name>ht-mani-pos-1</name>
<value>64.962501</value>
</device>
</devices>
Extremely Helpful. Worked for me to get data from WeeWx.
snmrvoge
(René)
August 24, 2024, 9:02am
8
Hi together, im struggling with my sensor, its a Smartfox xml page.
I tried a lot of, also these two
rest:
resource: http://10.xx.xx.xx/values.xml
scan_interval: 300
sensor:
name: ToGridNeu
unit_of_measurement: ‘kW’
device_class: energy
value_template: >
{{ value_json[‘values’]
|selectattr(‘value id’,‘eq’,‘toGridValue’)
|map(attribute=‘value’)
|first }}
sensor:
platform: rest
name: Grid_Neu
resource: http://10.xx.xx.xx/values.xml
value_template: “{{ value_xml.values.value[4] }}”
scan_interval: 300
headers:
Content-Type: application/xml
but no success:-(
The file looks like this:
This XML file does not appear to have any style information associated with it. The document tree is shown below.
<values>
<value id="hidWebinterfaceVersion">9435ac2084cfe0aaaa15</value>
<value id="macAddress">6827198D4842</value>
<value id="ipAddress">010.xx.xx.xx</value>
<value id="version">EM2 00.01.07.03</value>
<value id="toGridValue">-2.59 kW</value>
<value id="lang">de</value>
<value id="dateValue">2024-08-24</value>
<value id="timeValue">09:28:18 Uhr</value>
<value id="hidApiKey">c6258081339fe35770b066e9ae9931bf</value>
and so on....
</values>
Where is the vault?
Troon
(Troon)
August 27, 2024, 6:42am
9
snmrvoge:
Where is the fault?
Please link to the documentation that covers value_xml
. I suspect either you’ve invented it or you’ve made the mistake of asking AI.
From here :
Try this:
sensor:
- platform: rest
name: Grid_Neu
resource: http://10.xx.xx.xx/values.xml
scan_interval: 300
value_template: >
{{ (value_json['values']['value']
|selectattr('@id','==','toGridValue')
|first)
['#text']
|select('in','-0123456789.')
|join }}
unit_of_measurement: 'kW'
device_class: power
Rather than relying on its position in the list, this looks up the toGridValue
element and returns its text, with the units removed to allow it to be treated as a number.
Please format your configs correctly for the forum; and note there’s no need to hide private IP addresses.