Web scraper Sensor question

Configuration
1

Hi everyone.
I love all the great possibilities in Home Assistent. I came across the webscraper component in HassIO. And it is the perfect sollution for something I was looking for to automate.
I can monitor my heating installation from a webpage. My goal was to create a web scraper sensor for each temperature setting.
But after days of trying I had to gave up. Can someone help me please?

The webpage is: https://www.vbus.net/scheme/6749374848321867e0c29f7feed58176

Here is my entry in the configuration:

  - platform: scrape
    resource: https://www.vbus.net/scheme/6749374848321867e0c29f7feed58176
    name: "Boiler boven"
    select: '#ui-id-3'
#    select: "#ui-id-3.innerText"
#    select: "#ui-id-3.text"
#    select: "('#ui-id-3').text"
#    select: "('#ui-id-3')[0].text"
#    select: ".main-container div:nth-of-type(3)
    unit_of_measurement: "°C"
2

Hi @edwin1972, Welcome to HA.
I can’t help you with the scraper, but maybe this might help?

3

can you confirm what it is exactly that you’re trying to scrape?
I’m guessing it’s one of the temperatures but don’t know which one…

4

I want to capture them all (in several sensors). But if I have one to go with, I can fix the other ones.

5

@VDRainer Looks interesting. I will give it a try. Thank you

6

Silly question, have you tried this:

  - platform: scrape
    resource: https://www.vbus.net/scheme/6749374848321867e0c29f7feed58176
    name: "Boiler boven"
    select: '.ui-id-3'
    value_template: '{{ value.split(" ")[0] }}'
    unit_of_measurement: "°C"
7

Nope. Now I still get this value ‘Unkown °C’

Log Details (ERROR)

Thu Apr 25 2019 16:11:19 GMT+0200 (Midden-Europese zomertijd)

Unable to extract data from HTML

8

ok. Will have a try tomorrow if I get time…

9

I believe you can’t scrape that info.
if you set the scrape component to debug, you’ll see what data it pulls and the data does not include anything that’s displayed because most of the content is loaded after the page is loaded if that males sense:

logger:
  default: error
  logs:
    homeassistant.components.scrape: debug

Happy to be corrected if someone can…

10

I am having an issue trying to scrape some data from a ski resort. Is there a good tutorial on how to do this? I have tried a lot of different codes nothing works.

here is the site i want https://skirose.com/snow-report/
basically just trying to pull the current conditions, and lift status for each lift.

there must be a simple way to find what i need, this logger thing how do i scrape the entire page to see the data to pull?

11

that looks like a big job. If you want to scrape so much you may be better off either having a link to that page, or write your own python code to retrieve the data. So many calls (one per sensor/info) will likely result in your IP address being blocked from accessing the site…

12

I would need some help with configuring scrape from the following HTML Page: http://77.119.243.51:88
it is a temperature sensor where I could like to grap the current value (temperature) I have tried several different configurations but was not successfull. Maybe somone here could help me with this.
Thanks a lot

image
image804×355 18.4 KB

13

have you tried select: ".value"?

14

ok that looks much better, now, thanks a lot for your help.
Omly thing which is not correct now is that the results shows like this:
43.5 A °C so there is a strange “A” in between which I don’t know where it comes from.

15

It’s because the “°C” is included in the value.
You need to add a filter to remove the text and convert to a number:
value_template: '{{value.split(" ")[0] | float}}'

16

Ok thanks I have put that in and now the value shows 0.0
so the value doesn’t show anymore but the “A” is gone

17

then you probably did not copy the value correctly as in your example there is a space between the A and .5…
If the A is directly close to the number, replace
value_template: '{{value.split(" ")[0] | float}}'
with
value_template: '{{value.split("A")[0] | float}}'

If there is indeed what looks like a space but the value template doesn’t work, try to toy with this:
value_template: '{{value[:-4] | float}}'
( you may need to adjust the 4)

18

It works perfectly with the this line:
value_template: ‘{{value[:-4] | float}}’

Thank you so much, I will have to investigat more in this to understand what I have to do if I have to configure another one in the future. Looks a bit complex to me at the morment.

19

for scrape sensors, best is to look for classes (e.g. <div class="classname">).
Hopefully the value you’re after is after a class and it’s the only class with that name
For the actual value, value[:-4] means remove the last 4 chars from value

20

ok thanks for clarifying, this is at least a good start as I understood your explanation.

:+1: