Scrape, dynamic data

Configuration
1

I want to scrape weather data, the problem is that the css selector moves dependent on the weather status.
I made a humidity sensor, just to figure out if that block reports humidity or not.
I want something like below, but can’t figure out the syntax

 - platform: scrape
   resource: https://www.vackertvader.se/v%C3%A4derstation/h%C3%B6rby
   name: vv-wind-direction
   select: >
     {% if '%' in states('sensor.vv_humidity') %}
     {{'div.weather-station-box:nth-child(7) > div:nth-child(3)'}}
     {% else %}
     {{'div.weather-station-box:nth-child(8) > div:nth-child(3)'}}

I think I need data or value template, but I don’t understand the syntax

2

According to the documentation select does not support templates, however value_template does. So, what you can explore is if instead of selecting the 7th or 8th child, try selecting all children, and then use the value_template to select the 7th or 8th entry.

3

Thanks for putting into this direction.
I am still confused but I’ll make it a try.

4

Well, I just gave it a quick try and realised that my proposal won’t work. The reason is that a selector like select: 'div.weather-station-box' would only select the first instance (or the one instance defined by attribute index).

The best alternative that I could offer you is to have 2 different scrape sensors - one that gets the 7th child and one that gets the 8th child - that store both values temporarily, and then have a template sensor that does that if logic, and selects the value from either one of those scrape sensors.

5

You can do both of your approaches with multiscrape (full disclosure: i’m the author).
It supports templates in select and also select_list which would retrieve all children.

1 Like
6

Thanks, that was one of my thougths, but it will create in total 15 sensors in the end. Still manageable.
But now struggling with the selector between the two alternatives, trying with template sensor.

7

Thanks! Scraping works like a charm. And my first customer component added :grinning:
Next is to test the select_list.
Is it correct that I still need a separate sensor (template) to select between the two indexes?

8

I don’t think so. You probably can solve it with magic in the value_template. But if you now got it working with a select template, I wouldn’t bother anymore…?

9

No, I haven’t got the select template to work. I would preciate some help, would be nice if it can be done in the scrape.
My idea is if item1 contains ‘%’ return item2 else item3.

10

Ah, I see now. That’s not going to work, as with a select (template) you determine what to select. You cannot select something based on the result of it. So you would need to go down the select_list route.

11

OK I got it, no problem.
Anyway, nice component to collect the scrape items from one source!

Thanks for pointing to multiscrape and for all help
/ Lars