Web scrape to boolean

Configuration
1

Hiya

I’m trying to create a boolean (that can then be used in automations) from my local weather page, based on whether there is (any) rain forecast for the current day.

https://www.metservice.com/towns-cities/locations/auckland

The info I want to get is in text of course, and with a value template also need to be converted to a boolean; so any rain at all (>0 = on).
Happy to have a bash at that part later, but initially I can’t even work out how to return the scrape.

This is the bit I want to get:

web grab
web grab839×333 18 KB

It’s buried very deep under many tags…

<p class="u-mT-0 u-mB-0 fineprint u-textGrey u-mT-0 u-mB-0">Total rainfall so far today: 2.0 mm. Total rainfall forecast for today: 2.0 mm</p>

I’d like to show what I’ve tried - but it is so many things it would be pointless!

Many thanks!

2

You can’t use any of the weather integrations in HA?

3

Well - there is the wife factor that ‘trusts’ our local weather provider (I’ll stay out of that validity argument), but the other factor is the inbuilt weather doesn’t seem to have today’s rain.
And I’d rather it was as up to date as possible - rather than rely on yesterday’s version today, if that makes sense.

4

That depends on the integration.
The one I use has today’s date in the forecast and has rain on today.

Doesn’t this weather service you want to use have an API?

5

And you can also use TAF and that is as local as it gets.
This is probably what I would use since it updates very often (not sure how often since I don’t know the airport).

I’m not suggesting to scrape this page but this is what it looks like:
https://en.allmetsat.com/metar-taf/australia-oceania.php?icao=NZAA

And what we are looking for is:

METAR: NZAA 130600Z AUTO 24017KT 9999 SCT022/// 19/15 Q1009 NOSIG

Which is the current weather, and it says scattered clouds at 2200 feet and no significant weather.

And:

TAF: NZAA 130505Z 1306/1406 24020G30KT 9999 -SHRA SCT020 BECMG 1306/1308 24015KT BECMG 1310/1312 19005KT BECMG 1322/1400 23015KT

Which says the forecast between 13th at 6 UTZ to 14th at 6 UTZ predicts light rain showers (-SHRA).

I’m going to need to think for a while because it was a while since I decoded METAR and TAF, but I believe you just need to look for RA in either of these strings.

6

Going back to OPs question, you can scrape using this integation: Scrape - Home Assistant

Have you tried this?

7

I believe that is the question, how to scrape the specific part of the page.

8

To use scrape sensor, you give two parameters;

  • address (obviously)
  • html element to read (class of p)

In your case, use selector as p .u-mT-0 u-mB-0 fineprint u-textGrey u-mT-0 u-mB-0

If you check the selector, it starts with p, which means paragraph element in html and continues with related css tags. I am just not sure if we need dot (.) after p or not, please give it a try.

9 
p.u-mT-0.u-mB-0.fineprint.u-textGrey.u-mT-0.u-mB-0

If we use spaces, then it will be interpreted as hierarchical elements.

1 Like
10

So - forgive me in advance, but in it’s most basic form, this should return at least that whole line of text?

- platform: scrape
  resource: https://www.metservice.com/towns-cities/locations/auckland
  name: Rain Today
  select: "p.u-mT-0.u-mB-0.fineprint.u-textGrey.u-mT-0.u-mB-0"
  value_template: '{{ value }}'

The log on this:

Error fetching data: https://www.metservice.com/towns-cities/locations/auckland failed with [Errno -3] Try again
11

Have you tried again?
This usually happens when you don’t have internet signal at the moment of the request

12

Yeah, many times! :joy:

13

Thank anyway chaps.
Have gone off using this now, as I have realised it totals the precipitation for the day for the figure I need, including if it has rained already.
Been working on the WeatherAPI integration instead, and that looks to be promising.

Much appreciated!

1 Like