Help with Website Scrape

Hi brains trust,

I am trying to scrape water level and flow rate from the Murray Darling Basin Authority Website, but I am not getting any results when I copy the selector into HA.

https://riverdata.mdba.gov.au/list-view/corowa

image

Selectors appear to be:

#main > div > main > section.section > div > div > div.tab-header-wrapper > div.btn_map.water_level.tab-header > div > div.tab-header-info > div.tab-measurement-wrapper > span.tab-measurement

and

#main > div > main > section.section > div > div > div.tab-header-wrapper > div.btn_map.discharge.tab-header > div > div.tab-header-info > div.tab-measurement-wrapper > span.tab-measurement

But Home Assistant doesn’t seem to be able to pull the values?
Any ideas?

Thankyou!!

The problem was not your selectors but the headers on the (asuming that you use) multiscrape request:

I’ve tested both of the following sensors adding a user-agent header and they work as required:

multiscrape:
 - resource: 'https://riverdata.mdba.gov.au/list-view/corowa'
    name: Water Corowa
    scan_interval: 600
    log_response: true
    sensor:
      - select: "#main > div > main > section.section > div > div > div.tab-header-wrapper > div.btn_map.water_level.tab-header > div > div.tab-header-info > div.tab-measurement-wrapper > span.tab-measurement"
        name: water_level_corowa
      - select: "#main > div > main > section.section > div > div > div.tab-header-wrapper > div.btn_map.discharge.tab-header > div > div.tab-header-info > div.tab-measurement-wrapper > span.tab-measurement"
        name: water_flow_corowa
    headers: 
      user-agent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36'



Remember to use a decent and responsible scan_interval for your request.

Let me know if it works for you.

@luisgarcia87 Thankyou!! That worked perfectly! I was not aware that the user agent header was required.

You’ll also want a value_template on the flow rate if you need it as a number, as HA won’t deal with the thousands comma. Try:

value_template: "{{ value|select('in','-.0123456789')|join|int(0) }}"

@Troon Thankyou also, that Value template and a unit of measurement took care of it!

1 Like