Need help with scrape sensor from website

Configuration
1

Hi, I try to get a value from this website here:

scrape
scrape890×379 77.3 KB

I tried already two different two different select, but without success

  - platform: scrape
    name: Wahltrend CDU
    resource: https://dawum.de/Bundestag/
    select: ".pg_div_chart_horizontal > tbody:nth-child(1) > tr:nth-child(1) > td:nth-child(3)"
    scan_interval: 90

  - platform: scrape
    name: Wahltrend CDU
    resource: https://dawum.de/Bundestag/
    select: "body > main > article > section.bar_chart > div > table > tbody > tr:nth-child(1) > td:nth-child(3)"
    scan_interval: 90

Can anybody help? Thanks in advance!

2

I’m pretty sure that the nth-child selector doesn’t work here, but nth-of-type should. And, apparently tbody doesn’t work either.

    select: "body > main > article > section.bar_chart > div > table tr:nth-of-type(1) > td:nth-of-type(3)"
3

yes it seems so, i have another scrape sensor for the date at the top (17.08.2021) which works fine:

  - platform: scrape
    name: Wahltrend Datum
    resource: https://dawum.de/Bundestag/
    select: "body > main > article > section.bar_chart > header > h2 > time"
    scan_interval: 90

do you know what else I can choose for nth-child?

4

That really depends on the HTML structure. You can see if parameter index can help you to select an item from a list. However, this only works on the lowest level element you selected in the select definition.

5

ok, i couldnt find a solution for the scrape sensor, but made a little python script with pandas and request:

import pandas as pd
import requests

url = "https://dawum.de/Bundestag/"

r = requests.get(url)
df_list = pd.read_html(r.text) # this parses all the tables in webpages to a list
df = df_list[0].loc[0,2].rstrip("%")
print(df);

now I can use this script as a command_line sensor, that works fine so far:

  - platform: command_line
    name: Wahlumfrage CDU
    command: "python3 /config/python_scripts/wahl_cdu.py"
    command_timeout: 60
    unit_of_measurement: '%'
    scan_interval: 300
1 Like
6

This works with the multiscrape component:
body > main > article > section.bar_chart > div > table > tr:nth-child(1) > td:nth-child(3)