Scrape sensor improved - scraping multiple values

In your browser just right-click on the textboxes and inspect. You’ll see the id ‘app-input-username’ for the username and ‘app-input-password’ for the password field.

Edit: your input fields don’t have a name, see reply below.

I manage to do it with this code:

value_template: >-
      {% if (value | regex_findall_index("(\d)", index=0)) == "0" %}
        on
      {% else %}
        off
      {% endif %}

So it is not name but id of the component right?

No sorry, you are right it should be the name. But the form inputs on that page don’t have a name. Instead some javascript magic seems to happen and it then posts a json body like {"login":"xcvcx","passwordEncrypted":"5kMhR235O3swwWtcW0zqgw=="}

That’s not supported by the form-submit functionality :neutral_face:

Thanks anyway

Hello everybody, would you please help me how to get this value

image

image

from here

https://www.vizugy.hu/?mapModule=OpGrafikon&AllomasVOA=16495FDB-97AB-11D4-BB62-00508BA24287&mapData=Idosor#mapModule

using scrape sensor@hassio? thank you!

Release v5.1.0

I released v5.1.0 which creates a service that allows you to “manually” trigger an update or schedule it at specific times with automations :partying_face:

Hey,
You can ‘right click’ on the number > Copy > Copy Selector
image

That will give you the selector you need to pull the data

#_content > table > tbody > tr > td:nth-child(1) > table > tbody > tr:nth-child(2) > td:nth-child(2) > strong

hi 91JJ, thank you for your help,

I tried this in config.yaml:

but have:

is my select wrong?

You might need to remove the tbody’s from the select, check the scraping guide.

hello Daniel,

it was the problem, now it works like a charm!

thank you!

Hi,
I’m trying to get some results from Markets - Bitvavo
Used the ‘copy selector’ to get the value required.
removed the # before the socket to get unknown instead of unavailable.

multiscrape:
  - name: bitvavo
    resource: https://bitvavo.com/en/markets
    scan_interval: 600 # seconds
    sensor:
      - unique_id: bitvavo_btc_price
        name: bitvavo.btc_price
        icon: mdi:bitcoin
        select: "socket > div:nth-child(2) > div:nth-child(2) > div:nth-child(2) > div > table.interactive.responsive.sortable > tbody > tr:nth-child(8) > td:nth-child(4) > span"
      - unique_id: bitvavo_eth_price
        name: bitvavo.eth_price
        select: "socket > div:nth-child(2) > div:nth-child(2) > div:nth-child(2) > div > table.interactive.responsive.sortable > tbody > tr:nth-child(4) > td:nth-child(4) > span"
      - unique_id: bitvavo_ant_price
        name: bitvavo.ant_price
        select: "socket > div:nth-child(2) > div:nth-child(2) > div:nth-child(2) > div > table.interactive.responsive.sortable > tbody > tr:nth-child(42) > td:nth-child(4) > span"

removing the > tbody doesn’t have any effect, value remains at state ‘unknown’.
any tips?

didn’t get the one above working but aas the order of the rows changes that is fine. Furthermore how would one select this row?

<tr data-market="BTC-EUR" data-action="link" data-href="bitcoin/price"> SHIP </tr>

Not working is

multiscrape:
  - name: bitvavo
    resource: https://bitvavo.com/en/markets
    scan_interval: 600 # seconds
    sensor:
      - unique_id: bitvavo_btc_price
        name: bitvavo.btc_price
        icon: mdi:bitcoin
        select: 'tr{data-market="BTC-EUR"]'

also variation with

select 'tr.data-market="BTC-EUR"'

is a no go.
UPDATE 1423
Part working, it gets me the euro-sign only…

        select: 'tr[data-market="BTC-EUR"] > td:nth-child(4)'

adding > span gives me a blank state again.

Hi,
Im unable to get the data from the website. Can someone know why?

multiscrape:
  - resource: https://myaccount.mytnb.com.my/AccountManagement/ViewAccountProfile?selectedAccountID=e06c71b2-68d8-4ec6-89f5-b6f29d35812c&activate=PaymentHistory
    scan_interval: 30
    headers:
        User-Agent: Mozilla/5.0
    form_submit:
      submit_once: True
      resource: https://www.mytnb.com.my/
      select: "form-control-valid"
      input:
        email: [email protected]
        password: abcd
        extra: field
    sensor:
      - select: "#gridBillingHistory > tr.active > td:nth-child(6) > div"
        name: TNB

I think the select for your form should be #frm-login. You don’t need the ‘extra’ input usually.

Looks like the actual prices are injected by javascript after the initial page retrieval.

Thanks for the help. Is there any way to scrape data from javascript?

Not that I know of and not in multiscrape. Interpreting javascript is a very different matter.

I hope you don’t mind, is there any tutorial on youtube that i can follow on how to scrape data from javascript?.

Can’t help you with that, I’m afraid.