Scrape sensor improved - scraping multiple values

I’m kind of new with all scraping.
But is it possible with this addon to scrape multiple values from a website and show them in the same sensor

And perhaps a stupid question but I enabled debug log but where can I find these logs? :S

I’m not sure how you invented this configuration, but I would say, check the documentation here: https://github.com/danieldotnl/ha-multiscrape

Yes, you can scrape multiple values and add them as attributes of the same sensor. (see documentation link above)

1 Like

Invented?

I managed to get a whole way and almost have it how I wanted. As I said I’m kind of new to scraping but learning on the way. I wasn’t aware it was called ‘attributes’ and I see what it is doing now. Just need to get it fit into my config…thanks a lot!

It was a response to Jonsson9, not to you :wink:

Aah sorry lol haha.
Anyway still thanks!

Yes, Managed to get it working how I wanted it. Again much appreciated was the last piece of the puzzle. Cheers!

Hi!
I am trying to scrape fuel prices from these two sites fuelGR | Πρατήρια και Τιμές Καυσίμων and http://oil.gge.gov.gr/
but I can’t get it to work.

- resource: https://fuelgr.gr/web/?lat=37.849917&lng=23.768234
    scan_interval: 3600
    sensor:
      - unique_id: gas_eko
        name: EKO
        select: "#map > div.leaflet-pane.leaflet-map-pane > div.leaflet-pane.leaflet-marker-pane > div:nth-child(2) > div > div.custom_marker_label.avg.ratebar.green"

this doesn’t work at all

and

- resource: http://oil.gge.gov.gr/
    scan_interval: 3600
    sensor:
      - unique_id: gas_unleaded
        name: Βενζίνη αμόλυβδη
        select: "#post-15266 > div.entry-content > table > tbody > tr:nth-child(6) > td:nth-child(2)"

provides the price of a fixed date.

can anyone help me and tell me where the error is?

  - resource: https://www.printables.com/de/social/259978-johination/about
    scan_interval: 14400 # alle 4 Stunden > Update #
    sensor:
      - unique_id: printables_downloads
        name: "Printables Downloads"
        icon: mdi:download-network-outline
        select: "#app-root > app-baselayout > div > div > ng-component > app-sticky-sidebar > aside > div > app-social-sidebar > div > div > div > div.stats > div:nth-child(1) > div:nth-child(1) > span"
        attributes:
          - name: "likes:"
            select: "#app-root > app-baselayout > div > div > ng-component > app-sticky-sidebar > aside > div > app-social-sidebar > div > div > div > div.stats > div:nth-child(1) > div:nth-child(2) > span"
          - name: "folgende:"
            select: "#app-root > app-baselayout > div > div > ng-component > app-sticky-sidebar > aside > div > app-social-sidebar > div > div > div > div.stats > div:nth-child(2) > a:nth-child(1) > span"
          - name: "folgend:"
            select: "#app-root > app-baselayout > div > div > ng-component > app-sticky-sidebar > aside > div > app-social-sidebar > div > div > div > div.stats > div:nth-child(2) > a:nth-child(2) > span"

get the following errors

Logger: custom_components.multiscrape.sensor
Source: custom_components/multiscrape/sensor.py:163
Integration: Multiscrape scraping component ([documentation](https://github.com/danieldotnl/ha-multiscrape), [issues](https://github.com/danieldotnl/ha-multiscrape/issues))
First occurred: 15:20:25 (1 occurrences)
Last logged: 15:20:25

Scraper_noname_3 # Printables Downloads # Unable to scrape data: Could not find a tag for given selector. Consider using debug logging and log_response for further investigation.

and

Logger: custom_components.multiscrape.entity
Source: custom_components/multiscrape/entity.py:80
Integration: Multiscrape scraping component (documentation, issues)
First occurred: 15:20:25 (3 occurrences)
Last logged: 15:20:25

Scraper_noname_3 # Printables Downloads # likes # Unable to extract data from HTML
Scraper_noname_3 # Printables Downloads # folgende # Unable to extract data from HTML
Scraper_noname_3 # Printables Downloads # folgend # Unable to extract data from HTML

regards
johann

hi, i need your help.
does not work. what am I doing wrong? thank you

multiscrape:
  - name: Offerta Enel E-light
    resource: https://www.enel.it/it/luce-e-gas/luce/offerte/e-light
    scan_interval: 36
    sensor:
      - unique_id: enel_f1
        name: Enel E-light F1
        select: "#splide-flex-card-slide03 li:nth-child(1) > span.Pcwil-tab > span > span"
        #value_template: '{{ value | replace (",", ".") }}'
        unit_of_measurement: "€/KWh"
      - unique_id: enel_f23
        name: Enel E-light F23
        select: "#splide-flex-card-slide03 li:nth-child(2) > span.Pcwil-tab > span > span"
        #value_template: '{{ value | replace (",", ".") }}'
        unit_of_measurement: "€/KWh"

Hi there. Did you ever get this working with Powershop?

Hey!
Yes, it is working.

  - resource: 'https://secure.powershop.co.nz/customers/123456/balance'
    name: Powershop
    log_response: true
    scan_interval: 3600
    form_submit:
      submit_once: true
      resource: 'https://secure.powershop.co.nz'
      select: ".content > form"
      input:
        email: !secret powershop_user
        password: !secret powershop_pass
    sensor:
      - unique_id: powershop_daily_consumption
        name: Powershop daily consumption
        select: ".usage-text > span"
        device_class: energy
        unit_of_measurement: "kWh"
        value_template: '{{ value.split(" ")[0] }}'
        on_error:
          log: warning
          value: last

Legend! Would you mind sharing your code? I’m pulling my hair out, what’s left of it. :thinking:

Just edited my previous comment.
Sorry @Crashbash2020 , somehow I missed your question, in August.

Wow, never would have figured that out. I was way off the mark. Trying now.

Arrr, I see you are pulling the usage. I want to pull the price of my peak and offpeak rates. I’ll use your code base to trying figure that out. Thanks for sharing. Very much appreciated.

You’re welcome.
I was struggling too, but @danieldotnl was kind to help me out.
Scraping the rates is a good idea too, but I’m mostly using/buying “special packs”, so the price fluctuates. I’m more interested in usage.
If you manage to scrape the rates, please share the code.

I’m getting an error when using your code and the sensor value is unknown. Bedtime now so I’ll investigate tomorrow.

2022-09-26 23:30:48.861 DEBUG (MainThread) [custom_components.multiscrape.coordinator] Finished fetching multiscrape data in 4.305 seconds (success: True)
2022-09-26 23:30:48.888 DEBUG (MainThread) [custom_components.multiscrape.sensor] Powershop # Powershop daily consumption # Setting up sensor
2022-09-26 23:30:48.890 DEBUG (MainThread) [custom_components.multiscrape.sensor] Powershop # Powershop daily consumption # Start scraping to update sensor
2022-09-26 23:30:48.894 DEBUG (MainThread) [custom_components.multiscrape.scraper] Powershop # Powershop daily consumption # Tag selected: None
2022-09-26 23:30:48.894 DEBUG (MainThread) [custom_components.multiscrape.form] Powershop # Exception occurred while scraping, will try to resubmit the form next interval.
2022-09-26 23:30:48.894 WARNING (MainThread) [custom_components.multiscrape.sensor] Powershop # Powershop daily consumption # Unable to scrape data: Could not find a tag for given selector.
Consider using debug logging and log_response for further investigation.
2022-09-26 23:30:48.895 DEBUG (MainThread) [custom_components.multiscrape.sensor] Powershop # Powershop daily consumption # On-error, keep old value: None
2022-09-26 23:30:48.895 DEBUG (MainThread) [custom_components.multiscrape.entity] Powershop # Powershop daily consumption # Updated sensor and attributes, now adding to HA

Did you change the 123456 from the url?
I’m not at home for the next few days, so I have limited access to my config.

I sure did but I realised what why issue is. I’m on a peak/off-peak plan so there are four daily rates, not one. That means your select won’t work for me as my table is a little more complex. I’ll post a screenshot here and see if anyone can assist with the right select.