I’m kind of new with all scraping.
But is it possible with this addon to scrape multiple values from a website and show them in the same sensor
And perhaps a stupid question but I enabled debug log but where can I find these logs? :S
I’m kind of new with all scraping.
But is it possible with this addon to scrape multiple values from a website and show them in the same sensor
And perhaps a stupid question but I enabled debug log but where can I find these logs? :S
I’m not sure how you invented this configuration, but I would say, check the documentation here: https://github.com/danieldotnl/ha-multiscrape
Yes, you can scrape multiple values and add them as attributes of the same sensor. (see documentation link above)
Invented?
I managed to get a whole way and almost have it how I wanted. As I said I’m kind of new to scraping but learning on the way. I wasn’t aware it was called ‘attributes’ and I see what it is doing now. Just need to get it fit into my config…thanks a lot!
It was a response to Jonsson9, not to you
Aah sorry lol haha.
Anyway still thanks!
Yes, Managed to get it working how I wanted it. Again much appreciated was the last piece of the puzzle. Cheers!
Hi!
I am trying to scrape fuel prices from these two sites fuelGR | Πρατήρια και Τιμές Καυσίμων and http://oil.gge.gov.gr/
but I can’t get it to work.
- resource: https://fuelgr.gr/web/?lat=37.849917&lng=23.768234
scan_interval: 3600
sensor:
- unique_id: gas_eko
name: EKO
select: "#map > div.leaflet-pane.leaflet-map-pane > div.leaflet-pane.leaflet-marker-pane > div:nth-child(2) > div > div.custom_marker_label.avg.ratebar.green"
this doesn’t work at all
and
- resource: http://oil.gge.gov.gr/
scan_interval: 3600
sensor:
- unique_id: gas_unleaded
name: Βενζίνη αμόλυβδη
select: "#post-15266 > div.entry-content > table > tbody > tr:nth-child(6) > td:nth-child(2)"
provides the price of a fixed date.
can anyone help me and tell me where the error is?
- resource: https://www.printables.com/de/social/259978-johination/about
scan_interval: 14400 # alle 4 Stunden > Update #
sensor:
- unique_id: printables_downloads
name: "Printables Downloads"
icon: mdi:download-network-outline
select: "#app-root > app-baselayout > div > div > ng-component > app-sticky-sidebar > aside > div > app-social-sidebar > div > div > div > div.stats > div:nth-child(1) > div:nth-child(1) > span"
attributes:
- name: "likes:"
select: "#app-root > app-baselayout > div > div > ng-component > app-sticky-sidebar > aside > div > app-social-sidebar > div > div > div > div.stats > div:nth-child(1) > div:nth-child(2) > span"
- name: "folgende:"
select: "#app-root > app-baselayout > div > div > ng-component > app-sticky-sidebar > aside > div > app-social-sidebar > div > div > div > div.stats > div:nth-child(2) > a:nth-child(1) > span"
- name: "folgend:"
select: "#app-root > app-baselayout > div > div > ng-component > app-sticky-sidebar > aside > div > app-social-sidebar > div > div > div > div.stats > div:nth-child(2) > a:nth-child(2) > span"
get the following errors
Logger: custom_components.multiscrape.sensor
Source: custom_components/multiscrape/sensor.py:163
Integration: Multiscrape scraping component ([documentation](https://github.com/danieldotnl/ha-multiscrape), [issues](https://github.com/danieldotnl/ha-multiscrape/issues))
First occurred: 15:20:25 (1 occurrences)
Last logged: 15:20:25
Scraper_noname_3 # Printables Downloads # Unable to scrape data: Could not find a tag for given selector. Consider using debug logging and log_response for further investigation.
and
Logger: custom_components.multiscrape.entity
Source: custom_components/multiscrape/entity.py:80
Integration: Multiscrape scraping component (documentation, issues)
First occurred: 15:20:25 (3 occurrences)
Last logged: 15:20:25
Scraper_noname_3 # Printables Downloads # likes # Unable to extract data from HTML
Scraper_noname_3 # Printables Downloads # folgende # Unable to extract data from HTML
Scraper_noname_3 # Printables Downloads # folgend # Unable to extract data from HTML
regards
johann
hi, i need your help.
does not work. what am I doing wrong? thank you
multiscrape:
- name: Offerta Enel E-light
resource: https://www.enel.it/it/luce-e-gas/luce/offerte/e-light
scan_interval: 36
sensor:
- unique_id: enel_f1
name: Enel E-light F1
select: "#splide-flex-card-slide03 li:nth-child(1) > span.Pcwil-tab > span > span"
#value_template: '{{ value | replace (",", ".") }}'
unit_of_measurement: "€/KWh"
- unique_id: enel_f23
name: Enel E-light F23
select: "#splide-flex-card-slide03 li:nth-child(2) > span.Pcwil-tab > span > span"
#value_template: '{{ value | replace (",", ".") }}'
unit_of_measurement: "€/KWh"
Hi there. Did you ever get this working with Powershop?
Hey!
Yes, it is working.
- resource: 'https://secure.powershop.co.nz/customers/123456/balance'
name: Powershop
log_response: true
scan_interval: 3600
form_submit:
submit_once: true
resource: 'https://secure.powershop.co.nz'
select: ".content > form"
input:
email: !secret powershop_user
password: !secret powershop_pass
sensor:
- unique_id: powershop_daily_consumption
name: Powershop daily consumption
select: ".usage-text > span"
device_class: energy
unit_of_measurement: "kWh"
value_template: '{{ value.split(" ")[0] }}'
on_error:
log: warning
value: last
Legend! Would you mind sharing your code? I’m pulling my hair out, what’s left of it.
Wow, never would have figured that out. I was way off the mark. Trying now.
Arrr, I see you are pulling the usage. I want to pull the price of my peak and offpeak rates. I’ll use your code base to trying figure that out. Thanks for sharing. Very much appreciated.
You’re welcome.
I was struggling too, but @danieldotnl was kind to help me out.
Scraping the rates is a good idea too, but I’m mostly using/buying “special packs”, so the price fluctuates. I’m more interested in usage.
If you manage to scrape the rates, please share the code.
I’m getting an error when using your code and the sensor value is unknown. Bedtime now so I’ll investigate tomorrow.
2022-09-26 23:30:48.861 DEBUG (MainThread) [custom_components.multiscrape.coordinator] Finished fetching multiscrape data in 4.305 seconds (success: True)
2022-09-26 23:30:48.888 DEBUG (MainThread) [custom_components.multiscrape.sensor] Powershop # Powershop daily consumption # Setting up sensor
2022-09-26 23:30:48.890 DEBUG (MainThread) [custom_components.multiscrape.sensor] Powershop # Powershop daily consumption # Start scraping to update sensor
2022-09-26 23:30:48.894 DEBUG (MainThread) [custom_components.multiscrape.scraper] Powershop # Powershop daily consumption # Tag selected: None
2022-09-26 23:30:48.894 DEBUG (MainThread) [custom_components.multiscrape.form] Powershop # Exception occurred while scraping, will try to resubmit the form next interval.
2022-09-26 23:30:48.894 WARNING (MainThread) [custom_components.multiscrape.sensor] Powershop # Powershop daily consumption # Unable to scrape data: Could not find a tag for given selector.
Consider using debug logging and log_response for further investigation.
2022-09-26 23:30:48.895 DEBUG (MainThread) [custom_components.multiscrape.sensor] Powershop # Powershop daily consumption # On-error, keep old value: None
2022-09-26 23:30:48.895 DEBUG (MainThread) [custom_components.multiscrape.entity] Powershop # Powershop daily consumption # Updated sensor and attributes, now adding to HA
Did you change the 123456 from the url?
I’m not at home for the next few days, so I have limited access to my config.
I sure did but I realised what why issue is. I’m on a peak/off-peak plan so there are four daily rates, not one. That means your select won’t work for me as my table is a little more complex. I’ll post a screenshot here and see if anyone can assist with the right select.