It was a response to Jonsson9, not to you
Aah sorry lol haha.
Anyway still thanks!
Yes, Managed to get it working how I wanted it. Again much appreciated was the last piece of the puzzle. Cheers!
Hi!
I am trying to scrape fuel prices from these two sites fuelGR | Πρατήρια και Τιμές Καυσίμων and http://oil.gge.gov.gr/
but I can’t get it to work.
- resource: https://fuelgr.gr/web/?lat=37.849917&lng=23.768234
scan_interval: 3600
sensor:
- unique_id: gas_eko
name: EKO
select: "#map > div.leaflet-pane.leaflet-map-pane > div.leaflet-pane.leaflet-marker-pane > div:nth-child(2) > div > div.custom_marker_label.avg.ratebar.green"
this doesn’t work at all
and
- resource: http://oil.gge.gov.gr/
scan_interval: 3600
sensor:
- unique_id: gas_unleaded
name: Βενζίνη αμόλυβδη
select: "#post-15266 > div.entry-content > table > tbody > tr:nth-child(6) > td:nth-child(2)"
provides the price of a fixed date.
can anyone help me and tell me where the error is?
- resource: https://www.printables.com/de/social/259978-johination/about
scan_interval: 14400 # alle 4 Stunden > Update #
sensor:
- unique_id: printables_downloads
name: "Printables Downloads"
icon: mdi:download-network-outline
select: "#app-root > app-baselayout > div > div > ng-component > app-sticky-sidebar > aside > div > app-social-sidebar > div > div > div > div.stats > div:nth-child(1) > div:nth-child(1) > span"
attributes:
- name: "likes:"
select: "#app-root > app-baselayout > div > div > ng-component > app-sticky-sidebar > aside > div > app-social-sidebar > div > div > div > div.stats > div:nth-child(1) > div:nth-child(2) > span"
- name: "folgende:"
select: "#app-root > app-baselayout > div > div > ng-component > app-sticky-sidebar > aside > div > app-social-sidebar > div > div > div > div.stats > div:nth-child(2) > a:nth-child(1) > span"
- name: "folgend:"
select: "#app-root > app-baselayout > div > div > ng-component > app-sticky-sidebar > aside > div > app-social-sidebar > div > div > div > div.stats > div:nth-child(2) > a:nth-child(2) > span"
get the following errors
Logger: custom_components.multiscrape.sensor
Source: custom_components/multiscrape/sensor.py:163
Integration: Multiscrape scraping component ([documentation](https://github.com/danieldotnl/ha-multiscrape), [issues](https://github.com/danieldotnl/ha-multiscrape/issues))
First occurred: 15:20:25 (1 occurrences)
Last logged: 15:20:25
Scraper_noname_3 # Printables Downloads # Unable to scrape data: Could not find a tag for given selector. Consider using debug logging and log_response for further investigation.
and
Logger: custom_components.multiscrape.entity
Source: custom_components/multiscrape/entity.py:80
Integration: Multiscrape scraping component (documentation, issues)
First occurred: 15:20:25 (3 occurrences)
Last logged: 15:20:25
Scraper_noname_3 # Printables Downloads # likes # Unable to extract data from HTML
Scraper_noname_3 # Printables Downloads # folgende # Unable to extract data from HTML
Scraper_noname_3 # Printables Downloads # folgend # Unable to extract data from HTML
regards
johann
hi, i need your help.
does not work. what am I doing wrong? thank you
multiscrape:
- name: Offerta Enel E-light
resource: https://www.enel.it/it/luce-e-gas/luce/offerte/e-light
scan_interval: 36
sensor:
- unique_id: enel_f1
name: Enel E-light F1
select: "#splide-flex-card-slide03 li:nth-child(1) > span.Pcwil-tab > span > span"
#value_template: '{{ value | replace (",", ".") }}'
unit_of_measurement: "€/KWh"
- unique_id: enel_f23
name: Enel E-light F23
select: "#splide-flex-card-slide03 li:nth-child(2) > span.Pcwil-tab > span > span"
#value_template: '{{ value | replace (",", ".") }}'
unit_of_measurement: "€/KWh"
Hi there. Did you ever get this working with Powershop?
Hey!
Yes, it is working.
- resource: 'https://secure.powershop.co.nz/customers/123456/balance'
name: Powershop
log_response: true
scan_interval: 3600
form_submit:
submit_once: true
resource: 'https://secure.powershop.co.nz'
select: ".content > form"
input:
email: !secret powershop_user
password: !secret powershop_pass
sensor:
- unique_id: powershop_daily_consumption
name: Powershop daily consumption
select: ".usage-text > span"
device_class: energy
unit_of_measurement: "kWh"
value_template: '{{ value.split(" ")[0] }}'
on_error:
log: warning
value: last
Legend! Would you mind sharing your code? I’m pulling my hair out, what’s left of it.
Wow, never would have figured that out. I was way off the mark. Trying now.
Arrr, I see you are pulling the usage. I want to pull the price of my peak and offpeak rates. I’ll use your code base to trying figure that out. Thanks for sharing. Very much appreciated.
You’re welcome.
I was struggling too, but @danieldotnl was kind to help me out.
Scraping the rates is a good idea too, but I’m mostly using/buying “special packs”, so the price fluctuates. I’m more interested in usage.
If you manage to scrape the rates, please share the code.
I’m getting an error when using your code and the sensor value is unknown. Bedtime now so I’ll investigate tomorrow.
2022-09-26 23:30:48.861 DEBUG (MainThread) [custom_components.multiscrape.coordinator] Finished fetching multiscrape data in 4.305 seconds (success: True)
2022-09-26 23:30:48.888 DEBUG (MainThread) [custom_components.multiscrape.sensor] Powershop # Powershop daily consumption # Setting up sensor
2022-09-26 23:30:48.890 DEBUG (MainThread) [custom_components.multiscrape.sensor] Powershop # Powershop daily consumption # Start scraping to update sensor
2022-09-26 23:30:48.894 DEBUG (MainThread) [custom_components.multiscrape.scraper] Powershop # Powershop daily consumption # Tag selected: None
2022-09-26 23:30:48.894 DEBUG (MainThread) [custom_components.multiscrape.form] Powershop # Exception occurred while scraping, will try to resubmit the form next interval.
2022-09-26 23:30:48.894 WARNING (MainThread) [custom_components.multiscrape.sensor] Powershop # Powershop daily consumption # Unable to scrape data: Could not find a tag for given selector.
Consider using debug logging and log_response for further investigation.
2022-09-26 23:30:48.895 DEBUG (MainThread) [custom_components.multiscrape.sensor] Powershop # Powershop daily consumption # On-error, keep old value: None
2022-09-26 23:30:48.895 DEBUG (MainThread) [custom_components.multiscrape.entity] Powershop # Powershop daily consumption # Updated sensor and attributes, now adding to HA
Did you change the 123456 from the url?
I’m not at home for the next few days, so I have limited access to my config.
I sure did but I realised what why issue is. I’m on a peak/off-peak plan so there are four daily rates, not one. That means your select won’t work for me as my table is a little more complex. I’ll post a screenshot here and see if anyone can assist with the right select.
It would be more helpful to post logs and maybe the “JS path” - my two cents
Will do. Here are the JS Paths if that will help anyway define the correct select.
Off-peak Rate (both can be found in two locations)
document.querySelector("#main_container > div > div.row > div.rates-table-container.base-rates-table > table > tbody > tr:nth-child(1) > td.base-rates.current > span.rate.gst_inclusive > div")
document.querySelector("#unit-balance-container > div.tou-consumer-detail.white-box > section.bin-times > section > section > div.bin-detail-bar > span.bin-detail.bin-width-14 > span.expanded > span.rate")
Peak Rate
document.querySelector("#main_container > div > div.row > div.rates-table-container.base-rates-table > table > tbody > tr:nth-child(2) > td.base-rates.current > span.rate.gst_inclusive > div")
document.querySelector("#unit-balance-container > div.tou-consumer-detail.white-box > section.bin-times > section > section > div.bin-detail-bar > span:nth-child(2) > span.expanded > span.rate")
Cracked it! For those wanting to scrape their pricing (standard rates) if the have a peak and off-peak plan with Powershop in NZ, here is the full working code.
[EDIT] 9th November 2022. Adjusted code so that the peak and off-peak dollar values can be used in the HA Energy dashboard.
- resource: 'https://secure.powershop.co.nz/rates'
name: Powershop
log_response: true
scan_interval: 43200 #every 12hrs
form_submit:
submit_once: true
resource: 'https://secure.powershop.co.nz'
select: ".content > form"
input:
email: !secret powershop_user
password: !secret powershop_pass
sensor:
- unique_id: powershop_offpeak
name: Powershop Off Peak
select: "#main_container > div > div.row > div.rates-table-container.base-rates-table > table > tbody > tr:nth-child(1) > td.base-rates.current > span.rate.gst_inclusive"
unit_of_measurement: "NZD/kWh"
value_template: '{{ (value | int) / 100 | float | round(3)}}'
device_class: monetary
- unique_id: powershop_peak
name: Powershop Peak
select: "#main_container > div > div.row > div.rates-table-container.base-rates-table > table > tbody > tr:nth-child(2) > td.base-rates.current > span.rate.gst_inclusive"
unit_of_measurement: "NZD/kWh"
value_template: '{{ (value | int) / 100 | float | round(3)}}'
device_class: monetary
- unique_id: powershop_daily_charge
name: Powershop Daily Charge
select: "#main_container > div > div.row > div.rates-table-container.base-rates-table > table > tbody > tr:nth-child(4) > td.base-rates.current > span.rate.gst_inclusive"
unit_of_measurement: "NZD/kWh"
value_template: '{{ (value | int) / 100 | float | round(3)}}'
device_class: monetary
on_error:
log: warning
value: last