Scrape Sensor | How do I test secure connection?

Configuration
#1

Hi,

I’m trying to see if I can get power rates from my power company but these are only available after login. The class I’ve deciphered is .rate which identifies the four values shown below (and yes, I know our power is very expensive!).

image

I tried using the following but I get an unknown value from the sensor. I don’t know how to test if I’m even logging in let alone whether any data is returned. Anyone got any tips? Ideally all I need to extract is one value <$0.20c and one value >$0.20c as the morning and evening rates are always the same. The damn energy platform has sent me down a deep dark rabbit hole! :blush:

- platform: scrape
  resource: https://secure.powershop.co.nz/customers/<mycustomernumber>/balance
  name: Scrape PowerShop Rates
  authentication: basic
  username: !secret powershop_user
  password: !secret powershop_pw
  # verify_ssl: false
  select: ".rate"
  headers:
    User-Agent: Mozilla/5.0

P.S I’ve tried using digest authentication too without success.

#2

A login via a form is something else than HTTP basic/digest authenticating.
First find out if the values are present in the initial html and not being injected by Javascript (because then they cannot be scraped). Then try multiscrape form-submit.

#3

Thanks @danieldotnl , but pretty much all of that went over my head. Might just have to program some averages. :pleading_face:

#4

Did you manage to get you poweshop rates?
I tried using Scrape sensor improved - scraping multiple values , but didn’t manage to log in.

#5

I gave up. Too hard for me which is a shame. Would be a really handy thing to know.