Hi. I’ve been trying for the past few days to multiscrape the Jersey Central Power & Light (a FirstEnergy company) website to get electric usage stats without luck. Using GitHub - danieldotnl/ha-multiscrape: Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality..
The URL with the data is at Log In (specific kWh selector is “#usage-history-table > tbody > tr:nth-child(1) > td:nth-child(4) > span”) which, when accessed, redirects the user to a login page here: Log In. On the login page is a form for username and password.
Here is the configuration I’ve tried without luck:
multiscrape:
- resource: 'https://www.firstenergycorp.com/my_account/view_usage_history.html'
name: FirstEnergy Electric Data
scan_interval: 60
form_submit:
submit_once: True
resource: 'https://www.firstenergycorp.com/content/customer/log_in.html'
select: "#main-content-wrap > section > div.subpage-template.subpage-template2.subpage-template2--1col_centered > div > div:nth-child(2) > div.mainpar.parsys > div.loginwithregister.section > div > div:nth-child(2) > div:nth-child(1) > div > form"
input:
login: [email protected]
password: '12345678'
sensor:
- select: '#usage-history-table > tbody > tr:nth-child(1) > td:nth-child(4) > span'
name: kWh
I receive the following error in the log:
This error originated from a custom integration.
Logger: custom_components.multiscrape.coordinator
Source: custom_components/multiscrape/coordinator.py:75
Integration: Multiscrape scraping component (documentation, issues)
First occurred: 7:36:25 PM (2 occurrences)
Last logged: 7:36:59 PM
FirstEnergy Electric Data # Exception in form-submit feature. Will continue trying to scrape target page. Client error '403 Forbidden' for url 'https://www.firstenergycorp.com/log_in.fecorplogin.html' For more information check: https://httpstatuses.com/403
And,
This error originated from a custom integration.
Logger: custom_components.multiscrape.sensor
Source: custom_components/multiscrape/sensor.py:172
Integration: Multiscrape scraping component (documentation, issues)
First occurred: 7:36:28 PM (2 occurrences)
Last logged: 7:36:59 PM
FirstEnergy Electric Data # kWh Used Last Month # Unable to scrape data: Could not find a tag for given selector Consider using debug logging and log_response for further investigation.
Also, I cannot for the life of me figure out how to enable the debug logging…
I’d appreciate any help.