Hello to everybody. Is some months now that I try to scrape the values of my domestic heating/cooling system from the related web-portal to integrate the information in home assistant, unluckily the system that has its own plc is not integrable in Home Assistant.
This is the yaml file:
- name: IDROSISTEMI
resource: "https://supervisione.idrosistemi.it/station/TGREG_1200#tab-zone"
scan_interval: 60
log_response: true
headers:
User-Agent: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"
referer: "https://supervisione.idrosistemi.it/"
origin: "https://supervisione.idrosistemi.it"
host: "supervisione.idrosistemi.it"
form_submit:
submit_once: true
resource: "https://supervisione.idrosistemi.it/login"
select: "#app > div > div.row > div > div > div.panel-body > form"
input:
email: "[email protected]"
password: "xxxxxxxxx"
sensor:
- unique_id: t_soggiorno
name: T soggiorno RADIANTE
device_class: temperature
unit_of_measurement: °C
select: "#bt_z1 > a > span.badge.badge-light.ows-bool-remover > span"
Here below you can see the login page:
And this is the page were I want to extract the values from:
Log gives a positive feedback, but the sensor is “not available”
2024-03-25 02:37:42.091 DEBUG (MainThread) [custom_components.multiscrape.form] IDROSISTEMI # The page with the form parsed by BeautifulSoup has been written to file: form_page_soup.txt
2024-03-25 02:37:42.091 DEBUG (MainThread) [custom_components.multiscrape.form] IDROSISTEMI # Try to find form with selector #app > div > div.row > div > div > div.panel-body > form
2024-03-25 02:37:42.093 DEBUG (MainThread) [custom_components.multiscrape.form] IDROSISTEMI # Form looks like this:
<form action="https://supervisione.idrosistemi.it/login" class="form-horizontal" method="POST" role="form">
<input name="_token" type="hidden" value="qEl0SII7V3dL4BgLxpFyW2zzCnS63ZXLuHpr9wid"/>
<div class="form-group">
<label class="col-md-4 control-label" for="email">Indirizzo e-mail</label>
<div class="col-md-6">
<input autofocus="" class="form-control" id="email" name="email" required="" type="text" value=""/>
</div>
</div>
<div class="form-group">
<label class="col-md-4 control-label" for="password">Password</label>
<div class="col-md-6">
<input class="form-control" id="password" name="password" required="" type="password"/>
</div>
</div>
<div class="form-group">
<div class="col-md-6 col-md-offset-4">
<div class="checkbox">
<label>
<input name="remember" type="checkbox"/> Ricordami
</label>
</div>
</div>
</div>
<div class="form-group">
<div class="col-md-8 col-md-offset-4">
<button class="btn btn-primary" type="submit">
Login
</button>
<a class="btn btn-link" href="https://supervisione.idrosistemi.it/password/reset">
Hai dimenticato la tua password?
</a>
</div>
</div>
</form>
2024-03-25 02:37:42.095 DEBUG (MainThread) [custom_components.multiscrape.form] IDROSISTEMI # Finding all input fields in form
2024-03-25 02:37:42.095 DEBUG (MainThread) [custom_components.multiscrape.form] IDROSISTEMI # Found the following input fields: {'_token': 'qEl0SII7V3dL4BgLxpFyW2zzCnS63ZXLuHpr9wid', 'email': '', 'password': None, 'remember': None}
2024-03-25 02:37:42.095 DEBUG (MainThread) [custom_components.multiscrape.form] IDROSISTEMI # Found form action https://supervisione.idrosistemi.it/login and method POST
2024-03-25 02:37:42.095 DEBUG (MainThread) [custom_components.multiscrape.form] IDROSISTEMI # Merged input fields with input data in config. Result: {'_token': 'qEl0SII7V3dL4BgLxpFyW2zzCnS63ZXLuHpr9wid', 'email': '[email protected]', 'password': 'xxxxx', 'remember': None}
2024-03-25 02:37:42.095 DEBUG (MainThread) [custom_components.multiscrape.form] IDROSISTEMI # Determined the url to submit the form to: https://supervisione.idrosistemi.it/login
2024-03-25 02:37:42.095 DEBUG (MainThread) [custom_components.multiscrape.form] IDROSISTEMI # Submitting the form
2024-03-25 02:37:42.095 DEBUG (MainThread) [custom_components.multiscrape.http] IDROSISTEMI # Executing form_submit-request with a POST to url: https://supervisione.idrosistemi.it/login with headers: {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36', 'referer': 'https://supervisione.idrosistemi.it/', 'origin': 'https://supervisione.idrosistemi.it', 'host': 'supervisione.idrosistemi.it'}
2024-03-25 02:37:44.076 DEBUG (MainThread) [custom_components.multiscrape.http] IDROSISTEMI # Response status code received: 200
2024-03-25 02:37:44.078 DEBUG (MainThread) [custom_components.multiscrape.http] IDROSISTEMI # response_headers written to file: form_submit_response_headers.txt
2024-03-25 02:37:44.080 DEBUG (MainThread) [custom_components.multiscrape.http] IDROSISTEMI # response_body written to file: form_submit_response_body.txt
2024-03-25 02:37:44.081 DEBUG (MainThread) [custom_components.multiscrape.form] IDROSISTEMI # Form seems to be submitted succesfully (to be sure, use log_response and check file). Now continuing to retrieve target page.
2024-03-25 02:37:44.081 DEBUG (MainThread) [custom_components.multiscrape.coordinator] IDROSISTEMI # Request data from https://supervisione.idrosistemi.it/station/TGREG_1200#tab-zone
2024-03-25 02:37:44.081 DEBUG (MainThread) [custom_components.multiscrape.http] IDROSISTEMI # Executing page-request with a get to url: https://supervisione.idrosistemi.it/station/TGREG_1200#tab-zone with headers: {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36', 'referer': 'https://supervisione.idrosistemi.it/', 'origin': 'https://supervisione.idrosistemi.it', 'host': 'supervisione.idrosistemi.it'}
2024-03-25 02:37:44.083 DEBUG (MainThread) [custom_components.multiscrape.http] IDROSISTEMI # request_headers written to file: page_request_headers.txt
2024-03-25 02:37:44.085 DEBUG (MainThread) [custom_components.multiscrape.http] IDROSISTEMI # request_body written to file: page_request_body.txt
2024-03-25 02:37:44.734 DEBUG (MainThread) [custom_components.multiscrape.http] IDROSISTEMI # Response status code received: 200
2024-03-25 02:37:44.737 DEBUG (MainThread) [custom_components.multiscrape.http] IDROSISTEMI # response_headers written to file: page_response_headers.txt
2024-03-25 02:37:44.741 DEBUG (MainThread) [custom_components.multiscrape.http] IDROSISTEMI # response_body written to file: page_response_body.txt
2024-03-25 02:37:44.741 DEBUG (MainThread) [custom_components.multiscrape.scraper] IDROSISTEMI # Loading the content in BeautifulSoup.
2024-03-25 02:37:45.376 DEBUG (MainThread) [custom_components.multiscrape.scraper] IDROSISTEMI # page_soup written to file: page_soup.txt
2024-03-25 02:37:45.376 DEBUG (MainThread) [custom_components.multiscrape.coordinator] IDROSISTEMI # Data succesfully refreshed. Sensors will now start scraping to update.
2024-03-25 02:37:45.376 DEBUG (MainThread) [custom_components.multiscrape.coordinator] Finished fetching multiscrape data in 6.853 seconds (success: True)
Can somebody please suggest me where I’m wrong?