Issue with Multiscrape / Scrape

Hi Robi and Thomas, hats off to you! :tophat: You found a very creative solution. This works because connections to the same endpoint are reused by HA, no matter which component initiated the connection.

@Robi07 Your post made me realize there is much simpler solution to support json, than what I initially had in mind! So as a reward I implemented parsing json through the value_templates.
This basically means that Multiscrape is now not just an improved Scrape component but an improved Rest component as well!!

@Thomas_Gregg : sorry, it took a while…

Now available in v5.7.0! :partying_face:

:partying_face:will give it a try these days.

Hi Daniel. This is genious! Works great! Many thanks for this!!! I even moved some normal rest-sensors to the multiscrape platform given the unique_id functionality of Multiscrape and subsequent area-assignment (now hoping that the new Lovelace Area Card will be improved in Q1 22 i.e. manual configuration possibilities). Best, Robert

1 Like

Hi Thomas. The following works for me:

multiscrape:
  - resource: https://www.remocon-net.remotethermo.com/BsbPlantDashboard/GetPlantData/F0AD4E109C20?zoneNum=1&isFirstRoundTrip=true&rnd=1629792069874&_=1629792068973
    scan_interval: 900
    form_submit:
      submit_once: True
      resource: 'https://www.remocon-net.remotethermo.com/Account/Login'
      select: "#login-form"
      input:
        Email: !secret elco_username
        Password: !secret elco_password
        extra: field
    sensor:
      - name: Elco Aussentemperatur
        unique_id: Elco Aussentemperatur
        value_template: '{{ value_json.outsideTemp }}'
        unit_of_measurement: "°C" 
      - name: Elco Raumtemperatur
        unique_id: Elco Raumtemperatur
        value_template: '{{ value_json.zone.desiredRoomTemp }}'
        unit_of_measurement: "°C" 
      - name: Elco Wassertemperatur Ist
        unique_id: Elco Wassertemperatur Ist
        value_template: '{{ value_json.dhwStorageTemp }}'
        unit_of_measurement: "°C" 
      - name: Elco Wassertemperatur Soll
        unique_id: Elco Wassertemperatur Soll
        value_template: '{{ value_json.dhwComfortTemp.value }}'
        unit_of_measurement: "°C"

I can say that it also works perfectly fine for me. Thanks a lot @Robi07 and @danieldotnl .

@Robi07 It seems they changed the web interface. Could you get it working with the new interface?

I gave it a try here: Help with JSON value template - #7 by Hellis81 but no success yet.

Hi Thomas. I was on holiday and just realized the issue right now. I also had a look on your other post. Looks like the plant datas are not accessible anymore from external sources i.e. the link “https://www.remocon-net.remotethermo.com/R2/PlantHomeBsb/GetData/F…” generates a 404 error if you try to access via explorer. Unless there is another link to access json-values directly from the explorer, I don’t see any chance to get the values to HA. Best, Robert

Hi,
I am trying to scrape a website that requires login.
image

My Configuration:

multiscrape:
- resource: 'https://xxxxx.com/studportal/Student-Home.aspx'
  scan_interval: 3600
  form_submit:
    submit_once: True
    resource: 'https://xxxxx.com/studportal/frmLogin.aspx'
    select: "#aspnetForm"
    input:
      ctl00_ContentPlaceHolder1_txtUserID_Raw: 'user'
      ctl00$ContentPlaceHolder1$txtUserID: 'user'
      ctl00_ContentPlaceHolder1_txtPassword_Raw: 'passwd'
      ctl00$ContentPlaceHolder1$txtPassword: 'passwd'
      ctl00$ContentPlaceHolder1$BnLogin: Login
      #ctl00$ContentPlaceHolder1$BnForgotPassword: Login
  sensor:
    - select: '#studInfoPanelDetailTBL > tr > td'
      name: studentname
  log_response: true

From the log response I see that form is submitted for ‘Forget Password’ instead of ‘Login’.
Can anybody suggest how do I configure to login.

There has been an update in the html code of the login page (login-form replaced by loginForm).
You should replace

...
resource: 'https://www.remocon-net.remotethermo.com/Account/Login'
select: "#login-form"
...

with

...
resource: 'https://www.remocon-net.remotethermo.com/Account/Login'
select: "#loginForm"
...

Hi there,
Based on the suggestions above, i created a code to read the temperature of my Ariston Nuos Plus Wifi boiler in Home Assistant. I guess many suppliers use the same underlying resources to manage digital interactions with their boilers.
Anyway, here is the YAML I use

  - resource: https://www.ariston-net.remotethermo.com/R2/PlantHomeSlp/GetData/34AB9582FD2C?fetchSettings=true&fetchTimeProg=true&rnd=1657775300275
    scan_interval: 300
    log_response: true
    form_submit:
      submit_once: True
      resource: https://www.ariston-net.remotethermo.com/R2/Account/Login
      select: "#loginForm"
      input:
        Email: !secret ariston_username
        Password: !secret ariston_password
        extra: field
    sensor:
      - unique_id: boiler_temperature
        name: Boiler temperature
        value_template: '{{ value_json.data.plantData.waterTemp }}'
        unit_of_measurement: '°C'       

Hi Francois. Many thanks for your response. I tried to adapt your YAML code to my device - but I’m getting HTTP 500 errors through Home Assistant Log.

The login through
https://www.ariston-net.remotethermo.com/R2/Account/Login
works through a web browser but when I try to open the suggested final link (with replaced device number) then I’m getting an error message (see image below). May I therefore quickly ask:

  • If you open your suggested link (

https://www.ariston-net.remotethermo.com/R2/PlantHomeSlp/GetData/34AB9582FD2C?fetchSettings=true&fetchTimeProg=true&rnd=1657775300275

through the web browser (Safari, Chrome etc.), do you see the respective values? I only get the following error message

  • What device number are you using. I tried both (appliance serial and gateway serial) but none was working

Many thanks in advance for your quick response. Best, Robert

Hello, I have the same issue for elco link

I search the probleme but if you have any suggested I will take it

Many thanks in advance for your help

Hi,

I tested this and it woks perfectly. I was wondering, is there a way to switch the Ariston Nuos Plus Wifi from “time program” to “manual” using HA?
Thanks a lot!

Best regards,
Anthonie

Hi,
I haven’t really tried to control the Nuos from HA, just to read data from it. Interested in the answer, if someone has it !

Cheers,

François

Hi,

Well I searched a bit further and used the app ‘web proxy tool’ to sniffer the messages send by my ‘AquaAristonNET’ app.
Changing the Boiler from ‘Manual mode’ to ‘Automatic Mode’ gives me some data

POST.200 OK.
https://www.ariston-net.remotethermo.com/api/v2/velis/slpPlantData/C82B96BD387C/plantMode?appId=com.remotethermo.velis
BODY
{“new”:1,“old”:2} → it means going from Manual:2 to Automatic:1
HEADERS
Expect:
100-continue
ar.authToken:
xxxxxxxxxxxxxxx…
Connection:
Keep-alive

And so on…(ask me when you need it or try it by yourself using the app)

I took over all the data using rest command:

rest_command:
  aristonnuos_man:
    url: https://www.ariston-net.remotethermo.com/api/v2/velis/slpPlantData/xxxxxxxxxxxx/plantMode? 
    appId=com.remotethermo.velis
    method: POST
    headers:
      content-type: 'application/json'
      expect: '100-continue'
      accept-encoding: 'gzip, deflate'
      accept: "application/json, text/json, text/x-json, text/javascript, application/xml, text/xml"
      host: 'www.ariston-net.remotethermo.com'
      content-length: '17'
      ar.authtoken: 'blablabla'
      user-agent: 'RestSharp/106.0.0.0'
    payload: '{"new":2, "old":1}'
    content_type: 'application/json'

And it works…for a couple of days because then the ar.authtoken was changed… I tried with a basic login in my POST, but didn’t work.

I also saw in the link the refer to velis. So I tried the HA integration of aqua remotethermo, but I didn’t work.

Hi to all,
I am experiencing a problem with retrieving data from a web page.
Specifically I want to get the data from :
body > table:nth-child(1) > tbody:nth-child(1) > tr:nth-child(9) > td:nth-child(2) > font:nth-child(1) > strong:nth-child(1) > font:nth-child(1) > small:nth-child(1) retrieved from http://perachora-davis.meteoclub.gr/
I’ve tried everything but it doesn’t bring me any data.
I have tried with other fields and the result is the same.

Can you help me? What can go wrong?

Thanks

With this integration you can read and control Ariston Nuos Split Heatpump Boilers

Did you ever solved your PB ?

Yes and it is working fine