Scraping data from heat pump website

Configuration
1

Hi, new user of HA here. I’ve started using HA few weeks ago and I’ve been trying to scrape data from my heat pump. Heat pump website login is:

https://www.myupway.com/Welcome

and data is in:

https://www.myupway.com/System/23/Status/ServiceInfo

Can I use normal scraper on or do I need multiscrape to login to that site?

This is what I have at the moment:

sensor:
  - platform: multiscrape
    name: home assistant scraper2
    resource: https://www.myupway.com/System/23/Status/ServiceInfo
    username: -
    password: -
    selectors:
      outsidetemp:
        name: Outside temperature
        select: '.span.AutoUpdateValue.ID40004'

HTML on the service site:

<span class="AutoUpdateValue ID40004">13.7°C</span>
2

Hi

Do you have had some success with it? I have also myupway and would like to collect some data.

3

Hi,

I didn’t get it to work, login was the problem. I couldn’t verify that scraper had logged in.

4

Hi,
Any updates with this?
I`m having a same problem, cant get the scraper working with this.

5

None, I gave up on scraping data from MyUpway.

6

Does anybody have any good idea how to get information from myUpway or directly from the heatpump?

I see that you can add modbus module and then maybe some ESPHome modbus controller to integrate to HA.

Manual of the modbus module: https://www.nibe.no/assets/documents/15716/031725-7.pdf

EDIT: Also some people have had luck to put Nibe firmware to their unit, so you can use Nibe MyUplink API. Unfortunately non Nibe units (Novelan, Metro, Alfa Innotec etc), which are identical hardware uses myUpway which does not have API.

More about it in OpenHab: Nibe uplink binding - #201 by prybil - Bindings - openHAB Community
Thread on Nibe myUplink API: NIBE Uplink API component (non S-series)
Something has been done for no nibe myUPway as well, but it’s only to get data. With Nibe myUplink you can write as well: GitHub - Jalle19/pyupway: Myupway data management scripts

1 Like
7

Hi, I got notification for this thread. I haven’t used HA ever since failing to scrape data from MyUpway, but you could try nibePi, maybe it will work out for you?

1 Like
8

Hi, I managed to get all needed values from myupway by using multiscraper.

This what I have on configuration

- name: myupway Scraper
  resource: https://myupway.com/System/XXXXXXXXXX/Status/ServiceInfo
  log_response: false
  scan_interval: 60
  form_submit:
    submit_once: false
    resource: https://myupway.com/LogIn
    input:
      email: !secret myupway_username
      password: !secret myupway_password
  sensor:
    - unique_id: mlp_outside_temperature
      name: Outside temperature
      device_class: temperature
      unit_of_measurement: °C
      select: ".ID40004"
      value_template: "{{ value.replace('°C', '') }}"

....

Please find IDs needed (e.g. ID40004) from myupway web page with browser inspect -function.

<tr role="row">
    <td>outside temperature <span class="VariableDesignation">BT1</span></td>
     <td><span class="AutoUpdateValue ID40004">-0.8°C</span></td>
</tr>
2 Likes
10

@villeku This works fine for me also. Thanks!!! I have a Metro Therm Heatpump.

image
image1291×486 38 KB

I’m only want to know how to add the sensor degree minutes and how you can fine the right unique_id. Do you know that @villeku

11

For degree minutes I have yaml configuration like this. You may decide unique_id by yourself but modbus id is important and that can be found from myupway -service

    - unique_id: mpl_degree_minutes
      name: MLP degree minutes
      unit_of_measurement: "DM"
      select: ".ID43005"
      value_template: "{{ value.replace('DM', '') }}"

Screenshot 2022-12-06 150259
Screenshot 2022-12-06 1502591383×458 40.6 KB

1 Like
12

A challenge with the multiscrape is - if you want to use device_class - that you must provide a valid device_class with regards to the format of the scraped data.
For the degree minutes you must use device_class: duration. If you use e.g. frequency or timestamp you get an error.

13

Thanks! That works. Any idea how to get defrosting in yaml? It has no device class. Not that I know. The value is Yes or No.

15

I use following:

    - unique_id: wp_id44703
      name: Ontdooien EB101
      select: ".ID44703"
16

Hi im New to HA :sweat_smile:

Can someone explain to me how to get this data into HA.

Im New at this so plain and step by step :sweat_smile:

17

Hey guys. Can i get a hand to figuring this one out also :see_no_evil:

18

Hello I’ve copied your code and made an sensors.yaml file. (adjusted to my home)
i’ve pasted the code into the sensors.yaml file but keep getting an error:
"Invalid config for [sensor]: required key not provided @ data[‘platform’]. Got None. (See /config/configuration.yaml, line 11). "

in the config file line 11:

sensor: !include sensors.yaml

sensors.yaml:

> - name: myupway Scraper
>   resource: https://myupway.com/System/111111/Status/Serviceinfo
>   log_response: false
>   scan_interval: 60
>   form_submit:
>     submit_once: false
>     resource: https://myupway.com/LogIn
>     input:
>       email: !secret myupway_username
>       password: !secret myupway_password
>   Sensor:
>     - unique_id: mlp_buitentemp
>       name: buitentemp
>       device_class: temperature
>       unit_of_measurement: °C
>       select: ".ID40004"
>       value_template: "{{ value.replace('°C', '') }}"

I’don’t know how to solve this, can somebody help me please?

kind regards,
Peter

19

I think you have include multiscrape with different name. For example in configuration.yaml

multiscrape: !include multiscrape.yaml

and multiscrape.yaml contains code above. Hope it helps.

20

Thanks, that helped. Also a capitol error Sensor: replaced bij sensor :smiling_face:

1 Like