How to track Italian electricity prices

[MicheleMercuri]

“I have a photovoltaic system, and I need to assess whether it’s worthwhile to consume the energy myself or feed it into the grid, especially if there will be very low prices the following day. Since the GME (Italian Wholesale Electricity Market) publishes the “Day-Ahead Market Prices (MGP)” around 1:00 PM, I would need to create sensors that fetch data for the next day like this:”

resource_template: 'https://www.mercatoelettrico.org/It/WebServerDataStore/MGP_Prezzi/{{ (as_timestamp(now()) + (24*3600)) | timestamp_custom("%Y%m%d", True) }}MGPPrezzi.xml'

# PUN RID TOMORROW
  - resource_template: 'https://www.mercatoelettrico.org/It/WebServerDataStore/MGP_Prezzi/{{ (as_timestamp(now()) + (24*3600)) | timestamp_custom("%Y%m%d", True) }}MGPPrezzi.xml'
    scan_interval: 3600
    parser: 'lxml'
    name: 'PUN Domani'
    button:
      unique_id: 'aggiorna_pun_domani'
      name: 'Aggiorna PUN domani'
    log_response: true
    form_submit:
      submit_once: false
      resource: 'https://www.mercatoelettrico.org/It/Tools/Accessodati.aspx'
      select: '#form1'
      input:
        'ctl00$ContentPlaceHolder1$CBAccetto1': 'on'
        'ctl00$ContentPlaceHolder1$CBAccetto2': 'on'
        'ctl00$ContentPlaceHolder1$Button1': 'Accetto'
      input_filter:
        - 'ctl00$ContentPlaceHolder1$Button2'
        - 'ctl00$vai'
        - 'ctl00$LinkButton2'
        - 'ctl00$LoginButton'
    sensor:
      - select: 'NewDataSet:nth-child(1) > Prezzi:nth-child(2) > PUN:nth-child(4)'
        name: 'PUN Domani Ora 00'
        unique_id: 'tomorr_energy_pun_0'
        icon: 'mdi:currency-eur'
        unit_of_measurement: '€/kWh'
        value_template: '{{ value | replace (",", ".") |float | int / 1000}}'
      - select: 'NewDataSet:nth-child(1) > Prezzi:nth-child(3) > PUN:nth-child(4)'
        name: 'PUN Domani Ora 01'
        unique_id: 'tomorr_energy_pun_1'
        icon: 'mdi:currency-eur'
        unit_of_measurement: '€/kWh'
        value_template: '{{ value | replace (",", ".") |float | int / 1000}}'

# create sensor for tomorrow RID hour Calabria
  - sensor:
      - name: "MGP PO Calabria Tomorrow"
        unit_of_measurement: "€/kWh"
        unique_id: energy_pun_tomorrow
        icon: mdi:currency-eur
        state: >
          {{ states('sensor.tomorr_energy_pun_' + (now().hour | string)) }}

ps. Today’s sensor code is the same as yours

[110hs]

If you need to optimize the charge/discharge cycle of your battery, depending on costs, load, PV prod and so on I would suggest you to give a try to this add-on, I’m already playing with:

EMHASS add-on: An energy management optimization add-on for Home Assistant OS and supervised

Hello everyone! This is the first release of this add-on for Home Assistant!!! The add-on repository and installation instructions are here: https://github.com/davidusb-geek/emhass-add-on This add-on implements the EMHASS module for systems using Home Assistant OS or Home Assistant Supervised. EMHASS is a Python module for Energy Management Optimization in Home Assistant. You can check the project source code here: https://github.com/davidusb-geek/emhass Feel free to test this add-on as yo…

For more details about how to use it you can have a look at this thread (not add-on specific but everything you see there is valid for it as well - if you use the add-on, of course):
https://community.home-assistant.io/t/emhass-an-energy-management-for-home-assistant/338126

@MicheleMercuri

Since the GME (Italian Wholesale Electricity Market) publishes the “Day-Ahead Market Prices (MGP)” around 1:00 PM, I would need to create sensors that fetch data for the next day like this

Interesting, for the moment I’ve applied an average cost difference between buying and “ritiro dedicato (RID)” sales prices; I’ll have a look to see if it’s more precise and allows me to make this little step automatic.

[MicheleMercuri]

GSE RID price is calculated in this way: the Hourly Zonal Price (PO), which is the price in the electricity market and depending on the time at which the energy is fed grid and the market zone in which the facility is located, so RID is multiplied by kWh fed to grid.
This in my HA:
1)

  # energy meter fed to grid hourly
  hourly_energy_grid_exported:
    source: sensor.grid_exported
    cycle: hourly

# RID Price zone Calabria
multiscrape:
  - resource_template: 'https://www.mercatoelettrico.org/It/WebServerDataStore/MGP_Prezzi/{{ as_timestamp(now()) | timestamp_custom("%Y%m%d", True) }}MGPPrezzi.xml'
    scan_interval: 3600
    parser: 'lxml'
    name: 'PUN oggi'
    button:

Nodered at xx:59:58 of every hour Every hour will multiply RID by kWh.

[{"id":"d8ccf8aecbf986ec","type":"tab","label":"Flow RID CALA","disabled":false,"info":"","env":[]},{"id":"0e6fc28155b6e376","type":"api-current-state","z":"d8ccf8aecbf986ec","name":"gmepo","server":"1c37c350.3f926d","version":3,"outputs":1,"halt_if":"","halt_if_type":"str","halt_if_compare":"is","entity_id":"sensor.mgp_pun_attuale","state_type":"num","blockInputOverrides":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"entityState"},{"property":"topic","propertyType":"msg","value":"input_1","valueType":"str"}],"for":"0","forType":"num","forUnits":"minutes","override_topic":false,"state_location":"payload","override_payload":"msg","entity_location":"data","override_data":"msg","x":310,"y":40,"wires":[["df859bcd22d81e32"]]},{"id":"a12479a3c1d97fd0","type":"api-current-state","z":"d8ccf8aecbf986ec","name":"energy","server":"1c37c350.3f926d","version":3,"outputs":1,"halt_if":"","halt_if_type":"str","halt_if_compare":"is","entity_id":"sensor.hourly_energy_grid_exported","state_type":"num","blockInputOverrides":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"entityState"},{"property":"topic","propertyType":"msg","value":"input_2","valueType":"str"}],"for":"0","forType":"num","forUnits":"minutes","override_topic":false,"state_location":"payload","override_payload":"msg","entity_location":"data","override_data":"msg","x":310,"y":140,"wires":[["df859bcd22d81e32"]]},{"id":"ed370677f6b728e9","type":"function","z":"d8ccf8aecbf986ec","name":"function 1","func":"msg.payload = (msg.payload.input_1 * msg.payload.input_2).toFixed(4);\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":540,"y":180,"wires":[["f76cebf3f1c4efb9","c153013cd9ecd1e4"]]},{"id":"f76cebf3f1c4efb9","type":"debug","z":"d8ccf8aecbf986ec","name":"debug 2","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":640,"y":40,"wires":[]},{"id":"df859bcd22d81e32","type":"join","z":"d8ccf8aecbf986ec","name":"","mode":"custom","build":"object","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":true,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":470,"y":100,"wires":[["ed370677f6b728e9"]]},{"id":"c153013cd9ecd1e4","type":"ha-sensor","z":"d8ccf8aecbf986ec","name":"RIDCALA","entityConfig":"288828a3536cdefe","version":0,"state":"payload","stateType":"msg","attributes":[{"property":"last_update","value":"","valueType":"date"}],"inputOverride":"allow","outputProperties":[],"x":740,"y":120,"wires":[["43b86b7a647ad72f"]]},{"id":"43b86b7a647ad72f","type":"debug","z":"d8ccf8aecbf986ec","name":"debug 3","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":900,"y":120,"wires":[]},{"id":"e76bf60247ce0fb0","type":"cronplus","z":"d8ccf8aecbf986ec","name":"Every 59min","outputField":"payload","timeZone":"","persistDynamic":false,"storeName":"","commandResponseMsgOutput":"output1","outputs":1,"options":[{"name":"schedule1","topic":"topic1","payloadType":"default","payload":"","expressionType":"cron","expression":"58 59 * * * *","location":"","offset":"0","solarType":"all","solarEvents":"sunrise,sunset"}],"x":110,"y":60,"wires":[["a12479a3c1d97fd0","0e6fc28155b6e376"]]},{"id":"0be4d7a852d0df17","type":"inject","z":"d8ccf8aecbf986ec","name":"Test","props":[{"p":"debug","v":"true","vt":"bool"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":110,"y":200,"wires":[["0e6fc28155b6e376","a12479a3c1d97fd0"]]},{"id":"1c37c350.3f926d","type":"server","name":"Home Assistant","version":5,"addon":true,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true,"heartbeat":false,"heartbeatInterval":"30","areaSelector":"id","deviceSelector":"id","entitySelector":"id","statusSeparator":"at: ","statusYear":"hidden","statusMonth":"short","statusDay":"numeric","statusHourCycle":"h23","statusTimeFormat":"h:m","enableGlobalContextStore":true},{"id":"288828a3536cdefe","type":"ha-entity-config","server":"1c37c350.3f926d","deviceConfig":"3cf3d12d268fc427","name":"calcolo rid orario","version":"6","entityType":"sensor","haConfig":[{"property":"name","value":"calcolo rid orario"},{"property":"icon","value":""},{"property":"entity_category","value":""},{"property":"entity_picture","value":""},{"property":"device_class","value":"monetary"},{"property":"unit_of_measurement","value":"EUR"},{"property":"state_class","value":"measurement"}],"resend":false,"debugEnabled":false},{"id":"3cf3d12d268fc427","type":"ha-device-config","name":"calcolo rid orario","hwVersion":"","manufacturer":"Node-RED","model":"","swVersion":""}]

@110hs

If you need to optimize the charge/discharge cycle of your battery, depending on costs, load, PV prod and so on I would suggest you to give a try to this add-on, I’m already playing with:

many thanks I will try this component

[110hs]

I may be wrong but I fear it’s not just the Zonal Price, updated hour by hour and day by day, but it’s rather the average over the month, per zone, per band (now I remember why I applied an average delta using current PUN as a basis, instead of pulling from the XML).
Have a look here:

• https://www.gse.it/servizi-per-te/fotovoltaico/ritiro-dedicato/documenti
• https://www.gse.it/documenti_site/Documenti%20GSE/Servizi%20per%20te/RITIRO%20DEDICATO/Altri%20contenuti/Prezzi%20medi%20RID%20aggiornato%2008-2023.pdf

But please let me know if you have evidence/knowledge I’m wrong.

[MicheleMercuri]

In Italy we have 2 methods of compensating for energy fed to the grid:

1. RID
2. SSP

1. RID: PO x kWh fed to grid

I took my GSE RID contract as an example (the data was taken from the GSE website):

I took last invoice of august, year 2023:
kWh fed to grid: 307
Total payment by GSE: € 28,24 (VAT excl.) (the screenshot is for last august day: 31)

so, in GSE website i downloaded billing details:

as you can see for each day kWh fed to the grid is multiplied for PO price zone and every hour of every day matches PO GME price data (august, 31, 2023 PO Calabria in my example pictures)

On may 2023 I bought a Tesla Model 3 so energy fed to grid is very low .

2. For SSP (Scambio Sul Posto) price calculation is quite different and a bit more complicated.

[110hs]

I verified and you are right, but (at least for me) I think you may also need to consider the loss coefficient (coeff_perd = 1,052).
This is the formula I found in my detailed RID payments:

energia * coeff_perd * max(prz_ora;0)

where prz_ora is, as you said, the PUN of your geographic area.
If I sum the results of the formula, day by day, hour by hour, I get the amount I receive.

[andrix (Andre)]

Thank you both for the very useful inputs. I’ve successfully added the MULTISCRAPE integration from HACS and the code in configuration.yalm (see below).

Just one technical question: I’ve copied/pasted your code " - select: ‘NewDataSet:nth-child(1) > Prezzi:nth-child(2) > PUN:nth-child(4)’", I understood how to change the hour (increase the number after Prezzi:nth-child( ) , but how to change the zone? Currently all the values are the same, but maybe it will be different in the future…

Another question, why did you search for the day doc named of the day after? (as_timestamp(now()) + (24*3600) ) ?
Example: to calculate the price of the energy on at 00:30 on 29-01-2024 I should look in the document named 29-01-2024 and in the time range 1:

<Prezzi>
<Data>20240129</Data>
<Mercato>MGP</Mercato>
<Ora>1</Ora>
<PUN>75,110000</PUN>

Am I right?

Thank you again,
Andrea

-sensor:
# create sensor for tomorrow RID hour
  - platform: template
    sensors:
      mgp_po_tomorrow:
        friendly_name: "MGP PO tomorrow"
        unique_id: "MGP PO tomorrow"
        value_template: "{{ (states('sensor.tomorr_energy_pun_' + (now().hour | string))) }}"
        unit_of_measurement: €/kWh


multiscrape:
# PUN RID TOMORROW
  - resource_template: 'https://www.mercatoelettrico.org/It/WebServerDataStore/MGP_Prezzi/{{ (as_timestamp(now()) + (10)) | timestamp_custom("%Y%m%d", True) }}MGPPrezzi.xml'
    scan_interval: 3600
    parser: 'lxml'
    name: 'PUN Domani'
    button:
      unique_id: 'aggiorna_pun_domani'
      name: 'Aggiorna PUN domani'
    log_response: true
    form_submit:
      submit_once: false
      resource: 'https://www.mercatoelettrico.org/It/Tools/Accessodati.aspx'
      select: '#form1'
      input:
        'ctl00$ContentPlaceHolder1$CBAccetto1': 'on'
        'ctl00$ContentPlaceHolder1$CBAccetto2': 'on'
        'ctl00$ContentPlaceHolder1$Button1': 'Accetto'
      input_filter:
        - 'ctl00$ContentPlaceHolder1$Button2'
        - 'ctl00$vai'
        - 'ctl00$LinkButton2'
        - 'ctl00$LoginButton'
    sensor:
      - select: 'NewDataSet:nth-child(1) > Prezzi:nth-child(2) > PUN:nth-child(4)'
        name: 'PUN Domani Ora 00'
        unique_id: 'tomorr_energy_pun_0'
        icon: 'mdi:currency-eur'
        unit_of_measurement: '€/kWh'
        value_template: '{{ value | replace (",", ".") |float | int / 1000}}'
      - select: 'NewDataSet:nth-child(1) > Prezzi:nth-child(3) > PUN:nth-child(4)'
        name: 'PUN Domani Ora 01'
        unique_id: 'tomorr_energy_pun_1'
        icon: 'mdi:currency-eur'
        unit_of_measurement: '€/kWh'
        value_template: '{{ value | replace (",", ".") |float | int / 1000}}'
      - select: 'NewDataSet:nth-child(1) > Prezzi:nth-child(4) > PUN:nth-child(4)'
        name: 'PUN Domani Ora 02'
        unique_id: 'tomorr_energy_pun_2'
        icon: 'mdi:currency-eur'
        unit_of_measurement: '€/kWh'
        value_template: '{{ value | replace (",", ".") |float | int / 1000}}'

[110hs]

@andrix

Just one technical question: I’ve copied/pasted your code " - select: ‘NewDataSet:nth-child(1) > Prezzi:nth-child(2) > PUN:nth-child(4)’", I understood how to change the hour (increase the number after Prezzi:nth-child( ) , but how to change the zone?

That’s the “tricky” part of multiscrape. You have to look into the CSS code to retrieve that; try to follow the instructions of the component here.
Once you find the right column you can select the rows by increasing the index (or inspect all of them - as you prefer).

@andrix

Am I right?

You are right. We just needed tomorrow’s prices as well.

[110hs]

@andrix I could replicate (I told you it’s tricky). Use Firefox:

[MicheleMercuri]

@andrix

as_timestamp

I need today’s and tomorrow’s prices for the RID in the Calabria area (for my PV, to calculate the price of the energy fed into the grid). Tomorrow’s prices are uploaded by the GME at 14:00 every day so I can visualize them on the dashboard 24 hours in advance. It’s a pity that the limitations of Home Assistant can’t create a sensor with future data and therefore I can’t represent them graphically, but to manage they are more than fine

to scrape the hourly PUN instead I use this code (used for the energy purchased and not for the energy sold as the RID):

# MGP PUN Giornaliero
  - resource: "https://www.mercatoelettrico.org/it/default.aspx"
    name: PUN MGP Daily Average
    scan_interval: 3600
    log_response: true
    form_submit:
      submit_once: True
      resource: "https://www.mercatoelettrico.org/It/Tools/Accessodati.aspx?ReturnUrl=%2fIt%2fdefault.aspx"
      select: "#form1"
      input:
        "ctl00$ContentPlaceHolder1$CBAccetto1": "on"
        "ctl00$ContentPlaceHolder1$CBAccetto2": "on"
        "ctl00$ContentPlaceHolder1$Button1": "Accetto"
      input_filter:
        - "ctl00$ContentPlaceHolder1$Button2"
        - "ctl00$vai"
        - "ctl00$LinkButton2"
        - "ctl00$LoginButton"
    sensor:
      - select: "#ContentPlaceHolder1_lblMedia"
        name: "PUN INDEX daily average"
        unique_id: daily_pun_avg
        unit_of_measurement: "€/kWh"
        icon: mdi:transmission-tower-import
        device_class: monetary
        state_class: measurement
        value_template: '{{ (value | replace (",", ".") | float / 1000) | round(6) }}'
        on_error:
          value: last

and for the single-hour PUN on a monthly basis I use this (it is used to calculate the cost of the PUN on your bill if you have a single-hour rate):

multiscrape:
  # MGP PUN Mensile
  - resource: "https://www.mercatoelettrico.org/it/Statistiche/ME/DatiSintesi.aspx"
    name: PUN MPG
    scan_interval: 3600
    log_response: true
    form_submit:
      submit_once: True
      resource: "https://www.mercatoelettrico.org/It/Tools/Accessodati.aspx?ReturnUrl=%2fIt%2fStatistiche%2fME%2fDatiSintesi.aspx"
      select: "#form1"
      input:
        "ctl00$ContentPlaceHolder1$CBAccetto1": "on"
        "ctl00$ContentPlaceHolder1$CBAccetto2": "on"
        "ctl00$ContentPlaceHolder1$Button1": "Accetto"
      input_filter:
        - "ctl00$ContentPlaceHolder1$Button2"
        - "ctl00$vai"
        - "ctl00$LinkButton2"
        - "ctl00$LoginButton"
    sensor:
      - select: "#ContentPlaceHolder1_GridView2_lblMedia2_1"
        name: "January"
        unique_id: jan_pun_avg
        unit_of_measurement: "€/kWh"
        icon: mdi:transmission-tower-import
        device_class: monetary
        state_class: measurement
        value_template: '{{ (value | replace (",", ".") | float / 1000) | round(6) }}'
        on_error:
          value: last
      - select: "#ContentPlaceHolder1_GridView2_lblMedia2_2"
        name: "February"
        unique_id: feb_pun_avg
        unit_of_measurement: "€/kWh"
        icon: mdi:transmission-tower-import
        device_class: monetary
        state_class: measurement
        value_template: '{{ (value | replace (",", ".") | float / 1000) | round(6) }}'
        on_error:
          value: last
      - select: "#ContentPlaceHolder1_GridView2_lblMedia2_3"
        name: "March"
        unique_id: mar_pun_avg
        unit_of_measurement: "€/kWh"
        icon: mdi:transmission-tower-import
        device_class: monetary
        state_class: measurement
        value_template: '{{ (value | replace (",", ".") | float / 1000) | round(6) }}'
        on_error:
          value: last

[110hs]

just a heads-up that a new website was released for GME.
Unsure if the old one will be retired at some point but we may need to re-work our code to pull the data from the new one.