Scraping my electric provider's rates, Eversource in New England

jaxzin · October 29, 2021, 6:34am

Seeing that the new Energy dashboard can track the cost of electricity to and from the grid, I realized I wanted to import my energy providers supply and delivery rates as sensors into Home Assistant. Eversource in Connecticut publishes their rates on their website, so I created my first sensors that used the scrape integration to scrape these values from the HTML tables on eversource.com.

Here’s my sensor configuration for other folks in the Northeastern region of the US that also have Eversource as a provider. You’ll need to adjust the URLs for your state.

sensors:
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-supply-rates
    name: Eversource Supply Rate
    select: "#Main_Main_C005_Col00 .table-responsive tr:nth-child(1) td"
    index: 2
    value_template: "{{ value | float / 100.0 }}"
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Transmission Charge
    select: "#Main_Main_C005_Col00 .table-responsive tr:nth-child(1) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Distribution Customer Service Charge
    select: "#Main_Main_C005_Col00 .table-responsive tr:nth-child(2) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/month"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Electric System Improvements Charge
    select: "#Main_Main_C005_Col00 .table-responsive tr:nth-child(3) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Distribution Charge
    select: "#Main_Main_C005_Col00 .table-responsive tr:nth-child(4) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Revenue Adjustment Mechanism
    select: "#Main_Main_C005_Col00 .table-responsive tr:nth-child(5) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Competitive Transition Assessment
    select: "#Main_Main_C005_Col00 .table-responsive tr:nth-child(6) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float * -1.0 }}'
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Federally Mandated Congestion Charge
    select: "#Main_Main_C005_Col00 .table-responsive tr:nth-child(7) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Combined Public Benefits Charge
    select: "#Main_Main_C005_Col00 .table-responsive tr:nth-child(8) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
  - platform: template
    sensors:
      eversource_total_rate:
        friendly_name: "Eversource Total Rate"
        value_template: "{{ states('sensor.eversource_supply_rate') | float + states('sensor.eversource_transmission_charge') | float + states('sensor.eversource_electric_system_improvements_charge') | float + states('sensor.eversource_distribution_charge') | float + states('sensor.eversource_revenue_adjustment_mechanism') | float + states('sensor.eversource_competitive_transition_assessment') | float + states('sensor.eversource_federally_mandated_congestion_charge') | float + states('sensor.eversource_combined_public_benefits_charge') | float }}"
        unit_of_measurement: "USD/kWh"

And I added a card to display all these new entities:

Topslakr · October 29, 2021, 3:25pm

This is awesome! Thanks for doing the leg work. Sadly, very little of this translates the rates they charge in NH, where I am. I’m sure each state has different regulations, so it’s hard for them to have a ‘standard’ page that matches state to state.

I’ll see if I can get it all linked up for NH and I’ll post again for anyone else here.

jaxzin · October 29, 2021, 5:02pm

Oops. I shouldn’t have claimed it was just a URL change without looking first! I could take a crack at the CSS selectors for the NH pages if you need assistance.

Insearchofth1ngs · November 7, 2021, 4:44pm

Just wanted to say thanks so much for these sensors. I got one question. Is it possible to link this up with the energy dashboard so I could see a predicted cost of this month’s energy usage?

jaxzin · November 7, 2021, 5:11pm

Yes, under Configuration > Energy, you can “Use an entity with current price” for both the “Grid consumption” and “Return to grid” settings. Then on the Energy dashboard under “Sources” the “Cost” column should included data going forward. I haven’t found a way to track the cost retroactively.

Insearchofth1ngs · November 7, 2021, 5:16pm

Awesome. Yea I don’t have my grid meter hooked up yet so that’s probably why I couldn’t figure it out. I don’t need retroactive. I figured everything will be from now and into the future for logging.

Thanks so much for the Eversource scraper again.

JamieC · December 4, 2021, 9:00pm

Thanks for sharing this! Any chance anyone knows how to scrape the information for the supply rate for WMA? I’ve got the other sensors working, but since our rates in western MA are just a change twice a year, there isn’t a table to scrape from. I think I’m close, but can’t seem to get that sensor to populate correctly

ivolodin · May 29, 2022, 3:48am

Sorry for resurrecting an old topic, but was just doing this for Eastern Mass, and figured I’ll post it here, if anyone will need it:

  # Electricity Rates
  - platform: scrape
    resource: https://www.eversource.com/content/ema-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-supply-rates
    name: Eversource Supply Rate Winter
    select: "#Main_Main_C007_Col00 #collapseOne .panel-body>ul:first-of-type>li:nth-of-type(2)"
    index: 0
    value_template: "{{ value.split('$')[1] | float }}"
    unit_of_measurement: "USD/kWh"
    scan_interval: 43200
  - platform: scrape
    resource: https://www.eversource.com/content/ema-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-supply-rates
    name: Eversource Supply Rate Summer
    select: "#Main_Main_C007_Col00 #collapseOne .panel-body>ul:first-of-type>li:nth-of-type(1)"
    index: 0
    value_template: "{{ value.split('$')[1] | float }}"
    unit_of_measurement: "USD/kWh"
    scan_interval: 43200
  - platform: scrape
    resource: https://www.eversource.com/content/ema-c/residential/account-billing/manage-bill/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Distribution Energy Charge
    select: "#Main_Main_C005_Col00 .table-responsive:first-of-type tr:nth-child(2) td:nth-child(2)"
    index: 0
    value_template: '{{ value.split(" ")[0].split("$")[1] | float }}'
    scan_interval: 43200
  - platform: scrape
    resource: https://www.eversource.com/content/ema-c/residential/account-billing/manage-bill/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Transition Energy Charge
    select: "#Main_Main_C005_Col00 .table-responsive:first-of-type tr:nth-child(3) td:nth-child(2)"
    index: 0
    value_template: '{{ value.split(" ")[1].split("$")[1] | float * -1 }}'
    unit_of_measurement: "USD/month"
    scan_interval: 43200
  - platform: scrape
    resource: https://www.eversource.com/content/ema-c/residential/account-billing/manage-bill/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Transmission Charge
    select: "#Main_Main_C005_Col00 .table-responsive:first-of-type tr:nth-child(4) td:nth-child(2)"
    index: 0
    value_template: '{{ value.split(" ")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
    scan_interval: 43200
  - platform: scrape
    resource: https://www.eversource.com/content/ema-c/residential/account-billing/manage-bill/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Revenue Decoupling Charge
    select: "#Main_Main_C005_Col00 .table-responsive:first-of-type tr:nth-child(5) td:nth-child(2)"
    index: 0
    value_template: '{{ value.split(" ")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
    scan_interval: 43200
  - platform: scrape
    resource: https://www.eversource.com/content/ema-c/residential/account-billing/manage-bill/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Distributed Solar Charge
    select: "#Main_Main_C005_Col00 .table-responsive:first-of-type tr:nth-child(6) td:nth-child(2)"
    index: 0
    value_template: '{{ value.split(" ")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
    scan_interval: 43200
  - platform: scrape
    resource: https://www.eversource.com/content/ema-c/residential/account-billing/manage-bill/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Renewable Energy Charge
    select: "#Main_Main_C005_Col00 .table-responsive:first-of-type tr:nth-child(7) td:nth-child(2)"
    index: 0
    value_template: '{{ value.split(" ")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
    scan_interval: 43200
  - platform: scrape
    resource: https://www.eversource.com/content/ema-c/residential/account-billing/manage-bill/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Energy Efficiency Charge
    select: "#Main_Main_C005_Col00 .table-responsive:first-of-type tr:nth-child(8) td:nth-child(2)"
    index: 0
    value_template: '{{ value.split(" ")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
    scan_interval: 43200
  - platform: template
    sensors:
      eversource_total_rate:
        friendly_name: "Eversource Total Rate"
        value_template: >
          {% if now().strftime("%m") | int >= 7 %}
          {{ 
            states('sensor.eversource_distributed_solar_charge') | float +
            states('sensor.eversource_distribution_energy_charge') | float +
            states('sensor.eversource_energy_efficiency_charge') | float +
            states('sensor.eversource_renewable_energy_charge') | float +
            states('sensor.eversource_revenue_decoupling_charge') | float +
            states('sensor.eversource_supply_rate_summer') | float +
            states('sensor.eversource_transition_energy_charge') | float +
            states('sensor.eversource_transmission_charge') | float
          }}
          {% else %}
          {{ 
            states('sensor.eversource_distributed_solar_charge') | float +
            states('sensor.eversource_distribution_energy_charge') | float +
            states('sensor.eversource_energy_efficiency_charge') | float +
            states('sensor.eversource_renewable_energy_charge') | float +
            states('sensor.eversource_revenue_decoupling_charge') | float +
            states('sensor.eversource_supply_rate_winter') | float +
            states('sensor.eversource_transition_energy_charge') | float +
            states('sensor.eversource_transmission_charge') | float
          }}
          {% endif %}
        unit_of_measurement: "USD/kWh"

afp316 · September 4, 2022, 5:48am

For anyone living in New Hampshire:

# Electricity Rates
sensor:
  - platform: scrape
    resource: https://www.eversource.com/content/nh/residential/account-billing/manage-bill/about-your-bill/rates-tariffs/electric-supply-rates
    name: Eversource Supply Rate
    select: "#Main_Main_C005_Col00 > div:nth-child(1) > p:nth-child(4) > strong:nth-child(1)"
    index: 0
    value_template: '{{ value.split(" ")[0].split("$")[1] | float }}'
    unit_of_measurement: "¢/kWh"
    scan_interval: 43200
  - platform: scrape
    resource: https://www.eversource.com/content/nh/residential/account-billing/manage-bill/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Distribution Charge
    select: "#Main_Main_C005_Col00 .table-responsive:first-of-type tr:nth-child(2) td:nth-child(2)"
    index: 0
    value_template: '{{ value.split(" ")[0] | float }}'
    unit_of_measurement: "¢/kWh"
    scan_interval: 43200
  - platform: scrape
    resource: https://www.eversource.com/content/nh/residential/account-billing/manage-bill/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Regulatory Reconciliation Adjustment
    select: "#Main_Main_C005_Col00 .table-responsive:first-of-type tr:nth-child(3) td:nth-child(2)"
    index: 0
    value_template: '{{ value.split(" ")[0] | float }}'
    unit_of_measurement: "¢/kWh"
    scan_interval: 43200
  - platform: scrape
    resource: https://www.eversource.com/content/nh/residential/account-billing/manage-bill/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Transmission Charge
    select: "#Main_Main_C005_Col00 .table-responsive:first-of-type tr:nth-child(4) td:nth-child(2)"
    index: 0
    value_template: '{{ value.split(" ")[0] | float }}'
    unit_of_measurement: "¢/kWh"
    scan_interval: 43200
  - platform: scrape
    resource: https://www.eversource.com/content/nh/residential/account-billing/manage-bill/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Stranded Cost Recovery Charge
    select: "#Main_Main_C005_Col00 .table-responsive:first-of-type tr:nth-child(5) td:nth-child(2)"
    index: 0
    value_template: '{{ value.split(" ")[0] | float }}'
    unit_of_measurement: "¢/kWh"
    scan_interval: 43200
  - platform: scrape
    resource: https://www.eversource.com/content/nh/residential/account-billing/manage-bill/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource System Benefits Charge
    select: "#Main_Main_C005_Col00 .table-responsive:first-of-type tr:nth-child(6) td:nth-child(2)"
    index: 0
    value_template: '{{ value.split(" ")[0] | float }}'
    unit_of_measurement: "¢/kWh"
    scan_interval: 43200
  - platform: template
    sensors:
      eversource_total_rate:
        friendly_name: "Eversource Total Rate"
        value_template: "{{ states('sensor.eversource_supply_rate') | float +
          states('sensor.eversource_distribution_charge') | float +
          states('sensor.eversource_regulatory_reconciliation_adjustment') | float +
          states('sensor.eversource_transmission_charge') | float +
          states('sensor.eversource_stranded_cost_recovery_charge') | float +
          states('sensor.eversource_system_benefits_charge') | float }}"
        unit_of_measurement: "¢/kWh"
  - platform: statistics
    name: "Eversource Rate"
    entity_id: sensor.eversource_total_rate
    state_characteristic: mean

scottg489 · October 27, 2022, 10:48pm

Thanks for this! I was about to use this, but then I noticed that it seems like this information is actually outdated. It seems that this page wasn’t updated mid-year when rates changed.

I compared the figures on that page to my bill and my bill was using different figures. Figures which lined up with this document: https://www.eversource.com/content/docs/default-source/rates-tariffs/ema-cambridge-rates.pdf

Unfortunately PDFs aren’t as easily parsable.

Does this page seem outdated compared to the rates on your bill too?

ivolodin · October 28, 2022, 1:07am

You are right, it’s a bit off. But it’s close enough, the only charges that seems to differ are distribution and energy efficiency charges. And scraping PDF would be significantly more complicated and not worth it. Your power readings (at least if you are using clamps) are never going to fully match power meter readings anyways. So I think it’s close enough to get an idea of what to expect.

scottg489 · October 30, 2022, 6:51pm

Thanks for confirming. And you’re probably right. What do you mean by clamps, though?

jdiesel · February 11, 2023, 9:59pm

Has anyone ported this to the new Scape Integration yet?

davrod · February 20, 2023, 9:13pm

Looks like there was an update to Eversource’s page. This is mostly greek to me so it took me awhile to figure out how to update the css references. Posting here in case it helps anyone else (or future me)…

sensor:
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-supply-rates
    name: Eversource Supply Rate
    select: "#MainContentPlaceholder_TEF602CA6001_Col00 .table-responsive tr:nth-child(3) td"
    index: 2
    value_template: "{{ value | float / 100.0 }}"
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Transmission Charge
    select: "#MainContentPlaceholder_TEF602CA6001_Col00 .table-responsive tr:nth-child(5) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Distribution Customer Service Charge
    select: "#MainContentPlaceholder_TEF602CA6001_Col00 .table-responsive tr:nth-child(1) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/month"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Electric System Improvements Charge
    select: "#MainContentPlaceholder_TEF602CA6001_Col00 .table-responsive tr:nth-child(3) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Distribution Charge
    select: "#MainContentPlaceholder_TEF602CA6001_Col00 .table-responsive tr:nth-child(2) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Revenue Adjustment Mechanism
    select: "#MainContentPlaceholder_TEF602CA6001_Col00 .table-responsive tr:nth-child(4) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Competitive Transition Assessment
    select: "#MainContentPlaceholder_TEF602CA6001_Col00 .table-responsive tr:nth-child(7) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float * -1.0 }}'
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Federally Mandated Congestion Charge
    select: "#MainContentPlaceholder_TEF602CA6001_Col00 .table-responsive tr:nth-child(8) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
  - platform: scrape
    resource: https://www.eversource.com/content/ct-c/residential/my-account/billing-payments/about-your-bill/rates-tariffs/electric-delivery-rates
    name: Eversource Combined Public Benefits Charge
    select: "#MainContentPlaceholder_TEF602CA6001_Col00 .table-responsive tr:nth-child(6) td"
    index: 1
    value_template: '{{ value.split("(")[0].split("$")[1] | float }}'
    unit_of_measurement: "USD/kWh"
  - platform: template
    sensors:
      eversource_total_rate:
        friendly_name: "Eversource Total Rate"
        value_template: "{{ states('sensor.eversource_supply_rate') | float + states('sensor.eversource_transmission_charge') | float + states('sensor.eversource_electric_system_improvements_charge') | float + states('sensor.eversource_distribution_charge') | float + states('sensor.eversource_revenue_adjustment_mechanism') | float + states('sensor.eversource_competitive_transition_assessment') | float + states('sensor.eversource_federally_mandated_congestion_charge') | float + states('sensor.eversource_combined_public_benefits_charge') | float }}"
        unit_of_measurement: "USD/kWh"

davrod · February 20, 2023, 9:25pm

Again, I’ll probably forget this immediately, so for future reference…

I used the browser’s developer tools to find the table entry to the appropriate value, right-clicked and selected Copy selector.

That gave me the following output, where I was able to grab the required reference for the select option on the yaml sensor definition.

#MainContentPlaceholder_TEF602CA6001_Col00 > div:nth-child(4) > div > div > div > table > tbody > tr:nth-child(2) > td:nth-child(2)

JvdMaat · October 12, 2023, 8:15pm

Ok, I gotta ask. How did that convert from the Copy Selector to the Select statement for the Scrape integration?
I am using this as my source: Electric Delivery Rates | Eversource (Which for me is WMA).
And I need the 2nd table (Residential Heating).
I can copy your statements, but that gets me all the values from the first table (Residential Non-Heating).
This is my Selector for the first line (Distribution Charge):

#MainContentPlaceholder_TEF602CA6001_Col00 > div:nth-child(6) > div > div:nth-child(8) > div > table > tbody > tr:nth-child(2) > td:nth-child(2)

I tried various combinations, but none give me a result.

dshag86 · March 26, 2024, 10:15pm

I too am trying to figure this one out, did you get it to work?