Scrape Help - Electricity Rates

I receive electricity based on a variable rate that changes every 5 minutes. The current value is listed on the website here http://www.ercot.com/content/cdr/html/hb_lz The page provides a simple table, and I am trying to pull the value for “LZ_NORTH” for use in Home Assistant but having trouble getting this to work. Is there anyone with some skills in this area who could help?? Thank you!

30%20AM

I get this when I click on your link?


Also what have you tried so far config wise?

Sorry, I accidentally had a period at the end of the URL. I edited the URL above and it should work now. I am just not sure how to identify the number in the LZ_NORTH table for the “select” variable.

I don’t appear able to access the domain. Is it restricted to specific country(ies)?
Can you right click > Source, copy all and paste in hastebin.com and provide the link to it here?

Possibly… Here you go…

https://hastebin.com/bedawexate.xml

I think it would look like something like this:

- platform: scrape
  resource: http://www.ercot.com/content/cdr/html/hb_lz
  name: LZ_NORTH
  select: 'div[class="tdLeft"]:nth-of-type(55)'

Only I’m not 100% sure. Got no way to test at the moment. You may have to try to replace 55 with a different number around that (try different numbers between 50 and 60 if that doesn’t work).

@lolouk44 Thank you for your help with this. After entering your suggested configuration, HA doesnt load the sensor. I have the following error message in my home-assistant.log file that I believe is related to the sensor:

    2018-08-29 21:32:41 ERROR (MainThread) [homeassistant.components.sensor] scrape: Error on device update!
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/homeassistant/helpers/entity_platform.py", line 248, in _async_add_entity
    await entity.async_device_update(warning=False)
  File "/usr/local/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 322, in async_device_update
    yield from self.hass.async_add_job(self.update)
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.6/site-packages/homeassistant/components/sensor/scrape.py", line 120, in update
    value = raw_data.select(self._select)[0].text
IndexError: list index out of range

I also struggle with scrape sensors.if nobody comes up with a solution I should have some time tomorrow to try and find you one

Still unable to find a working config… Is there a good way to easily test the setting? Having to reboot after each try is very time consuming.

FYI–this worked

‘td:nth-of-type(64)’

Now–how do I add a value template to multiply by .1 to convert from MWh to kWh?

Sorry mate haven’t had much time for this recently.
Try to add this below your scrape sensor
value_template: '{{ value | float / 1000 }}'

Finally had this working and the updated scrape sensor is no longer working. Can anyone help me with this. I am looking for the value at LZ_NORTH

  - platform: scrape
    resource: http://www.ercot.com/content/cdr/html/hb_lz
    name: elec_rate_MWh
    select: 'td:nth-of-type(64)'
    unit_of_measurement: '$/MWh'

Here’s the source:

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="x-ua-compatible" content="IE=8" >
<meta name="viewport" content="width=device-width, initial-scale=1">
<script>
  setTimeout("window.location.reload(true);",300000);
</script>
<title>Real-Time LMPs for Load Zones and Trading Hubs Display</title>
<meta http-equiv="pragma" content="no-cache">
<meta http-equiv="cache-control" content="no-cache">
<meta http-equiv="expires" content="0">
<link rel="stylesheet" type="text/css" href="/content/styles/cdr_reports.css" />
<script>
function open_window(url) {
  window.open(url,"help",'toolbar=0,location=0,directories=0,status=0,menubar=0,scrollbars=yes,resizable=0,width=750,height=825');
};
</script>
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');

ga('create', 'UA-460876-10', 'auto');
ga('send', 'pageview');
</script>
<!-- End Google Analytics -->

</head>
<body class="bodyStyle">
<div style="display: inline-block;">
<div>
  <div class='header1'>Real-Time LMPs for Load Zones and Trading Hubs Display</div>
  <div class='helpLink'><a href="javascript:open_window('/help/popup/rtLmpsLoadZonesHubsHelp')">Help?</a></div>
</div>
<br>
<div class="schedTime rightAlign">Last Updated:&nbsp; Feb 10, 2019 21:25:12</div>
<span class="disclaimer">LMP values <b><u>do not include</u></b> the Real-Time price adders.</span>
<table class="tableStyle">
  <tbody>
  <tr class='pAdderRow'>
    <td colspan='5'>
      <table class='pAdderTbl' cellpadding='0' cellspacing='0'>
        <tr>
          <td class='tdLeft pAdderCell pAdderCellLabel'>Price Adders</td>
                    <td class='tdLeft pAdderCell pAdderCellFirst'>RTORPA: $0.00</td>
          <td class='tdLeft pAdderCell'>RTOFFPA: $0.00</td>
          <td class='tdLeft pAdderCell'>RTORDPA: $0.00</td>
        </tr>
      </table>
    </td>
  </tr>
  <tr>
    <td class='headerValueClass'>Settlement Point</td>
    <td class='headerValueClass'>LMP</td>
    <td class='headerValueClass'>5&nbsp;Min<br>Change to&nbsp;LMP</td>
    <td class='headerValueClass'>RTORPA&nbsp;+&nbsp;RTORDPA&nbsp;+&nbsp;LMP</td>
    <td class='headerValueClass'>5&nbsp;Min&nbsp;Change to<br>RTORPA&nbsp;+&nbsp;RTORDPA&nbsp;+&nbsp;LMP</td>
  </tr>
    <tr>
              <td class="tdLeft">HB_BUSAVG</td>
    
              <td class="tdLeft">       23.68</td>
    
                                    <td class="tdLeft red_text">-4.01</td>

              <td class="tdLeft">23.68</td>
    
                                    <td class="tdLeft red_text">-4.01</td>
  </tr>
    <tr>
              <td class="tdLeft">HB_HOUSTON</td>
    
              <td class="tdLeft">       23.16</td>
    
                                    <td class="tdLeft red_text">-1.38</td>

              <td class="tdLeft">23.16</td>
    
                                    <td class="tdLeft red_text">-1.38</td>
  </tr>
    <tr>
              <td class="tdLeft">HB_HUBAVG</td>
    
              <td class="tdLeft">       23.85</td>
    
                                    <td class="tdLeft red_text">-3.92</td>

              <td class="tdLeft">23.85</td>
    
                                    <td class="tdLeft red_text">-3.92</td>
  </tr>
    <tr>
              <td class="tdLeft">HB_NORTH</td>
    
              <td class="tdLeft">       23.84</td>
    
                                    <td class="tdLeft red_text">-3.67</td>

              <td class="tdLeft">23.84</td>
    
                                    <td class="tdLeft red_text">-3.67</td>
  </tr>
    <tr>
              <td class="tdLeft">HB_SOUTH</td>
    
              <td class="tdLeft">       22.48</td>
    
                                    <td class="tdLeft red_text">-6.36</td>

              <td class="tdLeft">22.48</td>
    
                                    <td class="tdLeft red_text">-6.36</td>
  </tr>
    <tr>
              <td class="tdLeft">HB_WEST</td>
    
              <td class="tdLeft">       25.92</td>
    
                                    <td class="tdLeft red_text">-4.27</td>

              <td class="tdLeft">25.92</td>
    
                                    <td class="tdLeft red_text">-4.27</td>
  </tr>
    <tr>
              <td class="tdLeft">LZ_AEN</td>
    
              <td class="tdLeft">       22.75</td>
    
                                    <td class="tdLeft red_text">-5.14</td>

              <td class="tdLeft">22.75</td>
    
                                    <td class="tdLeft red_text">-5.14</td>
  </tr>
    <tr>
              <td class="tdLeft">LZ_CPS</td>
    
              <td class="tdLeft">       22.21</td>
    
                                    <td class="tdLeft red_text">-6.61</td>

              <td class="tdLeft">22.21</td>
    
                                    <td class="tdLeft red_text">-6.61</td>
  </tr>
    <tr>
              <td class="tdLeft">LZ_HOUSTON</td>
    
              <td class="tdLeft">       23.14</td>
    
                                    <td class="tdLeft red_text">-1.30</td>

              <td class="tdLeft">23.14</td>
    
                                    <td class="tdLeft red_text">-1.30</td>
  </tr>
    <tr>
              <td class="tdLeft">LZ_LCRA</td>
    
              <td class="tdLeft">       22.59</td>
    
                                    <td class="tdLeft red_text">-5.54</td>

              <td class="tdLeft">22.59</td>
    
                                    <td class="tdLeft red_text">-5.54</td>
  </tr>
    <tr>
              <td class="tdLeft">LZ_NORTH</td>
    
              <td class="tdLeft">       23.87</td>
    
                                    <td class="tdLeft red_text">-3.72</td>

              <td class="tdLeft">23.87</td>
    
                                    <td class="tdLeft red_text">-3.72</td>
  </tr>
    <tr>
              <td class="tdLeft">LZ_RAYBN</td>
    
              <td class="tdLeft">       23.93</td>
    
                                    <td class="tdLeft red_text">-3.68</td>

              <td class="tdLeft">23.93</td>
    
                                    <td class="tdLeft red_text">-3.68</td>
  </tr>
    <tr>
              <td class="tdLeft">LZ_SOUTH</td>
    
              <td class="tdLeft">       22.03</td>
    
                                    <td class="tdLeft red_text">-10.26</td>

              <td class="tdLeft">22.03</td>
    
                                    <td class="tdLeft red_text">-10.26</td>
  </tr>
    <tr>
              <td class="tdLeft">LZ_WEST</td>
    
              <td class="tdLeft">       51.12</td>
    
                                    <td class="tdLeft red_text">-5.11</td>

              <td class="tdLeft">51.12</td>
    
                                    <td class="tdLeft red_text">-5.11</td>
  </tr>
    </tbody>
</table>
</div>
</body>
</html>