Ameren IL hourly price HACS multiscrape trouble

I am trying to use multiscrape from Hacs to scrape the web page for Ameren IL hourly energy price program and input them into home assistant.
(https://www.ameren.com/illinois/account/customer-service/bill/power-smart-pricing/prices)
There is a table of the prices at the bottom of the page. I am pretty new to the scrape world so I’m not sure where my logic is going wrong. My issue is with the CSS selector not pulling the price values as I would expect, just reads unavailable. I think the path is incorrect or something. Any help would be appreciated!

- resource: https://www.ameren.com/illinois/account/customer-service/bill/power-smart-pricing/prices
  scan_interval: 28800
  sensor: 
    - unique_id: PSP_0000
      name: PSP 0000
      select: "tbody > tr > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_0100
      name: PSP 0100
      select: "tbody > tr:nth-child(2) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_0200
      name: PSP 0200
      select: "tbody > tr:nth-child(3) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_0300
      name: PSP 0300
      select: "tbody > tr:nth-child(4) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_0400
      name: PSP 0400
      select: "tbody > tr:nth-child(5) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_0500
      name: PSP 0500
      select: "tbody > tr:nth-child(6) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_0600
      name: PSP 0600
      select: "tbody > tr:nth-child(7) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_0700
      name: PSP 0700
      select: "tbody > tr:nth-child(8) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_0800
      name: PSP 0800
      select: "tbody > tr:nth-child(9) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_0900
      name: PSP 0900
      select: "tbody > tr:nth-child(10) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_1000
      name: PSP 1000
      select: "tbody > tr:nth-child(11) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_1100
      name: PSP 1100
      select: "tbody > tr:nth-child(12) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_1200
      name: PSP 1200
      select: "tbody > tr:nth-child(13) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_1300
      name: PSP 1300
      select: "tbody > tr:nth-child(14) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_1400
      name: PSP 1400
      select: "tbody > tr:nth-child(15) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_1500
      name: PSP 1500
      select: "tbody > tr:nth-child(16) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_1600
      name: PSP 1600
      select: "tbody > tr:nth-child(17) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_1700
      name: PSP 1700
      select: "tbody > tr:nth-child(18) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_1800
      name: PSP 1800
      select: "tbody > tr:nth-child(19) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_1900
      name: PSP 1900
      select: "tbody > tr:nth-child(20) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_2000
      name: PSP 2000
      select: "tbody > tr:nth-child(21) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_2100
      name: PSP 2100
      select: "tbody > tr:nth-child(22) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_2200
      name: PSP 2200
      select: "tbody > tr:nth-child(23) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'
    - unique_id: PSP_2300
      name: PSP 2300
      select: "tbody > tr:nth-child(24) > td:nth-child(2)"
      unit_of_measurement: ¢
      value_template: '{{ value.split("¢")[0] }}'

The site uses JavaScript to populate the page. So when the scraper runs the prices aren’t on the page yet.

Your best bet would be to run something like this:

multiscrape:
  - resource: 'https://www.ameren.com/api/ameren/promotion/RtpHourlyPricesbyDate'
    name: 'Ameren Pricing'
    method: POST
    payload: '{"SelectedDate":"2023-04-27"}'
    headers:
      Content-Type : application/json
    scan_interval: 3600
    log_response: true
    sensor:
      - name: AmerenPricing
        unique_id: AmerenPricing_01
        value_template: '{{ value_json.hourlyPriceDetails[0].price }}'
        unit_of_measurement: "¢" 

Awesome help man. Works perfectly! Any way to use the current date? Rather than having to manually enter it. I’m very new to this

Try putting this for the payload:

'{"SelectedDate":"{{ now().year }}-{{ now().month }}-{{ now().day }}"}'

Yep that does the trick as far as I can tell. The true test will be tomorrow to see if the values change. Thank you so much for the help. Really appreciate it!

Apologies for the hijack, but is there any way to make that work for the downloaded version? Seems like a CSV would be a tiny bit more reliable, but I’m not familiar enough with JS to make this work. I did request an Ameren developer account, but we’ll see how that goes…

https://www.ameren.com/account/retail-energy

Not responding to your question, but ever since I’ve been using this configuration it has worked flawlessly.

Crossposting, but it looks like 2024.8 broke the updates. Anyone know how to fix?

This error originated from a custom integration.

Logger: custom_components.multiscrape.sensor
Source: custom_components/multiscrape/sensor.py:167
integration: Multiscrape scraping component (documentation, issues)
First occurred: 10:16:44 PM (24 occurrences)
Last logged: 10:16:44 PM

Ameren Pricing # AmerenPricing19 # Unable to scrape data: hass variable not set on template Consider using debug logging and log_response for further investigation.
Ameren Pricing # AmerenPricing20 # Unable to scrape data: hass variable not set on template Consider using debug logging and log_response for further investigation.
Ameren Pricing # AmerenPricing21 # Unable to scrape data: hass variable not set on template Consider using debug logging and log_response for further investigation.
Ameren Pricing # AmerenPricing22 # Unable to scrape data: hass variable not set on template Consider using debug logging and log_response for further investigation.
Ameren Pricing # AmerenPricing23 # Unable to scrape data: hass variable not set on template Consider using debug logging and log_response for further investigation.

Update your Multiscrape to the latest version; this was addressed yesterday

Thank you! That did indeed resolve the issue. What a strange bug.