Multiscrape: question for <span class=

Configuration
1

I’m trying to scrape the following.
I want to get this information:
image

When I run in the debugger and break on this area. I can see the code looks like

image
image638×531 21 KB

If I select and copy CSS Selector it give me:
span.msg_tline:nth-child(1)
This is for the line that has “Furnace Status”

For the next line, that says Idle
span.msg_tline:nth-child(2)

All the rest are nth-child(3-6)

The line with the actual value returns a CSS Selector of: #param_0, 1, 2, 3

There is an additional value called inner HTML that contains the value I want.

How do I go about getting these values?

When I try to use:

    sensor:
      - unique_id: loginPage
        name: welcome
        select: '#desktopUserName > h2:nth-child(1)'
      - unique_id: loggedIn
        name: logoff
        select: "#desktopUserName > p:nth-child(2) > a:nth-child(1)"
      - unique_id: siemenslogo
        name: logo
        select: "#logo_iPad"
      - unique_id: screen
        name: entireScreen
        select: "span.msg_tline:nth-child(2)"

I get an error: 2022-11-21 13:12:10.185 ERROR (MainThread) [custom_components.multiscrape.sensor] Furnace # entireScreen # Unable to scrape data: Could not find a tag for given selector

Debug Log Info:

2022-11-21 14:21:35.199 DEBUG (MainThread) [custom_components.multiscrape.sensor] Furnace # entireScreen # Start scraping to update sensor
2022-11-21 14:21:35.200 DEBUG (MainThread) [custom_components.multiscrape.scraper] Furnace # entireScreen # Tag selected: None
2022-11-21 14:21:35.200 DEBUG (MainThread) [custom_components.multiscrape.form] Furnace # Exception occurred while scraping, will try to resubmit
the form next interval.
2022-11-21 14:21:35.200 ERROR (MainThread) [custom_components.multiscrape.sensor] Furnace # entireScreen # Unable to scrape data: Could not find a
 tag for given selector
Consider using debug logging and log_response for further investigation.

Any insight would be appreciated.

thanks

2

I added some additional debug to print out self._data when I get the message that the tag was not found.

I was able to then view the html in mozilla and it shows:

image
image697×952 19.8 KB

Coincidently this is what the webpage looks like when it is first rendered and the data hasn’t been updated from the server yet from POST AJAX.

so I can see that I’m authenticated, but none of the data I want to scrape is all empty.

3

I believe the root of the problem is that the webpage is updated via javascript, and never really rendered to be scraped. After rendering what is shown above, the data is continuously sent from the server to the browser every 500ms.

Is there any solution for this?