Dynamic Scrape Select

Configuration
1

Hello all
I am trying to do a scrape sensor with needs a dynamic argument in the select.
To be more specific, I need to select an element where the attribute data-weeknumber is the current week number and the data-dayofweek is the current weekday.
I tried doing a select like:

        select: >
            {% set wn = now().isocalendar()[1] %}
            {% set wd = now().weekday() %}
            tr[data-weeknumber="{{ wn }}"][data-dayofweek="{{ wd }}"] td:nth-child(2)

But I always get:
soupsieve.util.SelectorSyntaxError: Invalid character '{' position 0 line 1:
Is there a way to use templates in the select of a scrape?

2

Not saying this is a solution, but there is a custom component scraper also.
I don’t know if that does templates but could be worth trying.

3

Thanks for the hint. I am acutally already using that one :slight_smile: Looking at the code, it seems to use the exact same method as the scrape, so It has the same issue.

4

Alright…
I don’t know if Node red has a http tool, I believe it does. Maybe that is a solution.
Have that scrape and use function node to parse out what you need.

Or as I have done, set up a php webpage that does the parsing and delivers the parsed value to HA.

5

Here is a flow in node red that should do what you want.
It’s just the pattern and URL that needs to be written, but it will be a lot easier to write the pattern if you could share some of the HTML from the page

[{"id":"df9e7d98.0ad43","type":"inject","z":"ebaa69a9.649708","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"true","payloadType":"bool","x":90,"y":480,"wires":[["1a33be89.2c1791"]]},{"id":"1a33be89.2c1791","type":"http request","z":"ebaa69a9.649708","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"SOME_URL","tls":"","persist":false,"proxy":"","authType":"","x":70,"y":560,"wires":[["c4747b4e.8f5898"]]},{"id":"c4747b4e.8f5898","type":"moment","z":"ebaa69a9.649708","name":"","topic":"","input":"","inputType":"date","inTz":"Europe/Brussels","adjAmount":0,"adjType":"days","adjDir":"add","format":"dddd","locale":"C","output":"weekday","outputType":"msg","outTz":"Europe/Brussels","x":260,"y":560,"wires":[["94260a7a.e900f8"]]},{"id":"94260a7a.e900f8","type":"moment","z":"ebaa69a9.649708","name":"","topic":"","input":"","inputType":"date","inTz":"Europe/Brussels","adjAmount":0,"adjType":"days","adjDir":"add","format":"ww","locale":"C","output":"week","outputType":"msg","outTz":"Europe/Brussels","x":480,"y":560,"wires":[["126a6ba9.396244"]]},{"id":"126a6ba9.396244","type":"function","z":"ebaa69a9.649708","name":"","func":"\nvar match =[];\nmatch = (\"\" + msg.payload).match(/some_pattern/);\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":660,"y":540,"wires":[["777caa3e.1ec134"]]},{"id":"777caa3e.1ec134","type":"debug","z":"ebaa69a9.649708","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":790,"y":480,"wires":[]}]
6

For future readers: this has been added to the multiscrape component.