I’m trying to get a value from a website into a sensor. Scraping the regular website (with CSS etc.) didn’t work because the value is injected with javascript (or some other magic ;-)). So I took a deep dive with the developer tool and found the website with the data I wanted to extract. Unfortunately no selector. Just a big ‘cloud’ of raw text data. My next step was figuring out the Regex to extract the data.
The data to extract:
precip_accum_local_yesterday: 2.4734
(or something else depending on actual data).
The regex I managed to came up with is:
precip_accum_local_yesterday":([0-9]*\.?[0-9]+)
Which gives me data mentioned above.
How do I implement this in a scrape sensor? My current code for this is:
- platform: scrape
#resource: "https://tempestwx.com/station/32351/"
resource: "https://swd.weatherflow.com/swd/rest/better_forecast?callback=jQuery22400765091875898638_1616934542421&api_key=a8f5dbda-af0a-4b57-99b9-f10baa88f27b&build=11&station_id=32351&lat=56.43546&lon=9.95076&units_temp=c&units_wind=mps&units_pressure=hpa&units_distance=km&units_precip=mm&units_other=metric&_=1616934542422"
name: Rain Accumulation Today
value_template: '{{ value | regex_search(precip_accum_local_yesterday":([0-9]*\.?[0-9]+)) }}'
scan_interval: 3600 # SECONDS - be nice; once per hour only
unit_of_measurement: "mm"
Which gives me the following errors:
Invalid config for [sensor.scrape]: invalid template (TemplateSyntaxError: unexpected char '"' at 52) for dictionary value @ data['value_template']. Got '{{ value | regex_search(precip_accum_local_yesterday":([0-9]*\\.?[0-9]+)) }}'
required key not provided @ data['select']. Got None. (See ?, line ?).
The documentation for the scrape sensor states that the select string is REQUIRED. But because we have no selector this scrape intergration isn’t going to work for the website mentioned above. So that explains the error.
Any help would be very appreciated.
