Hi
Can someone please help me to scrape the price of pellets after selecting poatalcode, size and amount?
Source: hxxps://www.harjeans.se/pellets/
Postalcode: 75243
Size: 8mm
Amount pallet: 3
Would much appreciate help to get this code and sensor up and running.
See attached picture
If you open the page in Chrome with “Developer Tools” open and go to the Network tab you can see what is going on. Once you enter the postal code and click the button, it does another round-trip to get the data in JSON format using the following link (postal code is at the end):
https://lime-external.harjeans.se:8081/h%C3%A4rje%C3%A5ns%20kraft/solution-harjeanskraft/pellets/?zipcode=75243
And it returns:
{"response_dict":{"discount":{"6mm":[],"8mm":[]},"type_and_prices":{"6mm":{"pallets_2":4899.0,"pallets_3":4899.0,"pallets_4":4899.0,"pallets_5":4899.0,"pallets_6":4899.0,"pallets_7":4899.0,"pallets_8":4899.0},"8mm":{"pallets_2":4899.0,"pallets_3":4899.0,"pallets_4":4899.0,"pallets_5":4899.0,"pallets_6":4899.0,"pallets_7":4899.0,"pallets_8":4899.0}},"zipcode":"75243"}}
So you would hook this up to a REST sensor rather than using scrape, as that definitely won’t work. However, it’s not that simple. If you paste the link into a browser, you get a 401 error, meaning they’re blocking you from getting the data outside their own page. From a superficial look it doesn’t seem that it just needs a header set, and it’s doing a “preflight” call, which is beyond my knowledge. Short answer - it might not be possible, as plenty of shopping sites block this sort of activity.