Test scrape sensor?

Is there any way to test the resources of the scrape sensor? I’m not familiair with beautiful soup and having a hard time scraping certain data. At the moment I’m testing several options before (sometimes) succeeding and every time I want to test a resource I have to restart Hass which delays a lot…

https://home-assistant.io/components/sensor.scrape/

I created a notebook which is showing how the scrape sensor works and should help to identify the relevant part in the HTML.

http://nbviewer.jupyter.org/github/home-assistant/home-assistant-notebooks/blob/master/web-scraping.ipynb

2 Likes

Thanks, I’ll dive into this next weekend!

Thank you. The notebook is very good to help with this sensor.

I have a question because I’m trying to use with my web www.pcdemano.com and selecting 'b" it can found, for example, a data in 8 position.

Unfortunately the scrape sensor doesn’t work properly

and it seems there is an error

File “/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/sensor/scrape.py”, line 120, in update
value = raw_data.select(self._select)[0].text
IndexError: list index out of range

Not idea?

Thank you in advance :slight_smile:

Hard to tell if the site is gone.

?

Can’t you access to www.pcdemano.com?

It’s working fine :slight_smile:

I seem to be getting the same, tho i’m pretty sure i’m doing it wrong, as i’m trying to scrape from an RSS feed (as i can’t seem to get the feedreader to work)

Me have errors too:

sensor:

The object is a ID (actualPrice)…
Maybe because the web have JS? Ajax?

In a wonderful twist of irony, your notebook appears to now be gone. Any chance of a reshare? I’m trying to decipher the scrape sensor and would love to see your work!

1 Like

Hi,

is it possible to scrape from https://app.proofofhumanity.id ?
I can’t get it to work.

Kind regards
midijunkie

Found this when browsing the repository… :wink: