Thanks @icaman004 … looks like Dark Sky has gotten me what I need. Still not sure why I was getting the weird errors with the scrape sensor, but I will concentrate on other things for now and worry about scraping when I need to worry about it.
Can you share your code?
I’m thinking of writing a 7 day summary that uses mdi icons and the publicly available BoM FTP forecast data. It’s a lot easier to parse than the HTML. E.g.
<forecast-period index="3" start-time-local="2018-03-02T00:00:00+11:00" end-time-local="2018-03-03T00:00:00+11:00" start-time-utc="2018-03-01T13:00:00Z" end-time-utc="2018-03-02T13:00:00Z"><element type="forecast_icon_code">3</element><element type="air_temperature_minimum" units="Celsius">13</element><element type="air_temperature_maximum" units="Celsius">22</element><text type="precis">Partly cloudy.</text><text type="probability_of_precipitation">20%</text></forecast-period>You can look up your city here: http://reg.bom.gov.au/catalogue/anon-ftp.shtml
Very interested if you do this!
It’s still on my list of things to do but don’t hold your breath. It’s a bit of a low priority compared to the many other projects I have on the go at the moment. I expect it will be a few months before I get to it.
Instead of a custom component, I’m using a handful of commandline sensors to download XML forecasts from the BoM FTP site and parse out forecast info. Perhaps it might be of use? Another take on Australian weather forecasts using BoM feeds
I’ve just requested an API key.
Do you have a layman guide for setting this up?
Presuming you mean public transport and not BOM weather try here Transport NSW / Sydney Bus, Ferry and Train schedule
Thanks. I’m after PTV (Victoria) though, wich requires a dev id, api key and getting a session key. My python skills are rubbish!
Found http://timetableapi.ptv.vic.gov.au/swagger/ui/index
When I use this page (after entering my devID and devKey) I get 403 errors for the session key that their own website creates. Do you get any errors?
Essentially, you can then just copy and paste the URL that the webpage generates and feed this into a REST sensor.!
Code:
platform: rest
resource: http://timetableapi.ptv.vic.gov.au/v3 7
name: API
value_template: “{{value_json[‘departures’][0][‘estimated_departure_utc’]}}”
thn i add this code to split the time value.
Thanks Gillrishi.
How did you create the session key?
I’m using PTV website (and a few php scripts from github) but am getting 403 errors for session key being incorrect.
Thanks for the help. PTV fixed my dev id and key, and so now it’s working.
I’m parsing the value in a single sensor. Pasting here in case in helps someone else.
- platform: rest
resource: http://timetableapi.ptv.vic.gov.au/v3/departures/route_type/0/stop/.....
name: Departure_1
value_template: "{{ as_timestamp(value_json['departures'][0]['estimated_departure_utc']) | timestamp_custom('%H:%M')}}"I know I am re-opening an older thread, but I have a couple of questions about the PTV setup
Currently I have got it working using the API and it shows me the departure time from my local station in both directions…(using a different rest sensor for both directions)
I have a couple of questions about using the json filter, I would like to include both the scheduled time as well as the estimated time.
Also my link collects the next 5 departures, so I would like to some how display the next five departures, so any advice on how to configure the JSON to get it to display it…
Thanks all…
Has the BOM scape method stopped working for some people in the last day or two (or ever!)?
Mine just stopped working, and I’m wondering if they changed their code so it breaks everyone or if my IP has been identified as ‘scraper’?
1 Like
My BOM sensor is working… hasn’t missed a beat.
Same here, stopped working 2 days ago.
Gives the following error:
scrape: Error on device update!
Summary
scrape: Error on device update!
Traceback (most recent call last):
File “/usr/local/lib/python3.6/site-packages/homeassistant/helpers/entity_platform.py”, line 251, in _async_add_entity
await entity.async_device_update(warning=False)
File “/usr/local/lib/python3.6/site-packages/homeassistant/helpers/entity.py”, line 353, in async_device_update
yield from self.hass.async_add_job(self.update)
File “/usr/local/lib/python3.6/concurrent/futures/thread.py”, line 56, in run
result = self.fn(*self.args, **self.kwargs)
File “/usr/local/lib/python3.6/site-packages/homeassistant/components/sensor/scrape.py”, line 120, in update
value = raw_data.select(self._select)[0].text
IndexError: list index out of range
I get lots of scrape errors per day. Seems to be working though.
Added few more other scrape sensors from different sources, it works perfectly, but not from bom.gov.au. I am just using 5 sensors to get basic data like min/max temperature, chance of rain, worked for half a year and just stopped working few days ago.
Wondering if they can blacklist my IP address
The same has happened to me… It just stopped, I thought it was maybe pi-hole, but it isn’t even trying to connect. I wonder if it is been blocked from scraping…