Australian Weather Forecast using BOM Public FTP

BrendanMoran · October 2, 2018, 3:34am

Hi All,

First off, credit to @enzymes in this thread for inspiring me to do this work. I am not a developer, and this is my first exposure to Python, so hopefully I have created something at least partially useful for the Australian Home Assistant community. I am completely open to feedback and would love to make something that is used by our community

So it seems the BoM have recently started detecting and blocking web scrapers, fortunately they also provide an open, public FTP service. Unfortunately, Home Assistant doesn’t have an FTP component already. This is where I have tried to fill in the gap with my custom component, bom_forecast.

Please follow the setup instructions on github:

Friendly mode?

Friendly mode will avoid creating a sensor for each individual condition for each day of forecast and will neatly present each forecast day as one sensor.

grantc · October 2, 2018, 4:16am

Tried this through the Appdaemon app you release a week ago and still getting the same error now. Am I the only one getting this issue? Thanks again for your efforts Brendan.

2018-10-02 11:58:47 ERROR (MainThread) [homeassistant.components.sensor] Error while setting up platform bom_forecast
Traceback (most recent call last):
File “/srv/homeassistant/lib/python3.6/site-packages/homeassistant/helpers/entity_platform.py”, line 128, in _async_setup_platform
SLOW_SETUP_MAX_WAIT, loop=hass.loop)
File “/usr/lib/python3.6/asyncio/tasks.py”, line 358, in wait_for
return fut.result()
File “/usr/lib/python3.6/concurrent/futures/thread.py”, line 56, in run
result = self.fn(*self.args, **self.kwargs)
File “/home/homeassistant/.homeassistant/custom_components/sensor/bom_forecast.py”, line 134, in setup_platform
bom_forecast_data.update()
File “/srv/homeassistant/lib/python3.6/site-packages/homeassistant/util/init.py”, line 324, in wrapper
result = method(*args, **kwargs)
File “/home/homeassistant/.homeassistant/custom_components/sensor/bom_forecast.py”, line 295, in update
open(‘bom_forecast.xml’, ‘wb’).write)
PermissionError: [Errno 13] Permission denied: ‘bom_forecast.xml’

BrendanMoran · October 2, 2018, 4:26am

Hmmm… looks like on hass.io there is no permission to create files on the local system. I will have to come up with another way! Thanks for the feedback Grant.

xx_Nexus_xx · October 2, 2018, 4:27am

That looks promising, I will have a try tonight … great work!

I saw another weather related chat talking about weather alarms. Is it possible to have these weather warnings/events triggered with this as well ?

sparkydave · October 2, 2018, 4:29am

you can save to the local system but the directory needs to in the ‘whitelist’. I’ll be keen to try this out next week when I get the time

tom_l · October 2, 2018, 4:39am

Working perfectly for me (Hassio).

tom_l · October 2, 2018, 4:40am

This would be why it is working for me then. I have:

  whitelist_external_dirs:
    - /config

BrendanMoran · October 2, 2018, 4:41am

I have updated the code on github so that it doesn’t attempt to write a file to disk, it will store it in memory rather than save the file. Give it a go now.

grantc · October 2, 2018, 5:10am

Tried the whitelist option with no luck and now your revised version seems to have worked. Cheers

DavidFW1960 · October 2, 2018, 5:38am

How did you do the weather code Tom?

tom_l · October 2, 2018, 5:46am

Which one David?

DavidFW1960 · October 2, 2018, 5:47am

The graph with temp/dewpoint… bottom left…

tom_l · October 2, 2018, 5:49am

It’s a history graph of the BoM Australia sensors

groups.yaml

weather_now:
  name: Weather Now
  entities:
  - sensor.bom_hobart_air_temp_c
  - sensor.bom_hobart_feels_like_c
  - sensor.bom_hobart_wind_speed_kt
  - sensor.bom_hobart_wind_gust_kt
  - sensor.bom_hobart_wind_direction
  - sensor.bom_hobart_dew_point_c
  - sensor.bom_hobart_pressure_mb
  - sensor.bom_hobart_rain_today
  - sensor.bom_hobart_relative_humidity
  - history_graph.weather

history_graphs.yaml

weather:
  name: Weather
  entities:
  - sensor.bom_hobart_air_temp_c
  - sensor.bom_hobart_feels_like_c
  - sensor.bom_hobart_wind_speed_kt
  - sensor.bom_hobart_wind_gust_kt
  - sensor.bom_hobart_dew_point_c
  - sensor.bom_hobart_pressure_mb
  - sensor.bom_hobart_rain_today
  - sensor.bom_hobart_relative_humidity
  hours_to_show: 72
  refresh: 300

sensors.yaml

- platform: bom
  station: IDT60901.94970
  name: Hobart
  monitored_conditions:
    - apparent_t
    - gust_kt
    - air_temp
    - dewpt
    - press
    - rain_trace
    - rel_hum
    - wind_dir
    - wind_spd_kt

sparkydave · October 2, 2018, 7:24am

that looks awesome! How often does your radar update? I take it that you haven’t got the animated version in HA? I had to use the animated radar from another provider because I couldn’t figure a way to import the BOM one properly

tom_l · October 2, 2018, 7:27am

No it’s not animated. Just the latest snapshot of the radar using the generic camera component:

- platform: generic
  still_image_url: http://www.bom.gov.au/radar/IDR761.gif?{{ now().strftime("%Y%m%d%H%M") }}00
  name: Rain Radar
  limit_refetch_to_url_change: true

DavidFW1960 · October 2, 2018, 7:38am

yeah I use the same Tom but didn’t know about the limit_refresh! cool.

Mahko_Mahko · October 2, 2018, 9:27am

This is awesome! Stoked to have BOM forecasts back. Mine wasn’t working initially. I did a few hassio restarts and still wasn’t working. Then I just tried a hard reboot and voila! Thanks for your work!

Greenhouse · October 2, 2018, 10:07pm

Thanks heaps for your efforts with this. Will be using this heaps.

At the moment, though, I’m getting this error (with hassio, whitelist enabled, updated python code:

Wed Oct 03 2018 08:03:18 GMT+1000 (Australian Eastern Standard Time)

INFO:homeassistant.util.package:Attempting install of colorlog==3.1.4
Testing configuration at /config
ERROR:homeassistant.scripts.check_config:BURB
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/homeassistant/scripts/check_config.py", line 207, in check
    res['components'] = check_ha_config_file(hass)
  File "/usr/local/lib/python3.6/site-packages/homeassistant/scripts/check_config.py", line 372, in check_ha_config_file
    platform = loader.get_platform(hass, domain, p_name)
  File "/usr/local/lib/python3.6/site-packages/homeassistant/loader.py", line 60, in get_platform
    return get_component(hass, PLATFORM_FORMAT.format(domain, platform))
  File "/usr/local/lib/python3.6/site-packages/homeassistant/loader.py", line 92, in get_component
    module = importlib.import_module(path)
  File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 674, in exec_module
  File "<frozen importlib._bootstrap_external>", line 781, in get_code
  File "<frozen importlib._bootstrap_external>", line 741, in source_to_code
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/config/custom_components/sensor/bom_forecast.py", line 9
    <!DOCTYPE html>
    ^
SyntaxError: invalid syntax
Fatal error while loading config: invalid syntax (bom_forecast.py, line 9)
Failed config
  General Errors: 
    - invalid syntax (bom_forecast.py, line 9)
Successful config (partial)

BrendanMoran · October 2, 2018, 11:19pm

Hi Adam, looks like some HTML code from github has snuck in there. Try copying the code from this link:
https://raw.githubusercontent.com/bremor/bom_forecast/master/custom_components/sensor/bom_forecast.py

DavidFW1960 · October 2, 2018, 11:27pm

Github is a bit of a bugger to get code from. You have to open the file and then click on raw and save as… otherwise you get this happening.