Melbourne Pollen Forecast Scrape Sensor and Lovelace "Card"

Share your Projects!
1

Edit: 2023-11-05

The solution has changed over time due to the available data source changing in structure and content. So a lot of content in this thread may be obsolete.

There’s now a custom integration available for early adopters/testing - see here.

image
image1524×587 77.8 KB

For a lighter weight solution using multiscrape you can follow the thread from around here.

And there’s a few automation examples here.

The rest of this first post is legacy, which I’ll leave for completeness.

Edit: 2023-10-14
It’s broken again with no fix in sight ATM for the 6 day forecast. The API doesn’t seem to contain updated data.

Edit: 2021-12-04
Got something hacky working again for 6 day forecast for Melbourne + Thunderstorm Asthma by district for “today”. See this post.

Edit: 2021-09-18
Have re-established something basic based on what is now available (1 day forecast on Melbourne website). Context below. Link to revised sensor is in this post.

Edit: 2021-09-05
Melbourne Pollen no longer offer this forecast data for free on their website (via app only, and subscription only) . Therefore this sensor is broken and cannot be fixed at this point. It might continue to work for some of the other sister sites (https://www.canberrapollen.com.au/). I have limited motivation to maintain/develop anything for these other locations;)

I made a start on a pollen forecast sensor/card in readiness for the season.

Pollen

It scrapes the forecast from Melbourne Grass Pollen Forecast using the HACS MultiScrape custom component. I’ve enquired about an API too (edit 2021-07-23: they confirmed there is no public API yet but it is on their radar)

Then I’ve slapped together a bit of a lovelace “card” mostly using the custom button card, in a rough weather forecast layout.

Haven’t done much testing/polishing but welcome any thoughts/help in refining it.

In config:

multiscrape:
############################################################################
##Melbourne Pollen Forecast
############################################################################
#TODO
# - Icon templates aren't working properly
# - General testing
# - Make card look prettier
# - Get the thunderstorm athsma forecast.
  - resource: https://www.melbournepollen.com.au/
    scan_interval: 36000 #Hourly
    sensor:
      - unique_id: melbourne_pollen_forecast_fc1
        name: Pollen fc1
        select: "#fc1"
        attributes:
          - name: Forecast Day
            select: "#dy1"
          - name: Forecast Date
            select: "#dt1"
        value_template: '{{ value.strip() }}'
        icon: >-
          {% if is_state("sensor.melbourne_pollen_forecast_fc1", "Low") %}
            mdi:emoticon-happy
          {% elif is_state("sensor.melbourne_pollen_forecast_fc1", "Moderate")  %}
            mdi:emoticon-neutral
          {% elif is_state("binary_sensor.melbourne_pollen_forecast_fc1", "High")  %}
            mdi:emoticon-sad
          {% elif is_state("binary_sensor.melbourne_pollen_forecast_fc1", "Extreme")  %}
            mdi:emoticon-angry
          {% else %}
            mdi:help-circle
          {% endif %}
      - unique_id: melbourne_pollen_forecast_fc2
        name: Pollen fc2
        select: "#fc2"
        attributes:
          - name: Forecast Day
            select: "#dy2"
          - name: Forecast Date
            select: "#dt2"
        value_template: '{{ value.strip() }}'
        icon: >-
          {% if is_state("sensor.melbourne_pollen_forecast_fc2", "Low") %}
            mdi:emoticon-happy
          {% elif is_state("sensor.melbourne_pollen_forecast_fc2", "Moderate")  %}
            mdi:emoticon-neutral
          {% elif is_state("binary_sensor.melbourne_pollen_forecast_fc2", "High")  %}
            mdi:emoticon-sad
          {% elif is_state("binary_sensor.melbourne_pollen_forecast_fc2", "Extreme")  %}
            mdi:emoticon-angry
          {% else %}
            mdi:help-circle
          {% endif %}
      - unique_id: melbourne_pollen_forecast_fc3
        name: Pollen fc3
        select: "#fc3"
        attributes:
          - name: Forecast Day
            select: "#dy3"
          - name: Forecast Date
            select: "#dt3"
        value_template: '{{ value.strip() }}'
        icon: >-
          {% if is_state("sensor.melbourne_pollen_forecast_fc1", "Low") %}
            mdi:emoticon-happy
          {% elif is_state('sensor.melbourne_pollen_forecast_fc1', 'Moderate')  %}
            mdi:emoticon-neutral
          {% elif is_state('binary_sensor.melbourne_pollen_forecast_fc1', 'High')  %}
            mdi:emoticon-sad
          {% elif is_state('binary_sensor.melbourne_pollen_forecast_fc1', 'Extreme')  %}
            mdi:emoticon-angry
          {% else %}
            mdi:help-circle
          {% endif %}
      - unique_id: melbourne_pollen_forecast_fc4
        name: Pollen fc4
        select: "#fc4"
        attributes:
          - name: Forecast Day
            select: "#dy4"
          - name: Forecast Date
            select: "#dt4"
        value_template: '{{ value.strip() }}'
        icon: >-
          {% if is_state("sensor.melbourne_pollen_forecast_fc1", "Low") %}
            mdi:emoticon-happy
          {% elif is_state('sensor.melbourne_pollen_forecast_fc1', 'Moderate')  %}
            mdi:emoticon-neutral
          {% elif is_state('binary_sensor.melbourne_pollen_forecast_fc1', 'High')  %}
            mdi:emoticon-sad
          {% elif is_state('binary_sensor.melbourne_pollen_forecast_fc1', 'Extreme')  %}
            mdi:emoticon-angry
          {% else %}
            mdi:help-circle
          {% endif %}
      - unique_id: melbourne_pollen_forecast_fc5
        name: Pollen fc5
        select: "#fc5"
        attributes:
          - name: Forecast Day
            select: "#dy5"
          - name: Forecast Date
            select: "#dt5"
        value_template: '{{ value.strip() }}'
        icon: >-
          {% if is_state("sensor.melbourne_pollen_forecast_fc1", "Low") %}
            mdi:emoticon-happy
          {% elif is_state('sensor.melbourne_pollen_forecast_fc1', 'Moderate')  %}
            mdi:emoticon-neutral
          {% elif is_state('binary_sensor.melbourne_pollen_forecast_fc1', 'High')  %}
            mdi:emoticon-sad
          {% elif is_state('binary_sensor.melbourne_pollen_forecast_fc1', 'Extreme')  %}
            mdi:emoticon-angry
          {% else %}
            mdi:help-circle
          {% endif %}
      - unique_id: melbourne_pollen_forecast_fc6
        name: Pollen fc6
        select: "#fc6"
        attributes:
          - name: Forecast Day
            select: "#dy6"
          - name: Forecast Date
            select: "#dt6"
        value_template: '{{ value.strip() }}'
        icon: >-
          {% if is_state('sensor.melbourne_pollen_forecast_fc1', 'Low') %}
            mdi:emoticon-happy
          {% elif is_state('sensor.melbourne_pollen_forecast_fc1', 'Moderate')  %}
            mdi:emoticon-neutral
          {% elif is_state('binary_sensor.melbourne_pollen_forecast_fc1', 'High')  %}
            mdi:emoticon-sad
          {% elif is_state('binary_sensor.melbourne_pollen_forecast_fc1', 'Extreme')  %}
            mdi:emoticon-angry
          {% else %}
            mdi:help-circle
          {% endif %}

In lovelace:

    #### Melbourne Pollen Card ##############################################################################################################
          - type: horizontal-stack
            cards:
               #mother Mode
              - type: "custom:button-card"
                entity: sensor.melbourne_pollen_forecast_fc1
                label: >
                  [[[return (states['sensor.melbourne_pollen_forecast_fc1'].attributes.forecast_day);]]]
                tap_action: 
                  action: more-info
                show_label: true
                show_state: true
                show_name: false                
                state:
                  - value: 'Low'
                    color: rgb(52, 235, 85)
                  - value: 'Moderate'
                    color: rgb(235, 235, 52)
                  - value: 'High'
                    color: rgb(235, 147, 52)
                  - value: 'Extreme'
                    color: rgb(235, 52, 52)
                size: 50%
                aspect_ratio: 1.8/1
                layout: icon_state
                styles:
                  name:
                    - font-weight: bold
                    - font-size: 0.5vw
                    - color: grey
                  icon:
                    - color: grey
                  label:
                    - font-weight: bold
                    - font-size: 0.6vw
                    - color: white
              - type: "custom:button-card"
                entity: sensor.melbourne_pollen_forecast_fc2
                label: >
                  [[[return (states['sensor.melbourne_pollen_forecast_fc2'].attributes.forecast_day);]]]
                tap_action: 
                  action: more-info
                show_label: true
                show_state: true
                show_name: false                
                state:
                  - value: 'Low'
                    color: rgb(52, 235, 85)
                  - value: 'Moderate'
                    color: rgb(235, 235, 52)
                  - value: 'High'
                    color: rgb(235, 147, 52)
                  - value: 'Extreme'
                    color: rgb(235, 52, 52)
                size: 50%
                aspect_ratio: 1.8/1
                layout: icon_state
                styles:
                  name:
                    - font-weight: bold
                    - font-size: 0.5vw
                    - color: grey
                  icon:
                    - color: grey
                  label:
                    - font-weight: bold
                    - font-size: 0.6vw
                    - color: white
              - type: "custom:button-card"
                entity: sensor.melbourne_pollen_forecast_fc3
                label: >
                  [[[return (states['sensor.melbourne_pollen_forecast_fc3'].attributes.forecast_day);]]]
                tap_action: 
                  action: more-info
                show_label: true
                show_state: true
                show_name: false                
                state:
                  - value: 'Low'
                    color: rgb(52, 235, 85)
                  - value: 'Moderate'
                    color: rgb(235, 235, 52)
                  - value: 'High'
                    color: rgb(235, 147, 52)
                  - value: 'Extreme'
                    color: rgb(235, 52, 52)
                size: 50%
                aspect_ratio: 1.8/1
                layout: icon_state
                styles:
                  name:
                    - font-weight: bold
                    - font-size: 0.5vw
                    - color: grey
                  icon:
                    - color: grey
                  label:
                    - font-weight: bold
                    - font-size: 0.6vw
                    - color: white
              - type: "custom:button-card"
                entity: sensor.melbourne_pollen_forecast_fc4
                label: >
                  [[[return (states['sensor.melbourne_pollen_forecast_fc4'].attributes.forecast_day);]]]
                tap_action: 
                  action: more-info
                show_label: true
                show_state: true
                show_name: false                
                state:
                  - value: 'Low'
                    color: rgb(52, 235, 85)
                  - value: 'Moderate'
                    color: rgb(235, 235, 52)
                  - value: 'High'
                    color: rgb(235, 147, 52)
                  - value: 'Extreme'
                    color: rgb(235, 52, 52)
                size: 50%
                aspect_ratio: 1.8/1
                layout: icon_state
                styles:
                  name:
                    - font-weight: bold
                    - font-size: 0.5vw
                    - color: grey
                  icon:
                    - color: grey
                  label:
                    - font-weight: bold
                    - font-size: 0.6vw
                    - color: white
              - type: "custom:button-card"
                entity: sensor.melbourne_pollen_forecast_fc5
                label: >
                  [[[return (states['sensor.melbourne_pollen_forecast_fc5'].attributes.forecast_day);]]]
                tap_action: 
                  action: more-info
                show_label: true
                show_state: true
                show_name: false                
                state:
                  - value: 'Low'
                    color: rgb(52, 235, 85)
                  - value: 'Moderate'
                    color: rgb(235, 235, 52)
                  - value: 'High'
                    color: rgb(235, 147, 52)
                  - value: 'Extreme'
                    color: rgb(235, 52, 52)
                size: 50%
                aspect_ratio: 1.8/1
                layout: icon_state
                styles:
                  name:
                    - font-weight: bold
                    - font-size: 0.5vw
                    - color: grey
                  icon:
                    - color: grey
                  label:
                    - font-weight: bold
                    - font-size: 0.6vw
                    - color: white
              - type: "custom:button-card"
                entity: sensor.melbourne_pollen_forecast_fc6
                label: >
                  [[[return (states['sensor.melbourne_pollen_forecast_fc6'].attributes.forecast_day);]]]
                tap_action: 
                  action: more-info
                show_label: true
                show_state: true
                show_name: false                
                state:
                  - value: 'Low'
                    color: rgb(52, 235, 85)
                  - value: 'Moderate'
                    color: rgb(235, 235, 52)
                  - value: 'High'
                    color: rgb(235, 147, 52)
                  - value: 'Extreme'
                    color: rgb(235, 52, 52)
                size: 50%
                aspect_ratio: 1.8/1
                layout: icon_state
                styles:
                  name:
                    - font-weight: bold
                    - font-size: 0.5vw
                    - color: grey
                  icon:
                    - color: grey
                  label:
                    - font-weight: bold
                    - font-size: 0.6vw
                    - color: white
1 Like
2

Check this for more info about the icon templates: add icon or icon template to the sensors. · Issue #29 · danieldotnl/ha-multiscrape · GitHub

1 Like
3

HI. I also live in old lockdown melbourne and have tried out this scrape sensor. I set it up and it was working fine for about 4-5 days, but it stopped working…

The only error is
[custom_components.multiscrape.entity] Sensor Pollen fc1 was unable to extract data from HTML

I pretty much copied your config without any changes, so not sure if it is still working for you.

Anyway, no rush, just though you might want to know about it

Any thanks again for an awesome sensor.

1 Like
4

Hey happy lockdown! Yeah mine has broken too, thanks for letting me know.

I’ll take a look in a bit (I’m busy setting up a particulate matter sensor to complement it:) - definitely would like to have it running reliably for the season.

It’s possible the site might block ip’s that are hitting it up too much?

Cheers…

5

Oh I see what’s happened now. They’ve removed the 6 day forecast from the page! So there’s nothing there to scrape… Hopefully something will return closer to the season?

6

Thanks for the follow up… I guess we just have to wait until spring…

Thanks again.

7

Just noticed that the ANU here in Canberra has a station, other than changing the url of the station are there any changes I need to make ??

Gaz

8

I got an email from them confirming they don’t have a public API yet.

The sensor is broken at the moment because they are redeveloping their webpage before the season starts. They indicated they should have something juicy out soonish.

I think we’ll have to wait and see what the new page looks like.

Then I can prob help you check if one for Canberra can be set up.

9

Actually I just checked the Canberra site and it still looks ok and may just work by changing the URL.

If not it is super easy to generate the required “select” via a browsers developer mode tools and I can give some pointers on that if needed…

10

The scraper seems to be working as I seem to getting data from the ANU site in Canberra, there are entities for 6 forecast days and all are low as per what is on the site.

The dashboard not so much, it doesn’t look like what you posted above. I haven’t play around much with dashboards yet so will continue to poke at it.

BTW, there is a note in one of the news posts on the Canberra site that they will be continuing counts through winter.

Gaz

1 Like
11

I’ll tidy up the Lovelace card when they deploy the new Melbourne site and that may help you.

Hopefully the data for the different locations will be consistent so we can reuse/collaborate on development.

12

Hi @obrien.craig maybe you can help raise support for the return of the 5 day forecast by commenting on this post/page?

I’m not clear if and when it is returning at this point. Thank you.

https://www.facebook.com/melbournepollen/posts/1999176443573899

13

I have just updated to 2021.9.2 and am now getting ‘unknown’ for all the pollen entities.

It was working fine from the Canberra site just before the update and the data on the site is still available.

Any ideas ?

Gaz

14

Hey. I had a quick tinker and had the same behaviour (using Canberra site) but couldn’t quickly diagnose/resolve.

Inrerestingly you can still scrape the forecast value as an attribute (maybe you can work with that?)

image
image992×145 12.3 KB 

      - unique_id: melbourne_pollen_forecast_fc1
        name: Pollen fc1
        select: '#fc1'
        attributes:
          - name: Forecast Day
            select: "#dy1"
            value_template: "{{ value[:3] }}"
          - name: Forecast Date
            select: "#dt1"
            #################NEW#################
          - name: Forecast Value
            select: "#fc1"
            #####################################
        value_template: "{{ value.strip() }}"
        icon: >-
          {% if is_state("sensor.melbourne_pollen_forecast_fc1", "Low") %}
            mdi:emoticon-happy
          {% elif is_state("sensor.melbourne_pollen_forecast_fc1", "Moderate")  %}
            mdi:emoticon-neutral
          {% elif is_state("binary_sensor.melbourne_pollen_forecast_fc1", "High")  %}
            mdi:emoticon-sad
          {% elif is_state("binary_sensor.melbourne_pollen_forecast_fc1", "Extreme")  %}
            mdi:emoticon-angry
          {% else %}
            mdi:help-circle
          {% endif %}

It’s looking unlikely the Melbourne site will return, and I so I won’t be very active in developing/maintaining work to date.

You may get some assistance on the Multiscape thread, or have a go at tinkering with the scaping. It’s easy once you know how. See the Multiscrape wiki.

Hope that is of some help.

15

Thanks for taking a look.

The Melbourne site states that they will start again in October and I also noticed that there is an Android app but it does not support the data from Canberra.

Just noticed your edit at the top, no longer free is a bugger.

Gaz

16

Issue is as per here.

Solution is post after…

17

After upgrading to V5.5 of multiscrape it is working again.

Gaz

1 Like
18

Revised sensor for “today” based on what is currently available for Victoria.

image

This is where it scrapes from.

image
image1110×915 257 KB

If you want another area, take a look at some of the multiscrape tutorials. It’s easy once you know how. If you try and fail, I can help.

In config…

############################################################################
##Melbourne Pollen Forecast
############################################################################
multiscrape:
  - resource: https://www.melbournepollen.com.au/
    scan_interval: 3600 #3600 sec = hr
    name: Pollen Forecast
    sensor:
      - unique_id: melbourne_pollen_forecast_today
        name: Pollen Forecast Today
        select: '#district-pollen-div > div > div > div > div > div > div.uk-grid-match.uk-child-width-1-2\@s.uk-text-center.uk-grid-collapse.uk-grid > div:nth-child(2) > div:nth-child(1) > div:nth-child(2) > div'
        value_template: '{{ value| trim}}'
        attributes:
          - name: Website Last Updated
            select: '#district-pollen-div > div > div > div > div > div > div.uk-grid-match.uk-child-width-1-2\@s.uk-text-center.uk-grid-collapse.uk-grid > div:nth-child(1) > div.ta-notice'
            value_template: "{{ value| replace('Last updated:', '')}}"
        icon: >-
          {% if value == 'Low' %}
            mdi:emoticon-happy
          {% elif value == 'Moderate' %}
            mdi:emoticon-neutral
          {% elif value == 'High' %}
            mdi:emoticon-sad
          {% elif value == 'Extreme' %}
            mdi:emoticon-angry
          {% else %}
            mdi:help-circle
          {% endif %}

in lovelace…

          - type: custom:button-card
            entity: sensor.melbourne_pollen_forecast_today
            name: Pollen Forecast
            label: >
              [[[return
              (states['sensor.melbourne_pollen_forecast_today'].attributes.website_last_updated);]]]
            tap_action:
              action: more-info
            show_label: true
            show_state: true
            color: auto
            size: 150%
            aspect_ratio: 3/1
            layout: icon_state
            state:
              - value: Low
                color: rgb(16, 140, 10)
              - value: Moderate
                color: rgb(235, 235, 52)
              - value: High
                color: rgb(235, 147, 52)
              - value: Extreme
                color: rgb(235, 52, 52)
            styles:
              name:
                - font-weight: bold
                - font-size: 0.7vw
                - color: white
              state:
                - font-weight: bold
                - font-size: 3vw
                - color: white
              label:
                - font-weight: bold
                - font-size: 0.5vw
                - color: grey
19

@obrien.craig the above post may interest you. Cheers.

20

Thanks for that @Mahko_Mahko I have set it up. Works well. I did see that they didn’t answer you question on facebook about if the 5 day forecast would now be a paid for feature. I guess that non-answer probably means it will be.

Thanks again.

1 Like