Scraping values into a sensor from an XML file

Configuration
1

I am struggling for days I would like to scrape this location once per day, and display information on the dashboard - Help please !

The file containing the information is located at: cfa.vic.gov.au/cfa/rssfeed/central-firedistrict_rss.xml

What I want to obtain is one of the dates - located at thend - it basically shows the restrictive dates for burining off in country Victoria / Australia

<item>

<title>Fire restrictions by muncipality</title>

<link>https://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</link>

<description><p>BALLARAT:Restrictions in force: 25/11/2024 - 01/05/2025<br/>BANYULE:Restrictions in force: 25/11/2024 - 01/05/2025<br/>BASS COAST:Restrictions in force: 09/12/2024 - 01/05/2025<br/>BOROUGH OF QUEENSCLIFFE:Restrictions in force: 25/11/2024 - 01/05/2025<br/>CARDINIA:Restrictions in force: 09/12/2024 - 01/05/2025<br/>CASEY:Restrictions in force: 09/12/2024 - 01/05/2025<br/>FRANKSTON:Restrictions in force: 09/12/2024 - 01/05/2025<br/>FRENCH ISLAND:Restrictions in force: 09/12/2024 - 01/05/2025<br/>GOLDEN PLAINS:Restrictions in force: 25/11/2024 - 01/05/2025<br/>GREATER DANDENONG:Restrictions in force: 09/12/2024 - 01/05/2025<br/>GREATER GEELONG:Restrictions in force: 25/11/2024 - 01/05/2025<br/>HEPBURN:Restrictions in force: 25/11/2024 - 01/05/2025<br/>HUME:Restrictions in force: 18/11/2024 - 01/05/2025<br/>KINGSTON:Restrictions in force: 09/12/2024 - 01/05/2025<br/>KNOX:Restrictions in force: 23/12/2024 - 01/05/2025<br/>MACEDON RANGES:Restrictions in force: 25/11/2024 - 01/05/2025<br/>MANNINGHAM:Restrictions in force: 23/12/2024 - 01/05/2025<br/>MAROONDAH:Restrictions in force: 23/12/2024 - 01/05/2025<br/>MELTON:Restrictions in force: 18/11/2024 - 01/05/2025<br/>MOORABOOL:Restrictions in force: 25/11/2024 - 01/05/2025<br/>MORNINGTON PENINSULA:Restrictions in force: 09/12/2024 - 01/05/2025<br/>NILLUMBIK:Restrictions in force: 25/11/2024 - 01/05/2025<br/>SURF COAST:Restrictions in force: 25/11/2024 - 01/05/2025<br/>WHITTLESEA:Restrictions in force: 18/11/2024 - 01/05/2025<br/>WYNDHAM:Restrictions in force: 18/11/2024 - 01/05/2025<br/>YARRA RANGES:Restrictions in force: 23/12/2024 - 01/05/2025<br/></p></description>

<guid>https://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</guid>

</item>

Its the words ‘Restrictions in force: 25/11/2024 - 01/05/2025’ that follow HEPBURN that I am interested in displaying…

I need help in getting the file loaded / scraped / placed into a sensor value to display on a dashboard please… This is first scrape so detailed explanation might help…

2

Haven’t really attempted to do anything with neither template entity sensors nor RESTful Sensors, so getting this to work was an interesting challenge…!

There is surely an infinite number of ways to do this but I went with a RESTful Sensor as I knew it would automatically parse the XML into a dict. Probably there is another integration that could be used and run from within the action part of the template entity sensor instead.

rest:
  - resource: https://www.cfa.vic.gov.au/cfa/rssfeed/central-firedistrict_rss.xml
    sensor:
      - name: Fire restrictions
        unique_id: 94f76835-9f62-4b95-9bd2-84ac741ee356
        icon: mdi:fire-alert
        device_class: timestamp
        force_update: true
        value_template: "{{ now() }}"
        json_attributes_path: "$.rss.channel.item[?(@.title == 'Fire restrictions by muncipality')]"
        json_attributes:
          - title
          - description
          - link

template:
  - trigger:
      - trigger: state
        entity_id: sensor.fire_restrictions
    action:
      - variables:
          fire_restrictions: >-
            {% set fire_restrictions = namespace(by_muncipality=[]) %}
            {% for description in state_attr('sensor.fire_restrictions', 'description')
            .removeprefix('<p>').removesuffix('</p>').split('<br/>') if description %}
              {% set muncipality, daterange = description.split(':Restrictions in force: ') %}
              {% set fire_restrictions.by_muncipality = fire_restrictions.by_muncipality +
              [(muncipality | lower, daterange.split(' - ')
              | map('regex_replace', '(\d\d)\/(\d\d)\/(\d\d\d\d)', '\\3-\\2-\\1') | list)] %}
            {% endfor %}
            {{ dict(fire_restrictions.by_muncipality) }}
    sensor:
      - name: Fire restrictions - Hepburn - Start
        unique_id: 2a382552-5ffe-41ac-95c2-5fb84575c551
        icon: mdi:fire-alert
        device_class: timestamp
        availability: "{{ has_value('sensor.fire_restrictions') }}"
        state: "{{ fire_restrictions['hepburn'] | first | as_datetime | as_local }}"
      - name: Fire restrictions - Hepburn - End
        unique_id: b165c773-4a94-4b26-8484-5d427572bf86
        icon: mdi:fire-alert
        device_class: timestamp
        availability: "{{ has_value('sensor.fire_restrictions') }}"
        state: "{{ fire_restrictions['hepburn'] | last | as_datetime | as_local + timedelta(days=1, seconds=-1) }}"
3

Big thankyou… works wonderful. You even split the dates into seperate sensors for me… Not sure if I will continue with that or just leave it as a text string - as I am unsure of the format of the file once the restrictions are lifted - I will just need to wait and see…

4

I am hoping you could help with this one too… I am starting to understand your code above… how the file gets loaded, scanned. Finds the literal and sets the sensor… It is starting to make sense…

However I still like to see one last example before you let me loose on the world - this example is a little different - having multiple fields of interest - and its repeated for each day - so the data changes…

The XML file is similar information - but its actually different data… Its located at cfa.vic.gov.au/cfa/rssfeed/central-firedistrict_rss.xml

Current contents look like this:

This XML file does not appear to have any style information associated with it. The document tree is shown below.
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
<channel>
<title>Country Fire Authority - Victoria, Australia - Total Fire Ban & Fire Danger Ratings - Central Forecast</title>
<link>http://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</link>
<description>Country Fire Authority - Victoria, Australia - Total Fire Ban & Fire Danger Ratings - Central Forecast.</description>
<pubDate>Wed, 26 Feb 2025 19:50:57 GMT</pubDate>
<dc:date>2025-02-26T19:50:57</dc:date>
<item>
<title>Thursday, 27 February 2025</title>
<link>http://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</link>
<description><p>Today, Thu, 27 Feb 2025 is not currently a day of Total Fire Ban.</p><p>Fire Danger Ratings<br/>Bureau of Meteorology forecast issued at: Thursday, 27 February 2025 05:30 AM</p><p>Central: MODERATE</p><p><img src="http://www.cfa.vic.gov.au/cfa/images/fdr/central/moderate.gif" alt="" border="0" /></p><p><img src="http://www.cfa.vic.gov.au/cfa/images/tfb_icon_big.png" alt="" width="22" height="22" /> Displays when Total Fire Ban in force<br /><span><a href="http://www.cfa.vic.gov.au/warnings-restrictions/restrictions-during-fire-danger-periods">Restrictions may apply</a></span></p></description>
<guid>http://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</guid>
</item>
<item>
<title>Friday, 28 February 2025</title>
<link>http://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</link>
<description><p>Tomorrow, Fri, 28 Feb 2025 is not currently a day of Total Fire Ban.</p><p>Fire Danger Ratings<br/>Bureau of Meteorology forecast issued at: Thursday, 27 February 2025 05:30 AM</p><p>Central: HIGH</p><p><img src="http://www.cfa.vic.gov.au/cfa/images/fdr/central/high.gif" alt="" border="0" /></p><p><img src="http://www.cfa.vic.gov.au/cfa/images/tfb_icon_big.png" alt="" width="22" height="22" /> Displays when Total Fire Ban in force<br /><span><a href="http://www.cfa.vic.gov.au/warnings-restrictions/restrictions-during-fire-danger-periods">Restrictions may apply</a></span></p></description>
<guid>http://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</guid>
</item>
<item>
<title>Saturday, 01 March 2025</title>
<link>http://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</link>
<description><p>Sat, 1 Mar 2025 is not currently a day of Total Fire Ban.</p><p>Fire Danger Ratings<br/>Bureau of Meteorology forecast issued at: Thursday, 27 February 2025 05:30 AM</p><p>Central: HIGH</p><p><img src="http://www.cfa.vic.gov.au/cfa/images/fdr/central/high.gif" alt="" border="0" /></p><p><img src="http://www.cfa.vic.gov.au/cfa/images/tfb_icon_big.png" alt="" width="22" height="22" /> Displays when Total Fire Ban in force<br /><span><a href="http://www.cfa.vic.gov.au/warnings-restrictions/restrictions-during-fire-danger-periods">Restrictions may apply</a></span></p></description>
<guid>http://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</guid>
</item>
<item>
<title>Sunday, 02 March 2025</title>
<link>http://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</link>
<description><p>Sun, 2 Mar 2025 is not currently a day of Total Fire Ban.</p><p>Fire Danger Ratings<br/>Bureau of Meteorology forecast issued at: Thursday, 27 February 2025 05:30 AM</p><p>Central: MODERATE</p><p><img src="http://www.cfa.vic.gov.au/cfa/images/fdr/central/moderate.gif" alt="" border="0" /></p><p><img src="http://www.cfa.vic.gov.au/cfa/images/tfb_icon_big.png" alt="" width="22" height="22" /> Displays when Total Fire Ban in force<br /><span><a href="http://www.cfa.vic.gov.au/warnings-restrictions/restrictions-during-fire-danger-periods">Restrictions may apply</a></span></p></description>
<guid>http://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</guid>
</item>
<item>
<title>Monday, 03 March 2025</title>
<link>http://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</link>
<description><p>Mon, 3 Mar 2025 is not currently a day of Total Fire Ban.</p><p>Fire Danger Ratings</p><p>Central: NO RATING</p><p><img src="http://www.cfa.vic.gov.au/cfa/images/fdr/central/norating.gif" alt="" border="0" /></p><p><img src="http://www.cfa.vic.gov.au/cfa/images/tfb_icon_big.png" alt="" width="22" height="22" /> Displays when Total Fire Ban in force<br /><span><a href="http://www.cfa.vic.gov.au/warnings-restrictions/restrictions-during-fire-danger-periods">Restrictions may apply</a></span></p></description>
<guid>http://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</guid>
</item>
<item>
<title>Fire restrictions by muncipality</title>
<link>http://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</link>
<description><p>BALLARAT:Restrictions in force: 25/11/2024 - 01/05/2025<br/>BANYULE:Restrictions in force: 25/11/2024 - 01/05/2025<br/>BASS COAST:Restrictions in force: 09/12/2024 - 01/05/2025<br/>BOROUGH OF QUEENSCLIFFE:Restrictions in force: 25/11/2024 - 01/05/2025<br/>CARDINIA:Restrictions in force: 09/12/2024 - 01/05/2025<br/>CASEY:Restrictions in force: 09/12/2024 - 01/05/2025<br/>FRANKSTON:Restrictions in force: 09/12/2024 - 01/05/2025<br/>FRENCH ISLAND:Restrictions in force: 09/12/2024 - 01/05/2025<br/>GOLDEN PLAINS:Restrictions in force: 25/11/2024 - 01/05/2025<br/>GREATER DANDENONG:Restrictions in force: 09/12/2024 - 01/05/2025<br/>GREATER GEELONG:Restrictions in force: 25/11/2024 - 01/05/2025<br/>HEPBURN:Restrictions in force: 25/11/2024 - 01/05/2025<br/>HUME:Restrictions in force: 18/11/2024 - 01/05/2025<br/>KINGSTON:Restrictions in force: 09/12/2024 - 01/05/2025<br/>KNOX:Restrictions in force: 23/12/2024 - 01/05/2025<br/>MACEDON RANGES:Restrictions in force: 25/11/2024 - 01/05/2025<br/>MANNINGHAM:Restrictions in force: 23/12/2024 - 01/05/2025<br/>MAROONDAH:Restrictions in force: 23/12/2024 - 01/05/2025<br/>MELTON:Restrictions in force: 18/11/2024 - 01/05/2025<br/>MOORABOOL:Restrictions in force: 25/11/2024 - 01/05/2025<br/>MORNINGTON PENINSULA:Restrictions in force: 09/12/2024 - 01/05/2025<br/>NILLUMBIK:Restrictions in force: 25/11/2024 - 01/05/2025<br/>SURF COAST:Restrictions in force: 25/11/2024 - 01/05/2025<br/>WHITTLESEA:Restrictions in force: 18/11/2024 - 01/05/2025<br/>WYNDHAM:Restrictions in force: 18/11/2024 - 01/05/2025<br/>YARRA RANGES:Restrictions in force: 23/12/2024 - 01/05/2025<br/></p></description>
<guid>http://www.cfa.vic.gov.au/warnings-restrictions/total-fire-bans-and-ratings/central-fire-district</guid>
</item>
</channel>
</rss>

I would like to build a table that shows for each day - whether it is a total fire ban or not and the fire rating. Something like this

Thursday, 27 February 2025      No Fire Ban      Moderate
Friday, 28 February 2025      No Fire Ban      Moderate
Saturday, 1 March 2025      No Fire Ban      Moderate
Sunday, 2 March 20255      No Fire Ban      Moderate
Monday, 3 March 2025      No Fire Ban      Moderate

So Sensors created like
TFB_Date0 TFB_Ban0 TFB_Rating0
TFB_Date1 TFB_Ban1 TFB_Rating1
TFB_Date2 TFB_Ban2 TFB_Rating2
TFB_Date3 TFB_Ban3 TFB_Rating3
TFB_Date4 TFB_Ban4 TFB_Rating4

The values in the XML file - literals are dates, Ban is either “is not currently a day of Total Fire Ban” or can be ‘day of Total Fire ban", and Rating can be “No Rating”, “Moderate”, “High”, "Extreme’, Catastrophic" - but with the ratings and Bans I would just take the literal string of whatever is there…

Hints / Solutions on getting this done - I actually want to get two regions scraped from 2 different regions and present it all together, but getting one done, I am sure I can get it all put together and presented nicely.

5

Got it pretty much working except for one thing: I’m not exactly sure how to detect whether a day has a fire ban or not, as right now all days are described as “not currently a day of Total Fire Ban”. Right now I guess the easiest solution is to look for the presence of that exact string, or do you know exactly what it says when there is a fire ban in place?

6

Here is an updated sensor configuration.

Changed some things around since we now need a bit more data but still from the same RSS feed, and didn’t want to add surplus sensors. After inserting this code and reloading configuration you will need to manually remove the old entities in Settings > Devices & services > Entities.

Also made it so that the feed sensor updates every hour. Instead of its timestamp showing when the last request was made, it pulls the update timestamp from when the feed was last updated.

I choose to put both the new data points as attributes on the same sensor, as that way all values can be presented sequentially as a list. Fire ban is a boolean where true means there is a fire ban, and today’s value is also reflected as the sensors state and icon.

Should be very easy to add start/end sensors for additional municipalities, or an additional fire district provided they use the same data format.

rest:
- resource: https://www.cfa.vic.gov.au/cfa/rssfeed/central-firedistrict_rss.xml
  scan_interval: 3600
  sensor:
  - name: Central Fire District
    unique_id: e6290bff-5bd9-4a0d-8355-cc8f22df5295
    icon: mdi:fire-alert
    device_class: timestamp
    force_update: true
    value_template: "{{ value_json['rss']['channel']['dc:date'] | as_datetime | as_local }}"
    json_attributes_path: "$.rss.channel"
    json_attributes:
    - title
    - item
    - link

template:
- trigger:
  - trigger: state
    entity_id: sensor.central_fire_district
  action:
  - variables:
      fire_ban: >-
        {{ state_attr('sensor.central_fire_district', 'item')[:5]
        | map(attribute='description')
        | map('regex_findall', ' is not currently a day of Total Fire Ban.</p>')
        | map('iif', false, true) | list }}
      danger_ratings: >-
        {{ state_attr('sensor.central_fire_district', 'item')[:5]
        | map(attribute='description')
        | map('regex_findall', '<p>Central: (.+?)</p>')
        | map('first') | map('title') | list }}
      fire_restrictions: >-
        {% set fire_restrictions = namespace(by_municipality=[]) %}
        {% for description in state_attr('sensor.central_fire_district', 'item')[-1]['description']
        .removeprefix('<p>').removesuffix('</p>').split('<br/>') if description %}
          {% set municipality, daterange = description.split(':Restrictions in force: ') %}
          {% set fire_restrictions.by_municipality = fire_restrictions.by_municipality +
          [(municipality | lower, daterange.split(' - ')
          | map('regex_replace', '(\d\d)\/(\d\d)\/(\d\d\d\d)', '\\3-\\2-\\1') | list)] %}
        {% endfor %}
        {{ dict(fire_restrictions.by_municipality) }}
  binary_sensor:
  - name: Central Fire District
    unique_id: c3c893ee-ddd0-42c0-9a4c-22be6764a5e4
    availability: "{{ has_value('sensor.central_fire_district') }}"
    state: "{{ state_attr(this.entity_id, 'fire_ban')[0] }}"
    icon: mdi:fire{{ '-off' if state_attr(this.entity_id, 'fire_ban')[0] }}
    attributes:
      fire_ban: "{{ fire_ban }}"
      danger_ratings: "{{ danger_ratings }}"
  sensor:
  - name: Hepburn Fire Restrictions Start
    unique_id: 72aec94e-ca70-4c7c-81c4-925e6b92dac8
    icon: mdi:fire-off
    device_class: timestamp
    availability: "{{ has_value('sensor.central_fire_district') }}"
    state: "{{ fire_restrictions['hepburn'][0] | as_datetime | as_local }}"
  - name: Hepburn Fire Restrictions End
    unique_id: 8de8c724-2db0-4e1c-a0ed-b51bbac2315e
    icon: mdi:fire
    device_class: timestamp
    availability: "{{ has_value('sensor.central_fire_district') }}"
    state: >-
      {{ fire_restrictions['hepburn'][-1] | as_datetime | as_local
      + timedelta(days=1, seconds=-1) }}