Another 'How do I parse this JSON' question

Configuration
1

I thought I’d got on top of this.

I have rest sensor which is trying to parse some XML. I believe conversion to JSON is now automatic so why can’t I parse this correctly?

  - platform: rest
    name: Met Office South East Weather Warnings 
    unique_id: met_office_south_east_weather_warnings
    resource: 'http://metoffice.gov.uk/public/data/PWSCache/WarningsRSS/Region/se'
    value_template: >
      {{ value_json.rss.channel.pubDate }}
    json_attributes_path: value_json.rss.channel
    json_attributes:
      - item

This is the XML and for reference the JSON It (should) convert to is below that:

Thanks.

<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/">
    <channel>
        <title>Met Office warnings for London &amp; South East England</title>
        <link>https://www.metoffice.gov.uk/weather/warnings-and-advice/uk-warnings</link>
        <description>Weather warnings of severe and extreme weather from the Met Office</description>
        <language>en-gb</language>
        <copyright>(c) Crown copyright</copyright>
        <pubDate>Wed, 10 May 2023 10:04:53 GMT</pubDate>
        <dc:creator>[email protected]</dc:creator>
        <item>
            <title>Yellow warning of thunderstorm affecting London &amp; South East England</title>
            <link>https://www.metoffice.gov.uk/weather/warnings-and-advice/uk-warnings#?date=2023-05-10&amp;id=83993be1-fbe2-4a1c-9f3d-c371a454e169&amp;referrer=rss</link>
            <description>Yellow warning of thunderstorm affecting London &amp; South East England: Bracknell Forest, Buckinghamshire, East Sussex, Greater London, Hampshire, Kent, Medway, Milton Keynes, Oxfordshire, Reading, Slough, Surrey, West Berkshire, West Sussex, Windsor and Maidenhead, Wokingham valid from 1200 Wed 10 May to 1900 Wed 10 May</description>
            <guid isPermaLink="false">https://www.metoffice.gov.uk/weather/warnings-and-advice/uk-warnings#?date=2023-05-10&amp;id=83993be1-fbe2-4a1c-9f3d-c371a454e169&amp;referrer=rss&amp;region=London &amp; South East England</guid>
            <enclosure length="109337" type="image/png" url="https://cdn.prod.weathercloud.metoffice.gov.uk/warnings/rss/image/yellow-thunderstorm.png"/>
        </item>
    </channel>
</rss>

{
  "rss": {
    "channel": {
      "title": "Met Office warnings for London & South East England",
      "link": "https://www.metoffice.gov.uk/weather/warnings-and-advice/uk-warnings",
      "description": "Weather warnings of severe and extreme weather from the Met Office",
      "language": "en-gb",
      "copyright": "(c) Crown copyright",
      "pubDate": "Wed, 10 May 2023 10:04:53 GMT",
      "creator": {
        "__prefix": "dc",
        "__text": "[email protected]"
      },
      "item": [
        {
          "title": "RED warning of thunderstorm affecting London & South East England",
          "link": "https://www.metoffice.gov.uk/weather/warnings-and-advice/uk-warnings#?date=2023-05-10&id=83993be1-fbe2-4a1c-9f3d-c371a454e169&referrer=rss",
          "description": "Yellow warning of thunderstorm affecting London & South East England: Bracknell Forest, Buckinghamshire, East Sussex, Greater London, Hampshire, Kent, Medway, Milton Keynes, Oxfordshire, Reading, Slough, Surrey, West Berkshire, West Sussex, Windsor and Maidenhead, Wokingham valid from 1200 Wed 10 May to 1900 Wed 10 May",
          "guid": {
            "_isPermaLink": "false",
            "__text": "https://www.metoffice.gov.uk/weather/warnings-and-advice/uk-warnings#?date=2023-05-10&id=83993be1-fbe2-4a1c-9f3d-c371a454e169&referrer=rss&region=London & South East England"
          },
          "enclosure": {
            "_length": "109337",
            "_type": "image/png",
            "_url": "https://cdn.prod.weathercloud.metoffice.gov.uk/warnings/rss/image/red-thunderstorm.png"
          }
        },
        {
          "title": "Yellow warning of thunderstorm affecting London & South East England",
          "link": "https://www.metoffice.gov.uk/weather/warnings-and-advice/uk-warnings#?date=2023-05-10&id=83993be1-fbe2-4a1c-9f3d-c371a454e169&referrer=rss",
          "description": "Yellow warning of thunderstorm affecting London & South East England: Bracknell Forest, Buckinghamshire, East Sussex, Greater London, Hampshire, Kent, Medway, Milton Keynes, Oxfordshire, Reading, Slough, Surrey, West Berkshire, West Sussex, Windsor and Maidenhead, Wokingham valid from 1200 Wed 10 May to 1900 Wed 10 May",
          "guid": {
            "_isPermaLink": "false",
            "__text": "https://www.metoffice.gov.uk/weather/warnings-and-advice/uk-warnings#?date=2023-05-10&id=83993be1-fbe2-4a1c-9f3d-c371a454e169&referrer=rss&region=London & South East England"
          },
          "enclosure": {
            "_length": "109337",
            "_type": "image/png",
            "_url": "https://cdn.prod.weathercloud.metoffice.gov.uk/warnings/rss/image/yellow-thunderstorm.png"
          }
        }
      ]
    },
    "_xmlns:dc": "http://purl.org/dc/elements/1.1/",
    "_version": "2.0"
  }
}
2

You should see a warning message “REST result could not be parsed as JSON” in your log file.

This automated conversion only works if the response’s content-type header is one of

  • text/xml
  • application/xml
  • application/xhtml+xml
  • application/rss+xml.

Your feed there returns content-type application/atom+xml so that’s why it’s not automatically converted to JSON. It might be worth raising a feature request to add this content-type for automated XML conversion.

1 Like
3

I hit that URL and I get this:

<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/">
    <channel>
        <title>Met Office warnings for London &amp; South East England</title>
        <link>https://www.metoffice.gov.uk/weather/warnings-and-advice/uk-warnings</link>
        <description>Weather warnings of severe and extreme weather from the Met Office</description>
        <language>en-gb</language>
        <copyright>(c) Crown copyright</copyright>
        <pubDate>Wed, 10 May 2023 19:00:50 GMT</pubDate>
        <dc:creator>[email protected]</dc:creator>
    </channel>
</rss>

So no <item>

4

Thanks, yes I might do that.

@kbrown01
Yeah the weather warning was over. That’s why I posted the XML being returned while it was in place.
(I saved it for testing)

5

This seems wrong to me:

json_attributes_path: value_json.rss.channel

Likely it needs quotes but I do not know

6

Agreed. In the past I have used this:

json_attributes_path: "$.rss.channel"
7

Just to be clear, none of those suggestions will help because the rest integration is simply not converting that particular XML feed into JSON.

1 Like
8

Understood now. BUT … instead of waiting for someone you could certainly install “xq” in your environment. “jq” is already installed with Home Assistant.

Then read up here:

You could likely change to a command_line sensor with “curl” get the result and parse through the result. I use “xq” all the time to reorganize JSON coming in to something more palatable and that works great. Using “xq” is just a front end that will do XML to JSON and pass that JSON to xq for further processing.