Not possible to use REST sensor to read large json with no parent name?

[SadGamerGeek (Richard Harnwell)]

After hitting some limitations with the existing worldtidesinfo option, I was pleased to find a UK based tides API at https://www.admiralty.co.uk/ that’s free and meets my needs. I’m able to pull back the required json with curl or Python, but I don’t seem to be able to extract this with the REST sensor, I believe because the data doesn’t have a name at the parent level:

[{
  "EventType": "HighWater",
  "DateTime": "2021-01-05T04:04:00",
  "IsApproximateTime": false,
  "Height": 5.0115273457652361,
  "IsApproximateHeight": false,
  "Filtered": false,
  "Date": "2021-01-05T00:00:00"
}, {
  "EventType": "LowWater",
  "DateTime": "2021-01-05T10:35:00",
  "IsApproximateTime": false,
  "Height": 0.57660918404826356,
  "IsApproximateHeight": false,
  "Filtered": false,
  "Date": "2021-01-05T00:00:00"
}, {
  "EventType": "HighWater",
  "DateTime": "2021-01-05T16:43:00",

The data is too large to fit in a sensor state (larger than 255 characters) and I can’t find any suitable json_attributes parameter to capture this. Is there a way to do this without having to revert to a command line sensor?

[tom_l]

Similar issue solved today:

REST sensor: Where "x" equals "Y"

There is a way : - platform: rest resource: http://10.0.24.11:61208/api/3/processlist name: Glances - DeCONZ - CPU - Process value_template: '{{ "ok" }}' json_attributes_path: '$..[?(@.name=="deCONZ")]' json_attributes: - name - status - username

[SadGamerGeek (Richard Harnwell)]

Thanks for that. Unfortunately I can’t seem to get anything like that to work for me. I suspect my json needs something a little different. Based on that answer I went with:

  - platform: rest
    name: rest_tides_admiralty
    resource: https://admiraltyapi.azure-api.net/uktidalapi/api/V1/Stations/0124/TidalEvents?duration=7
    headers:
      Authorization: !secret api_key_admiralty
    value_template: '1'  # dummy value, not used; avoids the "State max length is 255 characters" error
    json_attributes_path: '$..[?(@.EventType=="HighWater")]'
    json_attributes: 
      - EventType
      - Height

… as $…[?(@.EventType==“HighWater”)] looked good when I tried it in https://jsonpath.com/

Unfortunately I get no attributes set, and see this warning in the log:

JSON result was not a dictionary or list with 0th element a dictionary

[gartenbauer (Marc)]

@SadGamerGeek

… as $…[?(@.EventType==“HighWater”)] looked good when I tried it in https://jsonpath.com/

“EventType” is the parameter you are filtering, because of this it is not possible so set this a as an attribute in HA.
What are you trying to do? Can you past me the output on “ressource” on eg. https://pastebin.ubuntu.com/?

[SadGamerGeek (Richard Harnwell)]

Thanks very much. I have also tried without this attribute:

sensor:
  - platform: rest
    name: rest_tides_admiralty
    resource: https://admiraltyapi.azure-api.net/uktidalapi/api/V1/Stations/0124/TidalEvents?duration=7
    headers:
      Authorization: !secret api_key_admiralty
    value_template: '1'  # dummy value, not used; avoids the "State max length is 255 characters" error
    json_attributes_path: '$..[?(@.EventType=="HighWater")]'
    json_attributes: 
      - DateTime
      - Height

but also got that same error.

The resource output is small enough to fit it in the post (though also in pastebin if that’s easier):

[
    {
        "Date": "2021-01-06T00:00:00",
        "DateTime": "2021-01-06T04:38:39.3",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 4.5645572811437045,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-06T00:00:00",
        "DateTime": "2021-01-06T10:57:45",
        "EventType": "LowWater",
        "Filtered": false,
        "Height": 0.6164370982704671,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-06T00:00:00",
        "DateTime": "2021-01-06T17:11:43.217",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 4.530521927172035,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-06T00:00:00",
        "DateTime": "2021-01-06T23:05:37.5",
        "EventType": "LowWater",
        "Filtered": false,
        "Height": 1.2180845949919092,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-07T00:00:00",
        "DateTime": "2021-01-07T05:31:33.333",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 4.4727851870133435,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-07T00:00:00",
        "DateTime": "2021-01-07T12:05:07.5",
        "EventType": "LowWater",
        "Filtered": false,
        "Height": 0.6464246556611379,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-07T00:00:00",
        "DateTime": "2021-01-07T18:11:13.333",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 4.387457911198173,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-08T00:00:00",
        "DateTime": "2021-01-08T00:20:52.5",
        "EventType": "LowWater",
        "Filtered": false,
        "Height": 1.2424560539342702,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-08T00:00:00",
        "DateTime": "2021-01-08T06:38:13.333",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 4.405241312964828,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-08T00:00:00",
        "DateTime": "2021-01-08T13:19:22.5",
        "EventType": "LowWater",
        "Filtered": false,
        "Height": 0.6664784228584787,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-08T00:00:00",
        "DateTime": "2021-01-08T19:26:46.667",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 4.328524879681343,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-09T00:00:00",
        "DateTime": "2021-01-09T01:43:00",
        "EventType": "LowWater",
        "Filtered": false,
        "Height": 1.1919323398634305,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-09T00:00:00",
        "DateTime": "2021-01-09T07:56:20",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 4.436229325981978,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-09T00:00:00",
        "DateTime": "2021-01-09T14:31:22.5",
        "EventType": "LowWater",
        "Filtered": false,
        "Height": 0.6893275961729552,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-09T00:00:00",
        "DateTime": "2021-01-09T20:42:46.667",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 4.395592077530186,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-10T00:00:00",
        "DateTime": "2021-01-10T03:01:45",
        "EventType": "LowWater",
        "Filtered": false,
        "Height": 1.077913796529528,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-10T00:00:00",
        "DateTime": "2021-01-10T09:11:16.667",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 4.605305470483266,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-10T00:00:00",
        "DateTime": "2021-01-10T15:42:15",
        "EventType": "LowWater",
        "Filtered": false,
        "Height": 0.7196127145963841,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-10T00:00:00",
        "DateTime": "2021-01-10T21:52:26.667",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 4.562212089460421,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-11T00:00:00",
        "DateTime": "2021-01-11T04:16:00",
        "EventType": "LowWater",
        "Filtered": false,
        "Height": 0.9198850012712186,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-11T00:00:00",
        "DateTime": "2021-01-11T10:15:40",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 4.84784122695461,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-11T00:00:00",
        "DateTime": "2021-01-11T16:49:45",
        "EventType": "LowWater",
        "Filtered": false,
        "Height": 0.7449087740503451,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-11T00:00:00",
        "DateTime": "2021-01-11T22:53:40",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 4.756685518327773,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-12T00:00:00",
        "DateTime": "2021-01-12T05:25:45",
        "EventType": "LowWater",
        "Filtered": false,
        "Height": 0.7416727994898556,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-12T00:00:00",
        "DateTime": "2021-01-12T11:13:43.333",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 5.0724574536621,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-12T00:00:00",
        "DateTime": "2021-01-12T17:52:45",
        "EventType": "LowWater",
        "Filtered": false,
        "Height": 0.7645267756405048,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    },
    {
        "Date": "2021-01-12T00:00:00",
        "DateTime": "2021-01-12T23:47:30",
        "EventType": "HighWater",
        "Filtered": false,
        "Height": 4.921005217884829,
        "IsApproximateHeight": false,
        "IsApproximateTime": false
    }
]

My end goal is to get a list of upcoming high tides with time & height, so I need all the “DateTime”, and “Height” entries where “EventType” is “HighWater”, in a form where I can then display them in some sort of table layout.

I’d already got this working and displaying perfectly using an alternative tides source of https://api.stormglass.io, so it’s probably easiest to explain my goal by showing what I already had working. The stormglass provider gives me json like this:

{
    "data": [
        {
            "height": 0.6845618910459339,
            "time": "2021-01-06T12:18:00+00:00",
            "type": "low"
        },
        {
            "height": 4.426741654879187,
            "time": "2021-01-06T18:02:00+00:00",
            "type": "high"
        },
        {
            "height": 1.3619747293854356,
            "time": "2021-01-07T00:44:00+00:00",
            "type": "low"
        },

… so I’d been able to read it all into a “data” attribute like this:

  - platform: rest
    name: rest_tides
    resource: https://api.stormglass.io/v2/tide/extremes/point?lat=51.778160&lng=0.928929&datum=MLLW
    headers:
      Authorization: !secret api_key_stormglass
    value_template: '1'  # dummy value, not used; avoids the "State max length is 255 characters" error
    json_attributes: 
      - 'data'

Which looked like this:

I then used used a markdown sensor with lots of template commands to extract the high tide info and format it in a simple table, to give me the output exactly as I wanted:

Unfortunately I then realised this was inaccurate, as the provider didn’t have a “station” that was close enough to the location I was interested in. My new “admiralty” provider has lots more stations and one in the exact location I need.

I’m probably suffering from being very unfamiliar with json in general - this is my first time using it. I think the problem is that the above “stormglass” api I got working is providing a top level dictionary (“data” : [) , so I can just reference that “data” to pull out, whereas this new admiralty api is just a load of list elements and I have nothing to reference…

[gartenbauer (Marc)]

@SadGamerGeek

providing a top level dictionary (“data” : [) , so I can just reference that “data” to pull out

You are probably right. Sorry that I was not a great help to you…

Maybe you want to learn python and make a script for this sensor.

[SadGamerGeek (Richard Harnwell)]

Thanks for looking.

I have got this working in a very simply python script - luckily the python knowledge required was very small! This was going to be my backup plan if I couldn’t get the rest sensor to work. In fact it’s working on a different environment, but when I put it on my HA box (supported NUC image), I couldn’t work out where the python executable was to reference - there is no python available in the path. Do you know where it lives, or is it outside of the HA container?

[gartenbauer (Marc)]

@SadGamerGeek

Do you know where it lives, or is it outside of the HA container?

No unfortunetly not!

[jasonkjennings]

Hey there, did you ever figure this out?