Parse a nested JSON with timeseries data using the RESTful integration

Configuration
1

I’d be able to do this easily using pure Python, but I want to use Home Assistant to save the data into the DB and track it over time.

The JSON looks something like this

{
   "creation_forecast": "2023-12-22 07:42:46+01:00",
   "curve_forecast": {
      "data": [
         {
            "timestamp": "2023-12-21 22:00:00+01:00",
            "astro": 651,
            "curveforecast": null,
            "measurement": 938
         },
         [...]
         {
            "timestamp": "2023-12-22 11:50:00+01:00",
            "astro": 726,
            "curveforecast": 999,
            "measurement": null
         }
         [...]
      ]
   }
}

The […] indicates that there are a lot of values in between. In fact this array is quite long and represents an object similar to what a weather forecast API would return.

I tried to add this in my configuration.yaml

rest:
  - resource: https://url_to/file.json
    scan_interval: 900
    sensor:
      - name: "Forecast"
        value_template: >-
          {% for index in value_json["curve_forecast"]["data"] -%}
            {{ index }}
          {%- endfor %}.
        json_attributes:
          - "timestamp"
          - "astro"
          - "curveforecast"
          - "measurement"

But I always get this error homeassistant.exceptions.InvalidStateError: Invalid state with length 157668. State max length is 255 characters.

2

You’re putting too much information into value_template. If you want the count to be the state…

        value_template: "{{ value_json.curve_forecast.data | length }}"

Right, you’re putting every index into the main state. So if your data is 500 listed items, your main state will be

0123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500

instead of 500

3

Ok I understand why now.
This works, but it doesn’t parse the attributes but only set the status with the length of the array.

Screenshot 2023-12-22 at 13.42.31
Screenshot 2023-12-22 at 13.42.311154×312 14 KB

Is there a way to parse the attributes in a way they are defined as a weather forecast-like entity? Do I have to maybe use this Template Weather Provider - Home Assistant?

4

You’d have to make a template weather entity for that. The rest integration does not support templating the attributes.

5

Ok but first of all I need to store the response JSON of data somewhere in Home Assistant, right? Otherwise I cannot use it into a template weather entity.

Unfortunately the response is too large to be stored into an attribute. I tried doing this

    sensor:
      - name: "forecast"
        value_template: "{{ value_json.curve_forecast.data | length }}"
        json_attributes_path: "$.curve_forecast"
        json_attributes:
          - "data"

but I get the obvious warning

WARNING (Recorder) [homeassistant.components.recorder.db_schema] State attributes for sensor.forecast exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored

So, I’m not sure how this could be tackled with the default integrations. The idea I had at the beginning was to write my own Python script to run with pyscript-hacs, organize the results and save them into a state.

6

Your json_attributes_path should be able to filter down to selected elements. There’s a whole syntax language for that.

7

https://goessner.net/articles/JsonPath/index.html#e2

8

I understand what you’re proposing, and this works well for a single element, for example

json_attributes_path: "$.curve_forecast.data[:1]"
json_attributes:
  - "timestamp"

But it cannot parse an array. If you remove the [:1] nothing get stored in the state attribute. I’m trying to parse the entire array into the attributes. This may just not be the ideal solution…I don’t know how the weather forecast is stored for a weather entity, but this is exactly what I’m trying to achieve.