Home Assistant and Fingrid open data API for energy shortage forecast, templating and curl?

honkkkis · December 17, 2022, 4:32pm

Hi,
our (rather) fair and decent national electric grid operator Fingrid (in Finland) gives away lot of information for free via open API (of course you paid it already in your electric bill). They have forecast for electricity shortage on three minute timeslices, output is XML or JSON, depending which one you want. Fingrid open API requires personal API token, but you can make 10000 requests a day, which should be enough for most. API and response use UTC-time and all times are formatted as YYYY-MM-DDThh:mm:ssZ

I managed to create sensor (curl / JSON) that returns single value for testing purposes:

  name: Fingrid forecast
  scan_interval: 3600
  command: "curl -X GET --header 'Accept: application/json' --header 'x-api-key: my_very_personal_api_key_goes_here' 'https://api.fingrid.fi/v1/variable/336/events/json?start_time=2022-12-17T06%3A00%3A00Z&end_time=2022-12-17T06%3A03%3A00Z'"

and response for three minute period (sensor state) is single value:

[{"value":0,"start_time":"2022-12-17T06:01:00+0000","end_time":"2022-12-17T06:01:00+0000"}]

Now, my questions:

how to pass starting time attribute with current time and end time as current time + 12h for example?
response has always “value”:0 if no blackouts are expected and other numeric values, if blackouts are expected, numeric state varies by severity. If there’s other state than 0, I’d like to pick first and last response with different value than 0 and their timestamps. I have no need for middle timestamps, because there will be electricity shortage. How to template this first and last value and their timestamps?

Thanks in advance, this may not be complicated, but I’m not expert in templates, even if can do basics.

EDIT:
Add some useful links for Fingrid API
General description on Fingrid API - Fingridin avoin data API - Fingridin avoin data)
URL generator / validator for Fingrid API Swagger UI (fingrid.fi)
Dataset descriptions and their IDs used for requests Dataset - Fingridin avoin data

honkkkis · December 17, 2022, 4:36pm

Here is example of complete response body with no expected blackouts (all value: 0):

[
  {
    "value": 0,
    "start_time": "2022-12-17T06:01:00+0000",
    "end_time": "2022-12-17T06:01:00+0000"
  },
  {
    "value": 0,
    "start_time": "2022-12-17T06:04:00+0000",
    "end_time": "2022-12-17T06:04:00+0000"
  },
  {
    "value": 0,
    "start_time": "2022-12-17T06:07:00+0000",
    "end_time": "2022-12-17T06:07:00+0000"
  },
  {
    "value": 0,
    "start_time": "2022-12-17T06:10:00+0000",
    "end_time": "2022-12-17T06:10:00+0000"
  },
  {
    "value": 0,
    "start_time": "2022-12-17T06:13:00+0000",
    "end_time": "2022-12-17T06:13:00+0000"
  }
]

vingerha · December 17, 2022, 5:52pm

You can try this… and if this is what you want then you can create multiple sensors with 1 resource using restful
RESTful - Home Assistant (home-assistant.io)

sensor:
  - platform: rest
    name: testing
    scan_interval: 3600
    resource_template: "https://api.fingrid.fi/v1/variable/336/events/json?start_time={{ (now()-timedelta(hours=4)).strftime('%Y-%m-%dT%H:%M:%SZ') }}&end_time={{ (now()+timedelta(hours=12)).strftime('%Y-%m-%dT%H:%M:%SZ') }}"
    value_template: "{{ value_json.0.value }}"
    json_attributes_path: $.0
    json_attributes:
      - value
      - start_time
      - end_time

vingerha · December 17, 2022, 6:05pm

That is not an answer to your 2nd question though… I would know how to do this (iterating) but not sure how I can parse the whole response in one attribute (as json). Rest does not allow much on the attributes side

EDIT: maybe you should indeed use your curl and creata file and from that one use this…maybe others will chime in

For the non-0 value you can paste this in developer tools > templates, which will give you the first…and if you change it all at the bottom to ‘last’ … guess

{% set value_json = [
  {
    "value": 0,
    "start_time": "2022-12-17T06:01:00+0000",
    "end_time": "2022-12-17T06:01:00+0000"
  },
  {
    "value": 2,
    "start_time": "2022-12-17T06:04:00+0000",
    "end_time": "2022-12-17T06:04:00+0000"
  },
  {
    "value": 0,
    "start_time": "2022-12-17T06:07:00+0000",
    "end_time": "2022-12-17T06:07:00+0000"
  },
  {
    "value": 1,
    "start_time": "2022-12-17T06:10:00+0000",
    "end_time": "2022-12-17T06:10:00+0000"
  },
  {
    "value": 0,
    "start_time": "2022-12-17T06:13:00+0000",
    "end_time": "2022-12-17T06:13:00+0000"
  }
] %}
  {% set y = value_json | count %}
  {% set ns = namespace(val=[]) %}
  {% for x in range(0,y) %}
  {% if value_json[x].value != 0 %}
  {% set ns.val = ns.val + [value_json[x]] %}
  {% endif %}
  {% endfor %}
  {{ns.val | first}}

vingerha · December 17, 2022, 7:28pm

Final message for the day…I found a way to read the whole feedback into a sensor attrib
This you can then query alike what I earlier posted

  - platform: command_line
    name: testevents
    scan_interval: 1500
    command: >
         echo "{\"events\":" $(
         curl 
         -s 
         'https://api.fingrid.fi/v1/variable/336/events/json?start_time={{ (now()-timedelta(hours=4)).strftime('%Y-%m-%dT%H:%M:%SZ') }}&end_time={{ (now()+timedelta(hours=12)).strftime('%Y-%m-%dT%H:%M:%SZ') }}'
         ) "}" 
    value_template: > 
        {{ value_json.events | length }}
    json_attributes:
        - events

Temez · December 18, 2022, 10:48am

Great to notice that you are interested in this topic! I’m hoping that literally everyone (who has extra resources to think about this - some have enough burden already from inflation etc.) would give a thought to these things.

Some notes about possible improvements:

Fingrid API expects timestamps to be in UTC. So utcnow() would most probably give more appropriate results.
If your start_time is in the past, then value_template: “{{ value_json.0.value }}” returns the first result. Which is 4h ago due to timedelta (or maybe just -2h as the original poster most probably lives in Finland in timezone UTC+2).

You can hijack XML version of the API with Restful sensor to get all values to the attributes. An example below to get values from the last 60 minutes with the most new one in the sensor state.

sensor:
  - platform: rest
    resource_template: https://api.fingrid.fi/v1/variable/336/events/xml
    name: Fingrid Electricity Shortage
    value_template: '{{ value_json.events.event[-1].value }}'
    headers:
      x-api-key: "CENSORED"
    params:
      start_time: "{{ (utcnow() - timedelta(minutes = 60)).strftime('%Y-%m-%dT%H:%M:%SZ') }}"
      end_time: "{{ (utcnow() + timedelta(days = 1)).strftime('%Y-%m-%dT%H:00:00Z') }}"
    json_attributes_path: "events"
    json_attributes:
      - "event"
    scan_interval: 180

Temez · December 18, 2022, 10:59am

I tried the API and it seems that the API https://api.fingrid.fi/v1/variable/336/events/json does not have forecast capability. It seems that it just has current value and historical values. I would love to get that forecast data also but it seems that the API does no have forecast. (Or have I missed something? Could be that also ).

I presume that your use case would have been to do something with expected blackout timestamps?

honkkkis · December 18, 2022, 11:11am

You may have it right about forecast. My idea was to provide early notification on expected blackout and do some load shedding, but without forecast, it will be just immediate load shedding on water heater and notification.

However, since Fingrid has announced blackouts are likely to be known couple hours in advance, they will probably rise blackout status immediately, maybe I should do load shedding immediately when status differs from value: 0. That should be very easy, since I need to get single value only. Fingrid has followng values for energy situation:

0 = Normal
1 = Electricity shortage possible
2 = High risk of electricity shortage
3 = Electricity shortage

honkkkis · December 18, 2022, 11:12am

And thanks a lot for your effort vingerha!

Perttu · December 18, 2022, 7:40pm

I have done this simple Fingrid sensor that I use to scale down electricity usage (heating) based on status of the Fingrid network. It also uses the RESTFull api. Have not found asny other forecasting from the API though. GitHub - iivana/home-assistant-fingrid-api: Fingrid API RESTfull sensor examples for home assistant

honkkkis · December 20, 2022, 4:40pm

That API variable 209 might actually be even more useful than 336 I’m using currently. 209 includes more detailed information about grid state.

honkkkis · May 11, 2024, 8:58am

Since last post, Fingrid has reworked their API. And HA has also changed somewhat. Here’s updated code, having separate config files, this is my sensor.yaml:

- platform: rest
  resource_template: https://data.fingrid.fi/api/datasets/336/data/latest
  name: Fingrid Electricity Shortage
  headers:
    x-api-key: "yourverypersonalkeylistedinFingrid"
  json_attributes:
    - "value"
  scan_interval: 180
  
- platform: rest
  resource_template: https://data.fingrid.fi/api/datasets/209/data/latest
  name: Fingrid Power System State
  headers:
    x-api-key: "yourverypersonalkeylistedinFingrid"
  json_attributes:
    - "value"
  scan_interval: 180

This ia working one in 11.5.2024, dataset IDs 336 and 209 haven’t changed.

And worth noting, since this is public service offered, even if you need to register to use it, it’s completely free to use. Fingrid provides wide variety of information about Finland’s national grid’s state, electricity production and connections to neighboring states, their API is real gem for everyone wihing to show this kind of information in their HA setups!