Reading energy tarif forecast from RESTful

StephaneCherpilloz · January 12, 2025, 9:48pm

Hello,

I’m trying to get a forecast of energy rates for the next day. They are available on the URL I indicate below, at the latest every day at 18:00. My automation to trigger the reading works, at 18:06 every day.

I think I understand how to read the data, but I don’t know how to present it to Home Assistant. In the general case, I’d like to fill a data structure with these rates to be able to query it later. Ideally, I’d also like to delete data older than a year.

For example, I’d like to be able to look up the 4 cheapest hours of the next night (usually from 1:00 to 5:00) and trigger actions at 1:00 and 5:00 (allowing the water heater to be switched on or off).

Even before writing code, I don’t see how a sensor can present values at times that don’t yet exist.

Thanks in advance for your guidance!

URL:
https://api.tariffs.groupe-e.ch/v1/tariffs/vario_plus?start_timestamp=2025-01-12T00:00:00+01:00&end_timestamp=2025-01-13T00:00:00+01:00

returns:

[
    {
        "start_timestamp": "2025-01-12T00:00:00+01:00",
        "end_timestamp": "2025-01-12T00:15:00+01:00",
        "vario_plus": 25.9,
        "unit": "Rp./kWh"
    },
    {
        "start_timestamp": "2025-01-12T00:15:00+01:00",
        "end_timestamp": "2025-01-12T00:30:00+01:00",
        "vario_plus": 26.21,
        "unit": "Rp./kWh"
    },
[...]
    {
        "start_timestamp": "2025-01-12T23:30:00+01:00",
        "end_timestamp": "2025-01-12T23:45:00+01:00",
        "vario_plus": 26.7,
        "unit": "Rp./kWh"
    },
    {
        "start_timestamp": "2025-01-12T23:45:00+01:00",
        "end_timestamp": "2025-01-13T00:00:00+01:00",
        "vario_plus": 25.64,
        "unit": "Rp./kWh"
    }
]

vingerha · January 13, 2025, 6:52am

My 2 cents
This does already exist, e.g. the weather forecast.
Thing is that this is stored in attributes and attributes fafaik do not end up in long term statistics so even if you load them (see below), you will not have these available longer than the recorder set cleanup (default 10d)… which blocks your wish to have them for about 1 year.
Extending the recorder just-for-this seems overkill as it will considerably grow the database since it will not be just-for-this.
Storing it externally (file) and adding with each call is also an option but at some point the attributes will likely grow beyond a length readable in HA

EDIT: maybe you can create other sensors based on this which do get stored in LTS?

command_line:
  - sensor: 
        name: test
        scan_interval: 1500
        command: >
             echo "{\"YOURATTRIBUTE\":" $(
             curl 
             -s 
             'https://api.tariffs.groupe-e.ch/v1/tariffs/vario_plus?start_timestamp=2025-01-12T00:00:00+01:00&end_timestamp=2025-01-13T00:00:00+01:00 '
             ) "}" 
        value_template: > 
            {{ value_json.YOURATTRIBUTE | length }}
        json_attributes:
            - YOURATTRIBUTE

Troon · January 13, 2025, 7:54am

This long topic might help you:

Octopus Agile is a UK tariff that varies half-hourly: there’s discussion on there of ways to find the cheapest n-block times.

As a starting point for your RESTful sensor, this code should return the JSON list sorted by price (must be an easier way but a straight dictsort doesn’t seem to work):

{% set ns = namespace(indexed={},out=[]) %}
{% for i in range(value_json|count) %}
  {% set s = value_json[i]['vario_plus']|string %}
  {% if ns.indexed.get(s,'no') == 'no' %}
    {% set ns.indexed = dict(ns.indexed, **{s:[i]}) %}
  {% else %}
    {% set ns.indexed = dict(ns.indexed, **{s:ns.indexed[s] + [i]}) %}
  {% endif %}
{% endfor %}
{% set d = ns.indexed|dictsort %}
{{ d }}
{% for i in d %}
  {% for j in i[1] %}
    {% set ns.out = ns.out + [value_json[j]] %}
  {% endfor %}
{% endfor %}
{{ ns.out }}

Your cheapest 15-minute block is then on the top of the list.

To find bigger blocks of time, you’d need to combine them together then sort. I did this with AppDaemon here.

Biscuit · January 13, 2025, 12:16pm

Yes, Home Assistant is great at what it does, which is being a state model and an ‘easy-to-use’ automation engine based on events.

What HA is not designed to do is capture, process, store and display lots of user data, particularly arrays.

Welcome to Node-RED (available as an HA add-on) which is a programming language with the ability to easily work with data.

OK, so Node-RED is not that popular, and it is YMTL (Yet More To Learn) but hey, it can be really easy to put something together in just 10-20 minutes…

The http node does the API call and returns a JSON object.
The change node processes it in JSONata

(
    $hours:=5;
    $array:=payload^(vario_plus)[[0..$hours*4-1]]^(start_timestamp);
    $array:=$array#$v.(
        $last:= $v<1 ? false : start_timestamp=$array[$v-1].end_timestamp;
        $next:= end_timestamp=$array[$v+1].start_timestamp;
        $position:= $last ?  $next ? "middle" : "end" : $next ? "start" : "only";
        $~>|$|{"index": $v, "link_last": $last, "link_next": $next, "position": $position}|
        );
    $switching:=$array[position!="middle"].(
        $append(position!="end"   ? {"switch": "ON", "at": start_timestamp},
                position!="start" ? {"switch": "OFF", "at": end_timestamp})
    );
    {"max_price": $max($array.vario_plus),
     "min_price": $min($array.vario_plus),
     "array": $array, "switching": $switching}
)

This sorts by price, takes (5) hours of 15 minute best-price periods, re-sorts by time, identifies contiguous periods so every period is either start/middle/end/singleton, picks out the group starts and ends, and creates an array of switch-on and switch-off times [the cheapest block periods].

With a bit more Node-RED you can store this into context or file, and easily pass the array of prices and/or best periods and/or switching times back to Home Assistant inside the attribute of a sensor, for display, graphing, or further automation.

Here is a binary sensor (for switching automations) and I am passing more data in the attributes, including the array of best-time blocks.

Of course, you may find that your energy company hold this tariff data in one large file, and you can request either the latest 24 hours, or as you are using, with timestamps you can pick just what you want. There is, therefore, no need in this situation to store pricing data, however with Node-RED you can always save data to file/data base as you wish.