Create new datetime that is datetime + 1 hour from object

I pull data from an API for dynamic energy tarrifs. Every hour has a startsAt-property in this format: 2024-10-18T10:00:00.000+02:00

But I am missing the end time. I want to create calendar events from these objects to trigger a few energy related things. I have been breaking my mind about this and searching for a very long time now to get this done in a lambda when creating the event by filling in the end time as “start time + 1hr”. But I have not been able to find anything. At least nothing that was an option to include in one row of jsonata with lambda. Also, documentation on jsonata on time objects is very limited so I assume the options are too. Is there any way or are my assumptions correct?

Another option I could think off is to create a function that for every object creates a new property endsAt which value is “startsAt + 1hr”. Then I can fully use JavaScript to first parse the datetime string, then add 1 hour (add 3600s to epoch value) and then create a datetime string, again with offset (which also seems to be a bit of a challenge if one wants that time offset included). Then add the value to the new property.

Something else would be to just replace part of the string. Very ugly if you ask me and maybe prone to error. But every datetime entry has the exact same amount of characters and every element (hour, minute, second and so on) is in exactly the same index everytime. So I could just add 59 minutes and 59 seconds to replace the 00:00 in the string. Should work but doesn’t feel right.

I am wondering what your ideas and opinions would be for the mentioned options. Has anyone got any other idea? I am trying to find the cleanest, simplest and easiest to understand option. I have been thinking and searching a lot but delay in my project is starting to become too much for this seemingly simple problem…

Example using JSONata with the moment library.

$moment("2024-10-18T10:00:00.000+02:00").add(1, 'hour')
1 Like

I found the moment library but I though I had to declare/import it. If I understand correctly that is not necessary? That will make things sooooo much easier, for other stuff too. I thought I’d have to use a function to import and use it. This will make my flow much cleaner, shorter and more efficient.

I will try the following when I got time to:

$moment($Object.startsAt).add(1, 'hour')

Just did a little test. Adding the hours work. The string format changes however. I will look up moment.js documentation and see if I can get the same string format as the source.

Adding .format() did the trick :smiley:

$moment($Object.startsAt).add(1, 'hour').format()

Time is a relative concept and very difficult to deal with in any computer language. Since you asked specifically about JSONata, I thought it worth a reply, as much for posterity since the ‘easiest and simplest’ answer is clearly to use the $moment() ‘function’.

Since the $moment() function is not a documented addition to JSONata, and since JSONata does not support methods in the way they are being used here, I am personally wary of using this in JSONata other than for the novelty it provides.

JSONata has very little date-time support. Since it runs under JavaScript (as does Node-RED) the underlying provision comes from JS, and follows the use of Unix Epoch Millisecond time. Alongside this parsing from and to an ISO timestamp is permitted, hence

$now() will give "2024-10-22T09:50:38.248Z"
$millis() provides the Unix millisecond equivalent of ‘now’
$toMillis("2024-10-22T09:50:38.248Z") will parse the string to Unix time
$fromMillis($millis()) will turn Unix milliseconds to the ISO string, and is therefore equivalent to $now()

Since all of this is done in base UTC or Zulu time, it can be difficult to work in local time, however there are a couple of tricks that permit UTC to local time conversion. JSONata uses XPath JS support, and this includes the ability to format a timestamp to a given timezone. Our challenge is to first get the correct timezone string.

$round(($millis()-$toMillis($substringBefore($now(),".")))/3600000)

This will get the system timezone offset in integer hours, and works because of the perculiarity in JavaScript, whereby any ISO timestamp with a designated timezone (either Z or +02:00) will be correctly processed for the given UTC time, but any timestamp without a designated timezone will be processed for the given UTC time but based on the local time. Using the JSONata $substringBefore() function to strip off the ‘Z’ timezone desginator for UTC ‘now’ results in the UTC for now but with the local machine timezone offset subtracted. A bit of simple division and rounding gets the local timezone offset hour count.

Some string manipulation can also be used to extract the integer timezone hour offset from an ISO timestamp string, and to reverse this to a string +0200 as required for the later XPath formatting.

The $fromMillis() JSONata function permits an optional XPath formatting string, and further an optional timezone, hence it is possible to cast any UTC time to any format and timezone as required. To add one hour is done by adding the required number of milliseconds to the Unix time.

In terms of actually using JSONata, since this is a functional language everything is an expression, and most code would be written as a single line. However this is very restrictive, and code-blocks open up the possibility of writing functions, using variables, and making code visually easier to work with and debug. A JSONata code block can be used anywhere, and the following is a demonstration of how I would approach dealing with time. I use this myself to process energy tariffs and solar forecasts where I want to get an end time for an array of objects, and to do so using local time with code that is DST aware.

(
    /* example starting string... */
    $base:="2024-02-08T10:00:00.000+02:00";

    /* get system timezone offset in integer hours */
    $tzsys:=$round(($millis()-$toMillis($substringBefore($now(),".")))/3600000);
    

    /* function: ISO timestamp extract timezone offset in integer hours */
    $strtotz:=function($str){($match($str, /[+-]\d*:00/).match~>$replace("+","")~>$split(":"))[0].$number()};

    /* function: timezone integer hours to required string +0h00 */
    $tztostr:=function($tz){($tz>=0 ? "+" : "") & $formatNumber($tz, '00') & "00"};

    /* function: timestamp to timezone string as a composed function */
    $timezone:=$strtotz ~> $tztostr;

    /* function: add x mins to a unix ms integer and return ISO string to given timezone offset +hh00 */
    $addmins:=function($time, $mins, $tzstr){$fromMillis($time+$mins*60000,'[Y]-[M01]-[D01]T[H01]:[m01]:[s01].[f001][Z]' , $tzstr)};


    /* example - add 1 hour to 'now' and display in the base timezone */
    $addmins($millis(), 60, $timezone($base));

    /* take base timestamp and create an hourly array of times from local midnight */
    $start:=$toMillis($substringBefore($base,"T")& "T00:00:00.000" & $timezone($base));
    $array:=[0..23].{"Hour": $, "Start": $addmins($start, $*60, $timezone($base))};

    /* take an object of this array, and add end times for each item */
    $object:={"Times": $array};
    $object ~> |Times|{"Endat": $addmins($toMillis(Start), 60, $timezone(Start))}|;

)

I have included the code to get the local machine timezone hour offset value, although this is not used later.

I have added a function for extracting the integer hour from the timestamp timezone string, using $match() to find the +hh:00 or -hh:00, $replace() to remove the “+” as this cannot be parsed correctly, $split() to get the hour from the hh:00, and then $number() to convert.

I have added a function for turning an integer timezone hour offset to a string for later use, in the format +hh00

I have added a function that is the composition of both of the above, and a function for adding a given number of minutes to a given Unix time, and formatting to ISO with a given timezone string.

After that it is relatively easy to use these functions, to get 1 hour from now but in a different timezone, to create an array of start times, and to update the array with end times, each based on the start time.

I hope this is of interest to anyone looking to use JSONata for working with datetime and local time, using the JSONata native functions only.

1 Like

Thank you for sharing your idea. Where in your code is daylight savings taken care of?

I do like your approach but I also like to keep things simple for semantics. So I will use $moment() but I will also remember this if for some reason that library stops working correctly in NR.

There is no simple alternative to $moment() in JSONata, so I may have another look at this again myself. I am just (I believe rightly) concerned as to how it is implemented!

Daylight Saving is probably the most complicated bit of coding I have ever done, and I have been chipping away at this problem for about two years. As well as reading in my electricity tariff and meter consumption, I am also reading in solar forecasts in arrays, and have to deal with ‘time’. My most recent project was to call Solcast by API, and this produces all the returned half-hour periods with an end-time, which is in UTC. Solcast, quite rightly, say that if you want the time in local time you have to do it yourself. So I have.

Living in the UK, I can (and have) used the ‘01:00 UTC on the last Sunday in March and October’ rule to find the DST switching times, but decided to make my code universal. I am using the moment-timezone library, as an extension to the moment library. This does not work in JSONata - I just tried $moment().tz - and since JSONata is not exactly efficient, I have written code to obtain local DST details using JavaScript. This sits in a function node, and the moment library is loaded using the node setup module load feature.

This is the JS code

// guess timezone but use NR TZ env variable and work in local time
// for Home Assistant addon, TZ should be set, otherwise set in flow properties

let NRzone = env.get("TZ");
let LCzone = moment.tz.guess(true);

let mzon = moment.tz(NRzone);
let mnow = moment();

let utcoff = mzon.tz(NRzone).utcOffset();

// look in January and June to see if UTC offset different => DST necessary
let testjan = moment.tz({month:0, day:1}, NRzone);
let testjun = moment.tz({month:5, day:1}, NRzone);

let dst_jan = {
    "test": testjan.format(),
    "offset": testjan.utcOffset(),
    "timezone": testjan.format('Z z') };
let dst_jun = {
    "test": testjun.format(),
    "offset": testjun.utcOffset(),
    "timezone": testjun.format('Z z') };

// fixed dates for year and spring & autumn defaults
let year_start = moment.tz(NRzone).startOf('year')
let year_end = moment.tz(NRzone).endOf('year');
let springDST = moment.tz({month: 2, day: 20, hour: 3, minute: 0}, NRzone);
let autumnDST = moment.tz({month: 8, day: 22, hour: 3, minute: 0}, NRzone);

let dstarea = dst_jan.offset != dst_jun.offset;    // does DST apply here
let dstnrth = dst_jun.offset > dst_jan.offset;     // north or south hemisphere
let dstison = (dstarea && utcoff == Math.max(dst_jan.offset, dst_jun.offset));   // is DST currently on

let dstmodes = dstarea == false ? ["OFF", "OFF", "OFF"] : dstnrth ? ["OFF", "ON", "OFF"] : ["ON", "OFF","ON"];

// function to scan for DST change day then quarter hour - start moment and days to scan
function dst(event, lookdays){
    let testhour = event.hour();
    let x=0;
    while (x<=lookdays){
        event.add(24, 'hours');
        if (event.hour() != testhour) {
            testhour = event.hour();
            x = lookdays;
        };
        x++;
    };
    event.hour(0);
    testhour = event.hour();
    let testmins = event.minute();
    while (event.hour() == testhour && event.minutes() == testmins && testhour < 6) {
        event.add(15, 'minute');
        testmins = testmins+15;
        if (testmins>45) {
            testmins = 0;
            testhour++;
        };
    };
    return event;
};

// scan from 1st March to end April, and 1st September to end November
if (dstarea) {
    springDST = dst(moment.tz({month:2, day:1, hour:12 }, NRzone), 61);
    autumnDST = dst(moment.tz({month:8, day:1, hour:12 }, NRzone), 93);
    };

// for DST set spring and autumn change to 1 second before exact time
// this makes from-upto times inclusive for each period
let xspring = springDST.clone().subtract(1, 'second').unix();
let xautumn = autumnDST.clone().subtract(1, 'second').unix();

// for both DST and non-DST, all three periods are contiguous
// period 'A' from January to DST (or 'spring')
let dst_a = {
    "period": "to spring",
    "from": {"local": year_start.format(),
        "utc": year_start.utc(),
        "seconds": year_start.unix()},
    "upto": {"local": springDST.format(),
        "utc": springDST.utc(),
        "seconds": xspring },
    "offset": dst_jan.offset,
    "timezone": dst_jan.timezone,
    "mode": dstmodes[0]
    };
// period 'B' from DST/spring to DST (or 'autumn')
let dst_b = {
    "period": "spring to autumn",
    "from": {"local": springDST.format(),
        "utc": springDST.utc(),
        "seconds": springDST.unix()},
    "upto": {"local": autumnDST.format(),
        "utc": autumnDST.utc(),
        "seconds": xautumn },
    "offset": dst_jun.offset,
    "timezone": dst_jun.timezone,
    "mode": dstmodes[1]
    };
// period 'C' from DST/autumn to end of the year
let dst_c = {
    "period": "from autumn",
    "from": {"local": autumnDST.format(),
        "utc": autumnDST.utc(),
        "seconds": autumnDST.unix()},
    "upto": {"local": year_end.format(),
        "utc": year_end.utc(),
        "seconds": year_end.unix() },
    "offset": dst_jan.offset,
    "timezone": dst_jan.timezone,
    "mode": dstmodes[2]
    };
 
// final result
msg.dst =
    {
     "MYzone": LCzone,
     "NRzone": NRzone,
     "now_loc": mnow.format(),
     "now_utc": mnow.utc(),
     "zone_loc": mzon.format(),
     "zone_utc": mzon.utc(),
     "for_year": mzon.year(),
     "utc_offset": utcoff,
     "dst_aware": dstarea,
     "dst_north": dstnrth,
     "dst_is_on": dstison,
     "dst_details": [dst_a, dst_b, dst_c]     
    };

return msg;

and this it what it produces, updated every time I make an API call to Solcast

{
  "MYzone": "Europe/London",
  "NRzone": "Europe/London",
  "now_loc": "2024-10-24T06:05:20+01:00",
  "now_utc": "2024-10-24T05:05:20.977Z",
  "zone_loc": "2024-10-24T06:05:20+01:00",
  "zone_utc": "2024-10-24T05:05:20.976Z",
  "for_year": 2024,
  "utc_offset": 60,
  "dst_aware": true,
  "dst_north": true,
  "dst_is_on": true,
  "dst_details": [
    {
      "period": "to spring",
      "from": {
        "local": "2024-01-01T00:00:00Z",
        "utc": "2024-01-01T00:00:00.000Z",
        "seconds": 1704067200
      },
      "upto": {
        "local": "2024-03-31T02:00:00+01:00",
        "utc": "2024-03-31T01:00:00.000Z",
        "seconds": 1711846799
      },
      "offset": 0,
      "timezone": "+00:00 GMT",
      "mode": "OFF"
    },
    {
      "period": "spring to autumn",
      "from": {
        "local": "2024-03-31T01:00:00Z",
        "utc": "2024-03-31T01:00:00.000Z",
        "seconds": 1711846800
      },
      "upto": {
        "local": "2024-10-27T01:00:00Z",
        "utc": "2024-10-27T01:00:00.000Z",
        "seconds": 1729990799
      },
      "offset": 60,
      "timezone": "+01:00 BST",
      "mode": "ON"
    },
    {
      "period": "from autumn",
      "from": {
        "local": "2024-10-27T01:00:00Z",
        "utc": "2024-10-27T01:00:00.000Z",
        "seconds": 1729990800
      },
      "upto": {
        "local": "2024-12-31T23:59:59Z",
        "utc": "2024-12-31T23:59:59.999Z",
        "seconds": 1735689599
      },
      "offset": 0,
      "timezone": "+00:00 GMT",
      "mode": "OFF"
    }
  ]
}

By pulling in the Node-RED (add-on) timezone from the environment variable, I can easily override this for testing, and have checked my code for all the strange places around the world where it does not change, changes by 15 minutes, is in the southern hemisphere and so on. Seems to work.

This DST object is stored in NR context. You will notice that the dst_details array is three periods with a from and upto milliseconds time, and these are contiguous for the full year. Given a full array of ‘times’ (from a tariff array, solar forecast array etc) it is easy to filter the DST details for the one period or two periods that include the times within the array. Thus, a DST time change will occur within the array of times if and only if there are two DST periods found.

I use the following bit of JSONata code to

  • identify the first and last time points (UTC) in the array of time periods
  • pull the covering DST details for this period
  • compile the DST offset and zone for the DST current / before a change
  • compile the DST offset and zone for the DST after change if a change is within the span of the array time period

Then I am in a position to transform the array of times. Here I am removing 30 minutes from the ‘end time’ to get a ‘start time’ as Solcast give an end time for all forecast periods, and personally I want to know when a period starts from, not when it ends. After that I pull the +00:00 or +01:00 bit of the timezone I need, depending on DST, and then create a new ‘period_start’ which is now 30 minutes before the end time, and in local time, and correct for DST.

I have tested this, and as we (here) are about to put our clocks back from BST to GMT this very Sunday, I will have the once-a-year opportunity to check the code again for correct DST change.

For solar forecasts, the timezone change at DST is pointless, since all time changes take place at night, and at night the sun does not shine (duh…) but this is particularly important for translating UTC time to local time in energy pricing tariffs.

Of course, it does lead to the issue of only 23 hours in one day in March, and 25 hours in one day in October…

(
/* April 2024 */
/* read first and last times in payload and get covering DST change info   */
/* calculate period start (-30 mins from period_end) and set to local time */

    $times:=(payload.**.period_end).($toMillis()/1000 -1800);
    $first:=$min($times); $last:=$max($times);
    $tz:=dst.dst_details[($first>=from.seconds and $first<=upto.seconds) or ($last>=from.seconds and $last<=upto.seconds)];
    $x:=$tz{
        "offset_a": $[0].offset,
        "zone_a": $substringAfter($[0].timezone," "),
        "hhmm_a": $substringBefore($[0].timezone, " ")~>$replace(/[: ]/, ""),
        "uptosec": $[0].upto.seconds*1000,
        "change": $exists($[1]),
        "offset_b": $[1].offset,
        "zone_b": $substringAfter($[1].timezone," "),
        "hhmm_b": $substringBefore($[1].timezone, " ")~>$replace(/[: ]/, "")   
        };
/* if each start time past DST change, use post change details */
/* use offset as 'hhmm' in $fromMillis to force time to local  */
/* tranform on 3rd layer object to add period_start and zone   */
/* also to delete period and period_end as no longer required  */

    payload~>|*.*|(
        $start:=$toMillis(period_end) -1800000; 
        $hhmm:= $start<$x.uptosec ? $x.hhmm_a : $x.hhmm_b;
        {"period_start": $fromMillis($start,'[Y]-[M01]-[D01]T[H01]:[m01]:[s][Z]', $hhmm),
         "zone": $start<$x.uptosec ? $x.zone_a : $x.zone_b}),
        ["period", "period_end"] |;

)
1 Like

Still had to read this :smiley: How did switching from BST to GMT go down?

Perfectly, allowing for the fact that 01:00 to 01:59 happened twice…