Unexpected behaviour of as_timestamp

Configuration
1

I’m puzzled by the results I’m getting from as_timestamp when fed datetimes over the change to BST tonight. The docs say that it returns the number of seconds since 1/1/1970 UTC so I had thought I could display this as local time with timestamp_custom/local=True. From experimenting, as_timstamp gives a different result if it is fed a TZ aware datetime string (ie it assumes the datetime string is in local time by default). I have a REST sensor that is returning non-TZ aware datetimes so I’m looking for an elegant way of getting the conversion right. This is what I’ve got:
Some of the raw input from REST:

[{'EventType': 'LowWater', 'DateTime': '2025-03-30T00:57:00', 'IsApproximateTime': False, 'Height': 0.3726451733974183, 'IsApproximateHeight': False, 'Filtered': False, 'Date': '2025-03-30T00:00:00'}, {'EventType': 'HighWater', 'DateTime': '2025-03-30T06:19:00', 'IsApproximateTime': False, 'Height': 8.551698547503939, 'IsApproximateHeight': False, 'Filtered': False, 'Date': '2025-03-30T00:00:00'}]

Looking at one just after the change to BST:

{{events[n].DateTime|as_timestamp}} 1743311940.0
{{(events[n].DateTime~'Z')|as_timestamp}} 1743315540.0

I had been processing the raw data to get a list of timestamps like this:

{{ events|map(attribute='DateTime')|map('as_timestamp',0)|map('int')|list }}

Which clearly is giving the “wrong” results, in that if I try to display them using the local=True option in timestamp_custom, I get the raw time not the BST time.
I’ve got this work around, but it strikes me as inelegant, surely there is a filter or a map that will avoid having to loop through the entire list?

          {% set ns = namespace(out=[]) %}
          {%- for tide in events %}
            {%- set ns.out = ns.out + [(tide.DateTime~'Z')|as_timestamp] %}
          {%- endfor %}
          {{ns.out}}

As ever, I’m grateful to the wonderful community for any guidance.

2

Just add utc offset to the as_timestamp value.

{{ events|map(attribute='DateTime')|map('as_timestamp',0)|map('add', now().utcoffset().total_seconds())|list }}
1 Like
3

I believe you need to map the add.

BTW, nice solution. :+1:

1 Like
4

Thanks, but surely that will only work after summer time has started - not on future dates which are after the change.

5

It will work all the time because utcoffset changes with the seasons and dst/bst.

6

Viz:

% set events = [{'EventType': 'LowWater', 'DateTime': '2025-03-30T00:57:00', 'IsApproximateTime': False, 'Height': 0.3726451733974183, 'IsApproximateHeight': False, 'Filtered': False, 'Date': '2025-03-30T00:00:00'}, {'EventType': 'HighWater', 'DateTime': '2025-03-30T06:19:00', 'IsApproximateTime': False, 'Height': 8.551698547503939, 'IsApproximateHeight': False, 'Filtered': False, 'Date': '2025-03-30T00:00:00'}] %}
{{ events[1].DateTime|as_timestamp }} gives 1743311940.0
{{ events|map(attribute='DateTime')|map('as_timestamp',0)|map('add', now().utcoffset().total_seconds())|list }}  gives [1743296220.0, 1743311940.0]

ie it is applying the UTC offset now() not at the time of the event.

7

Those events are tomorrow… which has the same utcoffset as today. If you want tomorrows offset, you can always add a day to now().

And when you reply to people, don’t reply to the thread, reply to them. Otherwise they do not get the notification.

image
image928×389 25.7 KB

8

I think what I need is to add the UTC offset of the time of each event in the list. In extremis I could be getting a years worth of events in January, with the offset only applying to the middle 6 months-ish which are in summer time. Can map(add) do that?
Sorry I just replied to the thread again before reading your reply

9

No, add cannot do that. What rest API are you using? Are you sure you can’t request the times in UTC? Most APIs allow TZ as a param.

10

It’s a free UKHO API. The paid for version replied in tzv-aware format.

11

Do you have a link to the documentation for the free version?

12

Is the REST API reporting the 00:57:00 to be local time or UTC?

In other words, is the free API reporting local time or is it reporting UTC time but purposely omitting the timezone portion?

1 Like
13
14

I believe it is the latter. I’ve always had to add ‘Z’ or ‘+00’ to the DateTime to process it. I’ve only just started with processing the list as a block and come across this problem.

15

if it’s already in UTC, then just use as_datetime and as_local to get the local datetime object and don’t use timestamps.

{{ events|map(attribute='DateTime')|map('as_datetime')|map('as_local')|list }}
16

That doesn’t change the result. I think it all comes down to the automatic assumption that DateTimes are always local unless they have tzinfo.
Isn’t there something like map('add') that will append ‘+00’ to each item that I can then process as_timestamp?

17

That will change the value properly if it’s UTC and it will account for the TZ properly. If the value is not correct, that means the datetime in the api response is local, not UTC.

You can use add with strings.

18

Maybe I have been staring at this for too long but is this the result you expected?

{{ '2025-03-30T00:00:00'
  | as_datetime }}
{{ '2025-03-30T00:00:00'
  | as_datetime
  | as_local }}

2025-03-30 00:00:00
2025-03-30 00:00:00-04:00

Given that the datetime string is supposed to be interpreted as UTC, the correct result would be:

2025-03-30 00:00:00+00:00
2025-03-29 20:00:00-04:00
19

Ah yes, forgot about non-tz aware datetimes. That is normal.

He and just use add to append the TZ then.

20

Yeah, easier if as_datetime had an option to “handle this non-TZ aware string as UTC time, not local time”.