Timestamp & timezones

Tags: #<Tag:0x00007f77ff7411d8> #<Tag:0x00007f77ff741020>

In a monitoring dashboard where I first check for hints when something isn’t working, I list last boot time for a bunch of devices. I load the UNIX epoch of a boot into value_json.lastboot and want it to then show in a relative manner such as “x hours ago”. For this, I use device_class: timestamp.

Problem

It seems timezones are causing a problem, as right after boot the dashboard shows a device has last booted ‘In 6 hours’.

Current setup

In the sensor configuration, I set value_template: {{ value_json.lastboot | timestamp_custom('%Y-%m-%dT%H:%M:%S') }}. This works fine, it results in a string of the format defined. Adding device_class: timestamp however, shows a relative statement which seems to be assume that time is GMT.

Already tried

  • Confirmed configuration.yaml contains the correct time zone.
  • Defining timestamp_custom('%Y-%m-%dT%H:%M:%S', local_time=True) results in an “Unknown” both with and without `device_class set.
  • Using timestamp_local instead yields the same time minus the “T” in the middle and when adding device_class: timestamp it says “Invalid date”.
  • While adding “Z” to the end of the value_template doesn’t throw an error (Zulu time = GMT), adding “ICT” results in an error.

Why bother converting the timestamp to a time string? If you just leave it as a Unix timestamp that should work with the timestamp device class.

You had me excited there for a second! For value_template, I tried value_json.lastboot and value_json.lastboot | int but neither of them worked. The only way I’ve been able to get an MQTT sensor to work with device_class: timestamp has been by printing it in the custom timestamp.

Do let me know if you have it working!

As per the first answer you received, you should store and do your calculations with timestamps which are stored in gmt. You only convert to local time for display purposes. That way you avoid the timezone issues completely in your calculations.

Ok, that’s a bummer. I hoped it would work with a timestamp. (After all, the device class is named “timestamp”.) Hmm.

Ok, plan B. Try adding a lower case “%z” at the end.

{{ value_json.lastboot | timestamp_custom('%Y-%m-%dT%H:%M:%S%z') }}

I tested that and it seems to work ok.

Thanks for your help.

I tried your suggestion, but it does not work for me. Please note these are sensor definitions.

The input for todays lastboot is 1585625355 which was my morning (I’m in +0700) when I booted my machine.

To test, I defined without the %z you suggested, with and with a trailing Z which is what someone else suggested. All 3 with and all 3 without a device_class: timestamp set.

Definitions as follows :

  name: Mac lastboot A
  state_topic: "systems/mac/status"
  icon: mdi:clock
  value_template: >-
    {{ value_json.lastboot | timestamp_custom('%Y-%m-%dT%H:%M:%S') }}
  #device_class: timestamp

- platform: mqtt
  name: Mac lastboot B
  state_topic: "systems/mac/status"
  icon: mdi:clock
  value_template: >-
    {{ value_json.lastboot | timestamp_custom('%Y-%m-%dT%H:%M:%S') }}
  device_class: timestamp

- platform: mqtt
  name: Mac lastboot C
  state_topic: "systems/mac/status"
  icon: mdi:clock
  value_template: >-
    {{ value_json.lastboot | timestamp_custom('%Y-%m-%dT%H:%M:%S%z') }}
  #device_class: timestamp

- platform: mqtt
  name: Mac lastboot D
  state_topic: "systems/mac/status"
  icon: mdi:clock
  value_template: >-
    {{ value_json.lastboot | timestamp_custom('%Y-%m-%dT%H:%M:%S%z') }}
  device_class: timestamp

- platform: mqtt
  name: Mac lastboot E
  state_topic: "systems/mac/status"
  icon: mdi:clock
  value_template: >-
    {{ value_json.lastboot | timestamp_custom('%Y-%m-%dT%H:%M:%S') }}Z
  #device_class: timestamp

- platform: mqtt
  name: Mac lastboot F
  state_topic: "systems/mac/status"
  icon: mdi:clock
  value_template: >-
    {{ value_json.lastboot | timestamp_custom('%Y-%m-%dT%H:%M:%S') }}Z
  device_class: timestamp

Which yielded the following rendered in HA :

lastboots

It’s driving me nuts.

Your timestamp is provided in local time while HA interprets it as utc time internally, but displays back in your local time. So you are off by exactly your time zone offset.

You can see this in that your timestamp is 3:29 but is being displayed at 10:29.

You can try

value_template: >-
    {{ value_json.lastboot | timestamp_custom('%Y-%m-%dT%H:%M:%S', local_time=False) }}
  device_class: timestamp

Alternatively get your mqtt device to report the timestamp correctly in utc time.

So now I’m wondering if maybe the clock in your display device (the device on which you’re showing the HA frontend) is set wrong. I just did a quick test. I went to the STATES tab and added this:

image

That is my local time, and it does not have any time zone suffix (just like you have.) I then added it to my frontend and it looks like this:

image

In your case you were seeing “in 6 hours.” That would seem to imply the clock on the display device is set to the wrong time and/or the wrong time zone.

Tried this before with local_time=True as per original post. Tried again today as suggested, but the value shown in the frontend is ‘Unknown’ (both with and without the device_class: timestamp set).

The MQTT message contains a UNIX epoch. This, by definition, is in UTC.

I think you’re on to something.

I just did the same test, entered a time of 15:40 (which is 5 minutes ago) in the same format as what you did, and it says ‘In 6 hours’ for sensor.test too.

Here is what I’ve done to check potential time zone errors :

  • Safari has no settings for time zone.
  • The Mac & iPhone both set to the correct time zone.
  • My configuration.yaml has time_zone: Asia/Bangkok set
  • Running date on the host OS where I run Hassio results in : Wed Apr 1 15:43:34 +07 2020
  • Running date inside the homeassistant container yields the same result
  • Running date +%s on every machine I have running, yields the same UNIX epoch
  • Home Assistant -> Logbook shows the correct timestamp

Sorry it should be

value_template: >-
    {{ value_json.lastboot | timestamp_custom('%Y-%m-%dT%H:%M:%S', False) }}

This is also why your version with True didn’t work.

Can you try a different browser on a different OS? I don’t generally use Macs (anymore), but I thought I heard once a while ago that Safari had issues with HA, and it might have been related to your problem.

Not 100% sure what’s going on, but this seems more like a band-aid, or a wrong correcting a different wrong. Date/time strings without a time zone suffix are supposed to be interpreted in the local time zone. This would create a date/time string, without a time zone suffix, in UTC, which, AFAIK, is not the right thing to do.

What this does as I understand is to report the timestamp in utc time rather than local time. This does indeed fix the problem on the ha side, if as I strongly suspect that the device is providing a time stamp as if the user’s local time is in utc time. But the real fix would be to get the device to report utc time correctly.

Timestamps, by definition, are in UTC. And the timestamps the OP showed seemed perfectly fine. The issue is, when it is properly converted into a date/time string, with no time zone suffix, and while using the timestamp device class, it is not being displayed properly. That has more to do with the browser and/or OS the browser is running on. At least, that’s how it appears to me.

If you look at the time stamp it is 3am utc, which shows you as 10am local. This is correctly applied. But this is 1 hour in the future. This is what leads me to believe the device providing the timestamp is doing it incorrectly

BTW, it’s strange that what works on my system (a date/time string with a time zone suffix in the format +/-nnnn) doesn’t on yours (i.e., results in “Invalid date.”) This seems to be another indication that the problem is with the browser.

The real test is for @robdejonge to force a boot and report the local time and the reported time stamp and the display time. Otherwise this is a lot of speculation.