Make typed states available for entities

Feature Requests
1

In the forum and on Discord, I often see people struggle with casting states to the appropriate type for a calculation or comparison (math, date, etc.). This stems from the fact that all states are represented as strings, and any calculation/comparison of a string with a typed value will likely either result in an unexpected error, or just fail silently. Either way, it adds a barrier for new users to not only understand that states are strings, but also how to convert that string into something more useful. datetime states are particularly annoying, as laid out here and here.

Rather than tackle “typed states” as described in the first thread, with all the challenges therein, I wonder if we can take advantage of the fact that attributes can retain their native type. If we look specifically at helpers such as input_boolean, input_datetime, and input_number, we could add an attribute that’s equal to the state, but with the corresponding type intact - boolean, datetime, and float, respectively. It could be called typed_value or similar, and be readily available for templates to consume with no additional conversion required. input_datetime does already have a timestamp attribute, but even that requires some processing for use with many date/time-related Jinja functions.

Another possibility is to add a property to the state object itself, similar to last_changed/last_updated, that corresponds to the typed state value. That would avoid having it pollute the attribute list and still be available in the state object, but those properties aren’t currently provided by the integration. The main challenge is that it would be up to the integration to provide a typed value and it can’t be reliably derived from the state automatically.

Providing the typed value would be relatively easy for the built-in helpers that have explicit types, and could be expanded to other integrations at the developer’s option. Perhaps without a provided typed state, the attribute or state property just returns the state string by default, ensuring that it at least has a value.

I’ll continue to think about how this might work, but wanted to put it out there to see what others think.

2

Ideally, an entity’s state value would allow for types other than string. However, at this point in Home Assistant’s evolution, it would be the Mother of All Breaking Changes.

Duplicating the state value in an attribute, or property, would be more feasible (although it seems like kluge; same data twice).

Where it would be welcome is in input_datetime. It always seemed odd to me that time/date in Home Assistant is typically handled as a string or “flattened” into a Unix timestamp. All of a datetime object’s methods are unavailable unless one first converts the string via strptime.

Even worse are entities like sun.sun whose attributes, which can contain datetime objects, contain string-based times!

A challenge with input_datetime is the time-only variant. Most entities store date&time in UTC which can then be easily converted to local time. Except the time-only input_datetime. It stores the time as local time. Time arithmetic can become error-prone if you’re unaware of this deviation from the norm.

1 Like
3

I guess that’s this bit, where you don’t specify a timezone?

        # If the user passed in an initial value with a timezone, convert it to right tz
        if current_datetime.tzinfo is not None:
            self._current_datetime = current_datetime.astimezone(
                dt_util.DEFAULT_TIME_ZONE
            )
        else:
            self._current_datetime = dt_util.DEFAULT_TIME_ZONE.localize(
                current_datetime
            )

Yeah, that’s weird, but I don’t know what else we could do.

4

If input_datetime contains date and time, and the user supplies a value as local time (as would normally be the case), the result is stored as a Unix timestamp in UTC.

If input_datetime contains time only, and the user supplies a value as local time (as would normally be the case), the result is also stored as a Unix timestamp but without converting it to UTC.

You can confirm it by entering a time like 13:00. If you convert its timestamp attribute with as_timestamp the resulting date is 1970-01-01 (root date for a Unix timestamp) but the time is 13:00 (unconverted to UTC).

There may be a good reason for doing it this way but I’m not sure why (it may have something to do with it being timezone naive). What I do know is that it makes it awkward (if not impractical) to use it as a datetime object because it’s not in UTC.

5

I’m not sure if this was fixed recently or not, but this works in the developer template editor without casting the input_datetime to something other than the string (okay, I do use the as_timestamp function):

date_time: {{states('input_datetime.test_date_time')}}
  >>> date_time: 2021-04-20

# Not sure what the type is at this point... Check with some math using `as_timestamp`:
{% set inst = as_timestamp(states('input_datetime.test_date_time')) %}
  inst: {{inst}}                                           --->>> 1618894800.0
# Looks like a number - but not done checking yet...

# Grab another value to do some math...
{% set dif = states('input_number.days_to_wait') | int * 86400 %}
dif days: {{states('input_number.days_to_wait') | int }}   --->>> 8
dif secs: {{dif}}                                          --->>> 691200
{% set nt = as_timestamp(states('sensor.date')) %}
{{states('sensor.date')}}                                  --->>> 2021-04-23
  nt:  {{nt}}                                              --->>> 1619154000.0
{{ (((inst + dif) - nt) / 86400) | round(0) }}             --->>> 5

What this shows is that, as long as it’s cast using as_timestamp, it is actually a number that can be used in math. Maybe there’s a use case where one would to use the raw value, but I always cast to a timestamp (gives everything the same baseline).

I’m not disagreeing with what Taras and Rob are saying, as dates and times are the things that mess me up the most. But this seems to work (??)

6

You’re demonstrating this part:

If it were available as a datetime.date object, you can use its methods to get the year, month, day, etc directly.

For example, if input_datetime.test_date_time contained an attribute called typed_value (as suggested in the first post), you could use it like this:

{% set dt = state_attr('input_datetime.test_date_time', 'typed_value') %}
{{ d.year }}
{{ d.month }}
{{ d.day }}
{{ d.weekday() }}
{{ d + timedelta(days=10) }}
{{ now() - d }}

Most of that can be achieved today (using its timestamp attribute and/or state value) but employing more templating gymnastics.

7

Got it - I missed that point of the discussion. Probably because I’ve become so accustomed to simply casting them into a Unix timestamp :upside_down_face:. Thanks for the clarifying remarks!

8

Evidently this won’t fly because states/attributes must be JSON serializable, and a datetime object is not. This PR implemented something similar with the sun.sun attributes and was closed for that reason. This could still be done for input_xxx types that are JSON serializable, but I think the killer feature is really datetime. Unless and until there’s a solution to serializing datetime objects for the recorder’s benefit, I don’t think this is worth pursuing.

9

A quick search of the interwebs shows it is possible to serialize a datetime object but there’s no official JSON standard for it, only defacto standards like adopting ISO8601.

It suggests it’s possible but would require an Architectural review to determine how to represent it.

10

I already have raw datetime objects in a couple of my custom integrations and my HA instance still hums along, so I wonder what evil now lurks in my database. I’ll change them to formatted strings to avoid future catastrophe.

11

FYI, this is coming in 2021.7 and should be helpful here by eliminating the need to do the cumbersome strptime dance: 2021.7: Beta release notes - Home Assistant