Strategy for creating last value 'cache' versions of sensors in case the sensor becomes unavailable

[digitalmacdonald]

I have automations that trigger when time is equal to a timestamp-class sensor. In this case, it’s a sensor managed by an integration that tells me the start time of my free power for the day. sensor.hour_of_free_power_start

The sensor occasionally becomes unavailable, like when the integration wants me to log back in. If the sensor is unavailable at the time the automation would otherwise trigger, the automation doesn’t trigger. No surprises there.

The sensor is a device_class: timestamp and is reset by the integration each day, and includes the date in the sensor value. 2024-07-26T18:00:00+00:00

The time portion of the timestamp does not typically change. i.e. the free power starts at typically the same time each day, so the date is somewhat superfluous. It might take me a day or two to realise that the sensor is unavailable and ‘fix it’ by re-authenticating the integration.

My original thinking was to create a cache version of the sensor that could be included in the automation trigger if the original sensor was unavailable. In fact as a general strategy I thought it made sense to have a cache of the last value of every sensor that could become unavailable, so that if a sensor becomes unavailable, I can at least use it’s last value as a trigger for an automation (or something else).

For the specific use case of the timestamp for starting my free power (sensor.hour_of_free_power_start), I’ve run into my first problem with this strategy. Because the timestamp includes the date, then a cached version of this sensor would not trigger at the same time the following day. I really only want to cache/use the time component of that sensor. And I don’t practically know how to capture, save and then use only the time portion in the automation trigger.

So, my questions for the community are as follows:

1. Am I approaching the problem of ‘automations missing their triggers when timestamp sensors become unavailable’ all wrong? Or does this approach seem sensible (I’m sceptical its the best one)
2. If somehow using a ‘last value’ cached version of a sensor is a good way to approach it, how would I practically save the ‘last value’ cache.
3. And if I had a ‘last value’ cache version of a timestamp sensor, how could I extract and use the time component as an automation trigger.

[kartcon (Art Davis)]

I’m a bit confused by what you said:

it’s a sensor managed by an integration that tells me the start time of my free power for the day.

then

The time portion of the timestamp does not typically change. i.e. the free power starts at typically the same time each day,

So why not create an automation to start at that time, everyday? Or does it not occur every day?

I really only want to cache/use the time component of that sensor. And I don’t practically know how to capture, save and then use only the time portion in the automation trigger.

{{ as_timestamp('2024-07-26T18:00:00+00:00')|timestamp_custom("%H:%M")}}
results in 14:00 in my timezone

You can replace the fixed time with a sensor or sensor attribute.

{{ as_timestamp(now())|timestamp_custom("%H:%M")}}
{{ as_timestamp(state_attr(('automation.a_critical_device_goes_offline'), 'last_triggered'))|timestamp_custom("%H:%M")}}

Am I approaching the problem of ‘automations missing their triggers when timestamp sensors become unavailable’ all wrong?

You can use the availability template to check the sensors status:

  - sensor: 
      name: "meter_a quantity" 
      state: "{{ state_attr('sensor.meter_a','waterquantity')|int(-1) }}" 
      availability: "{{ state_attr('sensor.meter_a','waterquantity')|int(-1) >= 0 }}" 
      state_class: "measurement" 
      unit_of_measurement: "l" 

** Note, not my code, copied from the forum.

https://www.home-assistant.io/integrations/template/

There are many ways to acomplish this. The simple approach is always the best. If this truly occurs at the time time every day, that’s you best option. If not then I’d need more info on exactly what changes, for example no new update to the sensor so no reduced rate today, or something similar. The availability template will solve that portion of your coding pretty easily. Extracting the date is easy and there are LOTS of threads on the forum on how to do it. GET FAMILIAR with the Template Editor in the Developer Tools. It will be your best friend!! (that’s where I pasted all the code above to verify before posting)

[digitalmacdonald]

@kartcon

I’m a bit confused by what you said:

it’s a sensor managed by an integration that tells me the start time of my free power for the day.

then

The time portion of the timestamp does not typically change. i.e. the free power starts at typically the same time each day,

So why not create an automation to start at that time, everyday? Or does it not occur every day?

It’s always hard to know how much detail/context to provide, balancing people’s reading time with clarity.

The world TYPICALLY is the important word in that sentence. The hour of the day is likely to be the same, but not guaranteed. That’s why I pull it from the sensor provided by the integration instead of making it a fixed time each day.

I appreciate the suggestions with the template formatting.

I’d gotten to a stage of having a template that resulted in ‘06:00:00’ in my timezone and created a template sensor called sensor.cache_hour_of_free_power_start.

The issue I’m having comes when trying to use that within an automation. I’m using a condition to test it, since I don’t want to wait for the actual time to equal the sensor as a real world test.

So I create a condition Confirm the time is after… and select Value of a date/time helper or timestamp-class sensor. I then search for the sensor called sensor.cache_hour_of_free_power_start and it does not appear.

I notice in the Developer Tool (see above screenshot) that the sensor does not have have a device_class: timestamp attribute. So my assumption is that HA does not think it is a timestamp and is therefore not treating it as a timestamp-class sensor that can be used in this condition.

I then go into the Template sensor UI and set the Device class to Timestamp, thinking that this may force it to be seen as a timestamp-class sensor.

However, by adding the device_class: timestamp attribute, the sensor result is now unknown instead of ‘06:00:00’. It does appear as a timestamp-class sensor option in the condition, but as it’s unknown, it’s not what we want.

In summary,

1. I have a template sensor called sensor.cache_hour_of_free_power_start with a value of ‘06:00:00’
2. Manually setting the device_class: timestamp attribute causes the sensor value to be unknown
3. Not setting the device_class: timestamp attribute causes the sensor to not be available within a condition that looks for a timestamp-class sensor.

Thank you for your help and suggestions.

[kartcon (Art Davis)]

Two thing from my perspective;

1. I prefer to code in yaml so can’t speak to what the UI presents to you, but I think the next point is why you don’t see it.
2. The sensor you are showing in the developer tool screenshot is NOT in a timestamp format. It is a string.

Try this, set the device_class back to the timestamp. The data in the sensor WILL disappear as it’s not in the right format. THEN, back in the developer tools click on the blue name of the sensor. It will open an editor window at the very top of the page. (you may have to scroll up to see it). See my pic. Change the state to an actual timestamp. Use the one you provided in your original post (2024-07-26T18:00:00+00:00). Click save. See if you can now access that sensor in the UI interface.

As a last resort, switch the UI view to yaml and copy/paste that in your reply.
Good luck.

[kartcon (Art Davis)]

As an afterthought, why not just put that converted timestamp data directly into a date/time helper. You can define just a time. Might be a simpler solution.
https://www.home-assistant.io/integrations/input_datetime/

# Sets time to 05:30:00
- service: input_datetime.set_datetime
  target:
    entity_id: input_datetime.XXX
  data:
    time: "05:30:00"

[digitalmacdonald]

Thanks @kartcon. I appreciate your time on this. Replying to your first post just so that others might be able to see what I was doing, but the real answer/value is in your second comment, so feel free to skip to there.

Yes, by manually setting the state of the sensor (using Developer Tools) to the original timestamp state and format (2024-07-26T18:00:00+00:00) and having the attribute device_class: timestamp, the sensor becomes available as a condition in the automation and is treated as a timestamp format. This would be what I would expect as well.

But the value includes the date, not just the time, so although it is a timestamp format, is not appropriate for triggering an automation that triggers based on a time of day, not a specific date. This is why I was formatting the data in the proposed caches version to remove the date in the first place with timestamp_custom(“%H:%M”).

I’ll try to restate the problem differently for clarity…

1. The (example) automation currently triggers “When time is equal to entity Hour of free power start”.

2. The entity is sensor.hour_of_free_power_start and is controlled by an integration that will occasionally become unavailable due to needing reauthentication (nothing I can do about that I think).

3. When the sensor is unavailable, the automation will not trigger. (expected behaviour)

4. I would propose to add an additional trigger to the automation that triggers when time is equal to the %H:%M:%S component a previously seen state of sensor.hour_of_free_power_start, for use when sensor.hour_of_free_power_start happens to be unavailable.

5. There is a high likelihood (but not certainty) that if the sensor.hour_of_free_power_start was available, the time of day component would be the same as the %H:%M:%S component of the previously seen state of sensor.hour_of_free_power_start. The likelihood is high enough that it’s better to trigger the automation based on that assumption than to not trigger it at all.

Example:

• On 2024-07-26 sensor.hour_of_free_power_start state = 2024-07-26T18:00:00+00:00 (06:00:00 time in my timezone) and automation triggers.
• On the following day (2024-07-27) sensor.hour_of_free_power_start state = unavailable. There is a high likelihood (but not a certainty) that had it been available, it would been 2024-07-27T18:00:00+00:00.

My two ways of approaching it have been either to

1. try to format the template sensor (previously seen state of sensor.hour_of_free_power_start) data in a way that it can be accepted as a trigger for a “When time is equal to…” trigger, or
2. have a different type of trigger that accepts state formatted as a string and knows it’s a time of day that should be compared to current time and trigger when it equals that time of day.

I have briefly considered trying to copy the last seen state and manually increment the date ahead by a day, but that seemed overly complicated for what I felt should be a relatively common problem that may have been solved by someone already.

“How do people deal with automations that trigger based on time based sensors that may become unavailable?”

[digitalmacdonald]

@kartcon

As an afterthought, why not just put that converted timestamp data directly into a date/time helper. You can define just a time. Might be a simpler solution.
Input Datetime - Home Assistant

# Sets time to 05:30:00
- service: input_datetime.set_datetime
  target:
    entity_id: input_datetime.XXX
  data:
    time: "05:30:00"

Yes, I think this is creating the date/time helper with the time we need.

FYI, here is the specific code:

service: input_datetime.set_datetime
target:
    entity_id: input_datetime.cache_hour_of_free_power_start_time
data:
    time: "{{ as_timestamp(states('sensor.hour_of_free_power_start')) | timestamp_custom('%H:%M:%S')}}"`

It results in a date/time helper with the value 06:00:00 (which is my timezone adjusted time).

What I’m curious about now is if I need to account for when the sensor.hour_of_free_power_start IS available or not within the automation triggers or if I can just put both triggers into the automation.

trigger:
  - platform: time
    at: sensor.hour_of_free_power_start
  - platform: time
    at: input_datetime.cache_hour_of_free_power_start_time

Do you think this will be an issue if I have the automation mode set to Single? I could theoretically change the mode to Restart if we are concerned that having two triggers that could happen almost (but not necessarily) at the same time might cause some issue.

[kartcon (Art Davis)]

I think I would approach this completely different than the route you’re taking. If automation #1 needs to trigger at a certain time every day, and that time is driven by a sensor that changes daily and may become unavailable, I would:

1. get the data from the utility company as soon as it is updated. I’m assuming it changes after midnight, so start soon after that. Create another automation (automation #2) to get the data, place it in a helper and keep trying until the data is updated. For example triggered at 2am, 4am and 6am. Send a notification to reauthenticate if it fails.
2. Set automation #1 to fire at the time collected from automation #2. This will either be yesterdays value or todays updated value. Either way automation #1 will trigger.

The guts of making it work are in step 1. Step 2 should ALWAYS trigger, even if it’s yesterday’s time value. The time helper should NEVER be empty once it’s created and updated for the first time.

The notification in step 1 should only fire if the data does not update. This reduces pesky, unnecessary alerts and is an unspoken way of knowing the data did update.

Look at the repeat syntax for some ideas on how to setup step #1.

Lastly, EMBRACE YAML. The UI is good for basic automations but working directly in yaml opens the door for more complex automations. Even though this is a pretty simple automation to work out, in yaml you have much greater abilities to track the flow. For example if I’m writing a very complex sequence I will add persistent notifications with sensor data at each step where I’m having an issue. It’s very similar to the TRACES but I see it happen in real time.

[digitalmacdonald]

You make a very good point that if I can be sure that the helper has valid data, I don’t need the original sensor at all as part of the trigger for Automation #1. I was over complicating it by including both.

And this is ultimately the answer to my original question around ‘strategy’ to deal with this type of thing.

Use another automation to prepare a helper to alway be available and then trigger off of that helper instead of the other sensor(s) that may be some level of flakey.

Thanks for your help.

[beausmith (Beau)]

FWIW, here is how I created a template helper to provide a cached value of our Tesla battery state of charge, normally a value 0–100%:

{% if has_value("sensor.tesla_battery") %}
  {{ states("sensor.tesla_battery") }}
{% else %}
  {{ this.state | float(default = "0") }}
{% endif %}

(The float with default value is necessary if the helper already has a unknown/unavailable value.)

Or a simpler version, if the source value currently “has_value”:

{{ states("sensor.tesla_battery") | float(default = this.state) }}

[jackjourneyman (Jack)]

Sorry to be a wet blanket, but might it not be a good idea to establish why these timestamp sensors are becoming unavailable, and address that? I don’t think it should be happening. It certainly shouldn’t be necessary to save ‘last value’ versions of sensors.