Add force_update support to template sensor

[sebk-666 (Sebastian)]

When using a template sensor to track some slow changing value (e.g. battery_level of a device), the sensor is only updated when that value actually changes.
If the sensor’s state is being exported to e.g. InfluxDB with the intention to graph its values using Grafana, it’s likely that not enough data points get created to draw a meaningful graph (think plotting a graph over a 1d time range for a value that only changes every other day).

In case of the RESTful sensor, a force_update option was implemented for exactly that reason (https://www.home-assistant.io/components/sensor.rest/#force_update).

I’d like to request force_update support for template sensors, too.

Sebastian

[bangom]

Me too Please implement force_update for template sensor! Thanks

[Mutt (Muttley)]

I thought that was done via : -

- service: homeassistant.update_entity

[sebk-666 (Sebastian)]

It isn’t.
See https://github.com/home-assistant/core/issues/22197 and Updating template sensor on attribute change? for background info.

Sebastian

[bangom]

@Mutt

service: homeassistant.update_entity

Service homeassistant.update_entity doesn’t update entity if value is still same.

[Mutt (Muttley)]

Yes it does.
My device trackers update attributes on an update even when the tracker itself still remains away (not_home)
I admit it may depend on the sensor and whether it has any attributes
And the template sensors based on it thus update. Distance, co-ords etc.
Where would you need an update if the value hasn’t changed anyway

[Mutt (Muttley)]

Give me the full yaml detail for your template sensor and I’ll show you how to get it to update if ANY contributing element updates, or we can just do it on 1 minute intervals if you require it to be as clockwork

[sebk-666 (Sebastian)]

Just read the links I posted earlier.
Feel free to reply to the thread behind the second link.

Sebastian

[msmits2011]

Is this already accomplished in some way? I’m using a trend sensor on a template sensor, but it looks like it doesn’t work properly since the update interval is very low.

[myhomeiot]

Please update topic, force_update also very important for statistics for slowly changes sensors.
Thanks.

[skrutt]

Yeah,i also have this problem. I set up a mqtt sensor for temperature in my car, and a template for checking when it was last updated. I thought that states.car_temp.last_updated would contain the time it was last updated, and last_changed would have the time for last change, but in reality they both just reflect when the value is updated AND changed.

Because of this, if the temperature is constant for over 5 minutes, the template shows car as away(template is updated every minute with sensor.time also, since we cant get car temp over wifi if it’s not here).

And there seem to be no way around this? how would you get a new timestamp for last recieved update from a sensor if there is no such timestamp registered if the value is not changed? even worse, the template does not seem to update on duplicate updates, i’m not a 100% on this since i did not try that specifically, but according to what i have seen so far, i could not even create an extra template to update now() into a diy extra car_temp_last_updated, since it only is run on changes or each new minute from the sensor.time.

But might be that my template is monitoring states.car_temp.last_updated, and monitoring states.car_temp itself might make a diy last_updated possible.

But yea, #rantover. i’m suprised that this was like this? OK if there was a universal force_update option, but not updating on new updates seems VERY counterintuitive, and not having an option for it just seems broken.

[sebk-666 (Sebastian)]

To illustrate the issue from my original post, here’s the graph of my sensors’ battery levels (from Grafana):
Last 30 days:

Last 2 days:

Last 24 hours:

Grafana can be configured to use the last valid data point inside the time frame to draw a graph (that’s why it’s usually lines and not just dots for each data point), but if no data point falls within the time frame, nothing can be drawn.

Sebastian

[skrutt]

Actually, this is ready implemented… and for my use as above, there is also as i found since then, an expire_after parameter that does exactly what i wanted to have… so i guess that HA is keeping track of last updated time in the background, that is in addition to the “official” last_updated, since it expires correctly even if the last values was the same for much longer. in my case, temp == 23 for one hour, timestamp does not update, but power off the temp sensor and it’s shown as unavailable after 5 more mins. Perfect!

MQTT Sensor

However, i updated the code on the sensor firmware itself to add this “expires_after” into mqtt discovery, now i wonder if you can add to a sensors configuration in configuration.yaml?

That is, if i have a sensor with mqtt discovery and updating the firmware myself is not an option, could i add just “force_update: true” in yaml and when that sensor is discovered my custom config would be added to the discovered one?

if you look in the supported abbreviation list you also find “force_update”

MQTT Discovery

Edit: After reading your initial question again, i guess this is still not quite what you wanted… but if you can append the configuration with yaml, i guess that might get you close enough?

[sebk-666 (Sebastian)]

I’m not sure what you’re suggesting. The MQTT sensors have nothing to do with template sensors.
Have a look at the referenced threads at the top.

To summarize:
I have a bunch of template sensors ("sensor.bat_device_xyz") that expose the battery levels of various (Z-Wave) devices. This data is fed into an InfluxDB instance.
Each time the state of one of those sensors changes, a new data point is created in the InfluxDB database - or, as long as the state does not change, no new data points are created.

Im graphing these sensor values using Grafana. Grafana needs valid data points to draw a graph for the selected time frame (e.g. 1 day). If there is no valid data for the selected time available, the graph cannot be drawn.

Due to their nature, the battery level values only change very slowly, so there’s often longer time spans where no new data points are created.

HA currently has no means to update a template sensor’s state (not even the last_updated attribute) if the sensor’s value does not change.
Ideally there would be a means to create new data points by updating timestamp of the template sensor’s last_updated attribute even if the sensor’s value hasn’t changed.

Sebastian

[sebk-666 (Sebastian)]

It seems I have to correct myself.
I just checked the template sensor documentation and it seems there’s now a way to not only template the sensor value but also specify templates for attributes. I had completely missed that.
With that it should be possible to add an attribute to each template sensor that updates regularly and thus changes the state of the sensor, causing a new data point to be created.
I just tried something like:

  bat_mydevice:
    value_template: {{ state_attr("device_tracker.mydevice", "battery") | int }}
    unit_of_measurement: '%'
    attribute_templates:
      hour_last_updated:  "{{ now().hour }}"

So even if the sensor value does not change, the hour_last_updated attribute should change hourly, which should solve this whole issue.

Sebastian

[sebk-666 (Sebastian)]

Talking to myself here…
Apparently the above does not work - a change in the attribute’s template does not trigger reevaluation of the template sensor, i.e. when now().hour changes at the top of an hour, nothing happens.
The template sensor attribute keeps its old value - I’m assuming an update still only happens when the sensor’s value changes.

Looking at influxdb shows that no additional data points are created:
(HA does create a data point when restarted, that’s why multiple records for the same value are present)

> select time,entity_id,hour_last_updated,value from "%" WHERE ("entity_id" = 'bat_aeotec_ms6_kitchen') order by time desc limit 5;
name: %
time                entity_id              hour_last_updated value
----                ---------              ----------------- -----
1603808842819112960 bat_aeotec_ms6_kitchen 15                100
1603807856891645952 bat_aeotec_ms6_kitchen                   100
1603628796677989120 bat_aeotec_ms6_kitchen                   100
1603368964196990976 bat_aeotec_ms6_kitchen                   100
1603190444560261888 bat_aeotec_ms6_kitchen                   100

So, unfortunately, adding a templated, regularly changing attribute doesn’t help anything.

Sebastian

[sebk-666 (Sebastian)]

And another update: With just released v0.117, the above does work now.
Likely it’s related to bdraco’s change: https://github.com/home-assistant/core/pull/41147

> select time,entity_id,hour_last_updated,value from "%" WHERE (entity_id = 'bat_aeotec_ms6_kitchen') order by time desc limit 3
name: %
time                entity_id              hour_last_updated value
----                ---------              ----------------- -----
1603962000013168128 bat_aeotec_ms6_kitchen 10                100
1603958400019803136 bat_aeotec_ms6_kitchen 9                 100
1603956896818607104 bat_aeotec_ms6_kitchen 8                 100

Sebastian

[skrutt]

Right, nice!

Before 117 you had to include a reference to some other sensor, such as sensor.time, and before 115 you could do that with entity_id: sensor.time.

as you noted now() did not trigger a refresh. A bit strange but since it’s fixed now no matter, same function but much clearer now!

[erik.aslaksen]

This is great Information. Thank you very much - helped me a lot! Would you also know how to update the hour_last_updated e.g every 15 minutes?

[sebk-666 (Sebastian)]

You can use

{{ (now().minute / 15) | round(0, 'floor') }}

as the expression which will change every 15 minutes; from 0 to 3 during the course of an hour.

Sebastian