Preserve the history of certain sensors for a longer period of time

Configuration
1

Hello,

I understood that by default the history is kept for 10 days on the sensors.

I would like to leave this rule as default and extend the duration for certain sensors like my power consumption and temperature to have a long-term view.

Storage space is not a problem for me.

Which option should I put in the recorder?

1 Like
2

No such option exists.

There’s only one option to control the retention period and it applies to all included entities.

purge_keep_days

There’s a Feature Request to provide control on a per-entity basis so may wish to vote for it.

3

You can take the opposite approach and periodically purge the entities that you don’t want long term history for by using the service calls described here:

So, what you’d do is set the recorders default purge days to your long term retention time (e…g a year), then periodically run automations to purge the entities you don’t want to keep.

4

If you supply a sensor entity with a state_class option, its history will be recorded in long-term statistics which is a section of the database exempt from purge_keep_days (i e. the sensor’s history is never purged).

2 Likes
5

I have 2 different Raspberry Pi4 setups, in both most of the sensor data can be retrieved for much longer period (whole history?). See example:

image

…and this is regardless of what i put to purge_keep_days. As a matter of fact the other system I have it set it to 35 days and in the other I have not set it at all. I still get the whole history

This behaviour applies to , as far as I have checked, to all numeral sensors., whether being temp, humidity, power etc…

…and no, I’m not using at least all of them in statistic graph cards etc. that should make then LTS

Why is this?

6

A sensor entity with a state_class attribute never has its history purged in order to create long-term statistics.

7

@123 Yes, I was just checking this, and seems to be, that state_class is defined in all of them. I should have checked this earlier, sorry.

One interesting thing is, that it seems to be the case that only some cards show the LTS data, some just the db data. Below is the same sensor as in my previous post seen via HA history:

image
image1666×471 26.1 KB

…and it is only 10 days.

However, I’m not quite aware of the logic and processes, how this state_class gets defined by the system with some sensors automatically and some not. In some cases I have added it manually via customize.yaml.

There is a post in the Configuration forum, where I try to do it with command line -sensor without success: Command line sensor => statistic sensor: History purged after 10 days

8

If the integration is the kind that automatically creates entities, it will do it.

If the integration is the kind that you must create the entity, then you’re responsible for adding it and only if that integration supports it.

Was this for a legacy format Template Sensor? That’s the only integration that I know of that supports the inclusion of state_class in that manner.

If you configure a Template Sensor using modern format, state_class can be defined directly in its configuration (as opposed to separately in customize.yaml).

9

@123

Was this for a legacy format Template Sensor? That’s the only integration that I know of that supports the inclusion of state_class in that manner.
If you configure a Template Sensor using modern format, state_class can be defined directly in its configuration (as opposed to separately in customize.yaml).

Hmm… did you check the link to Configuration forum post? I am pretty sure I used modern format. state_class got defined OK according to the Developer tools/ States, and I also tried to move the part where state_class is defined to customize.yaml as well (seems to work too?) , because I cannot see the command line sensor LTS data with the way I described in the earlier post (e.g. with the history-explorer-card nor anything else, if that matters).

This made me think it does not work.

However, I have used separtae files (see the link’s example) for sensor definition. In configuration-yaml I defined these file as follows:

homeassistant:
  customize: !include customize.yaml
  packages: 
     eh800: !include eh800.yaml

Could it be this separate file type of structure that is not valid? E.g. definition file and it’s location stated under “packages”?

EDIT: Summed just for clarity: In command line sensor, state_class gets defined. Sensor data gets purged after 10 days, like it probably should. This time only LTS data (if it even exists) is also only 10 days

image

10

In theory, it shouldn’t lose its history after 10 days because of the presence of the state_class attribute. Report it as an Issue in Home Assistant’s Core repository on GitHub.

11

@123 OK, done

12

I have various sensors, which have a state_class set, but none retain data for more than 10 days.

Here’s one example, it only has 10 days data.

    #Victron MQTT

    - name: "Victron (Battery Power)"
      unique_id: "victron_battery_power"
      state_topic: "BatteryWatts"
      device_class: "power"
      unit_of_measurement: "W"
      state_class: "measurement"
      #value_template: "{{ round(0) }}"
      icon: "mdi:flash"

Any idea way this is?

Quite disappointed that all this data HA has been collecting, has been thrown away after 10 days.

13

You need to configure the recorder properly. purge_keep_days defaults to 10 days.

14

@PeteRage Thanks, but its clearly stated above that sensors that have a state_class are not purged after the default 10 days, yet mine are, why is this?

15

That is incorrect information. The docs do not say that.

16

This Sensor Entity | Home Assistant Developer Docs implies it, by stating they are long term statistics.

17

How to add it manually?
I have an custom made ESPHome temperature and humidity sensor.
The initial sensor .yaml didn’t contain state_class and device_class. I added them these days after many months. But now the entity card still shows History data and not LTS.
Is there a way to make it work now or should I purge all data, delete the entity and reinstall it from scratch?

Thanks

18

Statistics are different. You are looking at recorder data.

19

Well, I’m lost, its data, what’s the difference?

1 Like
20

I set up Home Assistant in June 2023. I studied the documentation for months prior to that, and also used this forum for many answers to many issues. I created sensors related to energy and the cost of energy, to collect data so that I could see use, costs and trends over time. This was for monitoring things outside of the energy dashboard. Six months later at Christmas time, and ready to analyse my graphs and stats, some of the data is recorded back to June 2023, and some is purged after 30 days. As only some of the data is there, none of the data is useful.

All of the sensors are template sensors and have:
state_class: total
device_class: energy or monetary

I can’t find any rhyme nor reason why some are recorded in LTS and some not. I thought it may be the monetary sensors, but some are fine, some are purged.

Is there any way to find out if the sensor data is being recorded in LTS or not?
It’s unacceptable to have to wait 10 days, or 30 days in my case to find out! That’s no kind of acceptable development lifecycle timeframe.
In Developer Tools/Statistics, all of the sensors state the source as recorder, so I’m just as confused as Ron-ski regarding PeteRage’s comment.

I’d appreciate it if anybody can shed any light on the matter?

Thanks.