Recorder / History: Different Purge Settings for specific entities?

Configuration
1

Yesterday i was a bit shocked when i found out that my DB is growing like hell. (~ 200mb/Day). It didnt use to do that before - i think its the new four tasmota Plugs i integrated a couple of days ago, which report every 10secs 20 status each.

Since i want to use some sensors for a short timespan (like 24hrs) to observe if there is an unusual consumption, some would need to be recorded only once per day.

So, what would be your suggestion how to configure, whats your best practice hint?
Is there a way to keep specific entities for 24hrs and get logged every 10 secs, whereas others will be kept for 14 days and logged only every xx hrs, or just in case of changes? Can i just run multiple, different “recorder” scripts?

Or is everything just outdated, and i overlooked a way to configure those behaviors in the gui per Device/entites?

1 Like
2

There is only one global purge keep days setting.

200MB/day means you will have a 2GB database at the end of the default purge interval (10 days). This is not too bad. Home Assistant can handle that.

I keep 7 days data and while I use excludes extensively it still tops out at about 2.5GB.

Here’s some advice on excluding entities:

https://community.home-assistant.io/t/how-to-reduce-your-database-size-and-extend-the-life-of-your-sd-card/205299

You could also shorten the purge keep days and use InfluxDB for long term data storage.

You could also slow your sensor reporting intervals.

1 Like
3

Thanks! Well, sounds like i can relax a bit :wink:
Anyway - there are a lot of States which don’t change often (or at all), but get recorded every xx seconds. Is there a way to record specific entities only on change? I think i read it yesterday somewhere, but i just cant find it… :frowning:

4

They only get recorded when they change.

1 Like
5

It looks like i wanna open that again, after having a closer Look at my Database:
If just a Value of one Sensor changes, all Sensor States get written. Even the never changed Record “Start Time” exists as often as “Current” (per Device).

Right now, I do have roughly 1.000.000 Entries for States in Total, where around 800.000 are just from two Energy Meters. (Statistics: Around 7.000 per Hour :crazy_face: )

Is there a more elegant, yet not too complex way to only record States if they change, not if any Sensor of the Device changes?

6

+1 for this

I have like 10 entities that I would like to record for 30 days. For the rest, the standard setting is just fine. Since we cannot configure different purge intervals, I now have to record all of them for 30 days and the DB is growing. For most of the entities, this is just useless.

7

Any luck on finding something for this? I have only a few entities which I’d like to keep for a long period while the rest I’d still like to see the 10 days of data for.

8
9

I’ll use InfluxDB and some NodeRed magic in the meantime for the long term statistics I need. Quite a huge overhead, but it seems to work at least.

1 Like
10

I’d also like to see separate settings to keep more or less data for some sensors
maybe we need to add this to an enhancenment request :slight_smile:

11

There is already a feature request for this: Recorder Retention Period By Entity

As I understand it, it is unlikely to happen as it would require a major architectural change.

12

The good news is that there are ways to work around this limitation. The thread for that FR linked above goes into some detail. Basically, you can create an automation to call the purge_entities service to do whatever clean-up you’d like.

The issue is not really the architecture, but the efficiency of the query which would be required. Instead of simply deleting all records older than a certain date, one would have to select on different dates for different entities. Not insurmountable, but it doesn’t seem like something the HA database administrators want to take on.