WTH can't we specify the purge_keep_days in Recorder per entity?

I would like the history of some entities I use in graphs (like energy usage, temperature, etc.) to be kept for a whole year, while most of the other entitles I don’t need the history for a whole year.

Right now the purge_keep_days is a setting for all of the entities, it would be great to customize it per entity.

If you like this idea, please up-vote the existing FR, too:

Would be nice to have the option on the devices screen as well so we don’t have to code it in yaml for everything. Open the overview screen for a temp sensor see the option for data retention and then be able to choose between how many days, weeks, months…etc

2 Likes

This would be very useful for me, too. I want to keep the states of for example security relevant sensors a few weeks while I don’t need the temperature from more than 24h ago since I can use long term statistics for that.
I have a lot of sensors which create a lot of states, so I clean the history for these entities every night. But that’s not a good solution since I’d rather always keep the last 24h of history.

i was just thinking it’s weird that in order to capture seasonal variation in temperatures or power usage i also just end up with a creepy amount of location data on my partner and my roommates. As much as i trust myself more than some company, it’d be nice to be able to say that i don’t keep that data.

2 Likes

Just remove location data completely. Why would you need that in the history in the first place.

I mean that’s probably the one I’d keep longer. Not on roommates obviously, that’s not ideal. But I used to like Google’s location history feature other then it being Google, a private version of that is nice.

And even if you don’t want to keep that long term keeping it for at least a few days is really handy for debugging. Assuming you have automations around that.

Not sure when the functionality was added, but I’ve discovered the option of running a service to purge specific devices/entities !

So I guess with

  • the standard purge days,
  • many ways of filtering off what is recorded,
  • and to setup automation(s) that will purge some devices/entities with fewer keep_days than the standard

then all requested functionality is awailable ??

see my reference post on how I got back +10mill rows :wink:

Good write-up, thanks!

Of course, as you point out, this is just a workaround. In fact, this workaround doesn’t reduce the number of database writes at all, but increases them. From a data perspective, it has the same result. But if you’re using an SD card, each write is shortens the life of the card and brings you closer to an inevitable crash.

From a design perspective, it’s hard to argue that all data should be treated equally. It’s not an unusual use case where some data need different retention periods than others. It’s almost guaranteed for the vast majority of users. Yet for whatever reason this design choice has never been revisited, even as HA has grown and seen extensive changes in both functionality and UI.

Not sure I understand,
My referenve to workaround was the deletion of rows when I had an integration that somehow got triggerhappy. Have reconfigured it now, but looking for ways to cleanup I find it solves the issue of defining recorder-settings pr entity - hence this FR is actually solved (in a slightly different way)?

And this purge method can not increase number of writes - can it?

  • my sensor is getting a state change, and it is written to states-table (and atributes etc)
  • you can use Recorder-filter to avoid sensors to write to recorder, but if it is written - it is written only once?
  • Then the default recorder keep-days clean my states on that setting (currently 9 days)
  • and If I want to clean out something else I create a simple automation to clean out before (one device now after 2 days).
  • and I can create a new automation that purge for example lights-domain after 5 daysl

Then we all have some sensors with LongTermStatistics that is kept forever

So I have written to my DB

  • LTS forever
  • most states for 10days - then deleted
  • some states for 5 days - then deleted
  • some states for 2 days - then deleted
  • and some sensors excluded from any recording

states are written once, deleted once - it is just the timing/method for deletiion/purge that is different ??
STS/LTS created every 5min/hour, STS purged with states for entity?

If you want to reduce writes - you cant look at purge, then you must look to filter ?? - or as usual, there is something I’m missing :wink:
#learningcurve

introduced to support the energy dashboard is Long Term Statistics (LTS)

you can configure sensors to store LTS:

Now you can see states and LTS in history-tab (from 2023.12)

I’ve used custom history-explorer-card to show combination of lts/states in a more dynamical way

1 Like