I run the latest OS and Supervisor of Hass on a Raspi4 - currently migrating to Pi5.
I have mariaDB installed and had recorder settings to 320 Days. These settings gave me huge backups (12GB).
However, I tried to delete the recorder settings a few days ago (recorder.purge) as suggested in some topics here but my backups are just getting bigger.
How can I solve this?
I dont care anymore about long term stats of the recorder - the only entries I would like to keep are the ones from my energy dashboard. I have solarpanels and battery installed and would like to have these energy entries e.g. forever.
The backup is probably large due to a default configured recorder.
Look into how to manage the recorder, so you only store the data you need and want.
A quick fix right now is to make a service call for recorder.purge that limits the days kept and replace the database.
You can do this from Developer Tools → Service tab → Search for the service call “Recorder: Purge”
Try to open up the backup to see what is inside and find the culprit.
It is just a gzip file, which can be open with Winzip/WinRAR/7-Zip or similar.
Inside the backup you might find the different addons backed up and can see there size.
These addon backups can also be open up with the same programs to look further into the stored files.
Here we go again. I feel like I’m screaming into the wind sometimes.
The default settings for Recorder are inappropriate. The new user documentation gives few hints about how important it is to configure Recorder from the start. The UI gives no option to include or exclude each entity as it is added, or when it is displayed. There’s only one keep_days parameter for all entities, so no fine-tuning is possible without writing automations to selectively purge, which only increases writes instead of reducing them.
There are FRs about this. There threads about this, explaining how to configure the Recorder settings for the best compromise, given the lack of fine-tuning options. Yet nothing changes and users keep encountering problems.
I’m open to suggestions on how best to elevate this issue so future users don’t have to keep bumping into Recorder database size problems.
Ok Captain, thx for your word - however I did not hear you screaming in the first place
I did reconfigure my recorder settings to sth like follows (in configuration.yaml)
Do you have any suggestions on how to tweak this? Are there special configurations for energy dashboard so that I do not loose my e.g. solar production?
I have also read that only after uptime of more than 24h the purge will start →
recorder:
db_url: mysql://homeassistant:XXXXX@core-mariadb/homeassistant?charset=utf8
commit_interval: 60
purge_interval: 1
"#" purge_keep_days: 320 -> this was the old setting - commented it out
exclude:
domains:
- weblink
- updater
- input_boolean
- input_number
- input_select
- input_text
- light
- media_player
- sun
- timer
- weather
- camera
In Service, you can “Purge per entity” , Choose Purge All for the entities you want ( You can choose multiple in 1 Purge
You can even Purge per Domain and entity-Globs
Choose “keep 0 /zerro” Days
Then click Call Service
Then Choose Service “Purge”, un-tick “Apply Filter” And Tick " RePack "to the left, Make Sure to “toggle” the the “Repack” to the right.
Then click Call Service
If You Don’t choose “Repack” your DB remains roughly the same size
PS: If it turns out that your “Energy Datas” is the “culprit”, taken to much space, you are doomed, and need an external DB, where you can “shuffle” Datas to, and on monthly/yearly basis to another Table/DB
I went a different way, excluding entities rather than domains. I really just excluded the “heavy hitters” which were (1) spamming the database, and (2) I didn’t really care about. Not perfect, but good enough. It keeps any data I might care about or which doesn’t take up much space anyway. But really, there’s no right way to do this. The important thing is you pick a method which works for you and stick with it.
I don’t use the energy dashboard, but I suspect it uses the long-term statistics tables in the database. IMHO these tables should be in a different database, since they have different retention and performance requirements. I just send this type of data to .csv text files and manage/analyze them outside of HA.
Right, i did the opposite, still no exclude, Only Include !
Which i btw think should be a fairly Default settings, to remind people upon, if they want/need to track anything , Choose To
Could be an easy interface, or “tick box” i.e ( Want to track this entity ? ?
Many people have no idea what’s been tracking and how much data it “collects” even in default 10 days
And they start fast to implement various template-sensors, and tweek update-time-setting if possible to get every 1 seconds ( in worse case )
Basically, Move Recorder into UI
EDIT: Same with Purge … tick entity, glob, domain , click Purge&Repack, all the functions is there , it’s just a GUI Guru, with the knowledge of this parts which is “missing” , and im not that ( or only in my head
Hi, did it like you suggested. Could not see any effect. However I did not try to do a backup by now. Will do this tonight after HA is up for more than 24h.
Will keep you posted.
Good morning, automatic backup after uptime of 30h was created successfully.
Backup size after tweaking my configuration.yaml and calling the purge service as suggested is now 750MB.
This is perfect.
The statistics are also not fully gone-however the data shown is now divided in two parts: Long term statistics and history. The history shows data every minute, the long term stats displays hourly or daily - depending how long you go back.
