So after trying a manual recorder.purge last weekend without having any apparent effect, my DB has suddenly increased roughly ~1.2gb in the last week (growth looks to be ~250mb a day). This seems wrong since my db has been relatively stable for months and months at roughly 2.4gb. Am using postgres f…

Scroll down to, 2. Good-enough approach in this to see some SQL queries you can run to see what is causing the most data: [image] How to reduce your database size and extend the life of your SD card Community Guides Preamble: @tom_l is the original author of this communit…

Thank you. Just what I was hoping to be pointed towards. I will have a detailed look at that later this evening.

But in general most probably the reason is that you add some sensors that generates a lot of data (like CPU usage) - value changes at every measure - so a lot of records into the DB. And as the DB is not into the best format - it grows like a hell.

I considered that case with new sensors but that doesn’t seem to be it. Ironically, I do have a lot of cpu & memory related sensors that were not in my exclude list, so I have now added those. Bigger picture, I didn’t seem to have this runaway growth issue until I tried a manual recorder.purge serv…

[image] marwun01: what is the expected outcome after running that process manually? Definitely a decrease in size. Especially if you set repack: true. This can be quite an intensive operation for large databases though. Seems like you have a very chatty entity somewhere in there. Did you hav…

For the states table yes, but not on the events table. However, db size growth seems to be flattening since I updated the exclusion list so I’ll give this a bit more time to confirm. [chrome_AohBhoPJES]

That’s still quite enormous. What do you have purge_keep_days set to (10 days is the default if you have not set it)?

purge_keep_days is set to 14, so pretty tame I would say. 1.5gb of that size has been the runaway growth over the last week (there was a very slow creep prior to that…prob due to new sensors). Per my last post that seems to have abated but I don’t understand why it started in the first place. None o…

You might try and dig a bit deeper, this may help you discover where the size issues are at a more granular level. Just seeing the meta number of your database size growing is enough info I think to diagnose. Below are a couple things you could try and run over a couple days and see it give you a …

@dproffer , thank you for all the great detail! Very Helpful. I’ve apparently missed the logger integration and have since added. SQL queries - what I’ve been able to determine: Query 2 - Purge worked, event & state data only goes back 3 days (my setting is 14 days but I probably performed it manu…

DB size suddenly increasing ~250mb a day in the last week.....was previously stable (in postgres)

Configuration

tom_l February 19, 2021, 6:27pm 2

Scroll down to, 2. Good-enough approach in this to see some SQL queries you can run to see what is causing the most data: