What do you exclude from the recorder?

Hi all,

Some time ago I moved to using mySQL for the recorder in HA. Noticed my database is 1.6GB in size. That got me wondering, is there any low hanging fruit with regards to stuff I can exclude from the recorder without affecting HA?

For instance I use the time sensor and datetime, noticed every minute its writing its new time value… could I exclude that? This also got me thinking if there is any other useless stuff being written by default.

Thoughts?

See this good guide by tom_l

1 Like

Also you should notice an improvement after updating to (the yet to be released) v0.112. It has some optimisations in how it stores states.

And GLOB filtering support :slightly_smiling_face:

1 Like

so, like this?

exclude:
  entities:
    - sensor.*_voltage

Yes. Although technically the syntax will be this

exclude:
  entity_globs: 
    - sensor.*_voltage
2 Likes

some good hints here thanks chaps

As far as I am aware, excluding entities from recorder would not affect HA. Just that you would not be able to see the history state of that entity. I am running on a NUC, and I have influxdb running in parallel which records and stores all the sensors. As for recorder, I excluded most of the sensors that have a lot of data coming in (I have several Bruh DIY sensors), and I have set the purge_keep_days: 2 to keep only two days worth of data. If I need to see anything beyond that, I access influxdb through grafana. This way, the database size is ~700MB.

Finally! my recorder config is starting to get a bit unwieldy

It affects the reliability if using an SD card.

Yeah, One of the reasons I moved to a NUC.

@tom_l @bacco007 actually I may have been off about the timeline. The PR for glob filtering support was merged into dev about a week ago so I assumed that meant 0.112 release. But looking at the beta release notes I’m not seeing it so I think it may be 0.113. Still learning how the release schedule works, sorry about that.

2 Likes

As long as it’s in the pipeline, I can wait.

Just confirmed it’s in the 0.112 beta (I applied a glob filter to recorder). It is in the release notes, but it’s buried in All Changes at the bottom (search for “glob”): https://rc.home-assistant.io/blog/2020/06/24/release-112/#all-changes

2 Likes

Oh I did miss that. Thanks!

There are discussions about how to find stuff in the database that takes a lot of space in the forums. Some of the stuff you’ll find takes a lot is space and is crap. For example the sun component was a huge offender of this until it was recently fixed.

Dig around and you’ll find something I’m sure.

trouble is I have a pzm setup to monitor my mains… that’s about 10 sensors recording values every 10 seconds. I want to keep this setup as the values are important to me but it throws up a bit of a snowstorm in the db so its hard to spot anything else that might be contributing a lot :slight_smile:

One option would be to exclude that data from the HA DB and include it in InfluxDB instead. InfluxDB seems to be a lot more efficient in it’s data storage. Frenk did a setup video for Influx and Grafana:

1 Like

Everything!
Seriously, I exclude everything and then item by item, include those that have any purpose for me.