Dbstats - Addon for database usage analytics

Hi! That’s my forst post here, don’t be too strict :slight_smile:

I saw many people who were struggling to analyze, what’s taking up space in Home Assistant. We have Quick Start to Home Assistant Data Science | Home Assistant Data Science Portal - but… That’s a bit complex even for me, with my 20 years of web developent.

So I decided to make my own addon, anyway I thought it will be a fun experience.

So, meet DbStats - addon with several simple charts to make things clear.
Installation instructions are here - https://github.com/jehy/hass-addons/tree/master/dbstats .Addon works with mysql/maria, postgres and sqlite.

One of the charts for example:

BTW, it’s built using Node.JS, TypeScript, React and MUI.
Feel free to use, test, star and contrubute.

19 Likes

Wow… thats nice.

amazing, also shows long-term ain’t as small as one might think:

1 Like

Yup. Also, we can add any other charts, it’s really simple with addon.

Nice, how about a deep dive into the long term, as you can see, thats 5-6 times bigger than the rest of my db

How do I integrate this for Mariadb and Influxdb? The log shows “exec /init: exec format error”. Not quite sure what to put in the config options connectionString* area.

1 Like

Hi! Feel free to contact me on github. Please post your config and logs without sensitive data to help me reproduce any bugs.

Thanks for this nice add-on! For total noobs like me, it would be useful to add the default connection string in the doc (sqlite:////homeassistant/home-assistant_v2.db).

Just run it on the DB and apparently the persistent_notification domain is by far the largest event source. I didn’t even realise I was using this. Any idea where I can start digging to reduce this?

I moved my database to the share folder, how do I configure the addon to look for the db in the correct place?


[Nest] 177  - 07/25/2024, 11:00:24 PM   ERROR [ExceptionHandler] SQLITE_CANTOPEN: unable to open database file
Error: SQLITE_CANTOPEN: unable to open database file

The default connection string in the configuration is sqlite:////homeassistant/home-assistant_v2.db, you should adapt this to use the location you set.

I tried:

/share/ha_db/home-assistant_v2.db

and

sqlite:////share/ha_db/home-assistant_v2.db

None worked

very interesting addon.
thanks for your work!!

This is awesome,
I got some very 1 sided data. Im guessing I should start using recorder to exclude sensors??





mine is Input_number

@jehy ,
just wanted to say thanks for this addon!
It’s really nice to get this sort of breakdown.
The only gotcha is that the connection string to configure is not so trivial, it worked for me with sqlite:////homeassistant/home-assistant_v2.db but I suggest you make this as default value (most of add-on users use home assistant OS and thus have the /homeassistant folder shared)

I installed a number of Everything Presence Lite sensors and indeed my DB size has grown a lot.

I’m struggling however in excluding them from the recording.
Here’s my recorder config:

recorder:
  purge_keep_days: 30
  exclude:
    domains:
      - device_tracker
      - media_player
      - uptime
      - time_date
      - worldclock
    entity_globs:
      # this assumes that all EverythingPresenceLite sensors are named sensor.epl_<something>
      - sensor.epl*target*

Does anyone knows if it’s OK to put 2 wildcards in an entry within “entity_globs” ?
After restarting HA, I used the Developer Tools → Action panel to purge the database and then repack it. The size has decreased indeed, but using “dbstats” addon I still see that the sensors named sensor.epl*target* still account for the vast majority of the DB size:

What am I missing?

This is a really useful add-on. Thank you. I’m going to include this in the Community guide on keeping the Recorder under control, unless you have any objections.

One suggestion, if possible would it be possible to make the entity text in the bar charts selectable?

This would make it a lot easier to copy the items to my exclude lists.

1 Like

Yeah that should be fine. Not sure why but I’m seeing the same issue with plain entity excludes (not glob). I just excluded the sb6 entities shown here as I don’t use them:

Restarted, purged and repacked but the graphs have not changed.

The database did reduce in size though:

Tried restarting the add-on, no change.

@tom_l ,
I have a theory here.
Today I noticed that the amount of “states” associated with my top-scorer sensor has decreased from ~2M to 1.932M.
So instead of increasing it decreased.
My recorder is configured to hold 30days. I believe what’s happening is that the Recorder has stopped to record new values for the excluded sensors but it will still take 30days to purge all the ones already recorded (prior to the exclusion rule I put in place)…

Will let you know…

A purge with “repack” selected should delete all the states though.

1 Like

I confirm that after 30days, all the rows in the database related to the excluded sensors have been removed:

So it took a while but finally my DB shrinked from >5GB to 700MB

Thank you, @jehy ! DbStats has been very useful for me in identifying the entities taking up too much space in my Recorder database (already reduced from over 2.4GB to under 1.7GB). One question I had (and couldn’t find anything about in the add-on Configuration or the linked GitHub repo docs, though I haven’t looked at the code yet) - is there an option to show more than the top 30 results in the States table (or otherwise filtering by entity)?