Be Very Careful with the Unifi Integration...and I'm sure others as well

Third party integrations
1

When I installed the Unifi integration, I made the mistake of checking off the “uptime” sensor without filtering out sensor.*uptime in my recorder. As a result, in a matter of days, my database grew from 6GB to 17GB, recording multiple entries for every device on my network every second of every day.

There’s no warning that indicates that this is a potential result, and I think the integrations should all come with a rating or icon that indicates that it could be a data spam integration .

Perhaps when we install each add-on or integration, a prompt could remind us that “this integration will cause excessive data collection without proper measures in place. Please read more about how to filter entities from being recorded to your home assistant history.”

5 Likes
2

This just happened to me too. I’m in the process of trying to clean some records, but it’s very slow going. Plus, the integration’s created entities don’t share an easily filtered verbiage. I’ve since removed the integration and will get this good enough until the records get purged.

Have you found a good way to clear these records?

Edit: I created a feedback for the official Unifi integration page to give warning to others, Unifi Integration Options (Uptime/Rx/Tx) can cause large DB increases · Issue #18501 · home-assistant/home-assistant.io · GitHub

1 Like
3

I’m using MySQLWorkbench and phpmyadmin depending on how I feel that day. Using simple DELETE FROM query to target specific records. It’s taking forever.

4

Got it.

I ended up leveraging the Recorder’s purge settings. With the below in settings. I’m currently just running purge buck will try to repack eventually though it leverages more space which isn’t fun when my DB grew to 22 GBs

recorder: 
  exclude: 
    entity_globs: 
    - sensor.*_uptime 
    - sensor.*_tx 
    - sensor.*_rx
1 Like
5

It appears that the most recent update addresses something on the lines of this problem:

6

This is not an integration problem per say, but a fundamental architectural problem with the HA database. HA essentially uses the database as a flat file, storing huge amounts of redundant data with every state change that takes up 10,000x the space it should, and has 1% the performance it should.

We as users have no choice but to work around this problem by managing the recorder service and excluding data.

1 Like
7

I know your post was some time ago, but can you explain how you cleaned up the “uptime” mess created by the UniFi integration? What Recorder are you referring to? Where do you place that yaml code?
Thanks.
EDIT: I think this is what you were referring to: Recorder - Home Assistant

I have removed the UniFi integration, but my overall data file is about 300 MG larger than it was. I’m glad I caught it in one day’s time.
Using the Recorder to remove the data sounds more difficult than it’s worth. Can I clean up the data, then remove Recorder?