I recently connected some Tuya Zigbee Dual Clamp Energy Meters (Tuya PJ-1203A) through Zigbee@MQTT. Works nicely, but they explode the size of my database (home-assistant_v2.db). SQLite Web shows over x1000 data compared to other sensors! I changed the Update Frequency of these devices to 60s (max) but its change is marginal. Any idea what is going on, and how to resolve this issue? I prefer a solution for this particular device - and keep default purging etc.
The illustration below shows the impact of adding one Clamp Energy Meter with the lowest update frequency (60 s). Growth continues rapidly to several GB.
What is being stored? Is some attribute changing every 60s, forcing a whole new record, complete with all attributes (even those which haven’t changed)? That’s a bit of a design flaw with Recorder. You can see the attributes in Developer Tools / States.
If it’s recording more data, and/or more often, than you want, you could exclude this entity from Recorder and create a template containing just the data you want to keep.
I’m curious to see what you find. I’m thinking of adding a couple more power monitoring smart plugs and don’t want to run into something like this myself.
Good point. I put a screenshot for one the entities of my sensor (device/sensor has 9 entities). Remarkably, it comes with many irrelevant attributes, which I marked with a red line in the image below. I compared this with a another sensor of the same type in a different HA system (without data explosion) and this entity dos not show the red-marked attributes. Any idea why these irrelevant attributes are captured? This might cause the data explosion.Thanks.
Below, I also add the same entity of another device of the same type in a different HA system. No data explosion is seen in that setup.
No. But I have my suspicions. The way the DB is designed, “attribute” data values are duplicated in every new record for the entity, even those which haven’t changed.
Your use of the term “data explosion” is very descriptive of what happens when an entity is loaded up with attributes.
My solution, touched on above, is to exclude these “chatty” entities, and instead create a template entity to pull out just the value or values I actually want to record changes for.
For example, the climate entity has a ton of attributes I don’t care about. So I created a template to extract just the temperature:
- name: "guest_rm_temperature"
unique_id: "guest_rm_temp"
unit_of_measurement: "°F"
state: "{{ state_attr('climate.guest_rm', 'current_temperature') }}"
Update: After posting this I found some good news in the update notes thread. Apparently there’s some recognition that using attributes this way has negative effects, and there’s been a conscious decision to start moving away from using them.
2 Likes