I was starting to get a notification saying that there was no disk space left, but when i looked it would have like 10gb free. I am running HA as a VM on ProxMox and i had assigned it 50gb of space. The weird thing is i run backups nightly natively in HA and those are FULL backups, but they are only about 1.5gb. How can it report 40gb used disk space, but a full backup is only 1.5gb?
Its been like this for a while. reporting 20-30gb used, but the backup size is still less than 2gb. its been like this for a couple months. But in the last week or so is when i started getting the 0 disk space notification.
I bumped up the disk space to 100gb total today. The used space has ballooned up to 71.2gb now. So an increase of 30-40gb just today.
I have about 25-30 current clamps (2 emporia Vues and a couple Shelly power monitors) and a handfull of SONOFF smart plugs which all monitor circuit and device power usage. They all send the watts used like every 5-10 seconds, but i have influx set to only keep that level of accuracy for a day or two, then dump the records. I have Influx set to keep 1 minute averages for each sensor indefinitely for now.
I would have assumed that this would have been the main culprit, EXCEPT for the fact that my full backups are so small. I just cannot wrap my mind around this discrepancy. I just told HA to perform another full backup and it is still only 1.5gb even with over 70gb used being reported.
if anyone has any ideas why my disk usage claims it is so high with my backup size being so small, i would greatly appreciate it!
A huge component of the backup (a compressed Tarball) is your database. Databases are incredibly compressable because they are usually comprised of a lot of repeating plaintext structures. It’s entirely possible to compress a few gigabytes into a couple dozen megabytes in ideal conditions…
That is interesting, I would not have guessed that! I will look into what commands i can run to view database size raw on the disk. I believe i have looked at this at some point before and although the databases accounted for like 4 or 5gb, there was still 20gb+ unaccounted for.
I will check again, thanks for letting me know that the databases may still be the issue.
Found the problem. I run Frigate as an Addon for Home Assistant. I have a couple cameras feed into it. I had setup Frigate to save its data to a NAS, but apparently something happened about a week ago which caused Frigate to revert back to saving the recordings to the local disk.
If i remember right, I ran into this a couple times before, but it was when i was first setting up Frigate and I was having trouble getting it to behave. Once I had it setup, it never did this again… until a week ago. It is a bit frustrating that the Frigate addon REALLY wants to write local. I am not super familiar with Linux in general, but it seems that the way Frigate wants you to setup network storage is a bit of a hack…
It has you move the Frigate DB out of its default location, then rename or delete the Frigate folder it created in the Media directory. Then setup Network Storage named Frigate in HA, which in turn creates a Frigate directory in Media that is attached to the NAS which the Frigate addon will write to.
To me this all seems like basically tricking the Frigate addon into using the NAS. As I said though, I am not super familiar with Linux, so maybe this is standard operating behavior?
The problem seems to stem from when HA fails to establish a connection to the network storage. When it fails, I assume the frigate folder in Media disappears. Then when the frigate addon starts up, it creates a new frigate folder in Media and starts writing to it, which means it writing data locally. Also, since the folder exists now, HA cannot recover the connection to the NAS because it uses the same folder name as what the Frigate addon just created.
To be clear, I am not frustrated with how HA handles this situation, I am frustrated that the Frigate addon does not have a better/more robust setup for saving the video data to a network share. Even just having it “fail gracefully” by sending HA a notification saying “cannot access the network share” and then stop saving recordings until the connection is restored would be preferable to me. I probably should make my way over to the Frigate github and report a problem
this is typical because of the way that addons work in HA. It is much simpler when you have direct control of the docker stack but since addons set a static media location this is how it must be done.
that’s not how it works. Frigate does not know if the storage is network or local and that is by design. When running in docker frigate simply gets a location to write to, it is totally isolated from the OS and it doesn’t know how that storage is being provided. The robust handling needs to be done by the OS or docker stack, and frigate has access to neither of those. The user also does not have a way to fix that when running in HA OS (hence why running frigate in docker directly is recommended, because then you can configure the network storage to be required)
Thanks for the info, I have never operated with dockers directly. The main reason I went with Frigate as an addon to HA was because I assumed that integrating it with HA would be simpler this way. I might investigate running Frigate via ProxMox as either a VM or a container instead since that’s how i have HA setup anyway.
