Cleaning Up Unused Files to Free Up Disk Space

[GoodyGizmos]

I’m running HAOS on an RPi 4. Recently, the 32 GB SD card (this post isn’t about if you like HA running on an SD card) filled up and corrupted my database (no big deal), so a new database was started. This set me on a quest to find all of the files which were filling it up.

I found a few GB worth of Amazon package images in the custom_components and www folders from the Mail and Packages integration, but still couldn’t account for the majority of the used space as reported by HA. So, I ssh’d in and Google’d Linux commands to assess disk usage.

The backup folder appears to be the primary offender at 14.5 GB (/media is a mounted share network storage…not on the SD card)! Here is a detail file list of the backup folder:

1. Can I delete all of these files without issue?
2. Why aren’t these files listed under System>Backups?
3. Why does the disk usage only total ~16GB (~50%), but HA System>Storage indicates I’m using 90%?

[ShadowFist]

Delete the older (no longer relevant) backups from the front end, download the rest locally, then delete everything except the latest one in case you’ll ever need it.

That’ll give you some breathing room while you figure out where the remaining disk usage is going. My best guess is addons, since they live in a separate container.

[FriedCheese (Ryan)]

@ShadowFist

addons

Probably.

What's using my storage?

tl;dr - Running du -shc /* | sort -h from the SSH add-on will show most everything except add-ons. To see add-on sizes, run docker ps --size from console or SSH access into HAOS (not the add-on). Running this from the SSH add-on will only show you the storage seen by the HA container (I’m probably wording some of this technically incorrectly). [image] Runnng the same command from console access (or SSH access into HAOS) gives you the full results: [image] Then you can start iterating throu…

[GoodyGizmos]

See question #2. None of them are on listed on the front end. I want to delete all of them and start fresh, so my question #1 is just confirming that deleting all of them including the _swap.swap (a file type I don’t recognize but is taking up 2.0GB) won’t affect Home Assistant.

Using…

docker ps --size

…and totaling all of the virtual sizes (I think that’s correct) results in ~7.5-8.0GB (a couple containers use the same stack). Added to the 14.5GB backup folder and the homeassistant, var, and usr folders gives ~23.7GB, so let’s say 24GB. I know the math gets wonky when converting bits and bytes between MB and GB, but big picture, it seems I’m still missing ~8GB (32-24).

[GoodyGizmos]

We were typing at the same time. Please see my reply above and let me know what you think. The docker virtual size results are from ssh’ing in via Putty.

[francisp (Francis)]

@GoodyGizmos

Why does the disk usage only total ~16GB (~50%), but HA System>Storage indicates I’m using 90%?

Because you SSH to the Home Assistant Container, not the OS.

To SSH to the host, you need port 22222

Debugging the Home Assistant Operating System | Home Assistant Developer Docs

[GoodyGizmos]

OK. I just tried interpreting those instructions. Here’s my translation of the steps I took:

1. Formatted a USB drive in FAT and named it CONFIG
2. Using PuTTYgen and the default settings (RSA/2048)…generated a Public Key (and Private Key with passphrase…though seemingly not needed) and saved the Public Key with the name authorized_keys (no extension) to the USB drive.

*Admittedly, I don’t know how to set/verify this…File needs to be ANSI encoded (not UTF-8) and must have Unix line ends (LF), not Windows (CR LF). So, I just moved on.

3. Popped the USB drive into the RPi 4 running HA

4. SSH’d (PuTTY) via port 22 and executed the ha os import command resulting in a Command completed successfully message

5. Closed the port 22 SSH session

6. Re-opened PuTTY and the same SSH setting, but changed the port to 22222 and clicked Open. After a few seconds, I was prompted with a warning to accept the connection. I accepted, but was then presented with a login as: prompt. Entering , hassio, homeassistant, or root (neither of which should be required if I understand this right) then pressing Enter gives me this error:
   

7. Rebooted the host system and tried (6) again…same result

Any ideas?

[FriedCheese (Ryan)]

You need to import the key you generated into putty and use that for the credentials when you connect on port 22222. No username/password.

[ShadowFist]

I could be wrong here, but I seem to remember there are 2 different SSH add-ons. One is called terminal & ssh, while the other is called ssh & terminal.

Only 1 of those will let you access port 22222 - the other is only limited to port 22. It probably won’t apply if you’re trying to do everything via command line, but you might want to check the addons out since they’re simpler to configure.

[GoodyGizmos]

OK. I followed these instructions using the private key I generated in PuTTYgen in step (2) above, but was not given the option to input the passphrase I entered in PuTTYgen.

[GoodyGizmos]

I think both of them give access to port 22, but I did find this.

I tried configuring it multiple times as well, but still cannot seem to connect.

[ShadowFist]

Can’t offer much help on that I’m afraid. I could never get port 22222 to connect, but got around all that when I set up HAOS in proxmox instead of running direct.

The FAQ in the first post of the addon you linked to suggests a bunch of stuff, including no password, disabling protection mode & full restart of the physical hardware. Not sure if you’ve tried all that, but figured I’d give you a heads up just in case.

[FriedCheese (Ryan)]

That looks to me like HA doesn’t like the public key (maybe formatting). Instead of using a USB, I opted to go this route:

HOWTO: How to access the Home Assistant OS host itself over ssh

It took a while to figure this out, so I might as well document this. Looking at the amount of search hits on this subject, I wasn’t not the only one that needed access to the host itself. When debugging an issue related to the OS or docker, you might need access to the host itself. The Terminal & SSH Add-on drops you in a container, while this might be enough for users, you sometimes want to access the real operating system, to examine some docker issue for example. There is a well written do…

So, at step 2, you’ll open the publickey file and copy the contents into the key section under configuration for the Terminal & SSH add-on. Then, use the add-on to run the commands for steps 2 and 3.

I’d also use a password free private key.

[GoodyGizmos]

The public key from PuTTYgen looks like this:

  Comment: "rsa-key-20240310"
  abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12
  abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12
  abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12
  abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12
  abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12
  abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwx

I use the Advanced SSH & Web Terminal (not Terminal & SSH) add-on. It’s Configuration>ssh section looks like this currently:

username: hassio
password: mypassword
authorized_keys: []
sftp: false
compatibility_mode: false
allow_agent_forwarding: false
allow_remote_port_forwarding: false
allow_tcp_forwarding: false

So, do I just make it look like this?

username: hassio
password: mypassword
authorized_keys:
Comment: "rsa-key-20240310"
abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12
abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12
abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12
abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12
abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12
abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwx
sftp: false
compatibility_mode: false
allow_agent_forwarding: false
allow_remote_port_forwarding: false
allow_tcp_forwarding: false

After that’s properly configured and saved, what exactly do I do? Are you saying to restart the add-on? Should I change this to 22222 in the add-on Configuration as well?

[GoodyGizmos]

Thank you and yes, I had read/complied with all of that already.

[ShadowFist]

Try:

ssh:
  username: root
  password: ""
  authorized_keys: [
    - ssh-ed25519 AASDJKJKJFWJFAFLCNALCMLAK234234.....
]
  sftp: false
  compatibility_mode: true
  allow_agent_forwarding: true
  allow_remote_port_forwarding: false
  allow_tcp_forwarding: true

Not sure whether your key needs to be passed as a string within quotes or not. Can’t remember from when I had it set up. Also, it’s unclear whether the password is required or not. Documentation for the addon shows an empty password, but there are some github issues which state that the password is required.

[GoodyGizmos]

I tried a handful of variations. No dice. Meh, thanks for all your help . I can’t get any of this to work and it’s probably not worth it anyways.

It seems even for these supervised installations they could provide some way to get insight into what’s taking up space on the rest of the disk outside the HA container.

[donburch888 (Don Burch)]

@GoodyGizmos

I’m running HAOS on an RPi 4.

If you’re having problems SSHing to the console, can you just attach a monitor and keyboard ?

I am also trying to track down what has been eating my disk space - the samba exposed HA partitions only account for 11.3 of the 32GB. I’m now looking for program sizes, system logs (I have had periods of generating >10K warning messages per day, and wonder if they are hanging around), etc.

And yes, there should be a better tools within the HA GUI to see what is taking up the space, and allow you to clear it.

[GoodyGizmos]

I could, but it would be more of a PITA than just living with not knowing.