SecureTar is a HomeAssistant abstraction.
It looks like normal tar.gz archive but it isn’t - that’s why you can’t just uncompress it.
First 16 bytes - is a salt
Then goes AES 128 CBC encrypted data. (the data is a tar.gz archive) Key = your password. IV calcultaed based on salt and key by following code:
def _generate_iv(key: bytes, salt: bytes) -> bytes:
"""Generate an iv from data."""
temp_iv = key + salt
for _ in range(100):
temp_iv = hashlib.sha256(temp_iv).digest()
return temp_iv[:16]
I can’t decrypt it via command line tools because of too complex iv calculation function. Maybe we can just use python to decrypt it.
Do you think this changed with a release around January this year? Because I have no problems decrypting backups from before January 2019.
As long as I can actually restore from an encrypted backup this would be ok. Will give it a try. I would much prefer to have access to the files in the backup though.
I have to admit, I didn’t have the nerves to test it and just changed my backup to unencrypted.
But since you insisted I just tested it now
Yes, I was able to restore the encrypted backup. I am just not able to decrypt it manually with 7-zip.
Sometimes I wisch I could access single files from my backup.
For those who are still looking for a solution: i have created a simple python script to decrypt a secured snapshot. Off course it would be nicer to have the option to extract a snapshot from the ‘hassio’ tool, but this works. You will need to have python3 installed and the cryptography package.
#!/usr/bin/env python3
import sys
import getopt
import hashlib
import tarfile
import glob
import os
import shutil
from pathlib import Path
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.ciphers import (
Cipher,
algorithms,
modes,
)
def _password_to_key(password):
password = password.encode()
for _ in range(100):
password = hashlib.sha256(password).digest()
return password[:16]
def _generate_iv(key, salt):
temp_iv = key + salt
for _ in range(100):
temp_iv = hashlib.sha256(temp_iv).digest()
return temp_iv[:16]
class SecureTarFile:
def __init__(self, filename, password):
self._file = None
self._name = Path(filename)
self._tar = None
self._tar_mode = "r|gz"
self._aes = None
self._key = _password_to_key(password)
self._decrypt = None
def __enter__(self):
self._file = self._name.open("rb")
cbc_rand = self._file.read(16)
self._aes = Cipher(
algorithms.AES(self._key),
modes.CBC(_generate_iv(self._key, cbc_rand)),
backend=default_backend(),
)
self._decrypt = self._aes.decryptor()
self._tar = tarfile.open(fileobj=self, mode=self._tar_mode)
return self._tar
def __exit__(self, exc_type, exc_value, traceback):
if self._tar:
self._tar.close()
if self._file:
self._file.close()
def read(self, size = 0):
return self._decrypt.update(self._file.read(size))
@property
def path(self):
return self._name
@property
def size(self):
if not self._name.is_file():
return 0
return round(self._name.stat().st_size / 1_048_576, 2) # calc mbyte
def _extract_tar(filename):
_dirname = '.'.join(filename.split('.')[:-1])
try:
shutil.rmtree('_dirname')
except FileNotFoundError:
pass
print(f'Extracting {filename}...')
_tar = tarfile.open(name=filename, mode="r")
_tar.extractall(path=_dirname)
return _dirname
def _extract_secure_tar(filename, password):
_dirname = '.'.join(filename.split('.')[:-2])
print(f'Extracting secure tar {filename.split("/")[-1]}...')
try:
with SecureTarFile(filename, password) as _tar:
_tar.extractall(path=_dirname)
except tarfile.ReadError:
print("Unable to extract SecureTar - maybe your password is wrong or the tar is not password encrypted?")
sys.exit(5)
return _dirname
def print_usage():
print(f'{sys.argv[0]} -i <inputfile> -p <password>')
def main():
_inputfile = None
_password=None
try:
opts, args = getopt.getopt(sys.argv[1:],"hi:p:")
except getopt.GetoptError:
print_usage()
sys.exit(2)
for opt, arg in opts:
if opt == '-h':
print_usage()
sys.exit()
elif opt in ("-i"):
_inputfile = arg
elif opt in ("-p"):
_password = arg
if not _inputfile:
print ("Missing inputfile")
print_usage()
sys.exit(3)
if not _password:
print ("Missing password")
print_usage()
sys.exit(4)
_dirname = _extract_tar(_inputfile)
for _secure_tar in glob.glob(f'{_dirname}/*.tar.gz'):
_extract_secure_tar(_secure_tar, _password)
os.remove(_secure_tar)
print("Done")
if __name__ == "__main__":
main()
Thanks @Taapie !
This deserves to be pinned up somewhere - or better yet, included in the core toolset. After much digging around to figure out what this “diy crypto” format was that they’d introduced, I was about to start writing my own script until I found yours.
Appreciate your taking the time to write it, make it human-friendly (usage message, getopts etc.) and share it with us.
For anyone with their brain in neutral like I had for a moment: this script operates on the outer archive, not the individual .tar.gz files you get when you naively untar.
I need some help. How did you use the script to brute force yours?
EDIT: Never mind, I got it working after much hacking on windows. I was able to confirm that the password I thought should be right for the snapshot. Now I need to figure out why its not restoring
For all those struggling with password with special characters like ` or $ and using bash to run the script - use single quotes after -p, so […] -p ‘p`a$$word’ - single quotes will make parsing those skipped.
Maybe something’s change since the writing of the script in 2019?
I systematically got a “Unable to extract SecureTar - Maybe your password is wrong or the tar is not password encrypted?” message even on freshly created backups .
This doesn’t work for me. Has something changed since 2019 or am I using the script in the wrong way or for the wrong purpose?
I’ve downloaded a tar archive backed up to Google Drive by the Google Drive Backup add-on. That archive unpacks without issues. Inside it is a number of tar.gz archives, which I assume are encrypted, since tar can’t deal with them and file thinks they’re just “data”. However, the script can’t decrypt them:
% ./decrypt.py -i homeassistant.tar.gz -p redacted
Extracting homeassistant.tar.gz...
Traceback (most recent call last):
File "../decrypt.py", line 145, in <module>
main()
File "../decrypt.py", line 137, in main
_dirname = _extract_tar(_inputfile)
File "../decrypt.py", line 89, in _extract_tar
_tar = tarfile.open(name=filename, mode="r")
File "/Users/ehn/.pyenv/versions/3.8.3/lib/python3.8/tarfile.py", line 1604, in open
raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully
I have same issue - the file from the backup is homeassistant.tar.gz but when applying your script:
Extracting homeassistant.tar.gz...
Traceback (most recent call last):
File "/Users/papio/tmp/decrypt.py", line 145, in <module>
main()
File "/Users/papio/tmp/decrypt.py", line 137, in main
_dirname = _extract_tar(_inputfile)
File "/Users/papio/tmp/decrypt.py", line 89, in _extract_tar
_tar = tarfile.open(name=filename, mode="r")
File "/opt/homebrew/Cellar/[email protected]/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/tarfile.py", line 1625, in open
raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully
however the file homeassistant.tar.gz cannot be extracted from gz file as MacOS is not able to deal with that - what am I missing?
Yes, there have been changes on how the files are handled. The file format is now available as library.
The previous tar.py thus has been removed from the supervisor repo and the securetar files are now handled by the backup.py using the securetar dependency.
This is INCREDIBLY frustrating. I appreciate they are trying to protect our data but all this is doing is forcing me to not want to use encryption because I cannot just unzip anything with my password. I don’t want to have to restore a massive backup over 2 hours so I can get a hold of a SINGLE file to restore some mangled dashboard. The whole “you own your data” is kind of great in principle but in practice it’s so messy if you enable encryption with ZERO native tooling from HA. Why can we not browse our own backups easily damn it :*(. Ok sorry about the rant. I’m just waisting hours of my Sunday without really getting anything done.
Can somebody help me out. The script isn’t working and I need to restore a dashboard, so not the whole backup. I would really like to extract the homeassistant.tar.gz
This is what the script gives:
niels@niels-Inspiron-2350:~/Downloads/Domotica/Backupha$ ./script.py -i homeassistant.tar.gz -p <password>
Extracting homeassistant.tar.gz...
Traceback (most recent call last):
File "/home/niels/Downloads/Domotica/Backupha/./script.py", line 145, in <module>
main()
File "/home/niels/Downloads/Domotica/Backupha/./script.py", line 137, in main
_dirname = _extract_tar(_inputfile)
File "/home/niels/Downloads/Domotica/Backupha/./script.py", line 89, in _extract_tar
_tar = tarfile.open(name=filename, mode="r")
File "/usr/lib/python3.10/tarfile.py", line 1804, in open
raise ReadError(f"file could not be opened successfully:\n{error_msgs_summary}")
tarfile.ReadError: file could not be opened successfully:
- method gz: ReadError('not a gzip file')
- method bz2: ReadError('not a bzip2 file')
- method xz: ReadError('not an lzma file')
- method tar: ReadError('invalid header')
