EDIT: Added 10/1/2022 - To save You sometime I will say right here that the simplest solution Is that the SD card copier allows you to copy even a one terabyte SSD boot disk to a small microSD card if the amount of data on the SSD is not too much. I thought that did not work because I had a defective SSD before and it kept failing. So below is a very hardest way to do this but it was a very good educational exercise… and there are some backup solutions suggested by people below which were just fantastic and are a good read… The rest of this post is what was here before:
Is there a better, quicker, less painful full backup option (backstory below and then very onerous steps 1-7 below) to create a full useable backup copy of my RPI? (I do have the Google drive backup so I have Home Assistant backups, but I am talking about a full backup of the RPI boot partition, not just the Home Assistant instance that is running in a container on the RPI)…?
Backstory:
I have a RPI 4 booting off a 1TB* SSD.
it’s running Home Assistant Supervised in a container (not the Home Assistant OS) as it is already running WEEWX on the native host to record personal weather station statistics into a database and retransmit the readings to NOAA and various other weather repositories worldwide all over the net. I’m running it headless directly on ethernet (WiFi, Video (HDMI ports) and sound all turned off) - for maximum network stability and to ensure there is plenty of power.
Since I’ve been tinkering with this for a while and have mucked things up several times trying to do outrageous things I have from the getgo insisted on having a full backup which has saved my butt more than once.
Since the native RPI OS instantly takes up the entire space on the boot media upon first boot up*, even if much less is needed, that means even if I only have a tiny amount to back up, it’s always 1TB!
So this is my nightmare backup scenario, someone please let me know if there is an easier way!
The end goal is to have a full backup of everything on the SSD, and since it isn’t much in size, to be able to put it into a micro SD card to put into the pi to boot it up.
So here are the steps I go through and I wonder if there is an easier way?
- I’ve got the Adguard Home Assistant add-on running as my DNS server in my home so any device surfing the web automatically has all the advertisements stripped out of it. So if the pi goes down I lose internet access. So, I reset the DNS settings to defaults removing the RPI ip addresses as the DNS server from my router.
— So my “real” backup steps start below —
-
Update all the processes running on my RPI from a command line such that none of them start on boot up. (So I have control when starting this up from a backup copy.)
-
Reboot the rpi and make sure nothing starts automatically.
-
To minimize downtime (I am back up and running after steps 3 and 4), shut down the RPI and unplug the USB, then put a generic micro SD card running the default RPI os and nothing else on it, then once it is booted up and running off the micro SD card only, plug in the original 1TB SSD to be backed up, and another the same size in the other USB port, then use the included “SD Card Copier” app (that is on the RPI as part of the default OS) - to copy one SSD to the other. This copy takes about 45 minutes.
-
Shut down the RPI, remove the micro SD card, unplug only the second SSD copy, leave the original SSD in place, then when it is booted up, update the auto start processes again to restart on boot up, then reboot it and make sure all is working… At this point I am up and running after firing up the DNS server (mentioned in step 0) so the below does not disrupt the RPI server sending out weather reports etc…
-
Plug the new 1TB SSD copy into a Windows PC that has a separate drive with at least 1tb available (you will get pop-up error messages about the drive but ignore them as windows cannot understand the drive format), and then use the old app called “win32disk imager” on the PC to create a 1tb .img image file from the SSD on that PC drive. This takes about 90 minutes.
-
The only way I can find to shrink the 1tb .img file on the PC is a Linux utility called “pishrink.sh” so I put that file in the same directory as the 1tb .img image file. I only have a windows machine, so I created a VM on the PC which runs Ubuntu. Therefore from within the Ubuntu VM, at the command line, I mount that same drive in ubuntu and run the pishrink.sh utility on the .img file. The file typically will shrink from exactly 1tb to about 34MB. This takes about 25 minutes.
-
Then I plug a micro SD card into the PC , and using the free utility “Raspberry PI Imager” on the PC, select the .img file as the “Custom OS” and as the destination, select the microSD card drive letter that is plugged into the PC (in this case the MicroSD Card only has to be big enough to hold 34MB)… To play it safe every time make three MicroSD Card copies and label them all with the date.
— the core steps of the backup are 1-7 above —
- As an optional test I just boot up the RPI on just one of the MicroSD cards and it should come up sluggishly, but with nothing running. Then from a command line I can set weewx and Home Assistant to automatically start on boot, then reboot the device. Patience is key as it runs MUUUCCHH slower on a MicroSD Card instead of on an SSD… (It runs slowly the first time because it is expanding the boot mount to take up all the available space and creating temp swap space, etc. However - on a reboot and during running the os - EVERYTHING is MUCH faster on the SSD of course.)
So can anyone help me shorten the onerous manner of creating a full backup of my installation just running on the little raspberry PI? Having the second SSD which I can just swap in (from step 3) is a nice to have but not really necessary…
Thoughts on how to get a copy of my boot partition onto a MicroSD Card without jumping through a million hoops like the above (and without having to buy a 1TB MicroSD card which is total overkill - and without having to switch to a smaller SSD as it’s great to have that space available)?