mkstage4 is a bash script to backup a GNU/Linux installation. At the moment it's aimed for a Gentoo GNU/Linux installation - although it can be customised easily for any other distribution as well. The resulting backup file is called a stage4 (meaning a full system backup).
mkstage4 works right out of the box and merely uses the following tools:
It detects error/misconfiguration (wrong $PATH, wrong commands, wrong exclude list, overwriting existing files …). After checking that everything is set up the script will provide you with four options:
Backup script v3.5 ================== What do you want to do? (Use CONTROL-C to abort) Fast (tar.gz): (1) Minimal backup (2) Interactive backup Best (tar.bz2): (3) Minimal backup (4) Interactive backup Please enter your option:
This option will ask you for any file/directory listed in the custom_list variable if you want it to be included in your backup. This might result in the same stage4.tar.bz2/stage4.tar.gz if you answer all question with “no” or in a considerably bigger stage4 file.
The script only shows errors on stdout per default. If you want to run the backup in verbose mode and watch the files processed by tar, call the script like this:
./mkstage4.sh --verbose ./mkstage4.sh -v
If you intend to backup your system on a cd/dvd with a predefined size call the script like:
./mkstage4.sh --split ./mkstage4.sh -s
Don't forget to adjust the $split_options variable, i.o. to set the desired chunk size. Default is 685MB.
Of course you may combine command line parameters in any way.
Against some misconceptions you may use your stage4 on a box with a different CPU as well. If - and only if - you plan to use your stage4 for another CPU type (not just another box with the same CPU), you must check your /etc/make.conf for the CFLAGS entry:
( < GCC-3.4 )
CFLAGS="-Os -mcpu=athlon -funroll-loops -pipe"
( >= GCC-3.4):
CFLAGS="-Os -mtune=athlon -funroll-loops -pipe"
If your CFLAGS looks like this:
CFLAGS="-Os -march=athlon -funroll-loops -pipe"
you won't be able to use your stage4 on a box with a different CPU type. march breaks compatibility, mcpu/mtune doesn't. There is a simple workaround: replace “march” with mcpu/mtune and recompile your whole system:
# emerge -e world
As mentioned above there are several variables to customise. The script itself has a few comments which should get you started. You might want to change the location where the stage4.tar.bz2 is put (currently /mnt/backups/stage4), or you might not like the filename (hostname-stage4-date.tar.bz2), or you might think that the kernel sources are part of a minimal system backup and hence you won't exclude them (and so on) …
$default_exclude_list: put any file/directory in here which is never needed nor wanted for a minimal system backup.
$default_exclude_pattern: exclude patterns for the $default_include_folders/files.
$default_include_files: files which are needed for a minimal working system. Don't add folders here which should be included in the backup recursively. These files provide solely the needed folder structure.
$default_include_folders: folders which need to be included in the backup recursively for a minimal working system.
$custom_include_list: directories which are not imperative for a working system but which may be desirable to be also saved by your backup interactively (like /home or /usr/src/).
$custom_exclude_list: files/folders which are subfolders of a folder listed in $custom_include_list which should NOT be included.
$custom_exclude_pattern: exclude patterns for the $custom_include_list.
You'll get a stage4 which will take about half the time it does for a bzip2 stage4. On my laptop it takes about 20 minutes to create the stage4, my amd64 with a software raid0 needs 8 minutes. Unpacking takes about 2/3 of the time. A stage4.tar.gz will be about 10% bigger than a stage4.tar.bzip2.
You'll get a stage4.tar.bz2 (mine is about 800MB generated from 3.5GB of data) which will be a fully working backup of your system. On my laptop it takes some 50 minutes to create the
bz2 stage4.
#!/bin/bash # Backup script for Gentoo Linux # # mkstage4.sh is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # mkstage4.sh is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # Copyright: Reto Glauser aka blinkeye # Mailto: stage4 at blinkeye dot ch # Homepage: http://blinkeye.ch # Forum post: http://forums.gentoo.org/viewtopic-t-312817.html # Date: 2009-04-02 version=v3.7 basename=`basename $0` find=/usr/bin/find tar=/bin/tar # these are the commands we actually need for the backup command_list=(cut date echo $find grep hostname mount sh split $tar umount uname which) # verify that each command we use exists. if one can't be found use $PATH and make a suggestion if possible. for command in ${command_list[@]}; do if [ ! -x "`which $command 2>&1`" ]; then echo -e "\nERROR: $command not found! " base=`basename $command` if [ "`which $base 2>&1 | grep "no \`basename $command\` in"`" != "" ]; then echo -e "ERROR: $base is not in your \$PATH." fi exit -1 fi done help="\nUsage:\n\nsh `basename $0` [[-v]|[--verbose]] [[-s]|[--split]] \n\nTo run the script NOT in verbose mode comes in handy if you want to see only the errors that occur during the backup.\n" # Defaults to creating one tarball tar_output="--file" # split command split_options="--suffix-length=1 --bytes=685m" # options for the tar command tarOptions=" --preserve-permissions --create --absolute-names --totals --ignore-failed-read" # where to put the stage4 stage4Location=/mnt/backups/stage4 # name prefix stage4prefix=`hostname`-stage4-`date +\%Y.\%m.\%d` # patterns which should not be backed up (like iso files). # example: default_exclude_pattern="*.iso *.divx" # These pattern count only for files NOT listed in the $custom_include_list. default_exclude_pattern="" # these files/directories are always excluded. don't add trailing slashes. # don't touch it unless you know what you are doing! # /var/db and /var/cache/edb are intentionally added here. they are listed # in $default_include_folders default_exclude_list=" /dev /lost+found /mnt /proc /sys /tmp /usr/portage /usr/src /var/log /var/tmp /var/db /var/cache/edb $stage4Location `echo $CCACHE_DIR`" # files/devices/folders, which need to be backed up (preserve folder structure). # don't touch it unless you know what you are doing! no recursive backup of folders. # use $default_include_folders instead. default_include_files=" /dev/null /dev/console /home /mnt `find /mnt -name .keep` /proc /sys /tmp /usr/portage /usr/src /var/log/emerge.log /usr/src/linux-`uname -r`/.config" # folders, which need to be backed up recursively on every backup. # don't touch it unless you know what you are doing! the reason for this # variable is that some users add /var to the $default_exclude_list. here # we ensure that portage's memory is backed up in any case. default_include_folders=" /var/db" # IMPORTANT: A minimal backup will EXCLUDE files/folders listed here. A custom backup will # include/exclude these files/folders depening on your answer. custom_include_list=" /home/* /usr/src/linux-`uname -r`" # add files/folders here which are subfolders of a folder listed in $custom_include_list which should NOT # be backed up. eg. #custom_exclude_list="/home/foo/mp3 /home/foo/downloads /home/foo/.*" custom_exclude_list="" # Only files/folders within the $custom_include_list are checked against these patterns # custom_exclude_pattern="*.mp3 *.iso" custom_exclude_pattern="" # the find_command find_command="$find /*" # don't backup anything which matches pattern listed in $default_exclude_pattern for pattern in $default_exclude_pattern; do find_command="$find_command -not -name $pattern" done # assemble the find_command function find_files() { for folder in $default_exclude_list; do find_command="$find_command -path $folder -prune -o" done find_command="$find_command -print" for i in $default_include_files; do find_command="echo $i; $find_command" done for i in $default_include_folders; do if [ -d $i ]; then find_command="$find $i; $find_command" else find_command="echo $i; $find_command" fi done } # check the exclude/include variables for non-existing entries function verify() { for i in $1; do if [ ! -e "`echo "$i" | cut -d'=' -f2 | cut -d'*' -f1`" -a "$i" != "/lost+found" -a "$i" != "$stage4Location" ]; then echo "ERROR: `echo "$i" | cut -d'=' -f2` not found! Check your "$2 exit 0 fi done } # check input parameters while [ $1 ]; do case $1 in "-h" | "--help") echo -e $help exit 0;; "-v" | "--verbose") verbose=$1;; "-s" | "--split") tar_output="--split";; "");; *) echo -e $help exit 0;; esac shift done echo "" # check folder/files listed in $default_exclude_list exist verify "$default_exclude_list" "\$default_exclude_list" # check files listed in $default_include_files exist verify "$default_include_files" "\$default_include_files" # check folder listed in $default_include_folders exist verify "$default_include_folders" "\$default_include_folders" #check folder listed in $custom_include_list exist verify "$custom_include_list" "\$custom_include_list" #check folder listed in $custom_exclude_list exist verify "$custom_exclude_list" "\$custom_exclude_list" # print out the version echo -e "\nBackup script $version" echo -e "==================" # how do you want to backup? echo -e "\nWhat do you want to do? (Use CONTROL-C to abort)\n Fast (tar.gz): (1) Minimal backup (2) Interactive backup Best (tar.bz2): (3) Minimal backup (4) Interactive backup\n" while [ "$option" != '1' -a "$option" != '2' -a "$option" != '3' -a "$option" != '4' ]; do echo -en "Please enter your option: " read option done case $option in [1,3]) stage4Name=$stage4Location/$stage4prefix-minimal.tar;; [2,4]) stage4Name=$stage4Location/$stage4prefix-custom.tar for folder in $custom_include_list; do echo -en "\nDo you want to backup" `echo "$folder" | cut -d'=' -f2`"? (y/n) " read answer while [ "$answer" != 'y' -a "$answer" != 'n' ]; do echo -en "Do you want to backup" `echo "$folder" | cut -d'=' -f2`"? (y/n) " read answer done if [ "$answer" == 'n' ]; then find_command="$find_command -path $folder -prune -o" else custom_find="$find $folder" for i in $custom_exclude_pattern; do custom_find="$custom_find -name $i -o" done for i in $custom_exclude_list; do custom_find="$custom_find -path $i -prune -o" done find_command="$custom_find -print; $find_command" fi done ;; esac # add $custom_include_list to the $default_exclude_list as we assembled # $custom_find with $custom_include_list already. default_exclude_list="$default_exclude_list $custom_include_list" case $option in [1,2]) stage4postfix="gz" zip="--gzip";; [3,4]) stage4postfix="bz2" zip="--bzip2";; esac # mount boot echo -e "\n* mounting boot" mount /boot >/dev/null 2>&1 # find the files/folder to backup find_files find_command="($find_command)" # create the final command if [ "$tar_output" == "--file" ]; then tar_command="$find_command | $tar $zip $tarOptions $verbose --file $stage4Name.$stage4postfix --no-recursion -T -" else tar_command="$find_command | $tar $zip $tarOptions $verbose --no-recursion -T - | split $split_options - "$stage4Name.$stage4postfix"_" fi if [ "$verbose" ]; then echo -e "\n* creating the stage4 in $stage4Location with the following command:\n\n"$tar_command fi # everything is set, are you sure to continue? echo -ne "\nDo you want to continue? (y/n) " read answer while [ "$answer" != 'y' ] && [ "$answer" != 'n' ]; do echo -ne "Do you want to continue? (y/n) " read answer done if [ "$answer" == 'y' ]; then # check whether the file already exists. if [ "$tar_output" == "--split" ]; then overwrite="`ls "$stage4Name.$stage4postfix"_* 2>&1 | grep -v 'No such file'`" else overwrite="$stage4Name.$stage4postfix" fi if [ -a "`echo "$overwrite" | grep "$overwrite" -m1`" ]; then echo -en "\nDo you want to overwrite $overwrite? (y/n) " read answer while [ "$answer" != 'y' ] && [ "$answer" != 'n' ]; do echo -en "Do you want to overwrite $overwrite? (y/n) " read answer done if [ "$answer" == 'n' ]; then echo -e "\n* There's nothing to do ... Exiting" exit 0; fi fi # if necessary, create the stage4Location if [ ! -d "$stage4Location" ] ; then echo "* creating directory $stage4Location" mkdir -p $stage4Location fi echo -e "\n* Please wait while the stage4 is being created.\n" # do the backup. sh -c "$tar_command" # finished, clean up echo -e "\n* stage4 is done" echo "* umounting boot" umount /boot >/dev/null 2>&1 # Integrity check echo -e "* Checking integrity" if [ "$zip" == "--gzip" ]; then zip="gzip" else zip="bzip2" fi if [ "$tar_output" == "--split" ]; then if [ "`cat "$stage4Name.$stage4postfix"_*"" | $zip --test 2>&1`" != "" ]; then echo -e "* Integrity check failed. Re-run the script and check your hardware." exit -1 fi else if [ "`$zip --test $stage4Name.$stage4postfix 2>&1`" != "" ]; then echo -e "* Integrity check failed. Re-run the script and check your hardware." exit -1 fi fi # everything went smoothly echo -e "* Everything went smoothly. You successfully created a stage4." else echo -e "\n* There's nothing to do ... Exiting" fi
This will execute the script (you must be root to succesfully backup all folders).
For available parameters see section Parameters.
1 y
Just write down the exact commands you enter if you do the backup yourself. Save the file and call the script
$ mkstage4.sh < file
If you split your stage4 either with the command line parameter –split/-s or uncommented the split section within the script you need to re-assemble the split chunks if you intend to restore such a stage4:
# cat stage4.tar.gz_* > stage4.tar.gz
or
# cat stage4.tar.bz2_* > stage4.tar.bz2
substitute stage4.tar.gz/stage4.tar.bz2 with your stage4 name.