backup.org

Backups using rsync

Introduction

Here is my implementation of a backup strategy outlined by Mark Sanborn (https://www.marksanborn.net/howto/use-rsync-for-daily-weekly-and-full-monthly-backups/).

The general idea is to use rsync to mirror important directories daily on the same machine. At the end of every week, the daily backup is moved into a weekly folder and every month a tarball containing the current status is created. This strategy allows you to restore files that were accidentally overwritten but does not help if files get corrupted or the disk on your machine fails. To address these issues occasionally move the entire backup directory to a second (and third!) machine / hard disk.

To get started

Firstly create the target directories for the backups:

mkdir rsync
cd rsync 
mkdir daily 
mkdir weekly
mkdir monthly

Backup script

The script below implements the daily, weekly and monthly rsync commands. Since I generally do not change what I back up I list the directories included in my backup in the targets array at the start of the script. Modify these entries to point to the directories you intend to back up.

Since I am not sure where I’ll keep my monthly backups in the future, I use encrypt the tarballs using symmetric encryption.

To run the backup script:

backup_rsync.sh -m <d|w|m>  

for daily (d), weekly (w) or monthly (m) backups.

    targets=($HOME/code $HOME/work $HOME/life $HOME/travel)


    backup_daily()
    {
        printf "Running rsync\n"
        printf "%30s\n" $1;
        for item in ${targets[*]}
        do
            printf "rsync-ing %s\n" $item;
            rsync -av --stats  --progress $item $HOME/rsync/daily 2>&1
            echo "rsync -av --progress $item $HOME/rsync/daily"
        done
        printf "Done\n"
    }

    backup_weekly()
    {
        printf "Running rsync\n"

        rsync -av --stats  --progress --delete $HOME/rsync/daily $HOME/rsync/weekly 2>&1

        printf "Done\n"
    }

    backup_monthly()
    {

        printf "Pwd: "
        read_secret PASSWD
        printf "again: "
        read_secret PASSWD2

        if [ "$PASSWD" != "$PASSWD2" ]
        then
            printf "\nERROR: passwords do not match!\n\n" ;
            exit 1;
        fi

        if [ "${PASSWD}" = "" ]; then usage; fi


        DATE=$(date +%Y%m%d)

        OUTFILENAME=$HOME"/rsync/monthly/monthly"$DATE".tar.gz"
        OGPG=$OUTFILENAME".gpg"


        if [ -f $OGPG ]
        then
            printf "\nERROR: the file %s exists!\n\n" $OGPG;
            usage;
        fi

        printf "Creating archive: \n";

        printf "%30s\n" $OUTFILENAME ;

        cd
        tar -czf $OUTFILENAME $HOME"/rsync/daily/"

        if [ "$?" -eq "0" ]
        then
            printf "%30s\n" "Success";
        else
            printf "\nERROR: tar FAILED!\n\n";

            usage;
        fi




        printf "Encrypting: \n";
        printf "%30s\n" $OGPG ;

        echo $PASSWD | gpg2 --batch --passphrase-fd 0 --symmetric --s2k-cipher-algo AES256 --s2k-mode 3 --s2k-count 65000000 -o $OGPG $OUTFILENAME

        if [ "$?" -eq "0" ]
        then
            printf "%30s\n" "Success";
        else
            printf "\nERROR: gpg FAILED!\n\n";
            usage;
        fi

        #
        # delete unencrypted tar archive...
        #

        rm $OUTFILENAME;

    }

    read_secret()
    {
        stty -echo
        trap 'stty echo' EXIT
        read "$@"
        stty echo
        trap - EXIT
        echo
    }




    INLIST=
    PASSWD=
    MODE=
    function usage()
    {
        cat <<EOF
usage: $0  -m <d/w/m>
EOF
        exit 1;
    }

    while getopts m: opt
    do
        case ${opt} in
            m) MODE=${OPTARG};;
            *) usage;;
        esac
    done

    if [ "${MODE}" = "" ]; then usage; fi

    #
    #   Sanity check
    #

    programs=(rsync scp tar sha512sum gpg2)

    printf "Running Sanity checks:\n";

    for item in ${programs[*]}
    do
        if which $item >/dev/null; then
            printf "%15s found...\n"  $item;
        else
            printf "\nERROR: %s not found!\n\n" $item;
            exit 1;
        fi
    done


    case "$MODE" in
        d)
            backup_daily $INLIST
            ;;
        w)
            backup_weekly
            ;;
        m)
            backup_monthly
            ;;
        *)
            echo $"Usage: $0 -m <d/w/m> -i <list of dirs to back up>"
            exit 1
    esac

House keeping

To remove unwanted files from the backup

cd rsync 
find . -name  "vgcore.*" -print0 | xargs -0 -I {} mv {} ~/tmp

Mounting disks

To mount disks always to same location.

1. figure out the UUID of disk:

sudo blkid

Record UUID and TYPE

1. create mount point

mkdir -p /mnt/DISKXXXX 

1. mount

mount -t btrfs --uuid 40c13e5c-fe47-45e9-a88f-6ca33320b7b3 /mnt/removable