Rsync backup
From Noah.org
This is the small script I use for backups with rsync. This includes a one week rotating window. Prior days include only the files that changed, so each day does not take up the full amount of disk. The prior days' backups will appear complete because identical files are hard linked. There's about a hundred versions of this script on the Internet. This one isn't super fancy, but it's small and gets the job done.
When the script complete successfully it will create a file called BACKUP_TIMESTAMP in the root directory of the destination. If the script fails for any reason it will create a file called BACKUP_ERROR. The files will contain the finish date and exit code and message reported by `rsync`.
This script can change the permission of all directories to 755 and all files to 644. This is specified with the -n flag. This may be a bad thing in some cases. In my case, I am using this as a backup for a shared NAS through SaMBa. For restore purposes the difference between Windows permissions and UNIX permissions is worthless. Normalizing the permissions makes the the files easier to browse through the UNIX side and it removes the problem where someone might change permissions in the backup target directory which could cause rsync to fail.
Contents |
Setup
This script doesn't require much setup. It will automatically create destination backup directories if they don't exist. After the first time you run it you will see that it creates numbered copies of the source base directory. Each numbered copy appears to be a separate copy, but due to the use of hardlinks each copy takes no more room than one copy plus whatever files have changed between each run of the backup. If you run the backup script from a daily cron job then each numbered directory will correspond to one day in the past.
crontab
To use this script add something similar to the following line to /etc/crontab:
01 4 * * * backup /usr/bin/rsync_backup /media/shareddocs /home/backup
rsync_backup script
Click here to download: rsync_backup
#!/bin/sh
#
# This maintains a one week rotating backup. This will normalize permissions on
# all files and directories on backups. It has happened that someone removed
# owner write permissions on some files, thus breaking the backup process. This
# prevents that from happening. All this permission changing it tedious, but it
# eliminates any doubts. I could have done this with "chmod -R +X", but I
# wanted to explicitly set the permission bits.
#
# Pass two arguments: rsync_backup SOURCE_PATH BACKUP_PATH
#
# $Id: rsync_backup 222 2008-02-21 22:05:30Z noah $
usage() {
echo "usage: rsync_backup [-v] [-n] SOURCE_PATH BACKUP_PATH"
echo " SOURCE_PATH and BACKUP_PATH may be ssh-style remote paths; although,"
echo " BACKUP_PATH is usually a local directory where you want the"
echo " backup set stored."
echo " -v : set verbose mode"
echo " -n : normalize directory and file permissions to 755 for dirs and 644 for files."
}
VERBOSE=0
NORMALIZE_PERMS=0
while getopts ":vnh" options; do
case $options in
v ) VERBOSE=1;;
n ) NORMALIZE_PERMS=1;;
h ) usage
exit 1;;
\? ) usage
exit 1;;
* ) usage
exit 1;;
esac
done
shift $(($OPTIND - 1))
SOURCE_PATH=$1
BACKUP_PATH=$2
if [ -z $SOURCE_PATH ] ; then
echo "Missing argument. Give source path and backup path."
usage
exit 1
fi
if [ -z $BACKUP_PATH ] ; then
echo "Missing argument. Give source path and backup path."
usage
exit 1
fi
SOURCE_BASE=`basename $SOURCE_PATH`
PERMS_DIR=755
PERMS_FILE=644
if [ $VERBOSE ]; then
RSYNC_OPTS="-a --delete -v"
date
else
RSYNC_OPTS="-a --delete -q"
fi
# Create the rotation directories if they don't exist.
if [ ! -d $BACKUP_PATH ] ; then
mkdir $BACKUP_PATH
fi
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.0 ] ; then
mkdir $BACKUP_PATH/$SOURCE_BASE.0
fi
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.1 ] ; then
mkdir $BACKUP_PATH/$SOURCE_BASE.1
fi
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.2 ] ; then
mkdir $BACKUP_PATH/$SOURCE_BASE.2
fi
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.3 ] ; then
mkdir $BACKUP_PATH/$SOURCE_BASE.3
fi
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.4 ] ; then
mkdir $BACKUP_PATH/$SOURCE_BASE.4
fi
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.5 ] ; then
mkdir $BACKUP_PATH/$SOURCE_BASE.5
fi
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.6 ] ; then
mkdir $BACKUP_PATH/$SOURCE_BASE.6
fi
# TODO All these find operations to clean up permissions is going to add a lot
# of overhead as the backup set gets bigger. At 100 GB it's not a big deal. The
# correct thing would be to have an exception based system where I correct
# permissions when/if they cause a problem.
# Rotate backups.
if [ $NORMALIZE_PERMS ]; then
if [ $VERBOSE ]; then
echo "Normalizing file permissions."
fi
find $BACKUP_PATH/$SOURCE_BASE.6 -type d -exec chmod $PERMS_DIR {} \;
find $BACKUP_PATH/$SOURCE_BASE.6 -type f -exec chmod $PERMS_FILE {} \;
fi
rm -rf $BACKUP_PATH/$SOURCE_BASE.6
mv $BACKUP_PATH/$SOURCE_BASE.5 $BACKUP_PATH/$SOURCE_BASE.6
mv $BACKUP_PATH/$SOURCE_BASE.4 $BACKUP_PATH/$SOURCE_BASE.5
mv $BACKUP_PATH/$SOURCE_BASE.3 $BACKUP_PATH/$SOURCE_BASE.4
mv $BACKUP_PATH/$SOURCE_BASE.2 $BACKUP_PATH/$SOURCE_BASE.3
mv $BACKUP_PATH/$SOURCE_BASE.1 $BACKUP_PATH/$SOURCE_BASE.2
cp -al $BACKUP_PATH/$SOURCE_BASE.0 $BACKUP_PATH/$SOURCE_BASE.1
# Backup.
if [ $NORMALIZE_PERMS ]; then
if [ $VERBOSE ]; then
echo "Normalizing file permissions."
fi
find $BACKUP_PATH/$SOURCE_BASE.0 -type d -exec chmod $PERMS_DIR {} \;
find $BACKUP_PATH/$SOURCE_BASE.0 -type f -exec chmod $PERMS_FILE {} \;
fi
rsync $RSYNC_OPTS $SOURCE_PATH/. $BACKUP_PATH/$SOURCE_BASE.0/.
RSYNC_EXIT_STATUS=$?
if [ $NORMALIZE_PERMS ]; then
if [ $VERBOSE ]; then
echo "Normalizing file permissions."
fi
find $BACKUP_PATH/$SOURCE_BASE.0 -type d -exec chmod $PERMS_DIR {} \;
find $BACKUP_PATH/$SOURCE_BASE.0 -type f -exec chmod $PERMS_FILE {} \;
fi
# Ignore error code 24, "rsync warning: some files vanished before they could be transferred".
if [ $RSYNC_EXIT_STATUS = 24 ] ; then
RSYNC_EXIT_STATUS=0
fi
# Create a timestamp file to show when backup process completed successfully.
if [ $RSYNC_EXIT_STATUS = 0 ] ; then
rm -f $BACKUP_PATH/$SOURCE_BASE.0/BACKUP_ERROR
date > $BACKUP_PATH/$SOURCE_BASE.0/BACKUP_TIMESTAMP
else # Create a timestamp if there was an error.
rm -f $BACKUP_PATH/$SOURCE_BASE.0/BACKUP_TIMESTAMP
echo "rsync failed" > $BACKUP_PATH/$SOURCE_BASE.0/BACKUP_ERROR
date >> $BACKUP_PATH/$SOURCE_BASE.0/BACKUP_ERROR
echo $RSYNC_EXIT_STATUS >> $BACKUP_PATH/$SOURCE_BASE.0/BACKUP_ERROR
fi
if [ $VERBOSE ]; then
date
fi
exit $RSYNC_EXIT_STATUS
Ken's exclude.list file
My friend, Ken, uses this rsync exclude list.
### tmp ### - **/tmp/** ### transient ### - /proc/ - /mnt/ - /cdrom/ - /media/ - /sys/ - /var/run/ - /var/lock/ - /var/log/ - /var/tmp/ - /dev/ ### config ### - /etc/adjtime - /etc/motd - /etc/mtab - /etc/resolv.conf - /etc/lvm/.cache - /var/lib/alsa/asound.state - /var/lib/dhcp3/dhclient.*.leases - /var/lib/urandom/ - /var/lib/urandom/random-seed ### apps ### # firefox - **/.mozilla/firefox/*.default/Cache/** # bash - .bash_history # svn - .svn/ - .svn/** # vim - .viminfo - .*.swp # X + .cache/sessions/ - .cache/sessions/** - .xsession-errors - .ICEauthority - .serverauth.* # less - .lesshst ### backup ### - *~
