Difference between revisions of "Rsync backup"

From Noah.org
Jump to navigationJump to search
Line 2: Line 2:
 
[[Category:Free_Software]]
 
[[Category:Free_Software]]
 
This is the small script I use for online backups with rsync. This includes a one week rotating window. Prior days include only the files that changed, so each day does not take up the full amount of disk. The prior days' backups will appear complete because identical files are hard linked. There's about a hundred versions of this script on the Internet. This one isn't super fancy, but it's small and gets the job done.
 
This is the small script I use for online backups with rsync. This includes a one week rotating window. Prior days include only the files that changed, so each day does not take up the full amount of disk. The prior days' backups will appear complete because identical files are hard linked. There's about a hundred versions of this script on the Internet. This one isn't super fancy, but it's small and gets the job done.
 +
 +
== crontab ==
  
 
To use this script add something similar to the following line to /etc/crontab:
 
To use this script add something similar to the following line to /etc/crontab:
 +
 
     01 4 * * * backup /usr/bin/rsync_backup /media/shareddocs /home/backup
 
     01 4 * * * backup /usr/bin/rsync_backup /media/shareddocs /home/backup
  
 
== rsync_backup script ==
 
== rsync_backup script ==
  
−
<pre>
+
Click here to download: [http://www.noah.org/downloadsvn.php?src=file:///home/svn/src/shell/rsync_backup rsync_backup]
−
#!/bin/sh
+
<include svncat src="file:///home/svn/src/shell/rsync_backup" highlight="sh" />
−
#
 
−
# This maintains a one week rotating backup. This will normalize permissions on
 
−
# all files and directories on backups. It has happened that someone removed
 
−
# owner write permissions on some files, thus breaking the backup process. This
 
−
# prevents that from happening. All this permission changing it tedious, but it
 
−
# eliminates any doubts. I could have done this with "chmod -R +X", but I wanted
 
−
# to explicitly set the permission bits.
 
−
#
 
−
# Pass two arguments:
 
−
#    rsync_backup SOURCE_PATH BACKUP_PATH
 
−
#
 
−
# Noah Spurrier 2007
 
−
# $Id$
 
  
−
usage="usage: rsync_backup [-v] SOURCE_PATH BACKUP_PATH\n\
+
== Ken's exclude.list file ==
−
    SOURCE_PATH and BACKUP_PATH may be ssh-style remote paths; although,\n\
 
−
    BACKUP_PATH is usually a local directory where you want the\n\
 
−
    backup set stored."
 
−
 
 
−
while getopts ":vh" options; do
 
−
    case $options in
 
−
        v ) VERBOSE=1;;
 
−
        h ) echo -e $usage
 
−
            exit 1;;
 
−
        \? ) echo -e $usage
 
−
            exit 1;;
 
−
        * ) echo -e $usage
 
−
            exit 1;;
 
−
    esac
 
−
done
 
−
shift $(($OPTIND - 1))
 
−
SOURCE_PATH=$1
 
−
BACKUP_PATH=$2
 
−
if [[ ( -z $SOURCE_PATH ) || ( -z $BACKUP_PATH ) ]]; then
 
−
    echo "Missing argument. Give source path and backup path in this format:"
 
−
    echo "    $0 SOURCE_PATH BACKUP_PATH"
 
−
    echo -e $usage
 
−
    exit 1
 
−
fi
 
−
SOURCE_BASE=`basename $SOURCE_PATH`
 
−
PERMS_DIR=755
 
−
PERMS_FILE=644
 
−
if [ $VERBOSE ]; then
 
−
    RSYNC_OPTS="-a --delete -v"
 
−
    date
 
−
else
 
−
    RSYNC_OPTS="-a --delete -q"
 
−
fi
 
−
 
 
−
# Create the rotation directories if they don't exist.
 
−
if [ ! -d $BACKUP_PATH ] ; then
 
−
    mkdir $BACKUP_PATH
 
−
fi
 
−
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.0 ] ; then
 
−
    mkdir $BACKUP_PATH/$SOURCE_BASE.0
 
−
fi
 
−
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.1 ] ; then
 
−
    mkdir $BACKUP_PATH/$SOURCE_BASE.1
 
−
fi
 
−
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.2 ] ; then
 
−
    mkdir $BACKUP_PATH/$SOURCE_BASE.2
 
−
fi
 
−
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.3 ] ; then
 
−
    mkdir $BACKUP_PATH/$SOURCE_BASE.3
 
−
fi
 
−
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.4 ] ; then
 
−
    mkdir $BACKUP_PATH/$SOURCE_BASE.4
 
−
fi
 
−
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.5 ] ; then
 
−
    mkdir $BACKUP_PATH/$SOURCE_BASE.5
 
−
fi
 
−
if [ ! -d $BACKUP_PATH/$SOURCE_BASE.6 ] ; then
 
−
    mkdir $BACKUP_PATH/$SOURCE_BASE.6
 
−
fi
 
−
 
 
−
# TODO All these find operations to clean up permissions is going to add a lot
 
−
# of overhead as the backup set gets bigger. At 40GB it's not a big deal. The
 
−
# correct thing would be to have an exception based system where I correct
 
−
# permissions when they cause a problem.
 
−
 
 
−
# Rotate backups.
 
−
find $BACKUP_PATH/$SOURCE_BASE.6 -type d -exec chmod $PERMS_DIR {} \;
 
−
find $BACKUP_PATH/$SOURCE_BASE.6 -type f -exec chmod $PERMS_FILE {} \;
 
−
rm -rf $BACKUP_PATH/$SOURCE_BASE.6
 
−
mv    $BACKUP_PATH/$SOURCE_BASE.5 $BACKUP_PATH/$SOURCE_BASE.6
 
−
mv    $BACKUP_PATH/$SOURCE_BASE.4 $BACKUP_PATH/$SOURCE_BASE.5
 
−
mv    $BACKUP_PATH/$SOURCE_BASE.3 $BACKUP_PATH/$SOURCE_BASE.4
 
−
mv    $BACKUP_PATH/$SOURCE_BASE.2 $BACKUP_PATH/$SOURCE_BASE.3
 
−
mv    $BACKUP_PATH/$SOURCE_BASE.1 $BACKUP_PATH/$SOURCE_BASE.2
 
−
cp -al $BACKUP_PATH/$SOURCE_BASE.0 $BACKUP_PATH/$SOURCE_BASE.1
 
−
 
 
−
# Backup.
 
−
find $BACKUP_PATH/$SOURCE_BASE.0 -type d -exec chmod $PERMS_DIR {} \;
 
−
find $BACKUP_PATH/$SOURCE_BASE.0 -type f -exec chmod $PERMS_FILE {} \;
 
−
rsync $RSYNC_OPTS $SOURCE_PATH/. $BACKUP_PATH/$SOURCE_BASE.0/.
 
−
find $BACKUP_PATH/$SOURCE_BASE.0 -type d -exec chmod $PERMS_DIR {} \;
 
−
find $BACKUP_PATH/$SOURCE_BASE.0 -type f -exec chmod $PERMS_FILE {} \;
 
−
 
 
−
# Create a timestamp file to show when backup process was complete.
 
−
date > $BACKUP_PATH/$SOURCE_BASE.0/BACKUP_TIMESTAMP
 
−
 
 
−
# Create a shortcut to the latest backup.
 
−
ln -sf $BACKUP_PATH/backup.0 $BACKUP_PATH/latest
 
  
−
if [ $VERBOSE ]; then
+
My friend, Ken, uses this rsync exclude list.
−
    date
 
−
fi
 
−
</pre>
 
−
 
 
−
== Ken's exclude.list file ==
 
  
 
<pre>
 
<pre>

Revision as of 19:46, 28 January 2008

This is the small script I use for online backups with rsync. This includes a one week rotating window. Prior days include only the files that changed, so each day does not take up the full amount of disk. The prior days' backups will appear complete because identical files are hard linked. There's about a hundred versions of this script on the Internet. This one isn't super fancy, but it's small and gets the job done.

crontab

To use this script add something similar to the following line to /etc/crontab: 

   01 4 * * * backup /usr/bin/rsync_backup /media/shareddocs /home/backup

rsync_backup script

Click here to download: rsync_backup <include svncat src="file:///home/svn/src/shell/rsync_backup" highlight="sh" />

Ken's exclude.list file

My friend, Ken, uses this rsync exclude list. 

### tmp ###
- **/tmp/**
### transient ###
- /proc/
- /mnt/
- /cdrom/
- /media/
- /sys/
- /var/run/
- /var/lock/
- /var/log/
- /var/tmp/
- /dev/
### config ###
- /etc/adjtime
- /etc/motd
- /etc/mtab
- /etc/resolv.conf
- /etc/lvm/.cache
- /var/lib/alsa/asound.state
- /var/lib/dhcp3/dhclient.*.leases
- /var/lib/urandom/
- /var/lib/urandom/random-seed
### apps ###
# firefox
- **/.mozilla/firefox/*.default/Cache/**
# bash
- .bash_history
# svn
- .svn/
- .svn/**
# vim
- .viminfo
- .*.swp
# X
+ .cache/sessions/
- .cache/sessions/**
- .xsession-errors
- .ICEauthority
- .serverauth.*
# less
- .lesshst
### backup ###
- *~
Retrieved from "https://www.noah.org/mediawiki-1.34.2/index.php?title=Rsync_backup&oldid=3664"