Backing up your data is fun and profitable! For my main data drives I use either rsync or Dantz Retrospect to simply mirror them to external USB drives (powered on only for this express purpose). This has me about 80% covered, but my system files and home directories are not backed up. The following script is meant to pick up the missing pieces.

My script is a modified version of Mike Rubel’s rotating backup script, which can be found here. I twiddled around with his script a bit (to brush up on my scripting skills, if for no other reason), modifying it to accept the source directory, destination directory, and number of backups in the rotation as command line arguments. I also removed the remounting code, since I am the only one using my system and there is no danger of the backup directory being accessed during a backup. I recommend you surf to his site and re-insert this code. It is probably safer. You can also read a more thorough explanation of his script.

I don’t want to bang you over the head with details, but it’s important to know how backups work, so here are just a couple of points about this script:

  • It uses rsync, which is available for *nix, OS X, and Windows (cygwin)
  • The thing that sets this script apart is it’s use of snapshots. Snapshots are kept and rotated a set number of times (just like your log files). You will be able to retrieve any version of any file, up to the oldest snapshot. So, e.g., you will be able to retrieve yesterdays version of your .bashrc file.
  • Hard links are used for the snapshots to minimize the amount of space used. This will allow you to keep far more snapshots than you would be able to otherwise. In essence, only one version of each file will be stored. This can be verified using ls -i filename to check the inodes of the same file in different snapshot sets. It is important to note that hardlinks are not available on all filesystems. In particular, if your using cygwin on a FAT drive, making a hardlink simply makes a normal copy, so there is really no point. I’m not sure about other non-linux filesystems, though I’m pretty sure NTFS works.
  • The source directory can be on another machine, however the snapshot directory must be on the local machine. Use the -e remote-shell switch to specify which remote shell to use for transport. This simply gets passed to rsync as is.
  • I use SRC=${OPTARG%/}/ in the script to force a trailing slash on the source directory. I did this because to rsync the source directory /usr/bin/ is not the same as /usr/bin and I get tired of thinking about the difference. This was intellectually lazy of me but it is easy to change if it drives you nuts.
Some examples:

usage: /root/bin/bkupNrotate.sh [-hy] -s source -d snapshot_directory [-e remote_shell] [-n number_of_snapshots] [-x exclude_file]
-h, help
-y, no confirmation

Backup a folder on a remote machine using ssh as the transport:
bkupNrotate.sh -s barcelona:/misc/bin/ -d /mnt/hdb2/scratch/bak/barcelona/m/bin/ -e ssh

Backup a folder on the local machine, no confirmation (-y), use 10 snapshots:
bkupNrotate.sh -s barcelona:/misc/bin/ -d /mnt/hdb2/scratch/bak/barcelona/m/bin/ -y -n 10

An example crontab entry (makes backup every 6 hours):
0 */6 * * * bkupNrotate.sh -s /home -d /mnt/hdb1/bak/prague/home -n 14 -y >/dev/null

cygwin notes:

Unfortunately, depending on which cygwin dll your using and which version of rsync, you may or may not experience hangs. This is a particularly pernicious bug in the cygwin port of rsync. I have heard rsync runs better in daemon mode on cygwin, so that’s how I have mine set up now.

To install rsync as a service on cygwin:
cygrunsrv -I rsyncd -p /usr/bin/rsync -a “–daemon”

Then, to start it:
cygrunsrv -S rsyncd

That may or may not work.

You are going to need an rsyncd.conf file as well. To save you some grief, you should be aware this file is retarded when it comes to file paths with spaces in them. I use symbolic links to backup directories like “Documents and Settings”. So far I’ve had much better luck running rsync as a daemon on cygwin.

download

bkupNrotate