Today I'll post my fav rsync options.

rsync is a little program that automates the syncronization between two folders, filesystems or partitions, the beauty of it is that is even works remotely (by relying on the SSH Protocol - Quick Tip: Configure and use SSH keys for that!).

Its main use is to move every kind of data, either local (ie. between hdds) or within a network.

Synchronisation means that rsync checks for all the files that are on the Source location versus the files that are present on the Destination, efficiently transferring only the things that have changed or that are missing.

Every standard Linux System Administrator should know rsync (man rsync), but given there's too many options and most people are averse to cold mans, I'll post some useful settings I've used in the past.

At its basic, rsync may be used as follows:

rsync $SOURCE $DESTINATION

Were $SOURCE and/or $DESTINATION can either be a local folder or a remote folder (accessed via ssh, [email protected]:/home/andy/backup).

If you have time to tweak its settings a bit, you may gain more functionality or performance gains.

You'd normally play with rsync through bash scripts but occasionally you may wish to use it directly for the odd manual backup job.

When used from the console, you'll need to remember some of its useful settings, so here I'll share some.

The options.

For a basic rsync between two places without "deletion", you'd use somethin similar:

rsync-aHvzh --progress SRC DST

-a is shorthand for "-rlptgoD"-Options ("a" stands for archive - check past this section for a more detailed explanation).

-H stands for hard links preservation.

-v is for verbose - I wonder if you appreciate talkative programs?

-z specifies to compress the SOURCE files before transferring them to the DESTINATION - This is best used with highly compressible data (such as documents) and where the processor resources are abundant.

-h shows human readable file sizes (ie. instead of "10737418240 bytes transferred").

--progress shows a cute ASCII gfx progress bar.

"-rlptgoD" options.

Once you know what the above options do, I'll show you what does "-rlptgoD" mean (these are simpler):

-r is for recursive, essentially you'll need this if you'd like to copy or move entire directories and subdirectories (ie. not only files).

-l keeps the symbolic links as such.

-p copies the same UNIX file permissions to the destination (r,w + x).

-t keeps the same date and time of the SOURCE files (IMHO, it is recommended to retain the same date and time of the source files onto the destination - this way rsync knows if the files have changed).

-g Same Group (similar to -p) - it retains the same group membership as the Source on the destination..

-o Same Owner (similar to -p) - it retains the same owner as the Source on the destination.

-D preserves character devices and block devices (ls /dev to see what these are) on the receiving end.

 

Dry Run

Another useful option I've used in the past is -n.

-n stands for DRY RUN.

A Dry Run is when you are still unsure what will happen to your data with the options that you've chosen.

This is especially useful when you are still experimenting with your rsync options and you don't want to disrupt your data.

 

--delete-when?

If you feel confident of killing your data, there are many ways you may wish to go about it.

Rsync allows you to automate the destruction of your data in three specific ways (head, heart or jugular) - during an rsync transfer, you may delete your data before, during or after the transfer process.

Obviously, these options are useful only when scripting or if the Source data can easily be re-created, so my recommendation is to never use them while performing manual once-off transfers.

You may use the below (self-descriptive) options as follows:

--delete-before (this deletes data on the target before the file transfer begins).

--delete-after

--delete-during

 

exclude-from=files

rsync allows you to specify some files or dirs to exclude from the file transfer process.

The syntax is as follows:

--exclude file01.txt etc

All the files or dirs specified after the --exclude option won't be transferred to the destination.

For example, you might wish to exclude /sys, /dev, /tmp, etc.

--exclude is ok for excluding a bunch of odd folders and it allows you to go overboard with wildcards, 'though it easily messes up your command whenever the things to exclude become more than three.

 

To keep things tidy and your rsync command more readable, you can create an external text file that lists all the things to exclude (one per line).

To better explain what I mean, let's assume you're copying the whole slash filesystem ("/") for backup purposes.

Normally, you may not wish to transfer the following folders: tmp, dev, proc, sys, floppy, cdrom, rsync, mnt, media.

To exclude them all in one shot, you may create a simple text file that contains (one per line), the name of the folder you don't want to transfer:

echo -e "/tmp\n/dev\n/proc\n/sys\n/floppy\n/cdrom\n/rsync\n/mnt\n/media\n" > /tmp/excluded.txt

You may then recall the "excluded files list"-file with the option "--exclude-from=/tmp/excluded.txt".