1 Best Linux rsync command

For Linux-like operating systems that run on Unix-like operating systems, there is a the rsync command syncs files from the source to the destination, either on local computers or through the secure network connection. It’s a speedy and flexible alternative to the command rcp.

Description

Rsync is a speedy and extremely flexible tool for copying files. It allows local copying or to a different host via the internet, and even to/from an rsync server that is remote. It provides a wide range of options to control every aspect of its behavior . They also allows for very flexible selection of the files that are copied. It is renowned for its delta transfer algorithm, which decreases the volume of data transmitted over the network by only transmitting the difference between the source files and the files of the destination. Rsync is used extensively for backups , mirroring and also as an enhanced copy command that can be used for daily use.

rsync locates files that require to be transferred by using an “quick scan” method (by default) which looks at files which have been altered in size or have been modified in the last time. Any modifications to the other properties (as required by the options) are applied to the destination file immediately in the event that the quick check shows that the file’s information is not required to be updated.

The additional functions of the rsync program include:

• Support for linking links to copy owners, devices group, permissions and copying links.
• Exclude and exclude-from options that are similar as GNU Tar
• An CVS exclusion option to ignore the same types of files that CVS would not consider.
• Use any remote shell, such as sh or rsh.
• Does not require super-user privileges.
• Pipelining of file transfers in order to reduce delay costs
• The support to an anonymous as well as authenticated Rsync daemons (ideal for mirroring)

Uses with just an argument for SRC argument and without a DEST argument will list the source files rather than copying.

Options Information

rsync supports the lengthy (double-dash plus word) and short (single-dash + letter) choices. The complete list of options is provided below. If an option is described in more than one manner it is comma-separated. Certain options have only the long version, but not the short. If the option has the form of a parameter the parameter is only mentioned after the long variant but it has to also be defined for the shorter. When you specify a parameter you may utilize the format “-option=” parameter or substitute the word ‘ =‘ by whitespace. The parameter might require quotation marks in a specific way to be able to pass your script’s commands-line parsing. Be aware that the leading tilde () in a file’s name is replaced by the shell, and so –option=/foo would not alter the tilde in that directory you have created as your personal home (remove the “=” to do that).

–help

Print a brief help page that explains the options in the rsync program or exit. For backward-compatibility with older versions of rsync, the help outputs if you use the -h option without any other args.

–version

Print the number of rsync Version number, and then exit.

-v, –verbose

This option will increase the amount of data you receive when you transfer. As default the rsync program works quietly. The single -v will give you details about the type of files that are transferred and provides a summary at the conclusion. A second pair of options, namely a v options provide details about which files are being skipped as well as a bit more information at the conclusion. There are more than two options with a -v options are not recommended unless you are trying to debug Rsync.

Be aware that the names of the files transferred that are output are determined with the default output format that is ” %n%L” that only tells you that the file’s name, and when it’s an external link, where it’s located. With only the level of verbosity, namely the -v volume of audio it does not specify whether a file’s attributes modified. If you request an organized list of changes to attribute (either –itemize-changes or by adding ” %i” to the –out-format setting) your output (on the client) expands to include the items that have been modified or modified in some way. Look up for more information on the out-format option for further details.

-q, –quiet

This choice reduces the amount of information that you are provided during the transfer, particularly blocking information messages sent by your remote server. This feature is beneficial when you invoke the rsync command from the cron.

–no-motd

This option alters the information that is generated by your client prior to the beginning of a transfer by daemon. This will remove MOTD (message-of-the-day) text, however, it can also alter the modules the daemon transmits as a response to ” rsync host request” requests (due to an issue with the rsync protocol) therefore, you should not use this option if you wish to obtain the list of available modules from the daemon.

-I, –ignore-times

Normally, rsync does not skip any files that are of identical in size and also have the identical modification time stamp. This switch disables the “quick check” behaviour, which causes every file upgraded.

–size-only

This alters the rsync‘s “quick test” algorithm to identify files that must be transferred, shifting it from the transfer of files with the size of the file has changed or a changed date of last modification to just look for files that have grown in size. This can be useful when beginning using the rsync system following the use of a mirroring software that doesn’t keep the timestamps precisely.

–modify-window

If you compare two timestamps, the rsync program considers the two dates as being the same when they differ by more than the modified-window value. This usually is the value of 0 (for an exact match) however, you might prefer setting this value higher in certain situations. Particularly, when moving to or out of a Microsoft Windows FAT filesystem (which is a representation of times using two seconds of resolution), –modify-window=1 is a good option (allowing the times to vary by 1 second or more).

-c, –checksum

This alters the method by which rsync determines if the files have been modified and are required to be transferred. Without this option the rsync program makes use of an “quick test” to (by default) determines if each file’s size and the time of the last modification are compatible between the receiver and sender. This change will check a 128- bits checksum for every file that has an identical size. Making checksums will mean that both parties will use many I/O on the disk to read all of the data contained in the file during the transfer (and this is prior to any reading will be carried out to transfer altered files) which will slow things down a lot.

It generates the checksums as it’s performing the file system scan, which builds the list of available files. The receiver generates checksums as it scans for altered files and it will verify any file that is similar size to the sender’s file. files with a different size or with a different checksum are chosen to be transferred.

It is important to note that the rsync always ensures that each transferred file was properly reconstructed on the receiving side , by verifying a complete checksum for each file which is generated when you transfer the files however, this verification process is automatic and is not related to the option’s prior-to-transfer “Does the file need to be modified?” check. For protocol 30 and higher (first included in 3.0.0) The checksum employed is MD5. For protocols older than 3.0.0 the checksum employed is MD4.

-a, –archive

It is similar to the -rlptgoD. It’s a way to say that you’re looking for the recursion feature and would like to save nearly everything (with -H being an obvious exclusion). The only exception to this similarity is when the –files-from is explicitly specified and the file-from parameter is specified, in which the the word’re’ does not mean.

Be aware it is important to note that -a doesn’t preserve hardlinks since finding multi-linked files is costly. You need to explicitly define the -H option..

–no-OPTION

You can disable one or more implicit options by prefixing the names by ” no-“. There are a few options that can be prefixed by”no-“: only those options that are prefixed with ” no-” as only those options have been implied in other alternatives (e.g., –no-D or –no-perms) or have distinct defaults for different situations (e.g., –no-whole-file or –no-blocking, –no-dirs). You can choose one of the shorter or longer option’s name following”no ” no-” prefix (e.g., –no-R is identical to the option –no-relative).

For instance: If you’d like to utilize”-a” (–archive) but you don’t want the option of (–owner) instead of turning -a into -rlptgD you can use -a –no (or the corresponding -a, no-owner).

The order in which the options are specified is critical: if for example, you choose no-r in the -a option then the option with the -r option will turn on in the opposite direction to the option -a –no-r. Be aware that the side effects that result from using this files-from option are not positional because it affects how the system defaults to a number of options, and may alter what is meant by the word ‘a’ (see for more details on the option –files-from options for further information).

-r, –recursive

This instructs the rsync to replicate directories in a recursive manner. Check out “-d” for directories. ( -d).

Since the release of rsync 3.0.0 The Recursive algorithm implemented has been changed to an incremental scan, which consumes much less memory than it did prior to and initiates the transfer after the initial scan of a few directories has been completed. The incremental scan affects only the recursion algorithm we use, it does not alter the non-recursive nature of the transfer. This is only possible only if both sides that are transferred are running at least 3.0.0. 3.0.0.

Certain options require the rsync to be aware of the complete list of files, and these options block increment recursion in the mode. They include: –delete-before, –delete-after, –prune-empty-dirs and delay-updates. This is because the default delete mode you select when you select the option –delete is currently deleted-during the time that each end of your connection is at 3.0.0 (use –del or delete-during in order to ask for this enhanced deletion method explicitly). Check out the deletion delay option, which is a better option instead of delete-after.

Incremental recursion is possible to disable with the option –no-inc-recursive option, or the shorter –no-i-r alternative.

-R, –relative

Use relative paths. This signifies that all path names that are specified on the command line will be transmitted to the server, instead of just the final parts of file names. This is especially useful when you need to share multiple directories at the same at the same time. For instance, suppose you were to use this command:

Then a file with the name /tmp/foo/bar/baz.c would be created on the remote computer while preserving the full path. These additional path elements are known as “implied directories” (i.e.”the” directories “foo” directories and “foo/bar” directories as in the example above).

Starting with the release of 3.0.0, rsync always identifies these implicit directories as actual directories in the list of files, regardless of whether the path element is actually an symlink from the sending side. This can prevent some really bizarre behavior when copying the entire path of a file that you did not realize contained the symlink. If you’re looking copy a server-side symlink add both the symlink in its path, as well as the your referent directory through its actual path. If you’re using an older version of version of rsync on the receiving side, you might need to choose the option to not include any implicit directories. option. You can also restrict the amount of information about a path that is sent in implicit directories with each of the paths you define. If you are using a modern Rsync on the receiving side (beginning at 2.6.7) it is possible to include a dot or cut-off into the path from which you are sending it in the following manner: This would result in /tmp/bar/baz.c on the remote machine. Be aware that the dot needs to preceded by a space therefore ” /foo/.” is not abbreviated. For older versions of rsync versions, you’d require a chdir in order to limit your source paths. For instance the following scenario occurs when you are pushing files: The parens are used to put both commands in an additional shell, to ensure it’s possible that”cd” is a separate command “cd” command isn’t in use for any subsequent commands. If you’re extracting the files of an old Rsync using this method, you can use the following idiom (but only for not-daemon after all):

–no-implied-dirs

This option alters how the system behaves by default with the relative option. If this option is selected that way, the attributes of the implied directories that are derived from the source names will not be transferred. This means that the elements of the destination system remain unaltered if they do exist in the absence of implied directories get created using default attributes. This also allows for implied path elements to be able to display major differences, for instance, being a symlink for a directory from the receiving side.

For example when a command-line argument or a files-from entry instructs that rsync should transfer to the destination system file “path/foo/file” The directories “path” and “path/foo” are implied when using the relative option. When ” path/foo” is a symlink that points to ” bar” on the destination system when it receives the file, rsync will typically delete ” path/foo” create the directory and then transfer the file to this new directory. When using no-implied-dirs this rsync update ” path/foo/file” by using the path elements that are already present that indicate the file is created within ” path/bar“. Another option to achieve this preservation of links is by using an option called “keep-dirlinks” option (which can also affect symlinks to directories during the remainder of the file transfer).

If you’re pulling the files out of an Rsync earlier than 3.0.0 it is possible to select this method if your side that is sending includes a symlink on the path you want to transfer and you would like the implicit directories to be transferred in the normal way.

-b, –backup

This option lets you control which destination files will be renamed when the file is moved or deleted. You can determine where the backup file is stored and which (if any) suffix is added with the backup-dir and –suffix options..

Be aware that if you do not specify the backup-dir option, (1) the option –omit-dir times will be implicit and (2) when –delete is also active (without the option to exclude deletion) the rsync program adds the “protect” filter-rule to the backup suffix at the end of your excludes (e.g.”-f “P *~”). This stops previously backup files from being erased. Be aware that if you’re providing filtering rules, you might need to manually add an the exclude/protect rule higher in the list to ensure it is given sufficient value to make it effective (e.g. If your rules call for an exclusion/inclusion trailing with ‘ *‘, then the rule that is automatically added would not be met).

–backup-dir=DIR

Together with the backup option instructs the rsync to save all backups within the directory specified on the other side. This is a good option to make incremental backups. You can also add a suffix to backups via the –suffix option (otherwise the backups within the specified directory retain the original names of the files).

If you choose to specify an absolute path, the Backup directory is located relative to that directory. Therefore, you should probably specify the absolute or one which begins by a ” ../“. If you specify an Rsync daemon acts as the recipient, then the backup directory is not able to be moved outside of the hierarchy of path paths for the module be sure to take precautions not to delete or copy it into it.

–suffix=SUFFIX

This option lets you alter the backup suffix that is default by the –backup ( -b) option. Its default suffix will be when no backup-dir was set, otherwise it’s an unspecified value.

-u, –update

This causes the rsync to remove any files that are on the destination file and contain a modification time which is more recent that the original file. If a destination file has a modified time similar to that of the time of the source file the file is updated if the sizes differ.

This does not alter the copying process of symlinks or any other files. A difference in the file format between the sender and the receiver is usually considered sufficient to warrant an update, regardless of what date is displayed on the items. Also that if the source is an account in which the destination’s file is and the transfer takes place regardless of the datestamps.

This is a rule for transfer and not an exclude which means it doesn’t impact the files that are included in the file-lists and, therefore, it won’t impact the deletions. It is only limiting the file that the receiver wants to transfer.

–inplace

This change alters the way the rsync program moves a file when the data in it requires updating in lieu of the default method of making a new duplicate of the data, and shifting it to the destination file after it has been completed, rsync instead writes the updated information straight to the file destination.

This can have a number of consequences:

• Hard links aren’t broken. The new data will be accessible through another hard link to the destination. Furthermore attempts to copy data from different sources onto a multipli-linked destination file can result in an “tug of war” where the destination information shifting back and forth.
• The binaries in use cannot be upgraded (either they are not updated, or OS stops the process from occurring, or the binaries who attempt to update their data will behave improperly or even crash).
• The data in the file will be inconsistent throughout the transfer, and will remain that way in the event of a transfer interruption or if the update does not succeed.
• An unreadable file is rsync-compatible cannot write to can’t be modified. While superusers are able to edit every file in the system, a regular user must be granted write access for the file to be opened for writing to succeed.
• The effectiveness of the rsync‘s delta-transfer algorithm can be decreased if information inside the target file gets written over before it is transferred to another location within the file. This is not the case when you are using the –backup option because the rsync algorithm is intelligent enough to utilize this backup as the foundation file to transfer the data.

This feature is helpful to transfer large files that have appsended or block-based data, as well as for systems which can be hard disk bound, but not connected to the network. It may also prevent the copy-on-write snapshot of a filesystem from dispersing the entire content of a file that contains minor modifications.

The option is the option of –partial (since the interruption of a transfer will not erase the file) However, it is in conflict when used with partial-dir and delay-updates. Prior to Rsync 2.6.4 –inplace was also not compatible with the –compare-dest along with the –link-dest option..

–append

This results in the rsync to update a file , by appending information to an end-of-file which assumes that the data is already present at the other end is exactly the same as the beginning of the file on the side that is sending it. If a file has to be transferred, and the size on the receiver side is identical or greater than its size on the sender side, the file is not transferred. This does not affect the update of a file’s attributes that are not content-related (e.g. permissions, permissions, ownership, etc.) in the event that the file doesn’t require transfer and also does not affect the process of updating other files that are not regular. It is a reference to the –inplace option but doesn’t cause conflict with the –sparse option (since it’s always expanding the length of a file).

–append-verify

This functions like it works with the append option, however the data already present on the receiving end is part of the full-file checksum verification, which results in an application to be sent again in the event that the verification process is unsuccessful ( rsync uses an ordinary unappending in-place transfer to return).

Prior to rsync 3.0.0 the –append option was the same as –append-verify, and when you’re using an older version of rsync (or the transfer is using protocols prior to 30) using either option will trigger an –append-verify transfer.

-d, –dirs

The sending party should include any directories discovered. Contrary to the –recursive directories, the contents of a directory cannot be copied unless the name of the directory given contains “.” or is terminated by an slash trailing the name (e.g., ” .“, ” dir/.“, ” dir/” and so on.). If you do not specify this option or with the option to use the –recursive choice, the rsync program will ignore all directories it encounters (and generate an appropriate message for each). If you choose each of directories with –dirs in addition to “recursive”, –recursive is the preferred option.

The option –dirs will be implicitly implied in the option –files-from or using the option –list-only (including an implicit –list-only use) when –recursive isn’t specified (so directories will be visible in the list). You can specify the option –no-dirs (or –no-d) for those who want to disable this.

There is also a backward-compatibility helper option, –old-dirs (or –old-d) that tells rsync to use a hack of “-r –exclude=’/*/*'” to get an older rsync to list a single directory without recursing.

-l, –links

If symlinks are detected Recreate the symlink at the destination.

-L, –copy-links

When symlinks appear when they are encountered, the object they link to (the reference) will be copied not than the symbol. In earlier versions of rsync this feature resulted in instructing the receiver to follow symlinks for example, symlinks that point to directories. For a more contemporary rsync like this it is necessary to set the option –keep-dirlinks ( -K) to enable this additional behaviour. Only one exemption is you send data to an Rsync which isn’t able to comprehend -K . In this scenario the -L option is a side effect of -K for the older rsync that is receiving.

–copy-unsafe-links

This instructs the rsync program to replicate the referents of symbolic links which point away from the copied tree. Absolute symlinks also behave as normal files, as are any symlinks within the source path when the option –relative is specified. This option does not have any additional impact in the event that the option –copy-links was specified.

–safe-links

This instructs the rsync to not to consider any symbolic links that point beyond of the tree copied. All absolute symlinks will be not considered. This option when used when combined with the –relative option can result in unexpected results.

-k, –copy-dirlinks

This setting causes the sending party to consider a link as directories as if they was a real directory. This can be useful in cases where you don’t want links to non-directories affected since they’d be by using copies of links.