[Quick Tips: Backup]: How to resume partially downloaded or transferred files using Rsync

How to resume partially downloaded or transferred files using Rsync

There are chances that the large files which are being copied or downloaded over SSH using SCP command might be interrupted or cancelled or broken due to power failure or network failure or user intervention. Today I was copying my Ubuntu 16.04 ISO to my remote system. When the power is gone, my network connection is dropped immediately and the copy process is terminated. This is just a simple example. The ISO is not that big size, and I can copy it again in couple of mins when the power is restored. But in production environment, you don’t want to do it while you’re transferring a large important folders or files.

Also, you can’t always resume the aborted process using scp command. If you do, It will simply overwrite your existing files or folders. What will you do in such cases? Luckily, we have Rsync utility which is used to resume the aborted copy or download process where you left it off. Rsync is a fast, versatile file copying utility that can be used to copy and transfer files or folders to and from remote and local systems.

It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.

Just like SCP, rsync will also copy files over SSH. In case you wanted to download or transfer a big files and folders over SSH, I recommend you to use rsync utility. Be mindful that the rsync utility should be installed in both sides, remote and local systems in order to resume partially downloaded files.

This brief tutorial describes how to resume partially downloaded or transferred files using rsync.

Well, let us see an example. I am going to copy Ubuntu 16.04 ISO from my local system to remote system with command:

scp Downloads/xenial-desktop-amd64\ Alpha1.iso sk@192.168.1.103:/home/sk/

Here,

• sk is my remote system’s username
• 192.168.1.103 is the IP address of remote machine.

As you see in the below output, I have terminated the copy process when it reached 25%.

Sample output:

sk@192.168.1.103's password: 
xenial-desktop-amd64 Alpha1.iso 25% 353MB 16.6MB/s 01:03 ETA^Csk@sk:~$

If you re-run the above command, it simply will overwrite the existing file. It means that the copy process will not be resumed where we left it off.

In order to resume the copy process, use rsync command as shown below.

rsync -P -rsh=ssh Downloads/xenial-desktop-amd64\ Alpha1.iso sk@192.16.1.103:/home/sk/

Sample output:

sk@192.168.1.103's password: 
sending incremental file list
xenial-desktop-amd64 Alpha1.iso
                   320.10M 22% 1.95MB/s 0:09:33

As you see in the above output, the copying process is resumed where we left it. Here, you can use –partial instead of parameter -P.

For example, you can use –partial flag instead of -P as shown below.

rsync --partial -rsh=ssh Downloads/xenial-desktop-amd64\ Alpha1.iso sk@192.16.1.103:/home/sk/

Here, the parameter –partial or -P tells the rsync command to keep the partial downloaded file and resumes the process.

Alternatively, we can use the following commands as well.

rsync -avP Downloads/xenial-desktop-amd64\ Alpha1.iso sk@192.168.1.103:/home/sk/

Or,

rsync -av --partial Downloads/xenial-desktop-amd64\ Alpha1.iso sk@192.168.1.103:/home/sk/

That’s it. You know now how to resume cancelled, interrupted, and partially downloaded files using rsync command. If you find this tutorial helpful, please share it on your social networks and support us.

Thanks for reading. Cheers!