Thursday, April 6, 2017

How to set rsync speed limit from eating all bandwidth with –bwlimit option

How to set rsync speed limit from eating all bandwidth with –bwlimit option


The rsync command is eating all my bandwidth from a backup script and generating tons of I/O. How do I set a limit for file transfer speed of rsync command on Linux or Unix-like system? How can I keep my rsync script from using all of my bandwidth?

You can easily limit I/O bandwidth using --bwlimit option. In this tutorial you will learn how to setup I/O limit using rsync command on Linux, MacOS, FreeBSD and Unix-like systems.
rsync-bwlimit-examples

How to keep rsync command from using all of my bandwidth

The syntax is:
rsync --bwlimit=KBPS src dst
rsync --bwlimit=KBPS [options] src dst
rsync --bwlimit=KBPS [options] src user@dst
rsync --bwlimit=KBPS [options] user@src /path/to/dir
rsync --bwlimit=KBPS -a -P /path/to/dir/ user@server1.cyberciti.biz

You set I/O limit in 1000 KBytes per second:
$ rsync --bwlimit=1000 /path/to/source /path/to/dest/
$ rsync --bwlimit=1000 /var/www/html/ \
backups@server1.cyberciti.biz:~/mysite.backups/

In this example, pull files from FreeNAS server with 5000KB/s in current directory:
$ rsync --bwlimit=5000 --delete -P -az -H --numeric-ids \
vivek@192.168.2.30:linode-www .

Say hello to ionice

Use ionice command to set or get the I/O scheduling class and priority for a program such as rsync or your own backup script. So you can take control of I/O bandwidth using ionice utility on Linux as follows:
# /usr/bin/ionice -c2 -n7 /root/scripts/nas.backup.full
OR
# /usr/bin/ionice -c2 -n7 rsync \
-bwlimit=1000 /path/to/source /path/to/dest/

Where:
  1. -c or --class {class} : Name or number of scheduling class, 0: none, 1: realtime, 2: best-effort, 3: idle.
  2. -n or --classdata {num} : priority (0..7) in the specified scheduling class, only for the realtime and best-effort classes

No comments: