Blog

03/06/2019

How to set rsync speed limit from eating all bandwidth with –bwlimit option



The rsync command is eating all my bandwidth from a backup script and generating tons of I/O. How do I set a limit for file transfer speed of rsync command on Linux or Unix-like system? How can I keep my rsync script from using all of my bandwidth?

You can easily limit I/O bandwidth using --bwlimit option. In this tutorial you will learn how to setup I/O limit using rsync command on Linux, MacOS, FreeBSD and Unix-like systems.

How to keep rsync command from using all of my bandwidth

The syntax is:
rsync --bwlimit=KBPS src dst
rsync --bwlimit=KBPS [options] src dst
rsync --bwlimit=KBPS [options] src user@dst
rsync --bwlimit=KBPS [options] user@src /path/to/dir
rsync --bwlimit=KBPS -a -P /path/to/dir/ user@server1.sxi.io

You set I/O limit in 1000 KBytes per second:
$ rsync --bwlimit=1000 /path/to/source /path/to/dest/
$ rsync --bwlimit=1000 /var/www/html/
backups@server1.sxi.io:~/mysite.backups/

In this example, pull files from FreeNAS server with 5000KB/s in current directory:
$ rsync --bwlimit=5000 --delete -P -az -H --numeric-ids
vivek@192.168.2.30:linode-www .

Say hello to ionice

Use ionice command to set or get the I/O scheduling class and priority for a program such as rsync or your own backup script. So you can take control of I/O bandwidth using ionice utility on Linux as follows:
# /usr/bin/ionice -c2 -n7 /root/scripts/nas.backup.full
OR
# /usr/bin/ionice -c2 -n7 rsync
-bwlimit=1000 /path/to/source /path/to/dest/

Where:

  1. -c or --class {class} : Name or number of scheduling class, 0: none, 1: realtime, 2: best-effort, 3: idle.
  2. -n or --classdata {num} : priority (0..7) in the specified scheduling class, only for the realtime and best-effort classes

Posted by: SXI ADMIN

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

20/08/2019

Start your Bitcoin Exchange with our Software

Start your Bitcoin Exchange with our Software Setup your white label bitcoin exchange right away. Or create your own cryptocurrency using...
14/08/2019

How to KVM, QEMU start or stop virtual machine from command line (CLI)

KVM or Kernel Based Virtual Machine is a popular virtualization technology. It allows you to run virtual guest machines over a host machine. To start...
14/08/2019

How to Docker backup Saving and restoring your volumes

Running a Docker volume backup First, we spin up a temporary container, and we mount the backup folder and the target Docker volume to this container....