Blog

03/06/2019

How to test the network speed/throughput between two Linux servers



I have setup total four servers in cloud based data center. They run either CentOS or Ubuntu Linux. I need to make sure clustered file system performance is reasonable as my servers shared with other users. How do I check the speed between the two Linux servers using command line options for private LAN/VLAN?

You can test the network speed (throughput) between Ubuntu/CentOS/Debian/Fedora or Unix box using the iperf command. Iperf commands show info about bandwidth, delay, jitter, and datagram loss. From the man page:

iperf is a tool for performing network throughput measurements. It can test either TCP or UDP throughput. To perform an iperf test the user must establish both a server (to discard traffic) and a client (to generate traffic).

Here is my sample setup:

Fig.01 Test network speed between two Linux computers

Step 1 – Installation

You need to install iperf as per your Linux distro version. There are two versions of iperf:

  1. iperf
  2. iperf3

The iperf/iperf3 works on Linux, Unix, Windows, *BSD and other operating systems.

Install iperf on a Ubuntu/Debian Linux server

Type the following apt-get command/[nixmcd name=”apt”] to install iperf on both servers:
$ sudo apt-get update
$ sudo apt-get install iperf

Sample outputs:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  iperf
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 51.0 kB of archives.
After this operation, 184 kB of additional disk space will be used.
Get:1 http://mirrors.linode.com/ubuntu xenial/universe amd64 iperf amd64 2.0.5+dfsg1-2 [51.0 kB]
Fetched 51.0 kB in 0s (4921 kB/s)
Selecting previously unselected package iperf.
(Reading database ... 26108 files and directories currently installed.)
Preparing to unpack .../iperf_2.0.5+dfsg1-2_amd64.deb ...
Unpacking iperf (2.0.5+dfsg1-2) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up iperf (2.0.5+dfsg1-2) ...

Reading package lists… Done Building dependency tree Reading state information… Done The following NEW packages will be installed: iperf 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 51.0 kB of archives. After this operation, 184 kB of additional disk space will be used. Get:1 http://mirrors.linode.com/ubuntu xenial/universe amd64 iperf amd64 2.0.5+dfsg1-2 [51.0 kB] Fetched 51.0 kB in 0s (4921 kB/s) Selecting previously unselected package iperf. (Reading database … 26108 files and directories currently installed.) Preparing to unpack …/iperf_2.0.5+dfsg1-2_amd64.deb … Unpacking iperf (2.0.5+dfsg1-2) … Processing triggers for man-db (2.7.5-1) … Setting up iperf (2.0.5+dfsg1-2) …

A note about installing iperf3 on a Debian/Ubuntu Linux

It is also possible to install iperf3 (latest version) as follows:
$ sudo apt-get install iperf3
Sample outputs:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libiperf0
The following NEW packages will be installed:
  iperf3 libiperf0
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 58.5 kB of archives.
After this operation, 238 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://mirrors.linode.com/ubuntu xenial/universe amd64 libiperf0 amd64 3.0.11-1 [50.4 kB]
Get:2 http://mirrors.linode.com/ubuntu xenial/universe amd64 iperf3 amd64 3.0.11-1 [8090 B]
Fetched 58.5 kB in 0s (4796 kB/s)
Selecting previously unselected package libiperf0:amd64.
(Reading database ... 26120 files and directories currently installed.)
Preparing to unpack .../libiperf0_3.0.11-1_amd64.deb ...
Unpacking libiperf0:amd64 (3.0.11-1) ...
Selecting previously unselected package iperf3.
Preparing to unpack .../iperf3_3.0.11-1_amd64.deb ...
Unpacking iperf3 (3.0.11-1) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up libiperf0:amd64 (3.0.11-1) ...
Setting up iperf3 (3.0.11-1) ...
Processing triggers for libc-bin (2.23-0ubuntu9) ...

Install iperf on a Fedora Linux server

Type the following [nixmcd name=”dnf”] to install iperf on both servers:
$ sudo dnf install iperf

A note about installing iperf3 on a CentOS/RHEL/SL/Oracle Linux

Type the following command:
$ sudo dnf install iperf3

Install iperf on a CentOS/RHEL/Oracle/Scientific Linux server

Type the following [nixmcd name=”yum”] to install iperf on both servers (first turn on EPEL repo for CentOS/RHEL):
$ sudo yum install iperf
Sample outputs:

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.mirror.snu.edu.in
 * epel: mirror01.idc.hinet.net
 * extras: centos.mirror.snu.edu.in
 * updates: centos.mirror.snu.edu.in
Resolving Dependencies
--> Running transaction check
---> Package iperf.x86_64 0:2.0.8-1.el7 will be installed
--> Finished Dependency Resolution
 
Dependencies Resolved
 
=====================================================================
 Package       Arch           Version             Repository    Size
=====================================================================
Installing:
 iperf         x86_64         2.0.8-1.el7         epel         321 k
 
Transaction Summary
=====================================================================
Install  1 Package
 
Total download size: 321 k
Installed size: 396 k
Is this ok [y/d/N]: y
Downloading packages:
iperf-2.0.8-1.el7.x86_64.rpm                    | 321 kB   00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : iperf-2.0.8-1.el7.x86_64                          1/1 
  Verifying  : iperf-2.0.8-1.el7.x86_64                          1/1 
 
Installed:
  iperf.x86_64 0:2.0.8-1.el7                                         
 
Complete!

Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: centos.mirror.snu.edu.in * epel: mirror01.idc.hinet.net * extras: centos.mirror.snu.edu.in * updates: centos.mirror.snu.edu.in Resolving Dependencies –> Running transaction check —> Package iperf.x86_64 0:2.0.8-1.el7 will be installed –> Finished Dependency Resolution Dependencies Resolved ===================================================================== Package Arch Version Repository Size ===================================================================== Installing: iperf x86_64 2.0.8-1.el7 epel 321 k Transaction Summary ===================================================================== Install 1 Package Total download size: 321 k Installed size: 396 k Is this ok [y/d/N]: y Downloading packages: iperf-2.0.8-1.el7.x86_64.rpm | 321 kB 00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : iperf-2.0.8-1.el7.x86_64 1/1 Verifying : iperf-2.0.8-1.el7.x86_64 1/1 Installed: iperf.x86_64 0:2.0.8-1.el7 Complete!

A note about installing iperf3 on a CentOS/RHEL/SL/Oracle Linux

Type the following command:
$ sudo yum install iperf3

Step 2 – Start an iperf server on serverA (server mode)

Type the following command to run iperf in server mode:
iperf -s
OR
iperf3 -s
Sample outputs:

------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------

A note about firewall config

Please note that you must open TCP port 5001 on the serverA. See how to use ufw to open a port on Ubuntu/Debian Linux:
$ sudo ufw allow from 192.168.149.0/24 to 192.168.149.69 port 5001 proto tcp
It is also possible to change the port by passing the -p option as follows (e.g. open and use TCP port 2323):
iperf -s -p 2323

Step 3 – Start an iperf server on serverB (client mode)

Type the following command to run in client mode, connecting to 192.168.149.69:
iperf -c {ip-address-of-serverA-here}
iperf -c {ip-address-of-serverA-here} -p {tcp-port}
iperf -c 192.168.149.69 -p 2323
iperf -c 192.168.149.69

OR
iperf3 -c {ip-address-of-serverA-here}
iperf3 -c {ip-address-of-serverA-here} -p {tcp-port}
iperf3 -c 192.168.149.69 -p 2323
iperf3 -c 192.168.149.69

Sample outputs:

Fig.02: Linux network speed test result

How to use UDP rather than TCP

On the server side:
iperf -s -u
On the client side:
iperf -c 192.168.149.69 -u

See also: How To Install Speedtest-cli On a CentOS / RHEL / Scientific / Fedora Linux To Check Internet Speed From the Bash Shelll

How to set number of parallel client threads to run (client mode options)

Pass the -P option:
iperf -c 192.168.149.69 -P 3
You can also set time in seconds to transmit for (default 10 secs) as follows:
iperf -c 192.168.149.69 -P 3 -t 30
Sample outputs:

------------------------------------------------------------
Client connecting to 192.168.149.69, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  5] local 192.168.181.40 port 54198 connected with 192.168.149.69 port 5001
[  3] local 192.168.181.40 port 54194 connected with 192.168.149.69 port 5001
[  4] local 192.168.181.40 port 54196 connected with 192.168.149.69 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-30.0 sec  2.26 GBytes   647 Mbits/sec
[  3]  0.0-30.0 sec  2.89 GBytes   828 Mbits/sec
[  4]  0.0-30.5 sec  1.91 GBytes   539 Mbits/sec
[SUM]  0.0-30.5 sec  7.07 GBytes  1.99 Gbits/sec

———————————————————— Client connecting to 192.168.149.69, TCP port 5001 TCP window size: 85.0 KByte (default) ———————————————————— [ 5] local 192.168.181.40 port 54198 connected with 192.168.149.69 port 5001 [ 3] local 192.168.181.40 port 54194 connected with 192.168.149.69 port 5001 [ 4] local 192.168.181.40 port 54196 connected with 192.168.149.69 port 5001 [ ID] Interval Transfer Bandwidth [ 5] 0.0-30.0 sec 2.26 GBytes 647 Mbits/sec [ 3] 0.0-30.0 sec 2.89 GBytes 828 Mbits/sec [ 4] 0.0-30.5 sec 1.91 GBytes 539 Mbits/sec [SUM] 0.0-30.5 sec 7.07 GBytes 1.99 Gbits/sec

Here is output from iperf3:
## on the server side ##
iperf3 -s
## on the client side ##
iperf3 -c 192.168.149.69

Sample outputs:

Connecting to host 192.168.149.69, port 5201
[  4] local 192.168.181.40 port 55522 connected to 192.168.149.69 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   235 MBytes  1.97 Gbits/sec   32    609 KBytes       
[  4]   1.00-2.00   sec   230 MBytes  1.93 Gbits/sec   21    670 KBytes       
[  4]   2.00-3.00   sec   159 MBytes  1.33 Gbits/sec   19    635 KBytes       
[  4]   3.00-4.00   sec   235 MBytes  1.97 Gbits/sec    7    686 KBytes       
[  4]   4.00-5.00   sec   229 MBytes  1.92 Gbits/sec   86    757 KBytes       
[  4]   5.00-6.00   sec   206 MBytes  1.73 Gbits/sec    6    537 KBytes       
[  4]   6.00-7.00   sec   210 MBytes  1.76 Gbits/sec    5    556 KBytes       
[  4]   7.00-8.00   sec   235 MBytes  1.97 Gbits/sec   10    617 KBytes       
[  4]   8.00-9.00   sec   245 MBytes  2.06 Gbits/sec   11    670 KBytes       
[  4]   9.00-10.00  sec   234 MBytes  1.96 Gbits/sec   10    713 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  2.17 GBytes  1.86 Gbits/sec  207             sender
[  4]   0.00-10.00  sec  2.16 GBytes  1.86 Gbits/sec                  receiver

Connecting to host 192.168.149.69, port 5201 [ 4] local 192.168.181.40 port 55522 connected to 192.168.149.69 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 235 MBytes 1.97 Gbits/sec 32 609 KBytes [ 4] 1.00-2.00 sec 230 MBytes 1.93 Gbits/sec 21 670 KBytes [ 4] 2.00-3.00 sec 159 MBytes 1.33 Gbits/sec 19 635 KBytes [ 4] 3.00-4.00 sec 235 MBytes 1.97 Gbits/sec 7 686 KBytes [ 4] 4.00-5.00 sec 229 MBytes 1.92 Gbits/sec 86 757 KBytes [ 4] 5.00-6.00 sec 206 MBytes 1.73 Gbits/sec 6 537 KBytes [ 4] 6.00-7.00 sec 210 MBytes 1.76 Gbits/sec 5 556 KBytes [ 4] 7.00-8.00 sec 235 MBytes 1.97 Gbits/sec 10 617 KBytes [ 4] 8.00-9.00 sec 245 MBytes 2.06 Gbits/sec 11 670 KBytes [ 4] 9.00-10.00 sec 234 MBytes 1.96 Gbits/sec 10 713 KBytes – – – – – – – – – – – – – – – – – – – – – – – – – [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 2.17 GBytes 1.86 Gbits/sec 207 sender [ 4] 0.00-10.00 sec 2.16 GBytes 1.86 Gbits/sec receiver

Getting help

Type the following command:
$ man iperf
$ iperf --help

Usage: iperf [-s|-c host] [options]
       iperf [-h|--help] [-v|--version]
 
Client/Server:
  -f, --format    [kmKM]   format to report: Kbits, Mbits, KBytes, MBytes
  -i, --interval  #        seconds between periodic bandwidth reports
  -l, --len       #[KM]    length of buffer to read or write (default 8 KB)
  -m, --print_mss          print TCP maximum segment size (MTU - TCP/IP header)
  -o, --output    <filename> output the report or error message to this specified file
  -p, --port      #        server port to listen on/connect to
  -u, --udp                use UDP rather than TCP
  -w, --window    #[KM]    TCP window size (socket buffer size)
  -B, --bind      <host>   bind to <host>, an interface or multicast address
  -C, --compatibility      for use with older versions does not sent extra msgs
  -M, --mss       #        set TCP maximum segment size (MTU - 40 bytes)
  -N, --nodelay            set TCP no delay, disabling Nagle's Algorithm
  -V, --IPv6Version        Set the domain to IPv6
 
Server specific:
  -s, --server             run in server mode
  -U, --single_udp         run in single threaded UDP mode
  -D, --daemon             run the server as a daemon
 
Client specific:
  -b, --bandwidth #[KM]    for UDP, bandwidth to send at in bits/sec
                           (default 1 Mbit/sec, implies -u)
  -c, --client    <host>   run in client mode, connecting to <host>
  -d, --dualtest           Do a bidirectional test simultaneously
  -n, --num       #[KM]    number of bytes to transmit (instead of -t)
  -r, --tradeoff           Do a bidirectional test individually
  -t, --time      #        time in seconds to transmit for (default 10 secs)
  -F, --fileinput <name>   input the data to be transmitted from a file
  -I, --stdin              input the data to be transmitted from stdin
  -L, --listenport #       port to receive bidirectional tests back on
  -P, --parallel  #        number of parallel client threads to run
  -T, --ttl       #        time-to-live, for multicast (default 1)
  -Z, --linux-congestion <algo>  set TCP congestion control algorithm (Linux only)
 
Miscellaneous:
  -x, --reportexclude [CDMSV]   exclude C(connection) D(data) M(multicast) S(settings) V(server) reports
  -y, --reportstyle C      report as a Comma-Separated Values
  -h, --help               print this message and quit
  -v, --version            print version information and quit
 
[KM] Indicates options that support a K or M suffix for kilo- or mega-
 
The TCP window size option can be set by the environment variable
TCP_WINDOW_SIZE. Most other options can be set by an environment variable
IPERF_<long option name>, such as IPERF_BANDWIDTH.

Usage: iperf [-s|-c host] [options] iperf [-h|–help] [-v|–version] Client/Server: -f, –format [kmKM] format to report: Kbits, Mbits, KBytes, MBytes -i, –interval # seconds between periodic bandwidth reports -l, –len #[KM] length of buffer to read or write (default 8 KB) -m, –print_mss print TCP maximum segment size (MTU – TCP/IP header) -o, –output <filename> output the report or error message to this specified file -p, –port # server port to listen on/connect to -u, –udp use UDP rather than TCP -w, –window #[KM] TCP window size (socket buffer size) -B, –bind <host> bind to <host>, an interface or multicast address -C, –compatibility for use with older versions does not sent extra msgs -M, –mss # set TCP maximum segment size (MTU – 40 bytes) -N, –nodelay set TCP no delay, disabling Nagle’s Algorithm -V, –IPv6Version Set the domain to IPv6 Server specific: -s, –server run in server mode -U, –single_udp run in single threaded UDP mode -D, –daemon run the server as a daemon Client specific: -b, –bandwidth #[KM] for UDP, bandwidth to send at in bits/sec (default 1 Mbit/sec, implies -u) -c, –client <host> run in client mode, connecting to <host> -d, –dualtest Do a bidirectional test simultaneously -n, –num #[KM] number of bytes to transmit (instead of -t) -r, –tradeoff Do a bidirectional test individually -t, –time # time in seconds to transmit for (default 10 secs) -F, –fileinput <name> input the data to be transmitted from a file -I, –stdin input the data to be transmitted from stdin -L, –listenport # port to receive bidirectional tests back on -P, –parallel # number of parallel client threads to run -T, –ttl # time-to-live, for multicast (default 1) -Z, –linux-congestion <algo> set TCP congestion control algorithm (Linux only) Miscellaneous: -x, –reportexclude [CDMSV] exclude C(connection) D(data) M(multicast) S(settings) V(server) reports -y, –reportstyle C report as a Comma-Separated Values -h, –help print this message and quit -v, –version print version information and quit [KM] Indicates options that support a K or M suffix for kilo- or mega- The TCP window size option can be set by the environment variable TCP_WINDOW_SIZE. Most other options can be set by an environment variable IPERF_<long option name>, such as IPERF_BANDWIDTH.

See iperf home page for more info.

(adsbygoogle = window.adsbygoogle || []).push({});

Posted by: SXI ADMIN

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

14/08/2019

How to KVM, QEMU start or stop virtual machine from command line (CLI)

KVM or Kernel Based Virtual Machine is a popular virtualization technology. It allows you to run virtual guest machines over a host machine. To start...
14/08/2019

How to Docker backup Saving and restoring your volumes

Running a Docker volume backup First, we spin up a temporary container, and we mount the backup folder and the target Docker volume to this container....
12/08/2019

How to Start and Enable Firewalld on CentOS 7

In this article, we discuss how to start and enable firewalld. It is highly recommended that you have a firewall protecting your server.Pre-Flight CheckThese...