ttcp/nttcp/nuttcp/iperf

http://sd.wareonearth.com/~phil/net/ttcp/

ttcp/nttcp/nuttcp/iperf versions

ttcp was one of the first TCP throughput testing tools ever written. It was created by Mike Muuss at BRL to compare the performance of TCP stacks by U.C. Berkeley and BBN to help DARPA decide which version to place in the first BSD Unix release (Berkeley won). The name stands for "Test TCP", but it also supports UDP testing. Many variations have since been created with different defaults, new options, ports to new systems, etc.

What does the reported data rate really mean?

All variations of ttcp/iperf report payload or user data rates, i.e. no overhead bytes from headers (TCP, UDP, IP, etc.) are included in the reported data rates. When comparing to "line" rates or "peak" rates, it is important to consider all of this overhead. It is also important to understand what the tools mean by "K" or "M" bits or bytes per second. Versions of the tools differ on this point.

Computer memory is measured in powers of two, e.g. 1 KB = 2^10 = 1024 bytes; 1 MB = 2^20 = 1024*1024 = 1048576 bytes. Data communication rates however should always be stated in simple bits per second. For example "100 megabit ethernet" can send exactly 100,000,000 bits per second.

K and M in ttcp/nttcp/iperf
• ttcp original: K = 1024 (M is not used)
• nttcp: K = 1024, M = 1000*1024
• iperf 1.1.1 and earlier: K = 1024, M = 1024*1024
• iperf 1.2 and above: K=1000, M=1000000
• nuttcp: K=1000, M=1000000

Thus for some early tools, the reported K bits per second should be multipled by 1.024 to get the actual kilobits per second payload data rate. Reported M bits per second should be multipled by 1.024 for nttcp and 1.048576 for iperf to get the payload data rate in millions of bits per second. Iperf 1.2 and above and nuttcp correctly report bits per second rates in power of ten.

Differnet Versions

ttcp original

(source code)

RCSid[] = "@(#)$Header: ttcp.c,v 1.10 87/09/02 23:26:36 mike Exp $ (BRL)"

Usage: ttcp -t [-options] host  -l## length of bufs written to network (default 1024)
 -s source a pattern to network
 -n## number of bufs written to network (-s only, default 1024)
 -p## port number to send to (default 2000)
 -u use UDP instead of TCP
Usage: ttcp -r [-options] >out
 -l## length of network read buf (default 1024)
 -s sink (discard) all data from network
 -p## port number to listen at (default 2000)
 -B Only output full blocks, as specified in -l## (for TAR)
 -u use UDP instead of TCP

Example run:

% ttcp -r -s
ttcp-r: nbuf=1024, buflen=1024, port=2000
ttcp-r: socket
ttcp-r: accept
ttcp-r: 0.0user 0.5sys 0:10real 5% 0i+0d 0maxrss 0+0pf 0+0csw
ttcp-r: 74891264 bytes processed
ttcp-r:      0.54 CPU sec  =    135437 KB/cpu sec,  1.0835e+06 Kbits/cpu sec
ttcp-r:    10.067 real sec =   7264.92 KB/real sec,   58119.3 Kbits/sec

It is also worth noting that the "bytes processed" number can wrap for large data transfers.

nttcp

(source code)

Created by someone at Silicon Graphics (SGI), it added the important option (-w) to set the transmit and receive window size (actually socket buffer sizes, which indirectly set the window size). It also changed several things:

• changed default nbuf (-n) from 1024 to 2048
• changed default buflen (-l) from 1024 to 65536
• changed default port (-p) from 2000 to 5001
• added window size (-w) option
• added TCP nodelay (-D) option
• reversed the meaning of (-s) source/sink option!

The default buffer length increase to 65536 was probably an attempt to maximize performance by reducing the number of write system calls. I have found however that on modern systems larger buffers are not always faster because they may not fit in cache memory as well. Somewhere around 8kB seems like a good value in my experience for todays hardware.

The change in default port and the meaning of -s is unfortunate. At least the author changed the program name. Others however have created "ttcp" programs that also have these changes. iperf and nuttcp continued nttcp's use of port 5001.

Usage: ttcp -t [-options] host [  -l## length of bufs written to network (default 8192)
 -s don't source a pattern to network, use stdin
 -n## number of source bufs written to network (default 2048)
 -p## port number to send to (default 5001)
 -u use UDP instead of TCP
 -D don't buffer TCP writes (sets TCP_NODELAY socket option)
 -L set SO_LONGER socket option
Usage: ttcp -r [-options >out]
 -l## length of network read buf (default 8192)
 -s don't sink (discard): prints all data from network to stdout
 -p## port number to listen at (default 5001)
 -B Only output full blocks, as specified in -l## (for TAR)
 -u use UDP instead of TCP

Example run:

% ./nttcp -r
ttcp-r: buflen=65536, nbuf=2048, port=5001 tcp
ttcp-r: socket
ttcp-r: accept from 127.0.0.1
send window size = 65535
receive window size = 65535
ttcp-r: 455426048 bytes in 10.05 real seconds = 44243.41 KB/sec = 353.9473 Mb/s
ttcp-r: 55550 I/O calls, msec/call = 0.19, calls/sec = 5526.05
ttcp-r: 0.0user 2.5sys 0:10real 25% 0i+0d 0maxrss 0+14pf 0+0csw

nuttcp

ftp://ftp.lcp.nrl.navy.mil/pub/nuttcp/

One of the best! Highly recommended. The author calls it n-u-t-t-c-p, but many of us affectionately call it nut-c-p. nuttcp can run as a server and passes all output back to the client side, so you don't need an account on the server side to see the results.

Usage: nuttcp or nuttcp -h      prints this usage info
Usage: nuttcp -V                prints version info
Usage: nuttcp -xt [-m] host     forward and reverse traceroute to/from server
Usage (transmitter): nuttcp -t [-options] host [3rd-party] [ out]
       -4      Use IPv4
       -6      Use IPv6
       -l##    length of network read buf (default 8192/udp, 65536/tcp)
       -s      don't sink (discard): prints all data from network to stdout
       -n##    number of bufs for server to write to network (default 2048)
       -w##    receiver window size in KB (or (m|M)B or (g|G)B)
       -ws##   server transmit window size in KB (or (m|M)B or (g|G)B)
       -wb     braindead Solaris 2.8 (sets both xmit and rcv windows)
       -p##    port number to listen at (default 5001)
       -P##    port number for control connection (default 5000)
       -B      Only output full blocks, as specified in -l## (for TAR)
       -u      use UDP instead of TCP
       -m##    use multicast with specified TTL instead of unicast (UDP)
       -N##    number of streams (starting at port number), implies -B
       -R##    server transmit rate limit in Kbps (or (m|M)bps or (g|G)bps)
       -T##    server transmit timeout in seconds (or (m|M)inutes or (h|H)ours)
       -i##    client interval reporting in seconds (or (m|M)inutes)
       -Ixxx   identifier for nuttcp output (max of 40 characters)
       -F      flip option to reverse direction of data connection open
       -xP##   set nuttcp process priority (must be root)
       -d      set TCP SO_DEBUG option on data socket
       -v[v]   verbose [or very verbose] output
       -b      brief output (default)
Usage (server): nuttcp -S [-options]
               note server mode excludes use of -s
       -4      Use IPv4 (default)
       -6      Use IPv6
       -1      oneshot server mode (implied with inetd/xinetd), implies -S
       -P##    port number for server connection (default 5000)
               note don't use with inetd/xinetd (use services file instead)
       -xP##   set nuttcp process priority (must be root)
       --no3rdparty don't allow 3rd party capability
       --nofork     don't fork server
Format options:
       -fxmitstats     also give transmitter stats (MB) with -i (UDP only)
       -frunningtotal  also give cumulative stats on interval reports
       -f-drops        don't give packet drop info on brief output (UDP)
       -f-percentloss  don't give %loss info on brief output (UDP)
       -fparse         generate key=value parsable output

iperf

http://dast.nlanr.net/Projects/Iperf/

% iperf -v
iperf version 1.1.1 (23 Feb 2000) pthreads

Example run:

% iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[  6] local 127.0.0.1 port 5001 connected with 127.0.0.1 port 3639
[ ID] Interval       Transfer     Bandwidth
[  6]  0.0-10.0 sec   421 MBytes   336 Mbits/sec

---

P. Dykstra, phil@sd.wareonearth.com, March 2001 (update March 2004)