Designing, building and troubleshooting networks is often the topic of our internetworking classes, but network analysis and testing is often of interest to us as network administrators. We need tools that allow us to generate network traffic and analyze the network's throughput performance. Cisco has included a useful tool in recent IOS releases so we'll talk about this tool and how to use it.
Most often, we've been using tools like FTP and Ping to generate TCP/IP traffic between end systems, and then analyzing network performance while this traffic was flowing. The problem with this approach is that we often do not have access to the end systems or they may have FTP performance that is incapable of testing high bandwidth networks. In many cases, we find that we have to install dedicated hardware to perform the tests - a process that may be impractical if the network consists of many remote LAN and WAN segments.
The Unix community has had a good tool for many years for doing this kind of testing: the Test TCP (TTCP) program. The source for TTCP is available from a variety of sources on the Net, including Chesapeake's Web site.
To use TTCP, you start a copy of TTCP in receive mode at one place within the network, then start a second copy in transmit mode at another place within the network. The results of the transfer of data from the transmitter to the receiver indicate the approximate performance of the path between the source and destination. By selecting the source and destination at various points with the network, you can analyze critical portions of the path.
TTCP has a real advantage over tools like FTP. If you have a high performance network, it is difficult for any single computer system to transfer data to or from disk at rates which are sufficient for real network testing. TTCP achieves high performance by filling a memory buffer with data, then repeatedly transmitting this data. Since everything is running from memory, you have a traffic transmitter and receiver that can operate at true network speeds.
Cisco has implemented a copy of TTCP in IOS 11.2 and later, currently as an undocumented command. Since it is undocumented, you will not find it by using the interactive help function. Instead, just type the command ttcp, then press RETURN. If the router model you are using supports TTCP, it will respond with a series of questions for the TTCP parameters. Because TTCP can create enormous amounts of network traffic, it is a privileged command.
This gives us a more widely available resource within our networks for generating traffic when performing network analysis and tuning. Because this capability is in the router, we no longer have to install special traffic generators in the network. You'll find that Cisco routers, because they are very efficient at moving IP data, are very good traffic transmitters and receivers. Now you can perform high-speed network traffic analysis without the need for extra equipment.
The Cisco version of TTCP sends and receives TCP data (the Unix version also does UDP.) You have control over the number of packets sent, the packet size, the port number at which the transmitter and receiver rendezvous, and several other parameters. By changing the parameters, you can test various buffering mechanisms in the network equipment between the transmitter and receiver.
And the destination doesn't have to be another router. To test throughput to a remote server, start only a transmitter and specify the discard port (TCP port 9) on the remote server.
TTCP reports the amount of data transferred, the transfer time, and the approximate throughput. By comparing the actual throughput with the theoretical bandwidth between the transmitter and receiver, you can tell whether the network is operating as expected. Variations in throughput may indicate a significant amount of other traffic, overloaded network equipment, or perhaps communications errors which are causing packets to be corrupted or dropped. By performing tests on individual segments of the path, you can determine which links or devices should be examined in more detail.
TCP is a connection oriented protocol, so we must have a receiver listening before a transmitter can connect. So the first step is to start up a TTCP receiver (or use the discard port on a destination system.) Then the transmitter is started. TTCP uses the time and the amount of data transferred, to calculate the throughput between the transmitter and the receiver.
Now that we know a little about TTCP, let's take a look at how to setup and run a network test. In this example, we'll test the throughput of a point-to-point 56Kbps link between two routers.
First, we must start the receiver. On the Cisco router, we would enter the following commands. The default values are in brackets, so just pressing RETURN accepts the default values. In starting the receive session below, we are modifying the receiver's TCP window size to mimic that used by Sun workstations. The last line of the startup output shows the parameters TTCP is using and that it is ready to receive.
Seattle#ttcp transmit or receive [receive]: buflen [8192]: bufalign [16384]: bufoffset [0]: port [5001]: sinkmode [y]: rcvwndsize [2144]: 4096 show tcp information at end [n]: ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001, rcvwndsize=4096 tcp
Then, we start the transmitter.
Tokyo#ttcp transmit or receive [receive]: trans Target IP address: 137.112.32.66 buflen [8192]: nbuf [2048]: 50 bufalign [16384]: bufoffset [0]: port [5001]: sinkmode [y]: buffering on writes [y]: show tcp information at end [n]: ttcp-t: buflen=8192, nbuf=50, align=16384/0, port=5001 tcp -> 137.112.32.66
When the connection is made, the receiver outputs a status line showing that it has accepted the connection. Upon completion of the transfer, the receiver statistics are output, showing the amount of data transferred, the transfer time, the calculated throughput, and the number of I/O calls to read the data:
ttcp-r: accept from 137.112.32.65 (mss 1460, sndwnd 2144, rcvwnd 4096) ttcp-r: 409600 bytes in 61064 ms (61.064 real seconds) (~5 kB/sec) +++ ttcp-r: 301 I/O calls ttcp-r: 0 sleeps (0 ms total) (0 ms average) Seattle#
Similarly, the transmitter outputs a status line when it connects to the receiver. When the transfer is complete, the transmitter statistics are output:
ttcp-t: connect (mss 1460, sndwnd 4096, rcvwnd 2144) ttcp-t: 409600 bytes in 60504 ms (60.504 real seconds) (~5 kB/sec) +++ ttcp-t: 50 I/O calls ttcp-t: 0 sleeps (0 ms total) (0 ms average) Tokyo#
TTCP calculates the approximate throughput in kilobytes per second. In this case, our throughput is 5KBps, which translates to 40Kbps (5KBps * 8 bits/Byte = 40Kbps.) This is over a 56Kbps point-to-point link. Is this throughput is reasonable?
Using the receive stats, let's analyze the link. The TCP buffer size on the transmitter is 8192 bytes. When TCP hands this size packet to IP, it will be broken into five 1500 byte IP packets, plus one 693 byte packet (for a total of 8192 bytes.) To send all 50 buffers, the router will need to send 300 IP packets (note the 301 I/O calls in the receiver stats - one packet during set connection setup and 300 to transfer the data.) Performing an engineering approximation analysis, we estimate that each packet will have a 40 byte TCP/IP header. We use the following calculation to determine the approximate throughput:
50 buffers * 8192 bytes each = 409,600 bytes 301 IP packets * 40 bytes of header = 12,040 bytes Total data transmitted = 421,640 bytes 421,640 bytes * 8 bits/byte = 3,373,120 bits 3,373,120 bits / 61.064 seconds = 55,239 bits/second
This is very close to 56Kbps, indicating that our link is otherwise unloaded. If we had included the HDLC framing overhead, and more accurately measured the number of bytes in the TCP and IP headers, we could determine the exact protocol overhead and therefore the exact theoretical data rate. But for most network analysis, this calculation is good enough.
TTCP sends normal IP datagrams, which are handled just like any other user data within the network. If you run tests during periods when there is other user data on the network, you will be measuring the throughput that results from the combined traffic load. The results of the above test indicate a dedicated link with no other traffic. Be aware of long running tests on routers where you are also running routing protocols - they will affect the link and the router's ability to process the packets quickly and possibly affect your analysis.
Using the other features of TTCP, you can set the size of the buffers that are used for transmission, the number of times the buffer is sent, and many other variations. We've heard of people using TTCP to validate the delivered CIR on Frame Relay links. Think up some useful tests of your own using these parameters.
When running TTCP tests, don't forget to examine router CPU, memory utilization, and interface statistics to see if the routers are operating efficiently. Are you seeing input interface drops? Output interface drops? Are the error counters incrementing with each test? Is the throughput near the theoretical maximum of the data path, or is the throughput much lower, perhaps indicating a problem along the path?
The source code for TTCP for Unix is available on the Internet from a variety of sources. We have a copy on the Chesapeake FTP server with a link from our Web page http://www.ccci.com/tools/ttcp. Look for a Java version of TTCP on our web site soon.
Happy testing!
If you find this article useful, drop us an email message to "tools@ccci.com".