Trace as a Troubleshooting Tool (Part One)

By Kevin Hamilton, Consultant-Instructor

Possible Paths From Source to Destination

Are you dissatisfied with your network performance? Does the response seem to be slow? Maybe the network was functional, but now you're having problems reaching a particular network. A fairly common network diagnostic tool found in UNIX stations, Windows 95, and your Cisco routers may help you uncover the cause for the difficulties. The trace utility provides a method for determining which path packets traverse between two devices. (See Figure 1.) Since trace reveals IP routing decisions, it is useful in discovering if the routers converged on a path that traverses a low-bandwidth link or a heavily congested router or segment. The network administrator could then use this information to carefully adjust the routing protocol metrics to optimize the path between critical points. Another usage of trace indicates if a link (segment) failed between two points.

This multi-part article will cover the operation and usage of trace. Part One will focus on how trace works and how to use it in a Cisco router. Part Two will appear in the next issue of The Network Monitor(TM) and discuss trace operations in complex networks.

How Does it Work?

Unfortunately, there is, within the networking populace, a misunderstanding of how trace works. In fact, upon reviewing some of the more common books about TCP/IP, we discovered that trace was often incorrectly explained. Many believe trace uses ICMP echo requests and replies (ping) to discover the path. This is not the case. Although there is an option within IP to record routes, there are limitations in its functionality. For example, record route can only record nine hops. Further, not all routers support the record route option. W. Richard Steven's book, TCP/IP Illustrated, Volume 1, correctly describes trace and is a highly recommended source for understanding many relevant TCP/IP topics.

The trace utility generates a UDP message targeted to a destination device. The UDP header contains a PORT value that identifies the upper-layer application for which the data are intended. Most trace implementations use a port value greater than 30000, a value unlikely to be used by the destination. Cisco routers use a default port value of 33434. Selecting an unused port value is important to trace, as we'll see later.

To discover each hop between the source and destination, trace modifies the IP TTL (Time-to-Live) value. TTL resides in the IP header and ensures that packets do not continuously traverse the network. When a packet reaches a router, it decrements the TTL value by one. If the router decrements TTL to zero, then the router discards the packet. In normal operations, the source sets TTL to a value large enough to traverse the network. The actual value used varies depending upon the IP implementation. A default value of 32 was commonly used in early IP implementations. However, as networks expanded worldwide, this number proved to be too small in instances where the required number of hops became greater between end points. Now 64 or 128 is used as a typical default TTL value.

Trace With TTL Set to 0

Trace, however, starts by setting TTL to one rather than to a higher default value. Referencing Figure 2, assume that York initiates a trace to Rome. When the packet reaches the first router (Tacoma), Tacoma decrements TTL from one to zero and discards the packet. Tacoma then returns an ICMP Time Exceeded message to the source (York). This message includes Tacoma's address, which is displayed on the console along with the response time. Most trace utilities repeat the measurement two or three times, in case one of the UDP or ICMP messages is lost.

Progression Of Starting TTL Values

Next, trace will resend the UDP message, but with TTL set to two (Figure 3). The packet reaches the first router (Tacoma), which decrements TTL from two to one and forwards the packet to the next router (Seattle) determined in its routing table. Seattle decrements TTL from one to zero and discards the packet. Seattle then returns an ICMP Time Exceeded message to York. Now the second hop displays on the console. Trace will then resend the UDP message, but with TTL set to three. The TTL will be increased until the destination is reached. Figure 3 shows the progression of starting TTL values and how they are decremented as they traverse the network.

Trace Output:

How does trace know when it hits the destination? When the UDP message reaches its destination, the TTL value is valid, but the PORT is not since the source targeted a random, hopefully unused, port. Rather than returning a Time Exceeded message, the destination device returns an ICMP message indicating Port Unreachable. (See Figure 4.) As with the other ICMP messages, the IP address is included and the trace utility displays the final hop on the console, completing the trace operation. Cisco routers display an asterisk (*) whenever the returned ICMP message is not received within the timeout interval (default three seconds).

Sending Time To An Inactive Destination Port

Cisco Trace Syntax

Trace may be initiated with trace [destination]. The [destination] may be specified as an IP address or as a host name. However, if a host name is used, then the router must be able to resolve it to an address either through DNS or an ip host entry in the running­configuration. Note in the output example above that both a name and an address were displayed for each hop. This implies that trace resolved the address returned in each ICMP message into a name found through one of the methods just mentioned. If the router was unable to resolve the address into a name, then it would display only the address.

Alternatively, extended trace can be used by simply typing trace. When using the extended trace, the following options in Figure 5 are available. (Note that this is from IOS 11.2.)

Extended Trace Options

Trace gets more interesting when the network complexity increases. In Part Two of this article, we will look at some of the extended trace options and explore trace behavior in complex routed networks.

Volume 3, Number 2 Table Of Contents

Copyright © Chesapeake 1997