Internet Engineering Task Force (IETF) A. Zimmermann
Request for Comments: 6069 A. Hannemann
Category: Experimental RWTH Aachen University
ISSN: 2070-1721 December 2010
Making TCP More Robust to Long Connectivity Disruptions (TCP-LCD)
Abstract
Disruptions in end-to-end path connectivity, which last longer than
one retransmission timeout, cause suboptimal TCP performance. The
reason for this performance degradation is that TCP interprets
segment loss induced by long connectivity disruptions as a sign of
congestion, resulting in repeated retransmission timer backoffs.
This, in turn, leads to a delayed detection of the re-establishment
of the connection since TCP waits for the next retransmission timeout
before it attempts a retransmission.
This document proposes an algorithm to make TCP more robust to long
connectivity disruptions (TCP-LCD). It describes how standard ICMP
messages can be exploited during timeout-based loss recovery to
disambiguate true congestion loss from non-congestion loss caused by
connectivity disruptions. Moreover, a reversion strategy of the
retransmission timer is specified that enables a more prompt
detection of whether or not the connectivity to a previously
disconnected peer node has been restored. TCP-LCD is a TCP sender-
only modification that effectively improves TCP performance in the
case of connectivity disruptions.
Status of This Memo
This document is not an Internet Standards Track specification; it is
published for examination, experimental implementation, and
evaluation.
This document defines an Experimental Protocol for the Internet
community. This document is a product of the Internet Engineering
Task Force (IETF). It represents the consensus of the IETF
community. It has received public review and has been approved for
publication by the Internet Engineering Steering Group (IESG). Not
all documents approved by the IESG are a candidate for any level of
Internet Standard; see Section 2 of RFC 5741.
Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc6069.
Zimmermann & Hannemann Experimental [Page 1]
RFC 6069 Making TCP More Robust to LCDs December 2010
Copyright Notice
Copyright (c) 2010 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction ....................................................3
2. Terminology .....................................................4
3. Connectivity Disruption Indication ..............................5
4. Connectivity Disruption Reaction ................................7
4.1. Basic Idea .................................................7
4.2. Algorithm Details ..........................................8
5. Discussion of TCP-LCD ..........................................11
5.1. Retransmission Ambiguity ..................................12
5.2. Wrapped Sequence Numbers ..................................12
5.3. Packet Duplication ........................................13
5.4. Probing Frequency .........................................14
5.5. Reaction during Connection Establishment ..................14
5.6. Reaction in Steady-State ..................................14
6. Dissolving Ambiguity Issues Using the TCP Timestamps Option ....15
7. Interoperability Issues ........................................17
7.1. Detection of TCP Connection Failures ......................17
7.2. Explicit Congestion Notification (ECN) ....................17
7.3. TCP-LCD and IP Tunnels ....................................17
8. Related Work ...................................................18
9. Security Considerations ........................................19
10. Acknowledgments ...............................................20
11. References ....................................................20
11.1. Normative References .....................................20
11.2. Informative References ...................................21
Zimmermann & Hannemann Experimental [Page 2]
RFC 6069 Making TCP More Robust to LCDs December 2010
1. Introduction
Connectivity disruptions can occur in many different situations. The
frequency of connectivity disruptions depends on the properties of
the end-to-end path between the communicating hosts. While
connectivity disruptions can occur in traditional wired networks,
e.g., disruption caused by an unplugged network cable, the likelihood
of their occurrence is significantly higher in wireless (multi-hop)
networks. Especially, end-host mobility, network topology changes,
and wireless interferences are crucial factors. In the case of the
Transmission Control Protocol (TCP) [RFC0793], the performance of the
connection can experience a significant reduction compared to a
permanently connected path [SESB05]. This is because TCP, which was
originally designed to operate in fixed and wired networks, generally
assumes that the end-to-end path connectivity is relatively stable
over the connection's lifetime.
Depending on their duration, connectivity disruptions can be
classified into two groups [TCP-RLCI]: "short" and "long". A
connectivity disruption is "short" if connectivity returns before the
retransmission timer fires for the first time. In this case, TCP
recovers lost data segments through Fast Retransmit and lost
acknowledgments (ACKs) through successfully delivered later ACKs.
Connectivity disruptions are declared as "long" for a given TCP
connection if the retransmission timer fires at least once before
connectivity is resumed. Whether or not path characteristics, like
the round-trip time (RTT) or the available bandwidth, have changed
when connectivity resumes after a disruption is another important
aspect for TCP's retransmission scheme [TCP-RLCI].
The algorithm specified in this document improves TCP's behavior in
the case of "long connectivity disruptions". In particular, it
focuses on the period prior to the re-establishment of the
connectivity to a previously disconnected peer node. The document
does not describe any modifications to TCP's behavior and its
congestion control mechanisms [RFC5681] after connectivity has been
restored.
When a long connectivity disruption occurs on a TCP connection, the
TCP sender eventually does not receive any more acknowledgments.
After the retransmission timer expires, the TCP sender enters the
timeout-based loss recovery and declares the oldest outstanding
segment (SND.UNA) as lost. Since TCP tightly couples reliability and
congestion control, the retransmission of SND.UNA is triggered
together with the reduction of the transmission rate. This is based
on the assumption that segment loss is an indication of congestion
[RFC5681]. As long as the connectivity disruption persists, TCP will
repeat this procedure until the oldest outstanding segment has
Zimmermann & Hannemann Experimental [Page 3]
RFC 6069 Making TCP More Robust to LCDs December 2010
successfully been acknowledged or until the connection has timed out.
TCP implementations that follow the recommended retransmission
timeout (RTO) management of RFC 2988 [RFC2988] double the RTO after
each retransmission attempt. However, the RTO growth may be bounded
by an upper limit, the maximum RTO, which is at least 60 s, but may
be longer: Linux, for example, uses 120 s. If connectivity is
restored between two retransmission attempts, TCP still has to wait
until the retransmission timer expires before resuming transmission,
since it simply does not have any means to know if the connectivity
has been re-established. Therefore, depending on when connectivity
becomes available again, this can waste up to a maximum RTO of
possible transmission time.
This retransmission behavior is not efficient, especially in
scenarios with long connectivity disruptions. In the ideal case, TCP
would attempt a retransmission as soon as connectivity to its peer
has been re-established. In this document, we specify a TCP sender-
only modification to provide robustness to long connectivity
disruptions (TCP-LCD). The memo describes how the standard Internet
Control Message Protocol (ICMP) can be exploited during timeout-based
loss recovery to identify non-congestion loss caused by long
connectivity disruptions. TCP-LCD's reversion strategy of the
retransmission timer enables higher-frequency retransmissions and
thereby a prompt detection when connectivity to a previously
disconnected peer node has been restored. If no congestion is
present, TCP-LCD approaches the ideal behavior.
Experimental results of a Linux implementation of TCP-LCD have been
presented in [ZimHan09]. The implementation has been incorporated
into mainline Linux, and is already used within the Internet. Thus
far, no negative experiences have been reported that could be
attributed to the algorithm. However, we consider TCP-LCD as
experimental until more real-life results have been obtained.
Nevertheless, we encourage implementation of TCP-LCD under other
operating systems to provide for broader testing and experimentation
opportunities.
2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
The reader should be familiar with the algorithm and terminology from
[RFC2988], which defines the standard algorithm that Transmission
Control Protocol (TCP) senders are required to use to compute and
manage their retransmission timer. In this document, the terms
"retransmission timer" and "retransmission timeout" are used as
Zimmermann & Hannemann Experimental [Page 4]
RFC 6069 Making TCP More Robust to LCDs December 2010
defined in [RFC2988]. The retransmission timer ensures data delivery
in the absence of any feedback from the receiver. The duration of
this timer is referred to as retransmission timeout (RTO).
As defined in [RFC0793], the term "acceptable acknowledgment (ACK)"
refers to a TCP segment that acknowledges previously unacknowledged
data. The TCP sender state variable "SND.UNA" and the current
segment variable "SEG.SEQ" are used as defined in [RFC0793]. SND.UNA
holds the segment sequence number of the earliest segment that has
not been acknowledged by the TCP receiver (the oldest outstanding
segment). SEG.SEQ is the segment sequence number of a given segment.
For the purposes of this specification, we define the term "timeout-
based loss recovery", which refers to the state that a TCP sender
enters upon the first timeout of the oldest outstanding segment
(SND.UNA) and leaves upon the arrival of the *first* acceptable ACK.
It is important to note that other documents use a different
interpretation of the term "timeout-based loss recovery". For
example, the NewReno modification to TCP's Fast Recovery algorithm
[RFC3782] extends the period that a TCP sender remains in timeout-
based loss recovery compared to the one defined in this document.
This is because [RFC3782] attempts to avoid unnecessary multiple Fast
Retransmits that can occur after an RTO.
3. Connectivity Disruption Indication
If the queue of an intermediate router that is experiencing a link
outage can buffer all incoming packets, a connectivity disruption
will only cause a variation in delay, which is handled well by TCP
implementations using either Eifel [RFC3522], [RFC4015] or Forward
RTO-Recovery (F-RTO) [RFC5682]. However, if the link outage lasts
for too long, the router experiencing the link outage is forced to
drop packets, and finally may remove the corresponding next hop from
its routing table. Means to detect such link outages include
reacting to failed address resolution protocol (ARP) [RFC0826]
queries, sensing unsuccessful links, and the like. However, this is
solely the responsibility of the respective router.
Note: The focus of this memo is on introducing a method of how
ICMP messages may be exploited to improve TCP's performance; how
different physical and link-layer mechanisms below the network
layer may trigger ICMP destination unreachable messages are out of
scope of this memo.
Provided that no other route to the specific destination exists, an
Internet Protocol version 4 (IPv4) [RFC0791] router will notify the
corresponding sending host about the dropped packets via ICMP
destination unreachable messages of code 0 (net unreachable) or
Zimmermann & Hannemann Experimental [Page 5]
RFC 6069 Making TCP More Robust to LCDs December 2010
code 1 (host unreachable) [RFC1812]. Therefore, the sending host can
use the ICMP destination unreachable messages of these codes as an
indication of a connectivity disruption, since the reception of these
messages provides evidence that packets were dropped due to a link
outage.
For Internet Protocol version 6 (IPv6) [RFC2460], the counterpart of
the ICMP destination unreachable message of code 0 (net unreachable)
and of code 1 (host unreachable) is the ICMPv6 destination
unreachable message of code 0 (no route to destination) [RFC4443].
As with IPv4, a router should generate an ICMPv6 destination
unreachable message of code 0 in response to a packet that cannot be
delivered to its destination address because it lacks a matching
entry in its routing table.
Note that there are also other ICMP and ICMPv6 destination
unreachable messages with different codes. Some of them are
candidates for connectivity disruption indications, too, but need
further investigation (for example, ICMP destination unreachable
messages with code 5 (source route failed), code 11 (net unreachable
for TOS (Type of Service)), or code 12 (host unreachable for TOS)
[RFC1812]). On the other hand, codes that flag hard errors are of no
use for this scheme, since TCP should abort the connection when those
are received [RFC1122].
For the sake of simplicity, we will use, unless explicitly qualified
with ICMPv4 or ICMPv6, the term "ICMP unreachable message" as a
synonym for ICMP destination unreachable messages of code 0 or code 1
and ICMPv6 destination unreachable messages of code 0. This implies
that all keywords from [RFC2119] that deal with the handling of
received ICMP messages apply in the same way to ICMPv6 messages.
The accurate interpretation of ICMP unreachable messages as a
connectivity disruption indication is complicated by the following
two peculiarities of ICMP messages. First, they do not necessarily
operate on the same timescale as the packets, i.e., TCP segments that
elicited them. When a router drops a packet due to a missing route,
it will not necessarily send an ICMP unreachable message immediately,
but will rather queue it for later delivery. Second, ICMP messages
are subject to rate-limiting, e.g., when a router drops a whole
window of data due to a link outage, it is unlikely to send as many
ICMP unreachable messages as dropped TCP segments. Depending on the
load of the router, it may not even send any ICMP unreachable
messages at all. Both peculiarities originate from [RFC1812] for
ICMPv4 and [RFC4443] for ICMPv6.
Zimmermann & Hannemann Experimental [Page 6]
RFC 6069 Making TCP More Robust to LCDs December 2010
Fortunately, according to [RFC0792], ICMPv4 unreachable messages have
to contain, in their body, the entire IPv4 header [RFC0791] of the
datagram eliciting the ICMPv4 unreachable message, plus the first
64 bits of the payload of that datagram. This allows the sending
host to match the ICMPv4 error message to the transport connection
that elicited it. RFC 1812 [RFC1812] augments these requirements and
states that ICMPv4 messages should contain as much of the original
datagram as possible without the length of the ICMPv4 datagram
exceeding 576 bytes. Therefore, in the case of TCP, at least the
source port number, the destination port number, and the 32-bit TCP
sequence number are included. This allows the originating TCP to
demultiplex the received ICMPv4 message and to identify the affected
connection. Moreover, it can identify which segment of the
respective connection triggered the ICMPv4 unreachable message,
unless there are several segments in flight with the same sequence
number (see Section 5.1).
For IPv6 [RFC2460], the payload of an ICMPv6 error message has to
include as many bytes as possible from the IPv6 datagram that
elicited the ICMPv6 error message, without making the error message
exceed the minimum IPv6 MTU (1280 bytes) [RFC4443]. Thus, enough
information is available to identify both the affected connection and
the corresponding segment that triggered the ICMPv6 error message.
A connectivity disruption indication in the form of an ICMP
unreachable message associated with a presumably lost TCP segment
provides strong evidence that the segment was not dropped due to
congestion, but was successfully delivered as far as the reporting
router. It therefore did not witness any congestion at least on that
part of the path that was traversed by both the TCP segment eliciting
the ICMP unreachable message and the ICMP unreachable message itself.
4. Connectivity Disruption Reaction
Section 4.1 introduces the basic idea of TCP-LCD. The complete
algorithm is specified in Section 4.2.
4.1. Basic Idea
The goal of the algorithm is to promptly detect when connectivity to
a previously disconnected peer node has been restored after a long
connectivity disruption, while retaining appropriate behavior in case
of congestion. TCP-LCD exploits standard ICMP unreachable messages
during timeout-based loss recovery. This increases TCP's
retransmission frequency by undoing one retransmission timer backoff
whenever an ICMP unreachable message is received that contains a
segment with a sequence number of a presumably lost retransmission.
Zimmermann & Hannemann Experimental [Page 7]
RFC 6069 Making TCP More Robust to LCDs December 2010
This approach has the advantage of appropriately reducing the probing
rate in case of congestion. If either the retransmission itself or
the corresponding ICMP message is dropped, the previously performed
retransmission timer backoff is not undone, which effectively halves
the probing rate.
4.2. Algorithm Details
A TCP sender that uses RFC 2988 [RFC2988] to compute TCP's
retransmission timer MAY employ the following scheme to avoid over-
conservative retransmission timer backoffs in case of long
connectivity disruptions. If a TCP sender does implement the
following steps, the algorithm MUST be initiated upon the first
timeout of the oldest outstanding segment (SND.UNA) and MUST be
stopped upon the arrival of the first acceptable ACK. The algorithm
MUST NOT be re-initiated upon subsequent timeouts for the same
segment. The scheme SHOULD NOT be used in SYN-SENT or SYN-RECEIVED
states [RFC0793] (see Section 5.5).
A TCP sender that does not employ RFC 2988 [RFC2988] to compute TCP's
retransmission timer MUST NOT use TCP-LCD. We envision that the
scheme could be easily adapted to algorithms other than RFC 2988.
However, we leave this as future work.
RFC 2988 [RFC2988] provides in rule (2.5) the option to place a
maximum value on the RTO. When a TCP implements this rule to provide
an upper bound for the RTO, it MUST also be used in the following
algorithm. In particular, if the RTO is bounded by an upper limit
(maximum RTO), the "MAX_RTO" variable used in this scheme MUST be
initialized with this upper limit. Otherwise, if the RTO is
unbounded, the "MAX_RTO" variable MUST be set to infinity.
The scheme specified in this document uses the "BACKOFF_CNT"
variable, whose initial value is zero. The variable is used to count
the number of performed retransmission timer backoffs during one
timeout-based loss recovery. Moreover, the "RTO_BASE" variable is
used to recover the previous RTO if the retransmission timer backoff
was unnecessary. The variable is initialized with the RTO upon
initiation of timeout-based loss recovery.
(1) Before TCP updates the variable "RTO" when it initiates timeout-
based loss recovery, set the variables "BACKOFF_CNT" and
"RTO_BASE" as follows:
BACKOFF_CNT := 0;
RTO_BASE := RTO.
Proceed to step (R).
Zimmermann & Hannemann Experimental [Page 8]
RFC 6069 Making TCP More Robust to LCDs December 2010
(R) This is a placeholder for standard TCP's behavior in case the
retransmission timer has expired. In particular, if RFC 2988
[RFC2988] is used, steps (5.4) to (5.6) of that algorithm go
here. Proceed to step (2).
(2) To account for the expiration of the retransmission timer in the
previous step (R), increment the "BACKOFF_CNT" variable by one:
BACKOFF_CNT := BACKOFF_CNT + 1.
(3) Wait either
a) for the expiration of the retransmission timer. When the
retransmission timer expires, proceed to step (R); or
b) for the arrival of an acceptable ACK. When an acceptable
ACK arrives, proceed to step (A); or
c) for the arrival of an ICMP unreachable message. When the
ICMP unreachable message "ICMP_DU" arrives, proceed to
step (4).
(4) If "BACKOFF_CNT > 0", i.e., if at least one retransmission timer
backoff can be undone, then
proceed to step (5);
else
proceed to step (3).
(5) Extract the TCP segment header included in the ICMP unreachable
message "ICMP_DU":
SEG := Extract(ICMP_DU).
(6) If "SEG.SEQ == SND.UNA", i.e., if the TCP segment "SEG"
eliciting the ICMP unreachable message "ICMP_DU" contains the
sequence number of a retransmission, then
proceed to step (7);
else
proceed to step (3).
Zimmermann & Hannemann Experimental [Page 9]
RFC 6069 Making TCP More Robust to LCDs December 2010
(7) Undo the last retransmission timer backoff:
BACKOFF_CNT := BACKOFF_CNT - 1;
RTO := min(RTO_BASE * 2^(BACKOFF_CNT), MAX_RTO).
(8) If the retransmission timer expires due to the undoing in the
previous step (7), then
proceed to step (R);
else
proceed to step (3).
(A) This is a placeholder for standard TCP's behavior in case an
acceptable ACK has arrived. No further processing.
When a TCP in steady-state detects a segment loss using the
retransmission timer, it enters the timeout-based loss recovery and
initiates the algorithm (step (1)). It adjusts the slow-start
threshold (ssthresh), sets the congestion window (cwnd) to one
segment, backs off the retransmission timer, and retransmits the
first unacknowledged segment (step (R)) [RFC5681], [RFC2988]. To
account for the expiration of the retransmission timer, the TCP
sender increments the "BACKOFF_CNT" variable by one (step (2)).
In case the retransmission timer expires again (step (3a)), a TCP
will repeat the retransmission of the first unacknowledged segment
and back off the retransmission timer once more (step (R)) [RFC2988],
as well as increment the "BACKOFF_CNT" variable by one (step (2)).
Note that a TCP may implement RFC 2988's [RFC2988] option to place a
maximum value on the RTO that may result in not performing the
retransmission timer backoff. However, step (2) MUST always and
unconditionally be applied, no matter whether or not the
retransmission timer is actually backed off. In other words, each
time the retransmission timer expires, the "BACKOFF_CNT" variable
MUST be incremented by one.
If the first received packet after the retransmission(s) is an
acceptable ACK (step (3b)), a TCP will proceed as normal, i.e., slow-
start the connection and terminate the algorithm (step (A)). Later
ICMP unreachable messages from the just terminated timeout-based loss
recovery are ignored, since the ACK clock is already restarting due
to the successful retransmission.
On the other hand, if the first received packet after the
retransmission(s) is an ICMP unreachable message (step (3c)), and if
step (4) permits it, TCP SHOULD undo one backoff for each ICMP
Zimmermann & Hannemann Experimental [Page 10]
RFC 6069 Making TCP More Robust to LCDs December 2010
unreachable message reporting an error on a retransmission. To
decide if an ICMP unreachable message was elicited by a
retransmission, the sequence number it contains is inspected
(step (5), step (6)). The undo is performed by recalculating the RTO
with the decremented "BACKOFF_CNT" variable (step (7)). This
calculation explicitly matches the (bounded) exponential backoff
specified in rule (5.5) of [RFC2988].
Upon receipt of an ICMP unreachable message that legitimately undoes
one backoff, there is the possibility that the shortened
retransmission timer has already expired (step (8)). Then, TCP
SHOULD retransmit immediately. In case the shortened retransmission
timer has not yet expired, TCP MUST wait accordingly.
5. Discussion of TCP-LCD
TCP-LCD takes caution to only react to connectivity disruption
indications in the form of ICMP unreachable messages during timeout-
based loss recovery. Therefore, TCP's behavior is not altered when
either no ICMP unreachable messages are received or the
retransmission timer of the TCP sender did not expire since the last
received acceptable ACK. Thus, by definition, the algorithm triggers
only in the case of long connectivity disruptions.
Only such ICMP unreachable messages that contain a TCP segment with
the sequence number of a retransmission, i.e., that contain SND.UNA,
are evaluated by TCP-LCD. All other ICMP unreachable messages are
ignored. The arrival of those ICMP unreachable messages provides
strong evidence that the retransmissions were not dropped due to
congestion, but were successfully delivered to the reporting router.
In other words, there is no evidence for any congestion at least on
that very part of the path that was traversed by both the TCP segment
eliciting the ICMP unreachable message and the ICMP unreachable
message itself.
However, there are some situations where TCP-LCD makes a false
decision and incorrectly undoes a retransmission timer backoff. This
can happen, even when the received ICMP unreachable message contains
the segment number of a retransmission (SND.UNA), because the TCP
segment that elicited the ICMP unreachable message may either not be
a retransmission (Section 5.1) or does not belong to the current
timeout-based loss recovery (Section 5.2). Finally, packet
duplication (Section 5.3) can also spuriously trigger the algorithm.
Section 5.4 discusses possible probing frequencies, while Section 5.6
describes the motivation for not reacting to ICMP unreachable
messages while TCP is in steady-state.
Zimmermann & Hannemann Experimental [Page 11]
RFC 6069 Making TCP More Robust to LCDs December 2010
5.1. Retransmission Ambiguity
Historically, the retransmission ambiguity problem [Zh86], [KP87] is
the TCP sender's inability to distinguish whether the first
acceptable ACK after a retransmission refers to the original
transmission or to the retransmission. This problem occurs after
both a Fast Retransmit and a timeout-based retransmit. However,
modern TCP implementations can eliminate the retransmission ambiguity
with either the help of Eifel [RFC3522], [RFC4015] or Forward RTO-
Recovery (F-RTO) [RFC5682].
The reversion strategy of the given algorithm suffers from a form of
retransmission ambiguity, too. In contrast to the above case, TCP
suffers from ambiguity regarding ICMP unreachable messages received
during timeout-based loss recovery. With the TCP segment number
included in the ICMP unreachable message, a TCP sender is not able to
determine if the ICMP unreachable message refers to the original
transmission or to any of the timeout-based retransmissions. That
is, there is an ambiguity with regard to which TCP segment an ICMP
unreachable message reports on.
However, this ambiguity is not considered to be a problem for the
algorithm. The assumption that a received ICMP unreachable message
provides evidence that a non-congestion loss caused by the
connectivity disruption was wrongly considered a congestion loss
still holds, regardless of to which TCP segment (transmission or
retransmission) the message refers.
5.2. Wrapped Sequence Numbers
Besides the ambiguity whether a received ICMP unreachable message
refers to the original transmission or to any of the retransmissions,
there is another source of ambiguity related to the TCP sequence
numbers contained in ICMP unreachable messages. For high-bandwidth
paths, the sequence space may wrap quickly. This might cause delayed
ICMP unreachable messages to coincidentally fit as valid input in the
proposed scheme. As a result, the scheme may incorrectly undo
retransmission timer backoffs. The chances of this happening are
minuscule, since a particular ICMP unreachable message would need to
contain the exact sequence number of the current oldest outstanding
segment (SND.UNA), while at the same time TCP is in timeout-based
loss recovery. However, two "worst case" scenarios for the algorithm
are possible.
For instance, consider a steady-state TCP connection, which will be
disrupted at an intermediate router due to a link outage. Upon the
expiration of the RTO, the TCP sender enters the timeout-based loss
recovery and starts to retransmit the earliest segment that has not
Zimmermann & Hannemann Experimental [Page 12]
RFC 6069 Making TCP More Robust to LCDs December 2010
been acknowledged (SND.UNA). For some reason, the router delays all
corresponding ICMP unreachable messages so that the TCP sender backs
the retransmission timer off normally without any undoing. At the
end of the connectivity disruption, the TCP sender eventually detects
the re-establishment, and it leaves the scheme and finally the
timeout-based loss recovery, too. A sequence number wrap-around
later, the connectivity between the two peers is disrupted again, but
this time due to congestion and exactly at the time at which the
current SND.UNA matches the SND.UNA from the previous cycle. If the
router emits the delayed ICMP unreachable messages now, the TCP
sender would incorrectly undo retransmission timer backoffs. As the
TCP sequence number contains 32 bits, the probability of this
scenario is at most 1/2^32. Given sufficiently many retransmissions
in the first timeout-based loss recovery, the corresponding ICMP
unreachable messages could reduce the RTO in the second recovery at
most to "RTO_BASE". However, once the ICMP unreachable messages are
depleted, the standard exponential backoff will be performed. Thus,
the congestion response will only be delayed by some false
retransmissions.
Similar to the above, consider the case where a steady-state TCP
connection with n segments in flight will be disrupted at some point
due to a link outage at an intermediate router. For each segment in
flight, the router may generate an ICMP unreachable message.
However, for some reason, it delays them. Once the link outage is
over and the connection has been re-established, the TCP sender
leaves the scheme and slow-starts the connection. Following a
sequence number wrap-around, a retransmission timeout occurs, just at
the moment the TCP sender's current window of data reaches the
previous range of the sequence number space again. In case the
router emits the delayed ICMP unreachable messages now, spurious
undoing of the retransmission timer backoff is possible once, if the
TCP segment number contained in the ICMP unreachable messages matches
the current SND.UNA, and the timeout was a result of congestion. In
the case of another connectivity disruption, the additional undoing
of the retransmission timer backoff has no impact. The probability
of this scenario is at most n/2^32.
5.3. Packet Duplication
In case an intermediate router duplicates packets, a TCP sender may
receive more ICMP unreachable messages during timeout-based loss
recovery than sent timeout-based retransmissions. However, since
TCP-LCD keeps track of the number of performed retransmission timer
backoffs in the "BACKOFF_CNT" variable, it will not undo more
retransmission timer backoffs than were actually performed.
Nevertheless, if packet duplication and congestion coincide on the
path between the two communicating hosts, duplicated ICMP unreachable
Zimmermann & Hannemann Experimental [Page 13]
RFC 6069 Making TCP More Robust to LCDs December 2010
messages could hide the congestion loss of some retransmissions or
ICMP unreachable messages, and the algorithm may incorrectly undo
retransmission timer backoffs. Considering the overall impact of a
router that duplicates packets, the additional load induced by some
spurious timeout-based retransmits can probably be neglected.
5.4. Probing Frequency
One might argue that if an ICMP unreachable message arrives for a
timeout-based retransmission, the RTO shall be reset or recalculated,
similar to what is done when an ACK arrives during timeout-based loss
recovery (see Karn's algorithm [KP87], [RFC2988]), and a new
retransmission should be sent immediately. Generally, this would
result in a much higher probing frequency based on the round-trip
time to the router where connectivity has been disrupted. However,
we believe the current scheme provides a good trade-off between
conservative behavior and fast detection of connectivity
re-establishment. TCP-LCD focuses on long-connectivity disruptions,
i.e., on disruptions that last for several RTOs. Thus, a much higher
probing frequency (less than once per RTO) would not significantly
increase the available transmission time compared to the duration of
the connectivity disruption.
5.5. Reaction during Connection Establishment
It is possible that a TCP sender enters timeout-based loss recovery
while the connection is in SYN-SENT or SYN-RECEIVED states [RFC0793].
The algorithm described in this document could also be used for
faster connection establishment in networks with connectivity
disruptions. However, because existing TCP implementations [RFC5461]
already interpret ICMP unreachable messages during connection
establishment and abort the corresponding connection, we refrain from
suggesting this.
5.6. Reaction in Steady-State
Another exploitation of ICMP unreachable messages in the context of
TCP congestion control might seem appropriate, while TCP is in
steady-state. As the RTT up to the router that generated the ICMP
unreachable message is likely to be substantially shorter than the
overall RTT to the destination, the ICMP unreachable message may very
well reach the originating TCP while it is transmitting the current
window of data. In case the remaining window is large, it might seem
appropriate to refrain from transmitting the remaining window as
there is timely evidence that it will only trigger further ICMP
unreachable messages at that very router. Although this promises
improvement from a wastage perspective, it may be counterproductive
from a security perspective. An attacker could forge such ICMP
Zimmermann & Hannemann Experimental [Page 14]
RFC 6069 Making TCP More Robust to LCDs December 2010
messages, thereby forcing the originating TCP to stop sending data,
very similar to the blind throughput-reduction attack mentioned in
[RFC5927].
An additional consideration is the following: in the presence of
multi-path routing, even the receipt of a legitimate ICMP unreachable
message cannot be exploited accurately, because there is the
possibility that only one of the multiple paths to the destination is
suffering from a connectivity disruption, which causes ICMP
unreachable messages to be sent. Then, however, there is the
possibility that the path along which the connectivity disruption
occurred contributed considerably to the overall bandwidth, such that
a congestion response is very well reasonable. However, this is not
necessarily the case. Therefore, a TCP has no means except for its
inherent congestion control to decide on this matter. All in all, it
seems that for a connection in steady-state, i.e., not in timeout-
based loss recovery, reacting to ICMP unreachable messages in regard
to congestion control is not appropriate. For the case of timeout-
based retransmissions, however, there is a reasonable congestion
response, which is skipping further retransmission timer backoffs
because there is no congestion indication -- as described above.
6. Dissolving Ambiguity Issues Using the TCP Timestamps Option
If the TCP Timestamps option [RFC1323] is enabled for a connection, a
TCP sender SHOULD use the following algorithm to dissolve the
ambiguity issues mentioned in Sections 5.1, 5.2, and 5.3. In
particular, both the retransmission ambiguity and the packet
duplication problems are prevented by the following TCP-LCD variant.
On the other hand, the false positives caused by wrapped sequence
numbers cannot be completely avoided, but the likelihood is further
reduced by a factor of 1/2^32, since the Timestamp Value field
(TSval) of the TCP Timestamps option contains 32 bits.
Hence, implementers may choose to employ the TCP-LCD with the
following modifications.
Step (1) is replaced by step (1'):
(1') Before TCP updates the variable "RTO" when it initiates
timeout-based loss recovery, set the variables "BACKOFF_CNT"
and "RTO_BASE", and the data structure "RETRANS_TS", as
follows:
BACKOFF_CNT := 0;
RTO_BASE := RTO;
RETRANS_TS := [].
Zimmermann & Hannemann Experimental [Page 15]
RFC 6069 Making TCP More Robust to LCDs December 2010
Proceed to step (R).
Step (2) is extended by step (2b):
(2b) Store the value of the Timestamp Value field (TSval) of the TCP
Timestamps option included in the retransmission "RET" sent in
step (R) into the "RETRANS_TS" data structure:
RETRANS_TS.add(RET.TSval)
Step (6) is replaced by step (6'):
(6') If "SEG.SEQ == SND.UNA && RETRANS_TS.exists(SEQ.TSval)", i.e.,
if the TCP segment "SEG" eliciting the ICMP unreachable message
"ICMP_DU" contains the sequence number of a retransmission, and
the value in its Timestamp Value field (TSval) is valid, then
proceed to step (7');
else
proceed to step (3).
Step (7) is replaced by step (7'):
(7') Undo the last retransmission timer backoff:
RETRANS_TS.remove(SEQ.TSval);
BACKOFF_CNT := BACKOFF_CNT - 1;
RTO := min(RTO_BASE * 2^(BACKOFF_CNT), MAX_RTO).
The downside of this variant is twofold. First, the modifications
come at a cost: the TCP sender is required to store the timestamps of
all retransmissions sent during one timeout-based loss recovery.
Second, this variant can only undo a retransmission timer backoff if
the intermediate router experiencing the link outage implements
[RFC1812] and chooses to include, in addition to the first 64 bits of
the payload of the triggering datagram, as many bits as are needed to
include the TCP Timestamps option in the ICMP unreachable message.
Zimmermann & Hannemann Experimental [Page 16]
RFC 6069 Making TCP More Robust to LCDs December 2010
7. Interoperability Issues
This section discusses interoperability issues related to introducing
TCP-LCD.
7.1. Detection of TCP Connection Failures
TCP-LCD may produce side effects for TCP implementations that attempt
to detect TCP connection failures by counting timeout-based
retransmissions. [RFC1122] states in Section 4.2.3.5 that a TCP host
must handle excessive retransmissions of data segments with two
thresholds, R1 and R2, that measure the number of retransmissions
that have occurred for the same segment. Both thresholds might be
measured either in time units or as a count of retransmissions.
Due to TCP-LCD's reversion strategy of the retransmission timer, the
assumption that a certain number of retransmissions corresponds to a
specific time interval no longer holds, as additional retransmissions
may be performed during timeout-based-loss recovery to detect the end
of the connectivity disruption. Therefore, a TCP employing TCP-LCD
either MUST measure the thresholds R1 and R2 in time units or, in
case R1 and R2 are counters of retransmissions, MUST convert them
into time intervals that correspond to the time an unmodified TCP
would need to reach the specified number of retransmissions.
7.2. Explicit Congestion Notification (ECN)
With Explicit Congestion Notification (ECN) [RFC3168], ECN-capable
routers are no longer limited to dropping packets to indicate
congestion. Instead, they can set the Congestion Experienced (CE)
codepoint in the IP header to indicate congestion. With TCP-LCD, it
may happen that during a connectivity disruption, a received ICMP
unreachable message has been elicited by a timeout-based
retransmission that was marked with the CE codepoint before reaching
the router experiencing the link outage. In such a case, a TCP
sender MUST, corresponding to Section 6.1.2 of [RFC3168],
additionally reset the retransmission timer in case the algorithm
undoes a retransmission timer backoff.
7.3. TCP-LCD and IP Tunnels
It is worth noting that IP tunnels, including IPsec [RFC4301], IP
encapsulation within IP [RFC2003], Generic Routing Encapsulation
(GRE) [RFC2784], and others, are compatible with TCP-LCD, as long as
the received ICMP unreachable messages can be demultiplexed and
extracted appropriately by the TCP sender during timeout-based loss
recovery.
Zimmermann & Hannemann Experimental [Page 17]
RFC 6069 Making TCP More Robust to LCDs December 2010
If, for example, end-to-end tunnels like IPsec in transport mode
[RFC4301] are employed, a TCP sender may receive ICMP unreachable
messages where additional steps, e.g., also performing decryption in
step (5) of the algorithm, are needed to extract the TCP header from
these ICMP messages. Provided that the received ICMP unreachable
message contains enough information, i.e., SEG.SEQ is extractable,
this information can still be used as a valid input for the proposed
algorithm.
Likewise, if IP encapsulation like [RFC2003] is used in some part of
the path between the communicating hosts, the tunnel ingress node may
receive the ICMP unreachable messages from an intermediate router
experiencing the link outage. Nevertheless, the tunnel ingress node
may replay the ICMP unreachable messages in order to inform the TCP
sender. If enough information is preserved to extract SEG.SEQ, the
replayed ICMP unreachable messages can still be used in TCP-LCD.
8. Related Work
Several methods that address TCP's problems in the presence of
connectivity disruptions have been proposed in literature. Some of
them try to improve TCP's performance by modifying lower layers. For
example, [SM03] introduces a "smart link layer", which buffers one
segment for each active connection and replays these segments upon
connectivity re-establishment. This approach has a serious drawback:
previously stateless intermediate routers have to be modified in
order to inspect TCP headers, to track the end-to-end connection, and
to provide additional buffer space. This leads to an additional need
for memory and processing power.
On the other hand, stateless link-layer schemes, as proposed in
[RFC3819], which unconditionally buffer some small number of packets,
may have another problem: if a packet is buffered longer than the
maximum segment lifetime (MSL) of 2 min. [RFC0793], i.e., the
disconnection lasts longer than the MSL, TCP's assumption that such
segments will never be received will no longer be true, violating
TCP's semantics [TCP-REXMIT-NOW].
Other approaches, like the TCP feedback-based scheme (TCP-F) [CRVP01]
or the Explicit Link Failure Notification (ELFN) [HV02] inform a TCP
sender about a disrupted path by special messages generated and sent
from intermediate routers. In the case of a link failure, the TCP
sender stops sending segments and freezes its retransmission timers.
TCP-F stays in this state and remains silent until either a "route
establishment notification" is received or an internal timer expires.
In contrast, ELFN periodically probes the network to detect
connectivity re-establishment. Both proposals rely on changes to
intermediate routers, whereas the scheme proposed in this document is
Zimmermann & Hannemann Experimental [Page 18]
RFC 6069 Making TCP More Robust to LCDs December 2010
a sender-only modification. Moreover, ELFN does not consider
congestion and may impose serious additional load on the network,
depending on the probe interval.
The authors of "ad hoc TCP" (ATCP) [LS01] propose enhancements to
identify different types of packet loss by introducing a layer
between TCP and IP. They utilize ICMP destination unreachable
messages to set TCP's receiver advertised window to zero, thus
forcing the TCP sender to perform zero window probing with an
exponential backoff. ICMP destination unreachable messages that
arrive during this probing period are ignored. This approach is
nearly orthogonal to this document, which exploits ICMP messages to
undo a retransmission timer backoff when TCP is already probing. In
principle, both mechanisms could be combined. However, due to
security considerations, it does not seem appropriate to adopt ATCP's
reaction, as discussed in Section 5.6.
Schuetz et al. [TCP-RLCI] describe a set of TCP extensions that
improve TCP's behavior when transmitting over paths whose
characteristics can change rapidly. Their proposed extensions modify
the local behavior of TCP and introduce a new TCP option to signal
locally received connectivity-change indications (CCIs) to remote
peers. Upon receipt of a CCI, they re-probe the path characteristics
either by performing a speculative retransmission or by sending a
single segment of new data, depending on whether the connection is
currently stalled in exponential backoff or transmitting in steady-
state, respectively. The authors focus on specifying TCP response
mechanisms; nevertheless, underlying layers would have to be modified
to explicitly send CCIs to make these immediate responses possible.
9. Security Considerations
Generally, an attacker has only two attack alternatives: to generate
ICMP unreachable messages to try to make a TCP modified with TCP-LCD
flood the network, or to suppress legitimate ICMP unreachable
messages to try to slow down the transmission rate of a TCP sender.
In order to generate ICMP unreachable messages that fit as an input
for TCP-LCD, an attacker would need to guess the correct four-tuple
(i.e., Source IP Address, Source TCP port, Destination IP Address,
and Destination TCP port) and the exact segment sequence number of
the current timeout-based retransmission. Yet, the correct sequence
number is generally hard to guess, given the probability of 1/2^32.
Even if an attacker has information about that sequence number (i.e.,
the attacker can eavesdrop on the retransmissions) the impact on the
network load from the attacker may be considered low, since the
retransmission frequency is limited by the RTO that was computed
before TCP had entered the timeout-based loss recovery. Hence, the
Zimmermann & Hannemann Experimental [Page 19]
RFC 6069 Making TCP More Robust to LCDs December 2010
highest probing frequency is expected to be even lower than once per
minimum RTO, i.e., 1 s as specified by [RFC2988]. It is important to
note that an attacker who can correctly guess the four-tuple and the
segment sequence number can easily launch more serious attacks (i.e.,
hijack the connection), whether or not TCP-LCD is used.
There may be means by which an attacker can cause the suppression of
legitimate ICMP unreachable messages (e.g., by flooding the router
experiencing the link outage to trigger ICMP rate-limiting).
However, even if the attacker could suppress every legitimate ICMP
unreachable message, the security impact of such an attack is
negligible, since the TCP sender using TCP-LCD will behave like a
regular TCP would. Note that this kind of attack is
indistinguishable from a router experiencing a link outage that is
not sending ICMP unreachable messages at all (e.g., because of local
policy).
In summary, the algorithm proposed in this document is considered to
be secure.
10. Acknowledgments
We would like to thank Lars Eggert, Adrian Farrel, Mark Handley, Kai
Jakobs, Ilpo Jarvinen, Enrico Marocco, Catherine Meadows, Juergen
Quittek, Pasi Sarolahti, Tim Shepard, Joe Touch, and Carsten Wolff
for feedback on earlier versions of this document. We also thank
Michael Faber, Daniel Schaffrath, and Damian Lukowski for
implementing and testing the algorithm in Linux. Special thanks go
to Ilpo Jarvinen for giving valuable feedback regarding the Linux
implementation.
This work has been supported by the German National Science
Foundation (DFG) within the research excellence cluster Ultra High-
Speed Mobile Information and Communication (UMIC), RWTH Aachen
University.
11. References
11.1. Normative References
[RFC0792] Postel, J., "Internet Control Message Protocol", STD 5,
RFC 792, September 1981.
[RFC0793] Postel, J., "Transmission Control Protocol", STD 7,
RFC 793, September 1981.
[RFC1323] Jacobson, V., Braden, B., and D. Borman, "TCP Extensions
for High Performance", RFC 1323, May 1992.
Zimmermann & Hannemann Experimental [Page 20]
RFC 6069 Making TCP More Robust to LCDs December 2010
[RFC1812] Baker, F., "Requirements for IP Version 4 Routers",
RFC 1812, June 1995.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2988] Paxson, V. and M. Allman, "Computing TCP's Retransmission
Timer", RFC 2988, November 2000.
[RFC4443] Conta, A., Deering, S., and M. Gupta, "Internet Control
Message Protocol (ICMPv6) for the Internet Protocol
Version 6 (IPv6) Specification", RFC 4443, March 2006.
[RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion
Control", RFC 5681, September 2009.
11.2. Informative References
[CRVP01] Chandran, K., Raghunathan, S., Venkatesan, S., and R.
Prakash, "A feedback-based scheme for improving TCP
performance in ad hoc wireless networks", IEEE Personal
Communications vol. 8, no. 1, pp. 34-39, February 2001.
[HV02] Holland, G. and N. Vaidya, "Analysis of TCP performance
over mobile ad hoc networks", Wireless Networks vol. 8,
no. 2-3, pp. 275-288, March 2002.
[KP87] Karn, P. and C. Partridge, "Improving Round-Trip Time
Estimates in Reliable Transport Protocols", Proceedings
of the Conference on Applications, Technologies,
Architectures, and Protocols for Computer Communication
(SIGCOMM'87) pp. 2-7, August 1987.
[LS01] Liu, J. and S. Singh, "ATCP: TCP for mobile ad hoc
networks", IEEE Journal on Selected Areas in
Communications vol. 19, no. 7, pp. 1300-1315, July 2001.
[RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791,
September 1981.
[RFC0826] Plummer, D., "Ethernet Address Resolution Protocol: Or
converting network protocol addresses to 48.bit Ethernet
address for transmission on Ethernet hardware", STD 37,
RFC 826, November 1982.
[RFC1122] Braden, R., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC 1122, October 1989.
Zimmermann & Hannemann Experimental [Page 21]
RFC 6069 Making TCP More Robust to LCDs December 2010
[RFC2003] Perkins, C., "IP Encapsulation within IP", RFC 2003,
October 1996.
[RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", RFC 2460, December 1998.
[RFC2784] Farinacci, D., Li, T., Hanks, S., Meyer, D., and P.
Traina, "Generic Routing Encapsulation (GRE)", RFC 2784,
March 2000.
[RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
of Explicit Congestion Notification (ECN) to IP",
RFC 3168, September 2001.
[RFC3522] Ludwig, R. and M. Meyer, "The Eifel Detection Algorithm
for TCP", RFC 3522, April 2003.
[RFC3782] Floyd, S., Henderson, T., and A. Gurtov, "The NewReno
Modification to TCP's Fast Recovery Algorithm", RFC 3782,
April 2004.
[RFC3819] Karn, P., Bormann, C., Fairhurst, G., Grossman, D.,
Ludwig, R., Mahdavi, J., Montenegro, G., Touch, J., and
L. Wood, "Advice for Internet Subnetwork Designers",
BCP 89, RFC 3819, July 2004.
[RFC4015] Ludwig, R. and A. Gurtov, "The Eifel Response Algorithm
for TCP", RFC 4015, February 2005.
[RFC4301] Kent, S. and K. Seo, "Security Architecture for the
Internet Protocol", RFC 4301, December 2005.
[RFC5461] Gont, F., "TCP's Reaction to Soft Errors", RFC 5461,
February 2009.
[RFC5682] Sarolahti, P., Kojo, M., Yamamoto, K., and M. Hata,
"Forward RTO-Recovery (F-RTO): An Algorithm for Detecting
Spurious Retransmission Timeouts with TCP", RFC 5682,
September 2009.
[RFC5927] Gont, F., "ICMP Attacks against TCP", RFC 5927,
July 2010.
[SESB05] Schuetz, S., Eggert, L., Schmid, S., and M. Brunner,
"Protocol enhancements for intermittently connected
hosts", SIGCOMM Computer Communication Review vol. 35,
no. 3, pp. 5-18, December 2005.
Zimmermann & Hannemann Experimental [Page 22]
RFC 6069 Making TCP More Robust to LCDs December 2010
[SM03] Scott, J. and G. Mapp, "Link layer-based TCP optimisation
for disconnecting networks", SIGCOMM Computer
Communication Review vol. 33, no. 5, pp. 31-42,
October 2003.
[TCP-REXMIT-NOW]
Eggert, L., Schuetz, S., and S. Schmid, "TCP Extensions
for Immediate Retransmissions", Work in Progress,
June 2005.
[TCP-RLCI]
Schuetz, S., Koutsianas, N., Eggert, L., Eddy, W., Swami,
Y., and K. Le, "TCP Response to Lower-Layer Connectivity-
Change Indications", Work in Progress, February 2008.
[Zh86] Zhang, L., "Why TCP Timers Don't Work Well", Proceedings
of the Conference on Applications, Technologies,
Architectures, and Protocols for Computer Communication
(SIGCOMM'86) pp. 397-405, August 1986.
[ZimHan09]
Zimmermann, A., "Make TCP more Robust to Long
Connectivity Disruptions", Proceedings of the 75th IETF
Meeting slides, July 2009,
.
Authors' Addresses
Alexander Zimmermann
RWTH Aachen University
Ahornstrasse 55
Aachen, 52074
Germany
Phone: +49 241 80 21422
EMail: zimmermann@cs.rwth-aachen.de
Arnd Hannemann
RWTH Aachen University
Ahornstrasse 55
Aachen, 52074
Germany
Phone: +49 241 80 21423
EMail: hannemann@nets.rwth-aachen.de
Zimmermann & Hannemann Experimental [Page 23]
RFC, FYI, BCP