TCPM Working Group C. Gomez Internet-Draft UPC Intended status: Informational J. Crowcroft Expires: September 27, 2020 University of Cambridge March 26, 2020 Sender Control of Delayed Acknowledgments in TCP: Problem Statement, Requirements and Analysis of Potential Solutions draft-gomez-tcpm-delack-suppr-reqs-01 Abstract TCP Delayed Acknowledgments (ACKs) allow reducing protocol overhead in many scenarios. However, in some cases, Delayed ACKs may significantly degrade network and device performance in terms of link utilization, latency, memory usage and/or energy consumption. This document presents the problem statement regarding sender control of Delayed ACKs in TCP. The document discusses the scenarios and use cases in which sender control of Delayed ACKs offers advantages. Then, requirements for a potential solution are derived. Finally, a number of potential solutions are discussed, based on the requirements, and also considering pros and cons in each case. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on September 27, 2020. Copyright Notice Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents Gomez & Crowcroft Expires September 27, 2020 [Page 1] Internet-Draft Sender control of Delayed ACKs March 2020 (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Conventions used in this document . . . . . . . . . . . . . . 3 3. Problem statement: issues due to Delayed ACKs . . . . . . . . 3 3.1. Slow start . . . . . . . . . . . . . . . . . . . . . . . 3 3.2. High bit rate environments and short data segments . . . 4 3.3. IoT scenarios . . . . . . . . . . . . . . . . . . . . . . 4 3.4. Beyond classic ACK transmission behavior . . . . . . . . 4 4. Requirements for sender control of Delayed ACKs . . . . . . . 5 4.1. Sender-triggered mechanism . . . . . . . . . . . . . . . 5 4.2. Per-segment granularity . . . . . . . . . . . . . . . . . 5 4.3. Header/Message overhead . . . . . . . . . . . . . . . . . 6 4.4. Support for enabling generic ACK ratios . . . . . . . . . 6 4.5. Middlebox traversal . . . . . . . . . . . . . . . . . . . 6 4.6. Safe return to normal Delayed ACKs operation . . . . . . 6 4.7. Impact on existing TCP functionality . . . . . . . . . . 7 4.8. Impact on future TCP development . . . . . . . . . . . . 7 4.9. Avoidance of 'hacks' . . . . . . . . . . . . . . . . . . 7 4.10. Who is in control? . . . . . . . . . . . . . . . . . . . 7 5. Potential solutions for sender control of Delayed ACKs . . . 7 5.1. AckCC . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5.2. TLP . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5.3. TCP ACK Pull (AKP) flag . . . . . . . . . . . . . . . . . 8 5.4. A new 'ACK Pull' TCP option . . . . . . . . . . . . . . . 9 5.5. Reuse of existing TCP header fields . . . . . . . . . . . 9 5.6. 'Hacks' . . . . . . . . . . . . . . . . . . . . . . . . . 9 6. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 7. Security Considerations . . . . . . . . . . . . . . . . . . . 10 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 11 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 9.1. Normative References . . . . . . . . . . . . . . . . . . 11 9.2. Informative References . . . . . . . . . . . . . . . . . 12 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 12 1. Introduction Delayed Acknowledgments (ACKs) were specified for TCP with the aim to reduce protocol overhead [RFC1122]. With Delayed ACKs, a TCP delays sending an ACK by up to 500 ms (often 200 ms, with lower values in Gomez & Crowcroft Expires September 27, 2020 [Page 2] Internet-Draft Sender control of Delayed ACKs March 2020 recent implementations such as ~50 ms also reported), and typically sends an ACK for at least every second segment received in a stream of full-sized segments. This allows combining several segments into a single one (e.g. the application layer response to an application layer data message, and the corresponding ACK), and it also saves up to one of every two ACKs under many traffic patterns (e.g. bulk transfers). The "SHOULD" requirement level for implementing Delayed ACKs in RFC 1122, along with its expected benefits, has led to a widespread deployment of this mechanism. However, there exist traffic patterns and scenarios for which Delayed ACKs can actually be detrimental to performance. When a segment carrying a message of a size up to one Maximum Segment Size (MSS) is transferred, if the message does not elicit an application-layer response, and a second data segment is not transferred earlier than the Delayed ACK timeout, the ACK is unnecessarily delayed, with a number of negative consequences. Furthermore, there may be reasons to allow a sender communicate the ACK ratio to be used in a TCP connection, and thus dynamically override (or restore) use of Delayed ACKs at the receiver. This document presents the problem statement regarding sender control of Delayed ACKs. The document discusses the scenarios and use cases in which sender control of Delayed ACKs offers advantages. Then, requirements for a potential solution are derived. Finally, a number of potential solutions are discussed, based on the requirements, and also considering pros and cons in each case. 2. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL","SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 3. Problem statement: issues due to Delayed ACKs This section provides scenarios and use cases where performance issues arise due to Delayed ACKs. 3.1. Slow start During slow start, the congestion window (cwnd) increases by up to Sender Maximum Segment Size (SMSS) upon receipt of an ACK covering new data [RFC5681]. However, use of Delayed ACKs reduces the amount of ACKs received by the sender, thus reducing the rate of cwnd growth, increasing transfer time and reducing throughput, when compared with sending an ACK for each incoming data segment. Note that, while Appropriate Byte Counting (ABC) [RFC3465] might be used Gomez & Crowcroft Expires September 27, 2020 [Page 3] Internet-Draft Sender control of Delayed ACKs March 2020 to address this problem, it remains as an experimental mechanism, not fully included in RFC 5681, which specifies standard TCP congestion control. Furthermore, Delayed ACKs preclude using sender behaviors intended to quickly and non-intrusively probe for available capacity during slow start. One example of such behaviors is chirping, where a sender follows a predetermined pattern to send data segments (e.g. decreasing intersegment time gaps) and measures the gaps between ACKs [I-D.kuehlewind-tcpm-accurate-ecn] (see Appendix B.4 of revision -03 of the referenced document). 3.2. High bit rate environments and short data segments When the Nagle algorithm is used, in some cases the sender may be prevented from sending more data while awaiting a delayed ACK. In some high bit rate environment (e.g. Gigabit Ethernet) use cases, such a delay may be very large, and link utilitzation may be dramatically reduced, since the Delayed ACK timeout may be several orders of magnitude greater than the Round Trip Time (RTT) [RFC8490]. 3.3. IoT scenarios Delayed ACKs are also detrimental in Internet of Things (IoT) scenarios, where TCP is being increasingly used [I-D.ietf-lwig-tcp-constrained-node-networks]. Many IoT devices, such as sensors, transfer small messages (e.g. containing sensor readings) rather infrequently, therefore if the receiver uses Delayed ACKs, the ACK will often be unnecessarily delayed. The sender cannot release the memory resources associated to a transferred data segment until the ACK is received and processed. This may be a problem for many IoT devices, which are typically memory-constrained, and may even lead to subsequent packet drops if their scarce memory resources are blocked while awaiting an ACK. Moreover, if the IoT device uses a radio interface for communication, in some scenarios Delayed ACKs will lead to increased energy consumption (e.g. with the radio interface of the device staying in receive mode while awaiting the ACK). Since many IoT devices run on small batteries, the device lifetime may significantly decrease. Furthermore, the delay suffered by the ACK may interact negatively with layer two mechanisms, especially in wireless network technologies where devices remain in low-power states for long intervals [RFC8352], potentially leading to a further exacerbated delay (by even one or more orders of magnitude). 3.4. Beyond classic ACK transmission behavior In some scenarios, it may be desirable to enable ACK transmission behaviors beyond the classic ones. Gomez & Crowcroft Expires September 27, 2020 [Page 4] Internet-Draft Sender control of Delayed ACKs March 2020 For example, it may be beneficial to apply congestion control to ACKs [RFC5690]. Reducing the amount of ACKs on a congested reverse path may allow alleviating congestion on that path, with minimal impact on a relatively independent forward path. On the other hand, in some scenarios, the rate at which ACKs arrive at the sender limits the achievable performance of data transfer. This happens due to forward and reverse path asymmetric capacity (with the latter being significantly limited, e.g. in terms of bandwidth) [RFC3449]. In some environments, the issue is mitigated by using middleboxes that perform ACK thinning, i.e., deleting a subset of the ACKs. Examples of technologies where deployments have been reported to do ACK thinning include satellite links, DOCSIS cable networks, mobile cellular networks, among others. 4. Requirements for sender control of Delayed ACKs This section provides the requirements for a potential solution to enable sender control of Delayed ACKs. 4.1. Sender-triggered mechanism An assumption is that the sender knows when Delayed ACKs operation should be overriden. For example, the sender may know in advance the pattern of the traffic it will generate, or it may know whether an application-layer response will be sent by the receiving endpoint upon reception of a given message. Therefore, control of Delayed ACKs has to be sender-triggered. 4.2. Per-segment granularity One approach that cannot be recommended as a general solution for controlling Delayed ACKs is (permanently) disabling Delayed ACKs at the receiving TCP. In fact, the latter may interact with a wide variety of devices and many of those may still benefit from the advantages of Delayed ACKs. Another approach would be determining suppression of Delayed-ACKs per connection. However, within the same connection, a sender may offer a mixed traffic pattern comprising single data segments that will lead to unnecessarily delayed ACKs, with other data segments upon which Delayed ACKs will act as intended. Therefore, the solution has to be provided at a per-segment granularity. Gomez & Crowcroft Expires September 27, 2020 [Page 5] Internet-Draft Sender control of Delayed ACKs March 2020 4.3. Header/Message overhead Since the presented problem is about low performance in various scenarios, another requirement for the solution is to minimize incurring overhead in terms of header size increase or additional packets sent. For example, in IoT scenarios, every additional communicated byte consumes scarce resources (e.g. energy, bandwidth and computational resources). Another benefit of keeping a low header/message overhead is alleviating the processing workload of a receiving TCP. 4.4. Support for enabling generic ACK ratios For many of the scenarios and use cases described in Section 2, an ACK ratio of 1 (i.e. a receiver sending one ACK per incoming data segment) would solve the mentioned issues. However, the ability of enforcing a generic ACK ratio (including values different from 1 and 2) allows to enable a wider range of ACK behaviors, which may support congestion control for ACKs, sender behaviors not based on ACK- clocking, etc. The desired generic ACK ratio is intended to be in force for the current data segment, and for subsequent data segments (at least, during a time interval of a duration that may depend on several factors). The mechanism used to indicate the desired receiver Delayed ACK behavior might exploit soft state (i.e. using explicit information carried by data segments) or connection state (which needs to be stored by the receiver). 4.5. Middlebox traversal Deployment of new functionality for TCP faces the risk of packets being discarded by existing middleboxes upon detection of unexpected or discouraged formats, header field values or even traffic patterns. A solution for sender control of Delayed ACKs should offer relatively good middlebox traversal (to the extent possible). 4.6. Safe return to normal Delayed ACKs operation A solution for sender control of Delayed ACKs must ensure that normal Delayed ACKs operation is in force by default, and also once temporary action on Delayed ACKs needs to end. Gomez & Crowcroft Expires September 27, 2020 [Page 6] Internet-Draft Sender control of Delayed ACKs March 2020 4.7. Impact on existing TCP functionality A solution for sender control of Delayed ACKs should not reduce the space of existing TCP functionality. 4.8. Impact on future TCP development A solution for sender control of Delayed ACKs should not pose significant risk of preventing future TCP development. If an available resource (e.g. a reserved bit of the TCP header, a new TCP option, etc.) is used by a solution, careful analysis must be carried out regarding the risks and benefits of using such resource. 4.9. Avoidance of 'hacks' Sender control of Delayed ACKs might be achieved by using workarounds, such as implementation techniques that may produce the desired effect. However, such approaches may be suboptimal regarding implementation cleanliness, and may entail other performance issues (see section 5.6). 4.10. Who is in control? The receiver might not always be able to honour the ACK behavior desired by the sender. Therefore, the semantics of sender control of Delayed ACKs have to be of a hint, not a command. If the receiver is actually not able to apply the ACK behavior desired by the sender, then the former has a range of options with regard to communicating so to the latter, from remaining silent, to providing explicit feedback to the sender. Each option has associated trade-offs. For example, remaining silent might degrade performance if a sender relies on a receiver that uses the ACK behavior intended by the sender. 5. Potential solutions for sender control of Delayed ACKs This section enumerates and discusses potential solutions that might be considered to enable sender control of Delayed ACKs. The list of solutions is not necessarily comprehensive. This section intends to illustrate the trade-offs that arise when considering potential solutions for sender control of Delayed ACKs. (Note: the analysis needs to be completed for many of the solutions below.) Gomez & Crowcroft Expires September 27, 2020 [Page 7] Internet-Draft Sender control of Delayed ACKs March 2020 5.1. AckCC In Acknowledgment Congestion Control (AckCC) [RFC5690], the sender tells the receiver the ACK ratio R to use, where the receiver sends one ACK per R data packets received. AckCC defines a 2-byte "TCP ACK Congestion Control Permitted Option" for negotiating use of AckCC, whereas it defines a 3-byte "ACK ratio TCP option" to communicate the ACK Ratio value from the sender to the receiver. Middlebox traversal of a new TCP option is often regarded as 'bad' (to be confirmed). 5.2. TLP Tail Loss Probe (TLP) [I-D.ietf-tcpm-rack] is intended to avoid RTO- expiration-based retransmission when tail loss occurs by inducing additional ACKs at the receiver. This is achieved by sending a probe segment after a probe time-out (PTO) when data have been sent but not confirmed. Of course, this means sending a whole new packet to trigger ACKs, which adds significant overhead. This approach might offer good middlebox traversal (to be confirmed). 5.3. TCP ACK Pull (AKP) flag One solution that has been proposed for sender control of Delayed ACKs is called 'TCP ACK Pull' [I-D.gomez-tcpm-ack-pull]. TCP ACK Pull defines the AKP flag as bit number 6 of the 13th byte of the TCP header. When a TCP sender needs a data segment to be acknowledged by the receiving TCP without additional delay, the sender sets the AKP flag of the data segment TCP header. Upon reception of a segment with the AKP flag set, a conforming receiving TCP behaves accordingly by sending the corresponding ACK without additional delay. This solution would entail zero header or message overhead. However, it would consume a TCP header bit, leaving only two available TCP header reserved bits. A question is thus whether one TCP header bit should be dedicated to this purpose or not. Middlebox traversal characteristics of bit 6 of the TCP header need to be assessed. Gomez & Crowcroft Expires September 27, 2020 [Page 8] Internet-Draft Sender control of Delayed ACKs March 2020 5.4. A new 'ACK Pull' TCP option Another approach relies on defining a new option-kind-only TCP option with the same semantics as the AKP flag, which might be called 'ACK Pull Option' or 'AKP Option'. This solution would consume an available TCP Option Kind number. However, most of the 256 numbers in the TCP Option Kind number space are currently available. Therefore, consuming one such number does not appear to significantly limit future TCP development. The header overhead of the AKP Option is one byte. Middlebox traversal of a new TCP option is often regarded as 'bad' (to be confirmed). 5.5. Reuse of existing TCP header fields Another approach that might be used to enable sender control of Delayed ACKs is based on reusing existing TCP header fields. For example, use of the Urgent pointer has been suggested (e.g. by reserving 3 of its 16 bits to encode an ACK ratio exponent that may be communicated by the sender to the receiver), when URG=0. A problem with this approach is that the semantics of the reused TCP header field may become overloaded. Therefore, in some cases either the original intended use of the reused TCP header field may become limited, or if it prevails, then sender control of Delayed ACKs might not always be available for use. Middlebox traversal characteristics of this approach might be relatively good (to be confirmed). 5.6. 'Hacks' One approach that allows eliciting an immediate ACK after sending a data segment is sending a subsequent segment carrying a previously acknowledged data byte. However, in addition to the inefficiency of sending a byte that has previously been sent, this approach may require the transmission of a new packet (even carrying a single byte of data payload) just for that purpose, which represents significant overhead. Furthermore, sending a previously sent byte is not a clean solution from an implementation perspective. Another workaround intended to trigger an immediate ACK from the receiving TCP, which is used in the Contiki operating system (a popular operating system for constrained devices in IoT scenarios) is splitting the data to be sent into two segments of smaller size. A standard compliant TCP receiver will acknowledge the second MSS of Gomez & Crowcroft Expires September 27, 2020 [Page 9] Internet-Draft Sender control of Delayed ACKs March 2020 data. However, this 'split hack' may not always work since a TCP receiver is required to acknowledge every second full-sized segment, but not two consecutive small segments. Furthermore, the overhead of sending two IP packets instead of one is another downside of the 'split hack'. 6. Summary The next table summarizes whether the different solutions presented in Section 4 are able to satisfy the requirements stated in Section 4. +-------+-------+-------+-------+-------+------+------+ | Per- | Over- |Generic|Middle-|Impact |Impact|Hack | |segment| head |ACK rat|box tr.|current|future|Avoid.| +------------+-------+-------+-------+-------+-------+------+------+ | ACKcc | Yes | Low | Yes | Bad? | No | Low | Yes | +------------+-------+-------+-------+-------+-------+------+------+ | TLP | No | High | No | Good | No | No | Yes | +------------+-------+-------+-------+-------+-------+------+------+ | AKP flag | Yes | No | Yes | ? | No |Med/Hi| Yes | +------------+-------+-------+-------+-------+-------+------+------+ | AKP option | Yes | Low | Yes | Bad? | No | Low | Yes | +------------+-------+-------+-------+-------+-------+------+------+ |Reuse fields| Yes | No | Yes | Good? | Yes | ? | Yes | +------------+-------+-------+-------+-------+-------+------+------+ | Hacks | ? |Med/Hig| No | Good? | No | No | No | +------------+-------+-------+-------+-------+-------+------+------+ Note: all considered potential solutions satisfy the following requirements: i) sender control of Delayed ACKs, and ii) safe return to normal Delayed ACKs operation. A receiver may be unable to always honour the ACK behavior desired by the sender regardless of the specific potential solution considered. Figure 1: Summary of potential solutions for sender control of Delayed ACKs. 7. Security Considerations TBD Gomez & Crowcroft Expires September 27, 2020 [Page 10] Internet-Draft Sender control of Delayed ACKs March 2020 8. Acknowledgments Bob Briscoe, Jonathan Morton, Richard Scheffenegger, Michael Tuexen and Jana Iyengar provided useful input for this document. Stuart Cheshire, Ted Lemon, Michael Scharf, and Christoph Paasch participated in a discussion that was seminal to the TCP ACK Pull proposal, which eventually led to this document. Carles Gomez has been funded in part by the Spanish Government (Ministerio de Ciencia, Innovacion y Universidades) through Secretaria d'Universitats i Recerca del Departament d'Empresa i Coneixement de la Generalitat de Catalunya 2017 SGR 376. 9. References 9.1. Normative References [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - Communication Layers", STD 3, RFC 1122, DOI 10.17487/RFC1122, October 1989, . [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC3449] Balakrishnan, H., Padmanabhan, V., Fairhurst, G., and M. Sooriyabandara, "TCP Performance Implications of Network Path Asymmetry", BCP 69, RFC 3449, DOI 10.17487/RFC3449, December 2002, . [RFC3465] Allman, M., "TCP Congestion Control with Appropriate Byte Counting (ABC)", RFC 3465, DOI 10.17487/RFC3465, February 2003, . [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, . [RFC5690] Floyd, S., Arcia, A., Ros, D., and J. Iyengar, "Adding Acknowledgement Congestion Control to TCP", RFC 5690, DOI 10.17487/RFC5690, February 2010, . Gomez & Crowcroft Expires September 27, 2020 [Page 11] Internet-Draft Sender control of Delayed ACKs March 2020 9.2. Informative References [I-D.gomez-tcpm-ack-pull] Gomez, C. and J. Crowcroft, "TCP ACK Pull", draft-gomez- tcpm-ack-pull-01 (work in progress), November 2019. [I-D.ietf-lwig-tcp-constrained-node-networks] Gomez, C., Crowcroft, J., and M. Scharf, "TCP Usage Guidance in the Internet of Things (IoT)", draft-ietf- lwig-tcp-constrained-node-networks-09 (work in progress), November 2019. [I-D.ietf-tcpm-rack] Cheng, Y., Cardwell, N., Dukkipati, N., and P. Jha, "RACK: a time-based fast loss detection algorithm for TCP", draft-ietf-tcpm-rack-08 (work in progress), March 2020. [I-D.kuehlewind-tcpm-accurate-ecn] Briscoe, B., Kuehlewind, M., and R. Scheffenegger, "More Accurate ECN Feedback in TCP", draft-kuehlewind-tcpm- accurate-ecn-05 (work in progress), October 2015. [RFC8352] Gomez, C., Kovatsch, M., Tian, H., and Z. Cao, Ed., "Energy-Efficient Features of Internet of Things Protocols", RFC 8352, DOI 10.17487/RFC8352, April 2018, . [RFC8490] Bellis, R., Cheshire, S., Dickinson, J., Dickinson, S., Lemon, T., and T. Pusateri, "DNS Stateful Operations", RFC 8490, DOI 10.17487/RFC8490, March 2019, . Authors' Addresses Carles Gomez UPC C/Esteve Terradas, 7 Castelldefels 08860 Spain Email: carlesgo@entel.upc.edu Gomez & Crowcroft Expires September 27, 2020 [Page 12] Internet-Draft Sender control of Delayed ACKs March 2020 Jon Crowcroft University of Cambridge JJ Thomson Avenue Cambridge, CB3 0FD United Kingdom Email: jon.crowcroft@cl.cam.ac.uk Gomez & Crowcroft Expires September 27, 2020 [Page 13]