Network Working Group I. Johansson Internet-Draft Ericsson AB Intended status: Informational January 18, 2017 Expires: July 22, 2017 ECN support in QUIC draft-johansson-quic-ecn-00 Abstract This memo outlines the ECN support in QUIC. The intention is that most of the material ends up updating other new or existing QUIC protocol specifications, thus it may be possible that this draft does not warrant a working group status. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on July 22, 2017. Copyright Notice Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents Johansson Expires July 22, 2017 [Page 1] Internet-Draft ECN support in QUIC January 2017 carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Elements of ECN support . . . . . . . . . . . . . . . . . . . 2 2.1. ECN negotialtion . . . . . . . . . . . . . . . . . . . . 3 2.2. ECN bits in the IP header, semantics . . . . . . . . . . 3 2.3. ECN echo . . . . . . . . . . . . . . . . . . . . . . . . 3 2.4. Fallback in case of ECN fault . . . . . . . . . . . . . . 6 2.5. OS socket specifics, access to the ECN bits . . . . . . . 6 2.6. Monitoring . . . . . . . . . . . . . . . . . . . . . . . 7 3. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 4. Open questions . . . . . . . . . . . . . . . . . . . . . . . 8 5. Security Considerations . . . . . . . . . . . . . . . . . . . 8 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 8 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 8 7.1. Normative References . . . . . . . . . . . . . . . . . . 8 7.2. Informative References . . . . . . . . . . . . . . . . . 8 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 10 1. Introduction ECN support in transport protocols is a fundamental feature that should be included in the QUIC specification as a mandatory element. The benefits of ECN is described in [I-D.ietf-aqm-ecn-benefits]. The ECN support should be implemented to support both present and future ECN, the latter is outlined in [I-D.ietf-tsvwg-ecn-experimentation], of particular interest is the ability to discriminate between classic ECN and L4S ECN by means of an ability to differentiate between the use of the ECT(0) and ECT(1) code points. This draft does however not delve into the details of the congestion control implementation. 2. Elements of ECN support This draft covers the following aspects of ECN support: o ECN negotiation o ECN echo o ECN bits in the IP header, semantics o Fallback in case of ECN fault Johansson Expires July 22, 2017 [Page 2] Internet-Draft ECN support in QUIC January 2017 o OS socket specifics, access to the ECN bits o Monitoring 2.1. ECN negotialtion In TCP, ECN was added as a later feature than the original TCP specification, this necessitated negotiation as there was (and still are?) TCP stacks that do not support ECN. With QUIC, the situation is more straightforward. The suggested way forward is here to just add ECN support from a given version of QUIC, thus negotiation is not necessary. This means that the necessary protocol elements as well as the associated ECN support functions are included in the specification from that version. This does not necessarily mean that the underlying OS supports ECN, for instance it may be the case that the sockets do not give access to the ECN bits. Investigations (Piers O'Hanlon) have indicated that this is in certain cases an asymmetric property, for instance while it is possible to set the ECN bits it is not possible to read them. An additional bit in the QUIC header should indicate if the peer is able to fully access the ECN bits, if any of the peers is unable to do so then ECN should not be used in any direction. The mode mechanism in [RFC6679] can serve as in input to a solution. 2.2. ECN bits in the IP header, semantics The ECN bits in the IP header should be set according to the recommendations in [I-D.ietf-tsvwg-ecn-experimentation]. This means that the meaning of ECT(0) and ECT(1) differ. Question: Should the ECN bits be set in the IP packets for the initial QUIC handshake ? Setting these bits may make the handshake setup fail if for instance the network discards ECT or CE marked packets, on the other hand a congested node may discard the QUIC connection initialization packets if ECT is not set. These matters are elaborated upon in [Bagnulo]. 2.3. ECN echo The ECN echo should go into the ACK frame [I-D.ietf-quic-transport]. It is suggested that the 'U' bit in the ACK frame type is renamed 'E' to indicate the presence of an ECN field in the ACK frame [Question: is it necessary to indicate this or should an ECN field just be included by default?]. Johansson Expires July 22, 2017 [Page 3] Internet-Draft ECN support in QUIC January 2017 Currently there are three alternatives how to add ECN support to the ACK frames . The first alternative inserts a one octet field that contains a 2 bit ECN echo, followed by the ACK block length. The ACK block length then dictates the number of received contiguous frames with the indicated ECN echo. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ECE (8) | First Ack Block Length (8/16/32/48) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | [Gap 1 (8)] | ECE(8) |[Ack Blk 1 L (8/16/32/48)] ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | [Gap 2 (8)] | ECE(8) |[Ack Blk 2 L (8/16/32/48)] ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | [Gap N (8)] | ECE(8) |[Ack Blk N L (8/16/32/48)] ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 1: ECN field in ACK frame ACK block, alt 1 The second alternative encodes a variable length field that contains the ECN echoes for the frames listed in the ACK blocks. The length of the field is inferred from the ACK block lengths. No ECN echoes are indicated for the gaps (it is, after all, impossible to indicate status of the ECN bits for lost packets). For instance if the ACK blocks list 10 frames, then the length of the ECN echo field becomes 2*10=20bits, with additional 4 bits of padding the ECN echo field will then become 3 octets long. Johansson Expires July 22, 2017 [Page 4] Internet-Draft ECN support in QUIC January 2017 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | First Ack Block Length (8/16/32/48) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | [Gap 1 (8)] | [Ack Block 1 Length (8/16/32/48)] ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | [Gap 2 (8)] | [Ack Block 2 Length (8/16/32/48)] ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | [Gap N (8)] | [Ack Block N Length (8/16/32/48)] ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |ECE|ECE|... variable length, padded to full octets ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 2: ECN field in ACK frame ACK block, alt 2 The third alternative encodes the number of bytes that are marked ECT(0), ECT(1) and CE with 32 bits each, the total extra overhead is thus 12 octets. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | First Ack Block Length (8/16/32/48) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | [Gap 1 (8)] | [Ack Block 1 Length (8/16/32/48)] ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | [Gap 2 (8)] | [Ack Block 2 Length (8/16/32/48)] ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | [Gap N (8)] | [Ack Block N Length (8/16/32/48)] ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | # ECT(0) bytes (32) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | # ECT(1) bytes (32) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | # ECN-CE bytes (32) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 3: ECN field in ACK frame ACK block, alt 3 There are pros an cons with the three alternatives: o Alt 1: Is very compact in the case where the ECN bits are largely unchanged. However in the worst case where received frames flip Johansson Expires July 22, 2017 [Page 5] Internet-Draft ECN support in QUIC January 2017 forth and back between ECT and CE then each frame will require at least 3 octets overhead (ECE, ACK block length, Gap). o Alt 2: Is quite compact as it only requires two bits encoding per frame. The additional overhead amounts to ceil(N*2/8) octets where the N is the sum of the ACK block lengths. On the downside is that it is a less efficient format for the case that the ECN bits are unchanged. One uncertainty is if STOP_WAITING frames could make this encoding bulky. o Alt 3: Has a fixed 12 octet overhead which may be beneficial as it gives a deterministic overhead. The possible drawback is that it is not possible to know exactly which frames have been remarked, something that can limit the ability to detect network ECN faults based on the method to transmit a pattern on ECT and CE marked packets. Which of the three formats above (or something else) that is the best alternative is subject to discussion. 2.4. Fallback in case of ECN fault ECN can be subject to issues in network equipment, such as remarking to Not-ECN, remarking from ECT(0) to ECT(1) and vice versa or constant remarking to ECN-CE. Furthermore ECT marked packets may be discarded in the network. While these problems seem to be rare, see for instance [McQuistin-Perkins], it is still necessary to safeguard against such problems. A peer should disable ECN for its outgoing packets if ECN fault is detected, it is however still possible for the other peer to use ECN. TODO add more information as regards to how to detect network ECN faults. [ECN-fallback](expired) gives a few examples for fault detection. Examples on how to detect ECN faults include for instance the method to set ECT and CE for outgoing packets according to a given pattern. Fallback in case of ECN faults is not an issue only for QUIC, it is here suggested that mechanisms for this is described in a non QUIC related draft, for instance in TSVWG. 2.5. OS socket specifics, access to the ECN bits ECN support in QUIC comes with the additional challenge that it is necessary to somehow access the ECN bits in the IP headers. In TCP this is provided without major concerns as TCP is generally implemented in OS kernel space. QUIC can however be implemented both Johansson Expires July 22, 2017 [Page 6] Internet-Draft ECN support in QUIC January 2017 in user space or kernel space and is layered on top of UDP, which means that access to the ECN bits is not a given, instead various tricks are needed. The text below is copy-pasted from [OHanlon]. "To set ECN on Linux, BSD and OSX one can use IP_TOS socket option, with the setsockopt() call, to set the relevant ECN bits of the TOS byte. On Windows one can use a similar technique though firstly one has to enable TOS byte setting by enabling a particular Registry key ( DisableUserTOSSetting=0 (see https://msdn.microsoft.com/en- us/library/windows/desktop/dd874008%28v=vs.85%29.aspx One could also probably use the libpcap write functionality." "To obtain the ECN bits from a packet one needs a mechanism to retrieve the ECN bits from each packet. On Linux, one needs to firstly set the IP_RECVTOS socket option on the receiving socket, and use the recvmsg() call to receive a packet, and then retrieve the TOS byte from the associated csmg structure returned by the recvmsg() call. This still works with linux-4.2.3. On OSX/BSD there are no suitable socket options to retrieve the ECN/TOS bits and one cannot use raw sockets as they do not function for UDP/TCP sockets (they do work with ICMP), so one has to use alternatives such the bpf interface, or a REDIRECT socket. Whilst on Windows it seems that the only way to retrieve the ECN bits is via a raw socket, or custom NDIS driver, though it's possible there's an API I'm missing." TODO: Write a more detailed description on how to implement ECN support in QUIC for different OS stacks. 2.6. Monitoring A QUIC implementation should monitor the ECN functionality in order to provide input to e.g. service providers to improve ECN support in the networks. Items of interest are: o Black holes, ECT or CE marked packets are discarded. o Faulty remarking, e.g. ECT(0) is remarked to ECT(1) or Not-ECT. o Continuous CE marking, possible indication of faulty on/off ECN marking, but can also be an effect of severe congestion. 3. IANA Considerations T.B.D. Johansson Expires July 22, 2017 [Page 7] Internet-Draft ECN support in QUIC January 2017 4. Open questions A list of open questions: o Should the ECN field in the ACK frame be mandatory ? (in which case it is not necessary to indicate its presence) o Should all packets be ECT or should there be special patterns to improve fault detection. o Write up a more detailed description on how to implement ECN support in QUIC for different OS stacks. o Determine which ECN echo encoding in the ACK frame is the best alternative. o Is a completely new ACK frame an alternative ? o How do STOP_WAITING frames affect the ECN echo overhead. 5. Security Considerations T.B.D 6. Acknowledgements The following persons have contributed with comments and suggestions for improvements: Mirja Kuehlewind, Koen De Schepper, Michael Welzl 7. References 7.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . 7.2. Informative References [Bagnulo] "Adding Explicit Congestion Notification (ECN) to TCP control packets and TCP retransmissions", . Johansson Expires July 22, 2017 [Page 8] Internet-Draft ECN support in QUIC January 2017 [ECN-fallback] "A Mechanism for ECN Path Probing and Fallback", . [I-D.ietf-aqm-ecn-benefits] Fairhurst, G. and M. Welzl, "The Benefits of using Explicit Congestion Notification (ECN)", draft-ietf-aqm- ecn-benefits-08 (work in progress), November 2015. [I-D.ietf-quic-transport] Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed and Secure Transport", draft-ietf-quic-transport-01 (work in progress), January 2017. [I-D.ietf-tsvwg-ecn-experimentation] Black, D., "Explicit Congestion Notification (ECN) Experimentation", draft-ietf-tsvwg-ecn-experimentation-00 (work in progress), December 2016. [McQuistin-Perkins] ""Is Explicit Congestion Notification usable with UDP?", Proceedings of the ACM Internet Measurement Conference, Tokyo, Japan, October 2015. DOI:10.1145/2815675.2815716", . [OHanlon] "ECN support in different OS stacks", . [RFC6679] Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P., and K. Carlberg, "Explicit Congestion Notification (ECN) for RTP over UDP", RFC 6679, DOI 10.17487/RFC6679, August 2012, . [RFC6789] Briscoe, B., Ed., Woundy, R., Ed., and A. Cooper, Ed., "Congestion Exposure (ConEx) Concepts and Use Cases", RFC 6789, DOI 10.17487/RFC6789, December 2012, . [RFC7560] Kuehlewind, M., Ed., Scheffenegger, R., and B. Briscoe, "Problem Statement and Requirements for Increased Accuracy in Explicit Congestion Notification (ECN) Feedback", RFC 7560, DOI 10.17487/RFC7560, August 2015, . Johansson Expires July 22, 2017 [Page 9] Internet-Draft ECN support in QUIC January 2017 Author's Address Ingemar Johansson Ericsson AB Laboratoriegraend 11 Luleaa 977 53 Sweden Phone: +46 730783289 Email: ingemar.s.johansson@ericsson.com Johansson Expires July 22, 2017 [Page 10]