QUIC Working Group S. Dawkins, Ed. Internet-Draft Tencent America Intended status: Informational December 06, 2020 Expires: June 9, 2021 Questions for Multiple Paths In QUIC draft-dawkins-quic-multipath-questions-00 Abstract The IETF QUIC working group has been chartered to produce extensions that would "enable ... multipath capabilities" since the working group was formed in 2016, but because multipath was an extension, work on multipath, and the other extensions named in the charter, waited while work proceeded on the core QUIC protocol specifications. After the QUIC working group chairs requested publication for the core QUIC protocol specifications, they scheduled a virtual interim meeting to understand the use cases that various groups inside and outside the IETF were envisioning for multipath with QUIC. As part of that discussion, it became obvious that people had a variety of ideas about how multiple paths would be used, because they weren't looking at the same use cases, and so had different assumptions about how applications might use QUIC over multiple paths. This document is intended to capture questions that have come up in discussions, with some suggested answers, to inform further discussion in the working group. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on June 9, 2021. Dawkins Expires June 9, 2021 [Page 1] Internet-Draft QUIC Multipath Questions December 2020 Copyright Notice Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1. A 2500-year Side Trip . . . . . . . . . . . . . . . . . . 3 1.2. Back to the Introduction . . . . . . . . . . . . . . . . 4 1.3. Notes for Readers . . . . . . . . . . . . . . . . . . . . 4 1.4. Contribution and Discussion Venues for this draft. . . . 5 2. Metaquestions About QUIC Multipath . . . . . . . . . . . . . 5 2.1. MQ-01 Do We Need Multipath in QUIC? . . . . . . . . . . . 5 2.2. MQ-02 Do We Already Have Multipath in QUIC? . . . . . . . 6 3. Questions about Multipath in QUIC (#mpinq} . . . . . . . . . 6 3.1. Q-01 Is There One "Multipath"? . . . . . . . . . . . . . 6 3.2. Q-02 Do Transport Protocols Need to Support Multipath? . 7 3.3. Q-03 Does Multipath for QUIC Imply Multipath for HTTP/3-QUIC? . . . . . . . . . . . . . . . . . . . . . . 8 3.4. Q-04 What are the Expectations about Reordering? . . . . 8 3.5. Q-05 How Will We Measure the Benefits and Costs of Multipath? . . . . . . . . . . . . . . . . . . . . . . . 9 3.6. Q-06 What Are the Goals for using Multiple Paths? . . . . 9 3.7. Q-07 What are the API Considerations for Multipath? . . . 9 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 5. Security Considerations . . . . . . . . . . . . . . . . . . . 10 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 10 7. Document History . . . . . . . . . . . . . . . . . . . . . . 11 8. Informative References . . . . . . . . . . . . . . . . . . . 11 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 13 1. Introduction The IETF QUIC working group has been chartered to produce extensions that would "enable ... multipath capabilities" ([QUIC-charter]) since the working group was formed in 2016, but because multipath was an extension, work on multipath, and the other extensions named in the Dawkins Expires June 9, 2021 [Page 2] Internet-Draft QUIC Multipath Questions December 2020 charter, waited while work proceeded on the core QUIC protocol specifications ([I-D.ietf-quic-transport] and related specifications). After the QUIC working group chairs requested publication for the core QUIC protocol specifications, they scheduled a virtual interim meeting ([QUIC-interim-20-10]) to understand the use cases that various groups inside and outside the IETF were envisioning for multipath with QUIC. As part of that discussion, which continued on the QUIC working group mailing list and at IETF 109 [QUIC-IETF-109-minutes], it became obvious that people had a variety of ideas about how multiple paths would be used, because they weren't looking at the same use cases, and so had different assumptions about how applications might use QUIC over multiple paths. This document is intended to capture questions that have come up in discussions, with some suggested answers, to inform further discussion in the working group. 1.1. A 2500-year Side Trip The story that might have been titled "The People Unable to See and the Elephant" [Elephant-By-Touch] in 2021, is from the Indian subcontinent, and is at least 2500 years old (it is recorded in the Buddhist text Udana 6.4 [Udana-6-4]). The story, which has spread widely throughout the world, is about people unable to see, who are brought before the king, and shown an elephant - but they only touched the part of the elephant they were each standing in front of, and then began to argue about what an elephant was like (in [Udana-6-4]. Their descriptions were that "an elephant is like" a jar, a basket, a plowshare, the pole of a plow, a granary, a post, a mortar, a pestle, or a broom. None were completely wrong (they had their hands on the part of the elephant they were right about), but none were completely right, either (an elephant was more than the parts they hand their hands on). And the king pointed to the monks of different sects who had been arguing about the teachings of the Buddha, and said, "they keep on arguing, quarreling, and disputing, wounding one another with weapons of the mouth, saying, 'The teaching is like this, it's not like that. The teaching is not like that, it's like this.'" Dawkins Expires June 9, 2021 [Page 3] Internet-Draft QUIC Multipath Questions December 2020 1.2. Back to the Introduction Let us turn our attention from "describing an elephant", to "describing what multiple paths in QUIC might mean", based on o the QUIC working group virtual interim meeting minutes on multicast at [QUIC-interim-20-10-minutes], which include pointers to the video recording from the meeting, pointers to each presentation given, questions and answers after each presentation, o discussion that continued following the virtual interim meeting the QUIC working group mailing list, and o discussion during the QUIC working group session at IETF 109 [QUIC-IETF-109-minutes]. We are not unable to see, but we can't agree on the shape of multipath for QUIC without understanding what other people are looking at. 1.3. Notes for Readers It cannot be emphasized enough that multiple proposals for "QUIC multipath" have been submitted to the IETF (at a minimum, [I-D.an-multipath-quic], [I-D.deconinck-quic-multipath], and [I-D.huitema-quic-mpath-option]), but none have been adopted by the working group as of this writing. In this document, "QUIC multipath" means only "QUIC using multiple paths", and draws from the discussion on the QUIC working group mailing list, some of which has been guided by one or more of the current proposals. This document does reuse some terminology from [I-D.dawkins-quic-what-to-do-with-multipath], especially "Traffic Switching" and "Traffic Splitting", which are summarized in Section 3.2. This document is an informational Internet-Draft, not adopted by any IETF working group, and does not carry any special status within the IETF. Please note well that this document reflects the author's current understanding of working group discussions. It is likely that there are more questions than currently included in the document, and it is even more likely that some of the suggested answers are incomplete or (unlike the people in Section 1.1) completely wrong. Contributions Dawkins Expires June 9, 2021 [Page 4] Internet-Draft QUIC Multipath Questions December 2020 that add or improve questions and answers are welcomed, as described in Section 1.4. 1.4. Contribution and Discussion Venues for this draft. (Note to RFC Editor - if this document ever reaches you, please remove this section) This document is under development in the Github repository at https://github.com/SpencerDawkins/draft-dawkins-quic-multipath- questions. Readers are invited to open issues and send pull requests with contributed text for this document, but since the document is intended to guide discussion for the QUIC working group, substantial discussion of this document should take place on the QUIC working group mailing list (quic@ietf.org). Subscription and archive details are at https://www.ietf.org/mailman/listinfo/quic. 2. Metaquestions About QUIC Multipath We've had considerable discussion about the details of multipath implementation in QUIC, but it's worth starting out with a couple of metaquestions: o Do we need multipath in QUIC? (see Section 2.1) o Do we already have multipath in QUIC? (see Section 2.2) 2.1. MQ-01 Do We Need Multipath in QUIC? The most compelling explanation of the point of view that we don't need multipath is likely the [Chromium-Multipath] presentation given at [QUIC-interim-20-10]. The short summary of this presentation was that Google had included multipath in planning for their gQUIC implementation, but subsequently stopped working on this, choosing instead to focus on making connection migration work, given that this was what customers were asking for. As of the date of [QUIC-interim-20-10], few if any implementations were using connection migration, which was defined in [I-D.ietf-quic-transport]. So there was a well-supported point of view that the right thing to do was to get deployment experience with connection migration, and then see what use cases remain that could not be supported using connection migration. Suggested answer to MQ-01: "Maybe". Dawkins Expires June 9, 2021 [Page 5] Internet-Draft QUIC Multipath Questions December 2020 2.2. MQ-02 Do We Already Have Multipath in QUIC? We noted in discussions that [I-D.ietf-quic-transport] allows an implementation to open two or more connections on different paths, verify the paths, and even probe to ensure that a path is still valid before migrating from one connection to another. When an implementation does this proactively, there is no connection establishment delay when the implementation migrates from one path to another. So it's a reasonable question to ask, "do we already have multipath support in QUIC?" For some applications, yes, but this is highly application-dependent. The key point in discussion was that this level of support allows quick migration from one path to another (what [I-D.dawkins-quic-what-to-do-with-multipath] calls "traffic switching"}, but does not allow sustained simultaneous use of multiple paths (what [I-D.dawkins-quic-what-to-do-with-multipath] calls "traffic splitting"). Even for traffic switching, the QUIC implementation does not know the capacity of the path that the connection migrates to, and must learn it, and this means that a successful connection migration can lead to a detectable difference in performance if the migrated-from connection was carrying a significant amount of traffic. Suggested answer to MQ-02: "For applications that will perform traffic switching, possibly so. For applications that perform traffic splitting, no". 3. Questions about Multipath in QUIC (#mpinq} Once QUIC working group discussion moves beyond the metaquestions in Section 2, questions remain. Please note that this document only includes questions that are high- level. There have also been discussions about topics like whether a single number space is shared across paths, etc. that aren't included, even though they're important. 3.1. Q-01 Is There One "Multipath"? Based on mailing list discussion, the answer is almost certainly "No". "Multipath" is at best an umbrella term covering a variety of use cases that can make use of multiple paths to accomplish a variety of goals. Some examples are listed in Section 3.6. If an application can use connections over multiple paths independently from each other, the application could possibly make Dawkins Expires June 9, 2021 [Page 6] Internet-Draft QUIC Multipath Questions December 2020 use of multiple paths, without any "multipath" support in QUIC at all. But this is highly application-dependent. We noted that there are many strategies for using multiple paths (some described in [I-D.dawkins-quic-what-to-do-with-multipath]). It's worth also noting that these strategies may not have much in common with each other. There are use cases that expect to move from path to path, depending on the relative financial costs of those paths. Other use cases expect to move from path to path depending on the measured RTT of each path, whether picking a path with the lowest measured RTT. Still others may expect to use multiple paths for a single connection, if the measured path characteristics are "sufficiently similar" to each other. Suggested answer to Q-01: "It's not clear that one multipath scheduler can support all of these use cases". 3.2. Q-02 Do Transport Protocols Need to Support Multipath? This question seems to revolve around two sub-questions: o Whether we can identify use cases with enough in common with each other to reuse a single scheduler, whether or not there are other use cases that need different schedulers. From a code reuse perspective, we note that multipath is complicated enough to get right that depending on applications to get multipath right isn't desireable, although it's unavoidable. One reason QUIC was originally chartered to do multipath was because we noticed that both TCP [RFC0793] and SCTP [RFC4960] were retrofitted with support for some aspects of multipath operation ([RFC8684] adding multipath support for TCP, and [RFC5061] adding fast failover to another path for SCTP}}. o Whether we think there are enough use cases that don't rely on application-specific awareness to make scheduling decisions. We have been successful when we've included Traffic Switching in transport protocols, but we've only been successful including Traffic Splitting in transport protocols when relevant path characteristics have been similar between various paths, and the use case involved traffic that didn't require the transport to distinguish between different types of application-level information when selecting an appropriate path. If an application wants to send some types of video frames over one path and other types over a different path, that's not going to be easy to do using a general-purpose transport mechanism. And while the Internet is more than the World Wide Web, we noted that the World Wide Web has been moving away from bulk Dawkins Expires June 9, 2021 [Page 7] Internet-Draft QUIC Multipath Questions December 2020 transfers for some time, so that user-perceived latency matters much more. Suggested answer to Q-02: "Maybe. But it's important to have realistic expectations. We will likely end up with multiple schedulers, and we may very well end up with applications that can't use any of the schedulers we describe, and have to handle multiple paths on their own". 3.3. Q-03 Does Multipath for QUIC Imply Multipath for HTTP/3-QUIC? [QUIC-charter] has focused on supporting HTTP since the working group was chartered in 2016. Although it is certainly possible for applications to use [I-D.ietf-quic-transport] as their interface to the QUIC protocol, that's not what most applications we have implementation and deployment experience with have used, and the recent discussions in the MASQUE working group [MASQUE-charter] has pointed towards reusing HTTP/3, even for a tunneling and proxying application that may not make much use of HTTP beyond connection set- up. Some use cases include tunneling, so are reasonable candidates to use HTTP/3 to set up MASQUE tunnels anyway. Suggested answer to Q-03: "Probably so, especially if the goal is to provide a multipath capability for QUIC without duplicating testing that has already been carried out at scale for HTTP/3". 3.4. Q-04 What are the Expectations about Reordering? We've had at least two starting ppints for people in these discussions - one that expects traffic to be delivered in order to applications, and one that expects applications to handle their own out-of-order delivery, since the sending application needs to track what's actually being delivered, including being delivered significantly out of order, in order to select a path more effectively. For latency-sensitive applications, it's likely that out of order delivery across paths is something the application would want to be aware of. And it's worth noting that we're chatting about the desirability of reliable streams, versus partially reliable streams, versus datagrams, in at least a couple of use cases, and partial reliability isn't part of [I-D.ietf-quic-datagram] is underway in the QUIC working group, so if we think partially reliable streams are the right answer, there's still some specification work to do. Suggested answer for Q-04: "Answer unclear". Dawkins Expires June 9, 2021 [Page 8] Internet-Draft QUIC Multipath Questions December 2020 3.5. Q-05 How Will We Measure the Benefits and Costs of Multipath? This has come up most often in private conversations, but I've heard concerns about our ability to measure whether we are making the best use of multiple paths at scale, in a reproduceable way. It seems obvious that this question is important, and while it may be early in the process to expect a detailed answer, this question is recorded in this document so that we don't lose track of it. Suggested answer for Q-05: "Please check back later". 3.6. Q-06 What Are the Goals for using Multiple Paths? At least one point of view is that different use cases have different goals. Some goals that have been shared include o to migrate from one path to another, possibly because a path has stopped working, or because a "better" path becomes available. o using all available bandwidth betwen two endpoints, across multiple paths. o minimizing latency for the best possible user experience, especially for page loads on the Web. o maximizing reliability of specific traffic, by transmitting that traffic on multiple paths, either at all times or at specific times, such as "when radio signals are fading". Other goals almost certainly exist, and contributions describing them in a sentence or two are welcomed. See Section 1.4 for details. Note that actual use cases may be more nuanced about their goals - for instance, sending low-latency traffic over the lowest-latency path, and then using all remaining available bandwidth across all paths for other traffic. Suggested answer for Q-06: "It depends". 3.7. Q-07 What are the API Considerations for Multipath? We've noted a number of times that the most useful features of protocols won't be used if there is no way for an application to use them. HTTP/2 stream priorities [RFC7540] was mentioned frequently, but other features were mentioned. Dawkins Expires June 9, 2021 [Page 9] Internet-Draft QUIC Multipath Questions December 2020 We need a better understanding of the goals (see Section 3.6 to answer this question, but it's included in this document so we don't forget it. Suggested answer for Q-07: "Please check back later". 4. IANA Considerations This document does not make any request to IANA. 5. Security Considerations QUIC-specific security considerations are discussed in Section 21 of [I-D.ietf-quic-transport]. Section 6 of [I-D.ietf-quic-datagram] discusses security considerations specific to the use of the Unreliable Datagram Extension to QUIC. Some multipath QUIC-specific security considerations can be found in Section 8 of the individual draft [I-D.deconinck-quic-multipath]. 6. Acknowledgements I'd like to thank Lars Eggert and Lucas Pardue, the QUIC working group chairs, who called the QUIC virtual interim meeting on multipath. I'd also like to thank the presenters at the QUIC virtual interim, who put together valuable presentations on short notice. Many thanks to (your name could easily appear here) for reviews and comments. I've been through the QUIC working group archives on multipath discussions, along with the minutes from [QUIC-interim-20-10] and IETF 109 [QUIC-IETF-109-minutes], and too many people have commented on that for me to list them all. My apologies for that, but thank you all for contributing. And it's worth noting that the story described in Section 1.1 included the people who were trying to describe the elephant beating other people who disagreed with their description. Thanks for not going there in the QUIC working group. Dawkins Expires June 9, 2021 [Page 10] Internet-Draft QUIC Multipath Questions December 2020 7. Document History (Note to RFC Editor - if this document ever reaches you, please remove this section) Version -00: initial submission 8. Informative References [Chromium-Multipath] Fan Yang/Jana Iyengar, ., "Multipath in Chromium", October 2020, . [Elephant-By-Touch] "Wikipedia Page for Blind men and an elephant", December 2020, . [I-D.an-multipath-quic] An, Q., Liu, Y., Ma, Y., and Z. Li, "Multipath Extension for QUIC", draft-an-multipath-quic-00 (work in progress), October 2020. [I-D.dawkins-quic-what-to-do-with-multipath] Dawkins, S., "What To Do With Multiple Active Paths in QUIC", draft-dawkins-quic-what-to-do-with-multipath-02 (work in progress), November 2020. [I-D.deconinck-quic-multipath] Coninck, Q. and O. Bonaventure, "Multipath Extensions for QUIC (MP-QUIC)", draft-deconinck-quic-multipath-06 (work in progress), November 2020. [I-D.huitema-quic-mpath-option] Huitema, C., "QUIC Multipath Negotiation Option", draft- huitema-quic-mpath-option-00 (work in progress), October 2020. [I-D.ietf-quic-datagram] Pauly, T., Kinnear, E., and D. Schinazi, "An Unreliable Datagram Extension to QUIC", draft-ietf-quic-datagram-01 (work in progress), August 2020. [I-D.ietf-quic-transport] Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed and Secure Transport", draft-ietf-quic-transport-32 (work in progress), October 2020. Dawkins Expires June 9, 2021 [Page 11] Internet-Draft QUIC Multipath Questions December 2020 [MASQUE-charter] "IETF MASQUE Working Group Charter", n.d., . [QUIC-charter] "IETF QUIC Working Group Charter", n.d., . [QUIC-IETF-109-minutes] "IETF QUIC Working Group IETF 109 Meeting - November 2020 - Minutes", November 2020, . [QUIC-interim-20-10] "IETF QUIC Working Group Virtual Interim Meeting - October 2020", October 2020, . [QUIC-interim-20-10-minutes] "IETF QUIC Working Group Virtual Interim Meeting - October 2020 - Minutes", October 2020, . [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, DOI 10.17487/RFC0793, September 1981, . [RFC4960] Stewart, R., Ed., "Stream Control Transmission Protocol", RFC 4960, DOI 10.17487/RFC4960, September 2007, . [RFC5061] Stewart, R., Xie, Q., Tuexen, M., Maruyama, S., and M. Kozuka, "Stream Control Transmission Protocol (SCTP) Dynamic Address Reconfiguration", RFC 5061, DOI 10.17487/RFC5061, September 2007, . [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext Transfer Protocol Version 2 (HTTP/2)", RFC 7540, DOI 10.17487/RFC7540, May 2015, . [RFC8684] Ford, A., Raiciu, C., Handley, M., Bonaventure, O., and C. Paasch, "TCP Extensions for Multipath Operation with Multiple Addresses", RFC 8684, DOI 10.17487/RFC8684, March 2020, . Dawkins Expires June 9, 2021 [Page 12] Internet-Draft QUIC Multipath Questions December 2020 [Udana-6-4] "Udana 6:4 Sectarians (1) (Tittha Sutta)", December 2020, . Author's Address Spencer Dawkins (editor) Tencent America Email: spencerdawkins.ietf@gmail.com Dawkins Expires June 9, 2021 [Page 13]