INTERNET-DRAFT M. Cullen Intended Status: Informational Painless Security N. Leymann M. Boucadair C. Heidemann C. Jacquenet Deutsche Telekom AG France Telecom M. Zhang B. Sarikaya Huawei Expires: April 21, 2016 October 19, 2015 Problem Statement: Bandwidth Aggregation for Internet Access draft-zhang-banana-problem-statement-00.txt Abstract Bandwidth aggregation for Internet access can increase the access bandwidth and reliability. It has become a desirable network function, especially for those Service Providers who operates multiple heterogeneous networks. This document describes the issues with bandwidth aggregation for Internet access. The required network functions to be supported are specified. Several candidate solutions had been considered to support bandwidth aggregation for Internet access in IETF. These techniques are investigated. Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html BANANA Expires April 21, 2016 [Page 1] INTERNET-DRAFT Problem Statement October 19, 2015 Copyright and License Notice Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Acronyms and Terminology . . . . . . . . . . . . . . . . . . . 4 3. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.1. Per-packet Traffic Distribution . . . . . . . . . . . . . . 4 3.2. Per-flow Traffic Distribution . . . . . . . . . . . . . . . 4 4. Generic Reference Model . . . . . . . . . . . . . . . . . . . . 4 5. Problem Areas . . . . . . . . . . . . . . . . . . . . . . . . . 5 5.1. Addressing . . . . . . . . . . . . . . . . . . . . . . . . 5 5.2. Traffic Classification . . . . . . . . . . . . . . . . . . 5 5.3. Traffic Distribution . . . . . . . . . . . . . . . . . . . 5 5.4. Traffic Recombination . . . . . . . . . . . . . . . . . . . 6 5.5. Bypassing . . . . . . . . . . . . . . . . . . . . . . . . . 6 5.6. Measurement . . . . . . . . . . . . . . . . . . . . . . . . 7 5.7. Policy Control . . . . . . . . . . . . . . . . . . . . . . 7 6. Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 8 7. Related IETF Work . . . . . . . . . . . . . . . . . . . . . . . 9 7.1. GRE Tunnel Bonding . . . . . . . . . . . . . . . . . . . . 9 7.2. LISP . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 7.3. Mobile IP . . . . . . . . . . . . . . . . . . . . . . . . . 10 7.4. Multipath TCP . . . . . . . . . . . . . . . . . . . . . . . 10 7.5. Network Based Layer-2 Tunneling . . . . . . . . . . . . . . 11 8. Security Considerations . . . . . . . . . . . . . . . . . . . . 11 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 11 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11 10.1. Normative References . . . . . . . . . . . . . . . . . . . 11 10.2. Informative References . . . . . . . . . . . . . . . . . . 11 Appendix A. Additional Requirements . . . . . . . . . . . . . . . 12 Author's Addresses . . . . . . . . . . . . . . . . . . . . . . . . 14 BANANA Expires April 21, 2016 [Page 2] INTERNET-DRAFT Problem Statement October 19, 2015 1. Introduction Bandwidth aggregation across multiple Internet access links (a.k.a., the bonding service) provides several useful features. The working text [WT-348] being developed by Broadband Forum has documented several use cases of the bonding service: Service Providers may use the bonding service to offer customers with increased access bandwidth and higher access reliability; Service Providers may accelerate the service turn-up of a primary access (e.g., a link of Digital Subscriber Line) with a longer lead time by providing an additional access (e.g., a link of Long Term Evolution) which has a short lead time. Although host based bonding service is possible, this document restricts the scope to be network based only. Also, a bonding service has to be operated by a single provider. Host based or multi-provider cases can be discussed here but will be standardized in other places, such as the MIF Working Group. This document is meant to capture the common problems that are addressed by several ongoing initiatives. The goal is to identify commonalities among them and derive functional requirements. It is not clear at this stage whether one or multiple solutions may be required to accommodate various deployment contexts. Typically, this is one of the inputs that are expected from the IETF community. Alternatively, a common framework can be sketched, including required functional modules and interactions. The various solution proposals (e.g., GRE, LISP, MIP, MPTCP) can be viewed as applicability assessments of the framework. To support the bonding service, the problems to be addressed includes: addresses, traffic classification, distribution and recombination, bypassing, measurement, and policy control. To address the problems, the work should be done in IETF includes: - to determine and further detail the tunneling technology to be used; - to specify the sole encapsulation format; - to develop a common control plane; - to define or suggest the algorithms to be used in packet reordering, flow control and congestion control. Several groups have developed, and in some cases deployed, bandwidth aggregation solutions based on existing IETF technologies. These technologies are investigated in this document. Note that solutions are listed in an alphabetic order. No preference order should be assumed by the reader. BANANA Expires April 21, 2016 [Page 3] INTERNET-DRAFT Problem Statement October 19, 2015 2. Acronyms and Terminology Bonding Service: An alias for Bandwidth Aggregation for Internet Access in this document. ACC: Shorted for ACCess equipment. ACC connects user's end station or network to the bonding service. AGG: Shorted for AGGregation equipment. AGG faces to the Internet while aggregates and terminates the bonding service from ACCs. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 3. Use Cases Depends on the traffic distribution techniques, there are two kinds of use cases for the bonding service. 3.1. Per-packet Traffic Distribution For residential users, there are few number of applications to be transmitted over the bonded connections. Per-flow load balance techniques might not be able to achieve a fine grained load distribution, so that the per-packet load sharing is necessary. 3.2. Per-flow Traffic Distribution Per-flow load balancing methods are widely used in IP networks. These techniques may be used to implement the bonding service. The per-flow traffic distribution is suitable for enterprise users who usually owns a dense number of applications. The load balancing among the bonded connections could be fine grained. 4. Generic Reference Model +-+ +-+ | +----- bonding connection ----+ | |A| |A| | | | | |C|if ---- sub-connection ----if|G| | | . . . | | |C|if ---- sub-connection ----if|G| | | | | +-+ +-+ Figure 4.1: Reference model of the bonding service BANANA Expires April 21, 2016 [Page 4] INTERNET-DRAFT Problem Statement October 19, 2015 Customers access the Internet using the bonding service which comprises of several key component functions as shown in Figure 4.1: the access peer (shorted as ACC), the aggregation peer (shorted as AGG), the bonding connection between the two peers and the sub- connections that logically make up the bonding connection. Customers can only sense the bonding connection while the internal sub- connection between ACC and AGG are invisible to outside. The bonding connection works at layer 3. The sub-connections usually works at layer 3 but could work at layer 2. 5. Problem Areas The following subsections list the problems that need to be solved in the bonding service solutions. 5.1. Addressing ACC and AGG need allocate addresses to the interfaces attached to each sub-connection and the whole bonding connection. IPv4, IPv6 or dual-stack operation should be supported. Sub-connections can be internally demultiplexed by their interface addresses. When the bonding service is bootstrapped, these connection addresses are discovered at the other end through the control protocol. The connection addresses of ACC may be assigned by AGG, also through the control plane. 5.2. Traffic Classification Classification happens before the received packets are distributed to the connections. ACC and AGG MUST support the classification function that classify a packet into a class that is to be further processed by the traffic distribution function. The classification can be based on the IP address, application type, etc. ACC and AGG MUST support their classification function to be controlled by the policy from providers or customers. 5.3. Traffic Distribution For the traffic that is to be distributed into multiple sub- connections, by default it is distributed equally to the sub- connections according to their available bandwidth. Unequal load balancing should be supported as well. Traffic may be distributed to the connections in proportion to the available bandwidth of the sub- connections. Traffic may also be split in a way that one sub- connection is saturated first and then another sub-connection is used. For the per-flow use case, the load can be distributed using hashing BANANA Expires April 21, 2016 [Page 5] INTERNET-DRAFT Problem Statement October 19, 2015 methods. For the per-packet use case, the coloring mechanism specified in [RFC2698] can be used to classify customer's IP packets, both upstream and downstream, into the sub-connections. For example, packets colored as green is distributed into one sub-connection and packets colored as yellow is distributed into the other sub- connection. For the scenario that requires more than two sub- connections, multiple levels of token buckets might be realized. 5.4. Traffic Recombination For the per-packet use case, the recombination function at the receiver provides the in-order delivery of customers' traffic. The sender need to mark each packet with a sequence number. The sequence number MUST be set per the whole bandwidth aggregation rather than per sub-connection so all packets carried on these sub-connections actually share the same buffer. The receiver reorder the out-of- sequence data packets in the buffer by the packets' sequence number. The maximum allowed size of the buffer and the maximum time that a packet can stay in the buffer is up to implementations. For the per-flow use case, an acknowledge number field appear in the packets in order to integrate the congestion and flow control into the traffic recombination. In order to support such control, the sender need also maintain a sending buffer. 5.5. Bypassing Service Providers may require some traffic to bypass the Traffic Distribution function. For example, some delay sensitive applications such as IPTV carried over a lossy sub-connection would impair customers' Quality of Experience. Service providers would require such applications transmitted only on the wired sub-connection when the aggregation is a mix of wired and wireless bonded sub- connections. There are two types of bypassing: the bypassing traffic are transmitted on a sub-connection out of all the sub-connections between ACC and AGG; the bypassing traffic is still transmitted on a sub-connection between ACC and AGG, just that the occupied bandwidth of the bypassing traffic on this sub-connection can not used for bandwidth aggregation anymore. In either case, the bypassing traffic does not benefit from the Bandwidth Aggregation. ACC and AGG need timely exchange information about bypassing, such as the application types that need be handled by the bypassing function, the bandwidth occupied by the bypassing traffic (See also Section 5.6). BANANA Expires April 21, 2016 [Page 6] INTERNET-DRAFT Problem Statement October 19, 2015 5.6. Measurement ACC and AGG need monitor and exchange the performance parameters of the sub-connections between them. The following parameters fall into the problem space. - Operating state: The operating state has to be measured by out-of- band control messages. When a sub-connection is detected to be failed, this sub-connection has to be removed from the bonding connection. - End-to-end delay and delay variation: The measured result of this parameter are used in flow control and congestions control algorithms for the per-flow distribution. The per-packet distribution may make use of this measured parameter as well. For example, when the delay difference of two sub-connection exceeds a pre-defined threshold, the slower one can be suspended. The probing packet could be piggy-backed by data packets or could be carried by out-of-band control messages. - Maximum sub-connection bandwidth: This parameter may be used to determine the amount of traffic that can be distributed to each sub-connection. - Bypassing bandwidth: This parameter has to be timely monitored. Usually, this is measured for the opposite end to figure out the available sending bandwidth. For example, the ACC report the downward bypassing bandwidth for a sub-connection so that the AGG can calculate the remained available downward bandwidth of this sub-connection. 5.7. Policy Control Operators and customers may control the bonding service with policies. These policies will be interpreted into parameters or actions that impact traffic classification, distribution, combination, measurement and bypassing. The following policy control examples are in the problem space of this document. - Operators may provide the parameters or filter list used by the traffic classification function. - Operators may control the traffic distribution to be done either in a per-flow manner or a per-packet manner. - Operators may determine the maximum allowed size (See MAX_PERFLOW_BUFFER in [RFC2890]) of the buffer for reordering. They may also define the maximum time (See OUTOFORDER_TIMER in BANANA Expires April 21, 2016 [Page 7] INTERNET-DRAFT Problem Statement October 19, 2015 [RFC2890]) that a packet can stay in the buffer for reordering. These parameters impact the traffic recombination. - Operators may specify the interval for sending detecting messages and the retry times for ACC or AGG to send out detecting messages before it can declare a sub-connection failure. Operators may specify the allowed threshold of the delay difference for two sub- connections. - Operators and customers may specify the service types bypassing the traffic distribution function. 6. Requirements Requirements for the bonding service are considered in this section. Also, some additional requirements are listed for discussion in the Appendix. The solution MUST apply for both IPv4 and IPv6. Host based solutions are out of the scope, therefore - the solution MUST NOT require any specific function at the terminal's side. In the bonding service, flapping the flows among various interfaces may have a negative impact on the customers experience (e.g., hazardous log outs, broken HTTPS associations, etc.). The solution should be carefully designed, so that - activating the solution MUST NOT impact the stability, availability of the services delivered to the customer compared to customers who are serviced with traditional non-bonding service. "Roles" should be assigned to each supported network interface type (e.g., fixed or mobile access). This role is assigned by the network operator (e.g., fixed access can be set as the "primary"). A default setting can be considered. - Setting of the role of the sub-connections (primary or backup) SHOULD NOT be changed by the user. The solution should provide Service Providers with means to enforce policy control of the bonding service. For example, - the solution MUST allow to rate limit the traffic on a per interface basis. BANANA Expires April 21, 2016 [Page 8] INTERNET-DRAFT Problem Statement October 19, 2015 - the solution MUST support means to enable/disable activating multiple interfaces at the CPE ACC side. - the solution MUST support means to instruct the CPEACC device about the logic for mounting interfaces. - the solution MUST support means to bind a given traffic to a subset of interfaces. For the sake of policy enforcement or analytics at the network side, - the solution MAY ease correlating flows, conveyed over multiple access networks, that belong to the same customer. Some services such as VoIP may be available over all active interfaces, but distinct logins and credentials may be used. - The CPE SHOULD be provided with clear instructions about which account to use to place outgoing sessions. For the sake of simplicity, it is RECOMMENDED to use the login/credentials that are independent of the underlying access network used to access the service. 7. Related IETF Work The bonding service has been considered to be supported in several ways. Subsections of this section list the related work that fall into the scope of IETF. In the future, IETF may adopt one direction to work on. However, the sub-functions developed in other directions may be integrated into the adopted direction. 7.1. GRE Tunnel Bonding GRE Tunnel Bonding [GRE-HA] uses per-packet traffic distribution to achieve a fine-grained load sharing among the sub-connections. The receiver may receive out-of-sequence packets so that reordering function is provided. Users' IP packets are encapsulated in the GRE header which in turn encapsulated in an outer IP header and transported on the sub-connections. The Sequence Number field of the GRE header is used to number the packets at the sender while the receiver makes use of this sequence number to reorder the packets. A clean-slate control plane is defined. Control messages are transported in the same GRE tunnels that are used to transport data packets. The control messages and data packets are distinguished by the GRE Protocol Type. GRE tunnel bonding has been implemented and deployed. Flow and BANANA Expires April 21, 2016 [Page 9] INTERNET-DRAFT Problem Statement October 19, 2015 congestion control could be supported within the tunnel through extending the GRE header, though it is currently out of the scope. 7.2. LISP LISP has the basic capability to support the bonding service [LISP- HA] [ILNP]. LISP used to do traffic engineering based on static load balancing that is not agnostic to link characteristics. If the LISP architecture takes on the bonding service, performance of the sub- connections, such as their available bandwidth, end-to-end delay and packet loss, need to be measured, though they are not supported yet. To achieve such kind of dynamic flow based load balancing, some technique issues, such path symmetry and route oscillation, need to be considered and addressed. Packet based traffic distribution has been considered in [LISP-HA] as well. Detail specification of the reordering mechanism based on a "Reorder" flag is left as future work. 7.3. Mobile IP [MIP-HA] investigates how to support the bonding service by using IP mobility protocols. By treating the bonding service as a special scenario of IP mobility, some mechanisms (such as redundancy and load balancing) that have already been defined in the IP mobility protocols could be reused. At the same time, however, the IP mobility protocols have to be tailored in order to reduce the deployment complexity. A new multipath binding option is proposed as an extension of PMIPv6 in [MIP-HA]. This option can be used to exchange the binding information, such as the Access Technology Type [RFC5213], the Interface Label and Binding ID, between the peers. Currently, per flow traffic management is well supported by IP mobility protocols (such as [RFC6088] and [RFC6089]) while packet based traffic distribution is left as future work. 7.4. Multipath TCP Multipath TCP (MPTCP) had been considered as a candidate technology to support the bonding service. There are some implementations and deployments. MPTCP provides the ability to simultaneously use the sub-connections between the ACC and AGG peers. MPTCP "subflows" would be created, one per sub-connection, and are combined with the existing TCP session, which continues to appear as a single connection to the applications at both ends [RFC6824]. BANANA Expires April 21, 2016 [Page 10] INTERNET-DRAFT Problem Statement October 19, 2015 MPTCP operates at the transport layer. The MPTCP protocol is specified to manage the state of subflows. [MPTCP-HA] addresses the deployment concerns and specifies the extension to transport UDP datagrams in MPTCP packets. The UDP traffic can be distributed among the sub-connections using the MPTCP features, which are traditionally not supported by MPTCP. Currently, MPTCP only supports per-flow traffic distribution. Traffic is distributed among these subflows using the flow based 5-tuple demultiplexing [RFC6824]. In the future, MPTCP might be extended to support packet based traffic distribution. 7.5. Network Based Layer-2 Tunneling Network Based Layer-2 Tunneling assigns a single IP address at each peer for the bonding connection. Layer-2 tunnels are set up per sub- connections. Layer 2 load balancing techniques, such as Link Aggregation [802.1AX] can be used to achieve the traffic distribution function. Per-flow load balance can be well supported by various of implementations. Packet based distribution might be supported as well. However, per-packet distribution may cause the packets to be received as out-of-sequence, which is an issue remained to be addressed by the Network Based Layer-2 Tunneling. 8. Security Considerations This document describes the problem space and does not introduce any security risks. However, security issues should be considered by solutions that address this problem space. 9. IANA Considerations No IANA action is required in this document. RFC Editor: please remove this section before publication. 10. References 10.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2689] Bormann, C., "Providing Integrated Services over Low- bitrate Links", RFC 2689, September 1999. [RFC2890] Dommety, G., "Key and Sequence Number Extensions to GRE", RFC 2890, September 2000. 10.2. Informative References BANANA Expires April 21, 2016 [Page 11] INTERNET-DRAFT Problem Statement October 19, 2015 [WT-348] Broadband Forum Work on "Hybrid Access for Broadband Networks" (WT-348), October 21, 2014, . [GRE-HA] N. Leymann, C. Heidemann, M. Zhang, et al, "GRE Tunnel Bonding", draft-zhang-gre-tunnel-bonding, work in progress. [RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure, "TCP Extensions for Multipath Operation with Multiple Addresses", RFC 6824, January 2013. [MPTCP-HA] M. Boucadair and C. Jacquenet, "An MPTCP Option for Network-Assisted MPTCP Deployments: Plain Transport Mode", draft-boucadair-mptcp-plain-mode, work in progress. [MIP-HA] P. Seite, A. Yegin and S. Gundavelli, "Multihoming support for Residential Gateways", draft-seite-dmm-rg-multihoming, work in progress. [RFC5213] Gundavelli, S., Ed., Leung, K., Devarapalli, V., Chowdhury, K., and B. Patil, "Proxy Mobile IPv6", RFC 5213, August 2008. [RFC6088] Tsirtsis, G., Giarreta, G., Soliman, H., and N. Montavont, "Traffic Selectors for Flow Bindings", RFC 6088, January 2011. [RFC6089] Tsirtsis, G., Soliman, H., Montavont, N., Giaretta, G., and K. Kuladinithi, "Flow Bindings in Mobile IPv6 and Network Mobility (NEMO) Basic Support", RFC 6089, January 2011. [802.1AX] IEEE, "IEEE Standard for Local and Metropolitan Area Networks - Link Aggregation", 802.1AX-2014, 24 December 2014. [LISP-HA] M. Menth, A. Stockmayer and M. Schmidt, "LISP Hybrid Access", draft-menth-lisp-ha, work in progress. [ILNP] "ILNP - Identifier-Locator Network Protocol", online available: http://ilnp.cs.st-andrews.ac.uk/ Appendix A. Additional Requirements The following requirements are listed as record and may subject to change. - The solution MUST be valid for any kinds of interfaces that need to be aggregated. No dependency to the underlying media should BANANA Expires April 21, 2016 [Page 12] INTERNET-DRAFT Problem Statement October 19, 2015 be assumed. - The solution MUST comply with servers policy regarding IP addresses that are bound to (HTTP session) cookies. - The solution MUST NOT break TLS associations. - Activating the solution MUST NOT have negative impacts on the service usability (including the ACC management). - Service degradation MUST NOT be observed when enabling the solution. - Enabling the solution MUST increase the serviceability of the ACC. In particular, the solution MUST allow for the ACC to always establish a network attachment when the primary connectivity is out of service. - The solution SHOULD NOT alter any mechanism, to aggregate available resources or to ensure a service continuity among multiple access points, that is supported by end-devices connected to the ACC. - The ACC MUST bind the DNS server(s) discovered during the network attachment phase to the interface from which the information was received. - The ACC MUST bind the service information (e.g., SIP Proxy Server) discovered during the network attachment phase to the interface from which the information was received. - When sending the traffic via a given interface, the ACC MUST use as source address an address (or an address from a prefix) that was assigned for that interface. - For protocols such as RTP/RTCP, the same IP address MUST be used for both RTP and RTCP sessions. - Dedicated tools SHOULD be provided to the customer to assess the aggregated capacity (instead of link-specific). This can be included as part of the ACC UI, a dedicated portal, etc. BANANA Expires April 21, 2016 [Page 13] INTERNET-DRAFT Problem Statement October 19, 2015 Author's Addresses Margaret Cullen Painless Security 14 Summer St. Suite 202 Malden, MA 02148 USA EMail: margaret@painless-security.com Nicolai Leymann Deutsche Telekom AG Winterfeldtstrasse 21-27 Berlin 10781 Germany Phone: +49-170-2275345 EMail: n.leymann@telekom.de Cornelius Heidemann Deutsche Telekom AG Heinrich-Hertz-Strasse 3-7 Darmstadt 64295 Germany Phone: +4961515812721 EMail: heidemannc@telekom.de Mohamed Boucadair France Telecom Rennes 35000 France EMail: mohamed.boucadair@orange.com Christian Jacquenet France Telecom Rennes France EMail: christian.jacquenet@orange.com Mingui Zhang BANANA Expires April 21, 2016 [Page 14] INTERNET-DRAFT Problem Statement October 19, 2015 Huawei Technologies No.156 Beiqing Rd. Haidian District, Beijing 100095 P.R. China EMail: zhangmingui@huawei.com Behcet Sarikaya Huawei USA 5340 Legacy Dr. Building 3 Plano, TX 75024 EMail: sarikaya@ieee.org BANANA Expires April 21, 2016 [Page 15]