Internet Draft Document Marc Lasserre Layer 2 VPN Working Group Xipeng Xiao draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Riverstone Networks Yetik Serbest Cesar Garrido SBC Telefonica Marc Rapoport Completel Expires: Jan. 2005 Jul. 2004 VPLS Applicability draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract Virtual Private LAN Service (VPLS) is a layer 2 VPN service that provides multipoint connectivity in the form of an Ethernet emulated LAN, while usual L2 VPN services are typically point-to-point. Such draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 2] emulated LANs can span across metropolitan area networks as well as wide area networks. [VPLS-LDP] defines a method for signaling MPLS connections between member PEs of a VPN and a method for forwarding Ethernet frames over such connections. This document describes the applicability of such procedures to provide VPLS services. This document also compares the characteristics of this solution against the requirements specified in [L2VPN-REQ]. In summary, there are no architectural limitations to prevent the requirements from being met. But meeting certain requirements (e.g. QoS) is beyond the specification of [VPLS-LDP], and requires careful planning and precise implementation of the SPs. This document attempts to capture such issues, present the potential solutions to these issues, and discuss the pros and cons of each alternative. This document does not cover the applicability of [VPLS-BGP]. Conventions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 RELATED DOCUMENTS www.ietf.org/internet-drafts/draft-ietf-l2vpn-vpls-ldp-03.txt www.ietf.org/internet-drafts/draft-ietf-l3vpn-applicability- guidelines-00.txt Table of Contents 1. VPLS Overview..........................................4 2. Operation of Control And Data Planes.......................4 2.1. Control Plane.......................................4 2.1.1. Signaling.......................................5 2.2. Data Plane..........................................5 2.2.1. Ingress Processing................................5 2.2.2. Egress Processing................................5 2.2.3. Intermediate Node Processing.......................6 3. VPLS vs. Alternative Approaches...........................6 3.1. Ethernet Switching...................................6 3.2. BGP VPN............................................6 4. Provisioning...........................................6 4.1. PE Auto-Discovery....................................7 4.2. Other Related Provisioning............................7 5. Migration Impacts.......................................8 5.1. Interconnecting Existing L2 Ethernet Islands with a VPLS Core8 5.2. Migrating an Existing L2 Ethernet Core to a VPLS Core......9 5.3. Interconnecting a new VPLS Network with Existing ATM/FR Networks...............................................10 5.4. Adding VPLS Support to an IP Routed Network.............10 draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 3] 6. Multi-homing..........................................10 7. Loop Prevention........................................12 8. Packet Ordering........................................13 9. Multi-Domain VPLS Service...............................13 10. Maximum Transmission Unit (MTU) Issues...................13 11. Interoperability and Interworking.......................14 11.1. Interworking with BGP VPN..........................14 11.2. Interworking With Frame Relay & ATM Attachment Circuits.14 12. Quality of Service....................................14 13. Security............................................15 13.1. Customer Access Control and Authentication............15 13.2. Traffic Separation between VPLS Instances.............15 13.3. Protection of SP Networks..........................16 13.4. Protection of User Data............................16 14. Scalability.........................................17 14.1. Mesh topology....................................17 14.2. Signaling........................................17 14.3. MAC addresses and MAC learning......................17 14.4. Packet replication................................17 14.5. Broadcast limiting................................18 14.6. Multicast........................................18 15. Management..........................................18 15.1. VPLS OAM.........................................18 16. Acknowledgments......................................19 17. References..........................................19 18. Authors' Addresses....................................20 Intellectual Property Considerations This document is being submitted for use in IETF standards discussions. Full Copyright Statement Copyright (C) The Internet Society (2001). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 4] TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 1. VPLS Overview The primary motivation behind Virtual Private LAN Services (VPLS) is to provide connectivity between geographically dispersed customer sites across MAN/WAN network(s), as if they were connected using a LAN. The intended applications for the end-user can be divided into the following two categories: - Connectivity between customer routers - Connectivity between customer Ethernet switches In addition, VPLS can also be used by the service provider to deliver services (e.g. triple play) to connected end-users. Unlike L3 VPNs such as BGP VPNs where traffic exchanged between customers and service providers must be IP, VPLS only requires traffic to be Ethernet over which any protocol can be used, e.g. Netbios or IPX. The Service Provider Network is a packet switched network (PSN). The PEs are assumed to be fully meshed with transport tunnels over which customer frames that belong to a specific VPLS instance are encapsulated and forwarded. IP-in-IP, L2TPv3, GRE, and MPLS are examples of transport tunnels. Specific labels used to identify end-to-end paths over such tunnel LSPs are established via targeted LDP [VPLS-LDP]. These LSPs are known as pseudo-wires (PWs). VPLS defines the bridging rules required for PEs to provide an emulated Ethernet LAN service. In particular it defines how a loop- free topology must be built and the forwarding rules between PEs, along with the signaling method to set up PWs between PEs. The resulting service provides a unique broadcast domain per VPN, with the ability to send unicast, multicast and broadcast traffic (as well as flooding of unknown unicast traffic). 2. Operation of Control And Data Planes 2.1. Control Plane draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 5] 2.1.1. Signaling As with [PWE3-ETHERNET], [VPLS-LDP] specifies the use of targeted LDP for the signaling of PWs. PWs are established between PEs that are part of the same VPLS instance. 2.2. Data Plane 2.2.1. Ingress Processing VPLS provides an Ethernet emulated LAN service and hence customer frames are encapsulated as Ethernet frames (Ethernet DIX or 802.1). Note that such Ethernet frames can be carried over various access transport technologies (Frame Relay, ATM, etc). Ingress PEs will determine which Forwarding Information Base (FIB) to look up based on the port, VLAN or port/VLAN combination where frames come from. This port to FIB mapping is performed at provisioning time. The destination MAC address is then looked up to determine on which PW this address has been learned from. If the lookup fails, i.e. if this MAC address has not been learned yet, the frame needs to be sent on all the PWs that are part of the corresponding VPLS instance. If the address is known, the frame is sent only over the associated PW. Before actually transmitting the customer frame, it needs to be encapsulated as defined in [PWE3-ETHERNET], and is further encapsulated with the appropriate transport header (e.g. MPLS or GRE). 2.2.2. Egress Processing Once the tunnel header has been removed, the egress PE determines from the PW label which FIB to look up to determine the egress port, VLAN or port/VLAN combination. The original Ethernet frame is then encapsulated with the proper transmission header if necessary (e.g. Frame Relay header) and sent over the corresponding port. MAC addresses are learned dynamically as traffic is exchanged. New source MAC addresses are learned on a per PW label per VPLS service instance basis. An aging timer is used to remove such bindings after a period of time. When user topology changes occur, MAC withdrawal messages in the signaling plane may be used to unlearn MAC addresses to improve convergence time. Egress PEs might also be configured to perform specific egress encapsulation functions (e.g. VLAN translation). draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 6] 2.2.3. Intermediate Node Processing Intermediate nodes (P routers) only act as pure forwarders based on the outer tunnel header. Hence, they do not participate in any VPLS- related processing. Only PE routers maintain VPN specific information. This improves the scalability of VPLS service. 3. VPLS vs. Alternative Approaches 3.1. Ethernet Switching Ethernet can be used to provide multipoint connectivity within small geographical areas such as small metropolitan networks. Pure Ethernet based solutions have scalability issues (e.g. STP limitations, 4095 VLAN limitations). Some enhancements such as QinQ, STP extensions (RSTP, MSTP) provide additional scalability. VPLS overcomes several of Ethernet based solutions by supporting large numbers of VPNs, better traffic engineering, and better quality of service. It is not uncommon for VPLS networks to be complemented with Ethernet switched networks as an aggregation layer. 3.2. BGP VPN In metropolitan area networks (MANs), BGP is usually not enabled. MANs provide a transport service to end-users. When multiple sites need to be connected within a metro, VPLS offers the appropriate multipoint transport solution. When multipoint connectivity is required across wide area networks such as national backbones, BGP VPNs can be more appropriate. Section 11.1 describes how VPLS and BGP VPNs can be complementary. The following sections compare the characteristics of LDP-based VPLS solution against the requirements specified in [L2VPN-REQ]. Key deployment issues that require careful planning and precise implementation of SPs are highlighted. 4. Provisioning To provision a VPLS service for a customer, the first step is to create a VSI, and assign the customer access links (e.g. port, draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 7] port/VLAN, ATM VC with 1483b encapsulation, etc.) and PWs (including H-VPLS spokes) to it. The PWs interconnect VSIs at different PEs and MTUs together to form an emulated LAN for the customer. One challenge in doing this is, when a VPLS site needs to be added or removed at a PE, in addition to configuring that particular PE, the network operator needs to find out which other PEs participate in that VPLS instance, and re-configure those PEs. PE auto-discovery can automate this process. The pros and cons of several auto-discovery approaches are discussed in 4.1. 4.1. PE Auto-Discovery Currently there are several proposals for PE auto-discovery: the BGP- based approach [VPLS-BGP], the RADIUS-based approach [RADIUS-DIS], and the Provisioning System-based approach. The BGP and RADIUS-based approaches mandate the use of BGP or RADIUS in every PE, and rely on it to propagate the information of which PEs participate in a VPLS instance (Signaling can automatically happen after the other PEs belonging to the same VPLS instance are discovered). The pros of both approaches are reduced provisioning work and no need for a provisioning system. The con is BGP/RADIUS has to be in every PE, which may not be the case in reality. With the Provisioning System-based approach, network operators do not configure the PEs. Instead, they specify which PEs participate in which VPLS instances at the Provisioning System. The Provisioning System then translates such service information into PE configuration commands and telnet/ssh to the PEs to execute such commands. Because all information related to every VPLS instance is centralized at the Provisioning System, PE auto-discovery is automatically achieved. To add or remove a PE for a VPLS instance, a network operator simply specifies it at the Provisioning System which will then configure the PEs accordingly. For VPLS deployments that span across multiple domains, because the ASBRs (autonomous system border routers) of other domains can be treated as CEs of the current domain, these auto-discovery approaches can all work in the multi-domain case. However, the built- in scalability mechanism in BGP makes the BGP-based auto-discovery more scalable in this scenario [VPLS-BGP]. 4.2. Other Related Provisioning To meet the service level agreement (SLA) with their customers, SPs also need to provision the following: - Traffic management throughout the network and on customer facing ports in particular draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 8] - Traffic Engineering - Traffic protection (e.g. Fast reroute) - Service management (e.g. SLA measurement, OAM, accounting, billing, etc) Manual provisioning for these tasks can be tedious. A provisioning system is highly desirable. If a provisioning system is used, PE auto-discovery may be integrated into it. 5. Migration Impacts Migration in this document means replacing, or more often, supplementing, an existing metro Ethernet or ATM/Frame Relay network with a VPLS network. There are four likely scenarios: - Interconnecting existing L2 Ethernet islands with a VPLS core - Migrating an existing L2 Ethernet core to a VPLS core; - Interconnecting a new VPLS network with existing ATM/FR networks - Adding VPLS support to an IP routed network Migration impacts may be mitigated through the use of careful planning when building and migrating the network. Also, consideration must be taken when integrating with protocols such as STP/MSTP and how control packets (BPDUs) are handled. In addition, one must also consider ongoing standards efforts within various standards bodies such as the IEEE [802.1ad] and the Metro Ethernet Forum to assess future impact of any changes within the provider network. 5.1. Interconnecting Existing L2 Ethernet Islands with a VPLS Core Today, many existing metro Ethernet networks are relatively small and cover only specific districts in a metro area. Such networks may simply backhaul traffic to a routing backbone and not interconnected at L2. When metro Ethernet service grows and these networks need to be interconnected at L2, one approach that may be used for a migration strategy is to effectively utilize existing L2 (possibly .1Q based or QinQ) networks as ôislandsö attached to an MPLS based VPLS core network. In this particular case, the L2 network uses predetermined Provider .1Q tags (P-tags) to transport a given customers traffic. This P-tag is then utilized as a service delimiter that is then stripped prior to being transported across the MPLS cloud. The service delimiting P-tag is used to identify the VPLS instance to which the traffic should be mapped. ----CE1 ------- ------- / -------- CE2- / \ / PE1 / \ draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 9] \ / \ / \ / \ ---| QinQ \ / MPLS/ \ / QinQ | | Domain PE VPLS PE Domain | \ / \ Domain / \ /\ \ / \ / \ / \ ------- ---------- -------- --CE3 In this scenario, one issue that SPs needs to address is that different sites of a customer may have a mismatch of.1Q tags. For example, customer connectivity at one site will be tied to a port on a VPLS PE/MTU that will utilize a PW for tunneling this packet through the network. Customer connectivity at another site will be interconnected to a port on a QinQ switch that will utilize QinQ techniques (see the figure below). By mandating that ôthe Ethernet packet that traverses a VPLS is always a customer Ethernet packetö, the [VPLS-LDP] solution naturally accommodates this need. ----- / A1 \ ---- ----CE1 | / \ ------- ------- / | | | A2 CE2- / \ / PE1 \ / \ / \ / \ / \ ----- ---- ---| QinQ \ / MPLS/ | | Domain PE2 VPLS | \ / \ Domain / ----- \ / \ / |QinQ|_/ ------- ------- -| | ---- / ------ ---- / \/ \ / \ CE = Customer Edge Router | A3 CE3 --C4 A4 | PE = Provider Edge Router \ / \ / ---- ---- 5.2. Migrating an Existing L2 Ethernet Core to a VPLS Core Providers that have already deployed VLAN based core may choose to overlay an MPLS edge on top of this existing L2 domain. In this method, provider .1q tags maybe assigned to MPLS backbone links that are then used for carrying VPLS traffic. While this approach may allow for a simple transition to solve some immediate deficiencies of a pure L2 network, it still does not solve some of the underlying problems associated with protocols such as spanning tree. In this case, although MPLS may provide some scaling advantages, the limitations associated with spanning tree can still pose potential problems to the overall infrastructure. CE1 ------------------- ------ / draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 10] / \ -|VPLS| / / \ / | PE |- / \ ------ / \ | 802.1Q/ | | QinQ | \ / ----- \ /\ ------ |VPLS|_/ \ / \ |VPLS| -| PE | \ / -| PE |- / ------ ------------------- ------ \ / \ \ CE3 --CE4 CE2 Alternatively, a parallel VPLS core is built and connected to the existing 802.1Q/Q-in-Q core. The 802.1Q/Q-in-Q core is effectively treated as a super-island. Then one by one, each individual Ethernet access island is disconnected from the existing core (i.e. super- island) and connected to the VPLS core. The migration issues then become similar to those described in 5.1. 5.3. Interconnecting a new VPLS Network with Existing ATM/FR Networks If interworking at L2 is needed, the existing ATM/FR networks would need to carry bridge-encapsulated traffic. VPLS can support ATM and Frame Relay (FR) attachment circuits with Ethernet bridge encapsulation. Once the FR/ATM encapsulation has been stripped off, the resulting Ethernet frames can be processed as if they came from an Ethernet link. Therefore, interworking can be naturally achieved. If the existing ATM/FR networks do not carry bridge-encapsulated traffic, then interworking can only happen at L3. For example, if both VPLS and ATM/FR carry IP traffic, then an IP router can be used to interconnect the two networks. 5.4. Adding VPLS Support to an IP Routed Network In such a scenario, if existing PEs can support VPLS, then they can continue to serve as PEs. Otherwise, new VPLS PEs need to be added and existing IP routers will serve as Ps. Depending on whether the existing IP routers support MPLS or not, MPLS or some other tunneling mechanism such as GRE can be used. 6. Multi-homing Multihoming is necessary in order to remove a VPLS PE as a single draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 11] point of failure for all devices attached to it. There are two instances of multihoming that apply to VPLS: 1. When a CE device is connected to more than one PE, 2. In the case of hierarchical VPLS - when an MTU-s device is connected to more than one PE-rs. In both of these cases, the concern is that a particular MAC address will appear as a source on more than one PE device, causing other PE devices to continuously change their FIBs with regard to the true location of the MAC. This will cause constant table thrashing on the remote PEs, a behavior akin to a Layer 2 switch which participates in a loop. It is therefore required that any Layer 2 loops, created by multihoming of a CE or an MTU-s, be resolved within the group of devices participating in that loop. This group includes the multihomed CE or MTU-s, and all PEs to which it is attached. The PEs involved in such a loop are connected with a full mesh of pseudowires per VPLS instance. There are two approaches to resolving the loops created by the multihomed devices: 1. Running an MSTP instance between all devices in the group. In this case, the PEs within the group will need to utilize a P-VLAN for the purposes of running MSTP in the group. This P-VLAN can be re-used on non-overlapping groups of multihomed CE (or MTU-s) and its PEs. It must be clear that the MSTP process discussed here is a completely different and independent instance of STP than any STP the customer may be running. Such customer STP is always tunneled through the VPLS network, and is never acted upon by the PE or MTU-s devices. 2. The MTU-s or the CE can designate its link to one of the PEs it connects to as primary, and only send packets for this particular VPLS instance over that link. In this case the MTU-s (CE) is responsible for monitoring the state of that link and for switching to an alternate link if the primary fails. No action is required from the PEs participating in the group, though there should be an indication given from the MTU-s to its connected PEs as to whether the PE is connected to the primary or backup link. This is a very lightweight approach, which is quite useful given the simple and known topology between the CE (MTU-s) and its PEs. With this approach the operator must ensure that pseudowires in the core remain up, as long as the ingress PE they start from is up. This can typically be ensured with MPLS TE tools, such as fast re-route or back-up LSPs. If pseudowires in the core go down while their ingress PE is up and accepting customer traffic, blackholes can occur. In each case, the PE nodes are most likely in two different physical locations in the provider network providing network element protection, last mile protection, fiber diversity and provider draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 12] facility backup. Customer STP traffic is always tunneled through the provider network, and is never acted upon by the PE or MTU-s devices. Lastly, it should be observed that, since VPLS services provide Ethernet switch-like transport level services, the customer is free to connect any device they desire as a CE. This could be anything from a simple host, hub, L2 switch, or a router. The operator has to be cognizant of the different capabilities of each of those devices to ensure loop-free environment when multi-homed. 7. Loop Prevention Loops in the core VPLS network are prevented by creating a full mesh of transport circuits between PEs and by applying a split-horizon rule. The split-horizon approach prevents a frame received from the backbone network from being sent out anything other than the customer facing ports belonging to that VPLS instance on the receiving PE. The frame MUST not be forwarded out other PW connecting the receiving PE to other PEs participating in the VPLS instance. This provides the necessary protection, network bandwidth optimization and scalability in the carriers network as it does not rely on link blocking technologies, like spanning tree type protocols. This forwarding mechanism allows PEs to effectively protect the core network from data loops. Customer networks need to be able to transparently transport the protocol information that allows their network to properly converge. However, the provider should consider loop protection schemes between the CE and PE that do not affect the customer functions. This would be in addition to spanning tree when the PE connects to a VLAN based L2 metro or when the customer is directly connected to multiple PE nodes. Methodologies providers can use to avoid loops when multi-homing CE devices have been discussed in the previous section. Some of these mechanisms involved running STP (or MSTP) between groups of PEs. The provider should look at deploying a loop protection scheme that would intervene automatically when it detects a loop condition. This loop protection scheme serves as an additional line of defence against protocol failures or misconfigurations, which can result in data loops. The concern is that a particular MAC address will appear as a source on more than one PE device, causing other PE devices to continuously update there tables. An external loop protection scheme adds a level of insurance above the customer link protection schemes. Its function is to reduce unnecessary core bandwidth usage when a loop condition occurs in an adjacent network and provide an extra level of protection to multihomed networks. It is a compliment but not a replacement for traditional loop protection mechanisms, like spanning tree. draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 13] With directly connected customers, careful consideration needs to be given to backdoor connections. Backdoor connections provide an alternate path around a single provider. If a loop detection scheme is invoked here the customer may be forced to traverse a link that is not desired. 8. Packet Ordering Normally there is only one transmission path towards a destination with VPLS. So there is no packet re-ordering issue. But if some load sharing mechanism is enabled for traffic inside an LSP, or LSPs carrying VPLS traffic are rerouted, packets may be re-ordered inside the PSN. VPLS data packets use the encapsulation mechanism defined in [PWE3- ETHERNET]. An optional control word which contains a sequence number field can be used to assist in-order delivery. If the userÆs applications are sensitive to packet re-ordering, this option may be used. However, enabling sequencing usually cause forwarding performance degradation. Another alternative is to avoid load sharing for traffic inside a LSP and pin down LSPs to avoid rerouting. 9. Multi-Domain VPLS Service As the use of VPLS grows, it is expected that customers will require a single VPLS service delivered by different providers (e.g. either for redundancy or because none of the SPs has the presence to support all the sites of a customer). Different providers would then need to interconnect their VPLS domains for these customers. [VPLS-LDP] has provision for such a requirement, utilizing a full mesh of LSPs among the VPLS gateways of these domains. However, experience of such interconnection is not yet available. 10. Maximum Transmission Unit (MTU) Issues Because of the encapsulation and transport headers, the MTU for user applications will be smaller than the smallest MTU of all the physical links. In responding to path MTU discovery message, each network device must deduct the total header size from a physical linkÆs MTU. Since path MTU discovery is not always used, SPs must clearly communicate the potential MTU issue to their customers and ask for their cooperation. In reality, most applications will work fine but a small number of them may be affected. This is by no means specific to VPLS. Any networks that put additional header(s) on customerÆs packets will have the same issue. draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 14] 11. Interoperability and Interworking Interoperability should be ensured by proper implementation of the published standards. 11.1. Interworking with BGP VPN When interworking VPLS with BGP VPN, a BGP VPN (in the backbone) is typically used to interconnect VPLS domains in multiple metros. In this type of scenario, the BGP VPN will carry inter-metro traffic whereas VPLS will handle intra-metro traffic. A useful method for interconnecting a VPLS with a BGP VPN is to use a ôlinkö to interconnect the VSI and the VRF. Such a ôlinkö can be a physical port, a VLAN spanning across one or multiple physical hops, or 2 LSPs with one in each direction, etc. Analogously, this is like interconnecting a L2 switch with a router, with the VSI as the switch and the VRF as the router. Access/transport networks such as VPLS can also be interconnected with BGP VPNs using various mechanisms such as CarrierÆs Carrier as defined in [RFC-2547]. 11.2. Interworking With Frame Relay & ATM Attachment Circuits Frame Relay (FR) and ATM attachment circuits with Ethernet bridged encapsulation can be terminated within VPLS PEs. The resulting Ethernet frames (i.e. once the FR/ATM encapsulation has been stripped off) are processed as standard Ethernet frames. In order to support a complete interworking model between FR and Ethernet or between ATM and Ethernet, mapping service profiles and OAM traffic from one to the other are necessary. Additionally, circuit management (e.g. LMI to PW state mapping) between the various technologies are required. Such standards are being defined by other standard organizations such as the MPLS-FR-ATM Alliance. 12. Quality of Service The provision of appropriate QoS capabilities may require any combination of the following: - QoS in the access network. - Admission control by the PE router on the ingress access links. draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 15] - Classification by the PE, for traffic arriving from the CE. Once the PE classifies the user packets, this classification needs to be preserved in the encapsulation (MPLS EXP or IP DSCP) used to send the packet across the backbone. - Traffic conditioning (policing or shaping) by the PE router on the ingress access links. - DSCP/EXP-based queueing and WRED in the VPLS network - Traffic engineering in the VPLS network. - Fast reroute in the VPLS network None of these features are VPLS specific. The ability to support them depends on whether the features are available on the edge and core devices. It is up to the SPs to decide how to use such mechanisms to provide QoS. Such mechanisms can be used to support either the "hose model" or the "pipe model", although the hose model is a more natural fit and is usually the support model by default. 13. Security 13.1. Customer Access Control and Authentication Control of the customer access can be achieved by controlling physical access to the CEs, the PEs and the links between them. If multiple customers use service delimiting tags in the same trunk link to access VPLS service, and the tags are put on by the customers themselves, ACLs should be used to ensure that each customer only puts on the tag that it is supposed to put on - Packets with other tag(s) must be dropped. If the CE device is a router, then 802.1x may be used for CE device authentication. 13.2. Traffic Separation between VPLS Instances VPLS instances maintain separation of broadcast domains between themselves. Traffic entering a given VPLS instance at a given PE device does not, under any circumstances, cross the boundaries of the VPLS into another instance. VPLS devices (PEs and MTU-s) ensure that by maintaining a FIB table on a per-VPLS instance basis. The above statement is correct regardless of the learning mode employed by a particular VPLS instance (qualified or unqualified), or whether or not VLANs are treated as broadcast domain identifiers, or simply as circuit IDs which have no significance in determining the broadcast domain. In either of these cases, the VPLS instance is the outer-most "envelope" which ensures that traffic within it does not "leak" into another VPLS instance. draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 16] 13.3. Protection of SP Networks Two types of DoS attacks are of concern with VPLS: 1. Attacks against VPLS devices 2. Attacks against other devices, for which the VPLS network is a transport. Attacks of the first type are naturally of greater concern for a VPLS operator, because they can destabilize the VPLS network as a whole, and affect multiple customers. The tunneling nature of VPLS by itself limits the possibilities for attacks via the data plane, simply because such attacks will be tunneled through the VPLS network, and will create the same load on the VPLS equipment as legitimate traffic will. Operators must watch for exception packet handling in VPLS equipment. In many cases, exception packets are sent to the control plane for handling. If that is the case, the operator must ensure that such exception packets can be rate-limited in a fashion that guarantees that the control plane will not be significantly burdened by them. A SP should limit the amount of traffic that a customer can flood. The second type of DoS attacks, the ones that use the VPLS network as a transport, are not really a threat to the VPLS devices themselves, but are to devices behind them. VPLS PEs may be configured with rate-limiting and rate-shaping capabilities which permit them to limit the amount of traffic allowed into a particular VPLS instance. This prevents a VPLS customer from consuming excessive amount of network resources and starve other customers. Optionally, they can also be tasked with advanced processing of the traffic they tunnel. For example, they may impose access lists which deny traffic from particular sources or protocols. Such approaches however are highly vendor-specific and outside the scope of [VPLS-LDP]. In addition, they may have significant design and operational repercussions. Alternative approaches which hand- off DoS protection activities to non-VPLS devices (such as customer equipment) are preferred. 13.4. Protection of User Data VPLS does not have special provisioning for ensuring user data security. If a customerÆs traffic is IP traffic, that customer may provide its own user data security by using IPsec. In fact, VPLS is compatible with any use of security by the customer, as long as a clear text Ethernet header is passed from CE to PE. draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 17] 14. Scalability As per [L2VPN-REQ], a large SP may eventually require support of up to O(10^4) VPLS instances. In addition, some of these VPLS instances may need to support O(10^2) sites and O(10^3) users/MACs. This section describes the key scalability challenges and how VPLS-LDP addresses them. 14.1. Mesh topology A full mesh of tunnel LSPs, over which PWs are established û resulting in a full mesh of PWs, is created between participating PEs. When using hierarchical VPLS constructs, the size of this full mesh can be reduced to hub PEs aggregating point-to-point spokes as described in section 10 of [VPLS-LDP]. This reduces the number of tunnels and PWs from O(N*N) to O(N). 14.2. Signaling Using HVPLS constructs also allows the total number of targeted LDP sessions to be reduced from O(N*N) to O(N). 14.3. MAC addresses and MAC learning Depending on the type of CE devices used, i.e. switches or routers, the total number of MAC addresses to be learned by VPLS PEs can vary from one address per site to a large number of MAC addresses. When Ethernet networks exceed a large number of MAC addresses (e.g. hundreds), routers are introduced to limit the size of such broadcast domains. This reduces the total number of MAC addresses to learn to such routers only. In the case of large flat Ethernet networks, ingress PEs must be able to limit the number of MAC addresses that can be learned on a per VPLS basis. 14.4. Packet replication With VPLS, broadcast, multicast and unknown destination frames get replicated by the ingress PEs, i.e. close to the source of the frame. Ideally such frames should be replicated as close to the destination as possible to minimize bandwidth consumption. With hierarchical VPLS, the replication process is distributed between draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 18] several ingress and egress MTUs and PEs. This helps not only minimizing bandwidth resources but also improving multicast performance and reducing latency. 14.5. Broadcast limiting Ingress MTUs or PEs may be able to rate limit the amount of broadcast traffic generated by end users in order to protect core resources and to prevent a few users from using all the bandwidth available. 14.6. Multicast In order to optimize the replication of multicast traffic, it is highly desirable for PEs to support multicast snooping techniques in order to only forward traffic where needed. In the case where the CE device is an L2 switch, IGMP snooping would be required, however, if the CE device is a router PIM snooping would be more applicable. 15. Management Five major areas in management are: Fault, Configuration, Accounting, Provisioning, and Security. They are discussed below. VPLS introduces new configurations related to creation and removal of VSIs, etc. VPLS also introduces new provisioning challenges because the service needs to be delivered end-to-end and therefore many things such as access control, QoS, etc need to be provisioned accordingly. Achieving these via manual CLI configuration can be tideous and error prone. Therefore, it is advisable to use a provisioning system for configuration and provisioning. Although VPLS-specific MIBs are still under development, accounting information can usually be achieved via [IF-MIB] and [LSR-MIB]. Such information can then be processed by an accounting application to produce the accounting records. Security can be achieved by the measures described in Section 13. Managing fault with VPLS involves multi-point connectivity verification and locating the fault if there is one. Such mechanism is sometimes referred to as ôVPLS OAMö and is discussed below. 15.1. VPLS OAM Although VPLS OAM is still being defined, one of the approaches has gained more momentum than others. This approach proposes applying draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 19] Ethernet OAM mechanism that is being standardized by ITU, IEEE and the Metro Ethernet Forum (MEF) to an VPLS environment for L2 connectivity verification and fault locating, and applying MPLS OAM mechanism such as [LSP-PING] or [BFD] or [VCCV] to MPLS connectivity verification and fault locating. Of course, if IP tunnels (e.g. GRE) are used, IP ping and traceroute can be used in the place of MPLS OAM. VPLS OAM is therefore divided into 2 parts which are dealt with separately. A procedure can then be constructed, e.g. by writing a CLI script, to combine both Ethernet OAM and MPLS/IP OAM to achieve the VPLS OAM goal. The pro of this approach is, the L2 connectivity verification and fault locating part is agnostic to spoke type (Q-in-Q or MPLS PW) and the PSN tunnel type (MPLS or GRE). The con is that a combined procedure needs to be devised. With VPLS OAM, ideally the OAM packets should always follow the same path as the VPLS data packets. However, because the Ethernet MAC layer has no TTL support, something needs to be added to the OAM packets to achieve the traceroute capability unless existing network equipment can be enhanced with new OAM processing capability (which is unlikely). As such, traceroute packets may not always follow the same path as the VPLS data packets. Nevertheless, VPLS OAM achieves the practical purpose of verifying VPLS connectivity and locating fault to a good extent. In summary of this section: management of VPLS services involves many things and can be tideous. A complete suite of management software including EMS, NMS and a provisioning system can therefore be highly desirable. 16. Acknowledgments The authors wish to thank the following people for their constructive contributions to the text in this document: Javier Antich Ian Cowburn Richard Foote Rob Nath Nick Slabakov Some text was adapted from the Applicability Statement for BGP/MPLS IP VPNs [AS2547] document. 17. References [AS2547] ôApplicability Statement for BGP/MPLS IP VPNsö, draft-ietf- l3vpn-as2547-05.txt, Work in progress, May 2004. [BFD] D. Katz and D. Ward, ôBidirectional Forwarding Detectionö, draft-ietf-bfd-base-00.txt, Work in progress, Jul. 2004. draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 20] [IF-MIB] "The Interfaces Group MIB using SMIv2", McCloghrie, Kastenholtz, RFC 2233, November 1997 [LSP-PING]K. Kompella, P. Pan, et al, "Detecting Data Plane Liveliness in MPLS", , work in progress, Feb. 2004. [LSR-MIB] "MPLS Label Switch Router Management Information Base", Srinivasan, Viswanathan, Nadeau, draft-ietf-mpls-lsr-mib-14.txt>, November 2003 [L2FRAME] "L2VPN Framework", draft-ietf-ppvpn-l2-framework-05, Work in progress, Jun. 2003. [L2VPN-REQ] "Service Requirements for Layer 2 Provider Provisioned Virtual Private Networks", draft-ietf-ppvpn-l2vpn-requirements- 01.txt, Work in progress, Feb. 2004. [PWE3-CTRL] " Pseudowire Setup and Maintenance using LDP", draft-ietf- pwe3-control-protocol-08.txt, Work in progress, February 2003. [PWE3-ETHERNET] "Encapsulation Methods for Transport of Ethernet Frames Over IP/MPLS Networks", draft-ietf-pwe3-ethernet-encap- 02.txt, Work in progress, Jul. 2004. [RADIUS-DIS] "Using Radius for PE-Based VPN Discovery", Work in progress, Feb. 2004 [VCCV] T. Nadeau et al ôPseudo Wire Virtual Circuit Connectivity Verification (VCCV)ö, draft-ietf-pwe3-vccv-03.txt, Work in progress, Jun. 2004 [VPLS-LDP] "Virtual Private LAN Services over MPLS", draft-ietf- ppvpn-vpls-ldp-03.txt, Work in progress, Apr. 2004 [VPLS-BGP] "Virtual Private LAN Service", draft-ietf-ppvpn-vpls-bgp- 02.txt, Work in progress, May 2004 [Y.17ethoam] "OAM mechanisms for Ethernet based networks", ITU-T, SG13, Jul. 2003 [802.1ad] "IEEE standard for Provider Bridges", Work in progress, December 2002. [802.1ag] "IEEE Connectivity Fault Management", Work in progress. 18. Authors' Addresses Marc Lasserre Riverstone Networks Email: marc@riverstonenet.com draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004 Lasserre et al [Page 21] Xipeng Xiao Riverstone Networks Email: xxiao@riverstonenet.com Yetik Serbest SBC Communications serbest@tri.sbc.com Cesar Garrido, Telefonica cesar.garridosanahuja@telefonica.es Marc Rapoport Completel m.rapoport@completel.fr draft-lasserre-l2vpn-vpls-ldp-applic-01.txt Jul 2004