Internet Draft Document Olen Stokes Provider Provisioned VPN WG Extreme Networks Vach Kompella TiMetra Networks Giles Heron PacketExchange Ltd. Yetik Serbest SBC Communications Expires December 2002 June 2002 Testing Hierarchical Virtual Private LAN Services draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt 1. Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 1 2. Table of Contents 1. Status of this Memo......................................1 2. Table of Contents........................................2 3. Abstract.................................................3 4. Conventions..............................................3 5. Placement of this Memo in Sub-IP Area....................3 5.1. RELATED DOCUMENTS......................................3 5.2. WHERE DOES THIS FIT IN THE PICTURE OF THE SUB-IP WORK..4 5.3. WHY IS IT TARGETTED AT THIS WG.........................4 5.4. JUSTIFICATION..........................................4 6. Overview.................................................4 6.1. Connectivity verification..............................5 6.2. Service Verification...................................5 6.3. Topology Discovery.....................................5 6.4. Performance Monitoring.................................5 7. Terminology..............................................5 8. Structural Model.........................................6 8.1. Identification.........................................6 8.2. Addressing.............................................6 8.3. FIB Traversal..........................................6 8.4. Intermediate Failure Processing........................6 8.5. Control and Data Plane.................................7 9. OA&M Functions...........................................7 9.1. VPLS ping..............................................7 9.1.1. L2 VPN Prefix......................................8 9.1.2. Reply mode........................................10 9.1.3. VPLS ping encapsulation...........................10 9.1.4. Spoke node control plane processing...............11 9.2. VPLS traceroute.......................................12 9.2.1. HVPLS operational requirements....................12 9.2.2. VC FEC label TTL processing.......................13 9.2.3. HVPLS spoke node considerations...................13 9.2.4. VPLS traceroute format............................14 9.2.5. VPLS traceroute procedures........................14 10. Distribution of VPLS node information..................15 10.1. Ethernet VPLS node TLV...............................15 10.2. Ethernet VPLS node procedures........................16 10.3. Operation of limited HVPLS nodes.....................17 11. General VPLS testing procedures........................17 12. Load sharing considerations............................19 13. Security Considerations................................20 14. Scalability Issues.....................................21 15. Intellectual Property Considerations...................21 16. References.............................................22 17. Authors' Addresses.....................................22 draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 2 3. Abstract This document describes a methodology for testing the operation, administration and maintenance (OA&M) of a general VPN service, that is applied here to Hierarchical Virtual Private LAN Services (HVPLS) as described in [VPLS]. As part of this methodology, the MPLS ping concepts described in [LSP- PING] are extended to enable HVPLS spoke-to-spoke connectivity testing. A method to provide the information necessary for this spoke-to-spoke OA&M is also proposed. These are the goals of this draft: - checking connectivity between "service-aware" nodes of a network, - verifying data plane and control plane integrity, - verifying service membership There are two specific requirements to which we call attention because of their seemingly contradictory nature: - the checking of connectivity MUST involve the ability to use packets that look like customer packets - the OAM packets MUST not propagate beyond the boundary of the provider network 4. Conventions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 5. Placement of this Memo in Sub-IP Area 5.1. RELATED DOCUMENTS http://search.ietf.org/internet-drafts/draft-lasserre- vkompella-ppvpn-vpls-01.txt http://search.ietf.org/internet-drafts/draft-ietf-mpls-lsp- ping-00.txt http://search.ietf.org/internet-drafts/draft-martini- l2circuit-trans-mpls-08.txt draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 3 http://search.ietf.org/internet-drafts/draft-martini-ethernet- encap-mpls-00.txt http://search.ietf.org/internet-drafts/draft-ietf-mpls-rsvp- lsp-fastreroute-00.txt http://www.ietf.org/rfc/rfc3036.txt http://www.ietf.org/rfc/rfc3209.txt 5.2. WHERE DOES THIS FIT IN THE PICTURE OF THE SUB-IP WORK PPVPN 5.3. WHY IS IT TARGETTED AT THIS WG The charter of the PPVPN WG includes L2 VPN services and this draft specifies a method for testing Ethernet VPLS services over MPLS. 5.4. JUSTIFICATION There is no Internet document that fully provides a method for testing a Hierarchical VPLS. 6. Overview This document describes a methodology for the operation, administration and maintenance (OA&M) of a Hierarchical VPLS (HVPLS) as described in [VPLS]. Service providers wanting to deploy such a VPLS need to be able to test its operation across the entire hierarchy of the VPLS. While today's standards are able to test parts of an HVPLS, there are some aspects that cannot be tested. This OA&M methodology requires extensions to the MPLS ping concepts described in [LSP-PING] to enable VPLS spoke-to-spoke connectivity testing. It also requires extensions to the HVPLS concepts in [VPLS]. We define the following four functions that are needed to provide OA&M for services: - connectivity verification - service verification - topology discovery draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 4 - performance monitoring 6.1. Connectivity verification There are five logical steps to verifying the operation of an HVPLS: - verify that all HVPLS nodes are operational - verify that all HVPLS peer sessions are operational - verify that all HVPLS tunnel LSPs are transporting data packets correctly - verify that data packets can be exchanged between any two spokes in the HVPLS - verify that actual customer devices can communicate with devices at any site Existing tests can verify the first three steps. This document describes how to address the final two issues. 6.2. Service Verification Service verification has to do with whether the service aware nodes have been consistently configured for the service or not. These extensions will be described in a later version. 6.3. Topology Discovery Topology discovery has to do with the current layout of a VPN service, e.g., the PEs and MTUs participating in a VPLS. These extensions will be described in Section 9.2. 6.4. Performance Monitoring Performance monitoring, such as determining round-trip times, jitter, average packet throughput, etc. will be described in a future version. 7. Terminology The following terms are used in this text: - Service point: a provider device that has knowledge of the service. We note here that the transport tunnel, while an integral part of the service, only serves to carry the service. In draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 5 contrast, service-aware nodes participate in the signaling and maintenance of the VPN service. - Service ping: a protocol for testing end-to-end connectivity through the data plane - Service traceroute: a protocol for identifying intermediate nodes in an end-to-end service - OA&M FEC: a forwarding equivalence class for the purpose of identifying and forwarding OA&M packets 8. Structural Model We describe below the structural model that implements OA&M for VPN services. The encapsulation and traversal methodology is described. We note that while it is not possible to make an OA&M packet look indistinguishable from customer frames, a reasonable verisimilitude may be maintained. This allows the lookup methods and encapsulation used in the data plane to be preserved. 8.1. Identification We use loopback or "always on" IP addresses to identify the service points. This allows service points to address each other, which is especially important in case of last ditch error reporting. 8.2. Addressing We require the addressing in the OA&M packets to be configurable to match a customer VPN address, whether a MAC address, DLCI, private IP address, etc. While addresses in the service provider network MAY be usable, they may not have relevance in proving that customer packets can traverse the service domain. 8.3. FIB Traversal We require that OA&M packets following the data plane traverse the service points using the forwarding lookup tables just as customer packets are. 8.4. Intermediate Failure Processing Failures at intermediate service points should be reported back using the control plane. This provides the basis for a traceroute function. draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 6 8.5. Control and Data Plane Typically, connectivity should first be checked against the data plane. If the request packet makes it to the destination service point, the reply packet should be sent along the data plane. Otherwise, after some interval, the sender should send another packet along the data plane, requesting a reply back on the control plane. If this fails, a final attempt may be made, with the request sent along the control plane, and the reply back along the control plane. If this fails, then the network is probably partitioned. 9. OA&M Functions The OA&M functions described below have specific reference to a particular VPN service, i.e., HVPLS. A later revision will deal with general VPN service OA&M. 9.1. VPLS ping There are two parts to connectivity verification. The first is ensuring that a particular remote service point is reachable. The second is diagnosing failures in case the said service point is not reachable. The primary goal of connectivity verification is to check the data plane consistency. This requires intermediate service points to forward OAM messages using FIB lookups, next hop table matches, etc., just as they would for a conventional customer packet. We call this the VPLS (service) ping function. A secondary goal of connectivity verification is to provide information of the service points in the service. We call this the VPLS (service) traceroute function. In addition, it is good to have the ability to send replies through the control plane, so as to allow for error reporting when all else fails. Verifying that data packets can be exchanged between any two spokes in the HVPLS detects problems with the "bridging" aspects of the HVPLS. Doing a complete test requires each spoke node to verify that it can exchange packets with every other spoke node in the HVPLS. This requires that each spoke node know the MAC addresses of all of the other spokes. Using draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 7 this information, a spoke can encapsulate a MPLS ping frame using the same label stack as normal customer data, and then send the packet to a remote peer. When the packet is un- encapsulated at the remote end, the L2 destination MAC address will be that of the remote spoke and the IP address will be that of the ALL_ROUTERS multicast address (224.0.0.2). Such a VPLS ping requires defining the Target FEC Stack Type 8 in [LSP-PING]. This type is defined as a "L2 VPN prefix." A new reply mode is also needed to specify if the reply should use the HVPLS, i.e., the data plane. For security purposes, a service point receiving a VPLS ping SHOULD check the sender information to ensure that the sender is known to be in the HVPLS. See Section 13. for further discussion. 9.1.1. L2 VPN Prefix At a minimum, this "prefix" should contain sufficient information to identify the target remote spoke and to allow the remote spoke to be able to respond. A proposed value specification is shown below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | VC ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | L2 specific sub-TLVs | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The VC ID is the same as contained in the VC FEC TLV in [VPLS]. The "L2 specific sub-TLVs" use the same Type field as defined in [MARTINI-SIG] and extended in [VPLS]. These are: VC Type Description ------- ------------ 0x0001 Frame Relay DLCI 0x0002 ATM AAL5 VCC transport 0x0003 ATM transparent cell transport 0x0004 Ethernet VLAN 0x0005 Ethernet 0x0006 HDLC 0x0007 PPP draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 8 0x8008 CEM [8] 0x0009 ATM VCC cell transport 0x000A ATM VPC cell transport 0x000B Ethernet VPLS For Ethernet VPLS, the following sub-TLV is proposed: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type (0x000B) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Target Ethernet MAC address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | 802.1Q Tag | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sender Ethernet MAC address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | 802.1Q Tag | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Type Type field. The field MUST be set to a value of 0x000B. Length Length field. This field specifies the total length of the Value fields in octets. Target Ethernet MAC address Target Ethernet MAC address field. This field specifies the target Ethernet MAC of the remote spoke or customer device. It is included here to assist implementations when processing echo replies and VPLS traceroute as discussed in Section 9. . Sender Ethernet MAC address Sender Ethernet MAC address field. This field specifies the sender's Ethernet MAC that can be used in a reply. 802.1Q Tag The 802.1Q Tag fields. These fields specify the 802.1Q Tag associated with the Target and Sender Ethernet MACs described above. If the Ethernet VLAN encapsulation is used, the values MUST be zero. If the Ethernet encapsulation is draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 9 used, and the Ethernet address is associated with a VLAN, it MUST be set to the VLAN tag. If the Ethernet encapsulation is used, and the MAC address is not associated with a VLAN (untagged VLAN), it MUST be set to zero. Since an 802.1Q tag is 12-bits, the high 4 bits of the fields MUST be set to zero. 9.1.2. Reply mode [LSP-PING] defines multiple reply modes for MPLS pings. For example, reply mode Value 1, "Reply via an IPv4 UDP packet" allows the reply to be sent via an MPLS LSP but does not require it. The reply can be sent as an IP packet. Value 2, "Reply via an IPv4 UDP packet with Router Alert," requires that the reply be sent as an IP packet. For a VPLS ping, an additional reply mode is required. Since a VPLS implies bidirectional operation, a reply mode is required that specifies that the reply packet be sent using the VPLS. Therefore, Value 4, "Reply via a VPLS IPv4 UDP packet," is proposed. This provides for three different reply methods to help in failure detection. If the ping is successful using reply mode Value 1, but not successful using reply mode Value 4, this indicates that the ping is reaching the remote peer but there is a problem with the VPLS return path. 9.1.3. VPLS ping encapsulation It is desirable for both operational and security reasons to be able to easily recognize in the data plane that a received packet is a VPLS ping. Three methods are discussed to identify that the encapsulated packet is a VPLS ping packet: - use the EXP bits of the VC FEC label to set a flag - set the protocol/length (ethertype) field of the encapsulated MPLS ping packet to 0x0000 - add a unique label below the VC FEC label The first method confines the checking to the MPLS label stack but is contrary to other proposed uses of this field including [MARTINI-ENC]. The value chosen in the second method avoids conflicts with any valid customer data packet. The actual length of the encapsulated packet is determined from the length of the containing packet. Note that this encapsulated packet would draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 10 be invalid on a customer network since the indicated length is not the same as the actual length. Because this is the protocol/length field of a packet encapsulated inside a MPLS packet, this length field is not used as the packet moves through a tunnel LSP. However, this field would have to be checked on every packet, including regular customer packets, to ensure that OA&M packets do not exit into the customer network. This impacts the forwarding fast path. The third method confines the checking to the MPLS label stack. Since the bottom-of-stack bit SHOULD be checked on every VC label, regular customer packets will not take a hit. But when the bottom-of-stack bit is off, either something has gone wrong or the packet is an OAM packet. Both cases can be handled on a slow path by the control plane. However, this approach will affect how the data plane is checked in load sharing cases where the label stack is used in the load sharing hashing algorithm (see Section 12. for more details). This also requires the exchange of an OAM FEC, a FEC that is used to classify OAM packets. The final method chosen to easily recognize in the data plane that a received packet is a VPLS ping will be specified in a future version of the draft. At an egress node, when a VPLS ping packet is detected, the packet SHOULD be passed to the node's control plane. If it cannot be passed to the control plane, the packet SHOULD be discarded. The packet MUST NOT be forwarded to the customer network. Note that the rate at which packets are passed to the control plane should be regulated to prevent overloading. 9.1.4. Egress node control plane processing When an egress node control plane receives a VPLS ping packet, it checks the Target Ethernet MAC address field to determine if it is a MAC address belonging to that node. If it is, a successful reply is returned. If the MAC address does not belong to that node, an additional check is made of the forwarding database associated with the HVPLS indicated by the VC ID field. If the MAC is present and is associated with a locally attached customer network for this HVPLS, a successful reply is returned. If not, an unsuccessful reply is returned. This requires a new MPLS ping Return code. This new code, Value 6, is shown below along with those defined in [LSP- PING]: draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 11 Value Meaning ----- ------- 1 Replying router is an egress for the FEC 2 Replying router has no mapping for the FEC 3 Replying router is not one of the "Downstream Routers". 4 Replying router is one of the "Downstream Routers", and its mapping for this FEC on the received interface is the given label 5 Replying router is one of the "Downstream Routers", but its mapping for this FEC is not the given label 6 Replying router is one of the "Downstream Routers", but it does not have the given MAC address 7 Replying router is one of the "Downstream Routers", but it does not have the given MAC address and is unable to flood the request The new Value 7 shown above is explained in Section 9.2.4. . Note that this method of operation allows VPLS ping to check for both remote node MACs and remote customer device MACs. When a VPLS ping target MAC address is unknown in a core node, the VPLS ping packet is flooded. As such, it potentially reaches other nodes in addition to the spoke where the destination is actually present. As a result, the sending node may receive replies from multiple remote nodes. This allows an operator to determine which remote nodes would receive a flood of an actual customer data packet destined to the target MAC address. 9.2. VPLS traceroute When a VPLS ping using reply mode 4 is unsuccessful, additional capabilities are required to identify the problem location. A VPLS traceroute is therefore proposed. 9.2.1. HVPLS operational requirements To provide an HVPLS traceroute capability, the basic operation of an HVPLS needs further definition. In particular, the handling of the TTL field in VC FEC labels needs to be defined. This TTL handling is also dependent on VC FEC parameters specified when a spoke signals to a core node. draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 12 9.2.2. VC FEC label TTL processing A data packet normally goes from a HVPLS spoke node to a HVPLS core node. In the core HVPLS node, the VC FEC label received from the spoke node is used to determine the HVPLS with which the encapsulated frame is associated. The frame is un- encapsulated and a check is made for the frame's destination MAC address. If the destination MAC is known, the frame is forwarded to the core peer from which the MAC was learned [VPLS]. As the frame is forwarded, the frame is again encapsulated using the VC FEC label received from the peer core node. A similar process occurs at the remote core peer. The VC FEC label is checked, the frame is un-encapsulated, the destination MAC is checked, the frame is again encapsulated and sent to the remote spoke node. The packet containing the newly encapsulated frame uses the VC FEC label received from the destination spoke node. Each time that this process occurs, the received VC FEC label TTL value SHOULD be decremented and the result placed in the outgoing VC FEC label TTL field. If the resulting value is 0, the packet SHOULD be passed to the node's control plane. If it cannot be passed to the control plane, the packet SHOULD be discarded. Note that the rate at which packets are passed to the control plane should be regulated to prevent overloading. If the destination MAC is not known and the packet is flooded to multiple peer nodes or multiple spoke nodes, the same TTL described above is placed in each VC FEC label. In normal operation, the TTL value in the VC FEC label should be set to a value larger than the number of "HVPLS Hops" through which the data packet will pass, e.g., 255. Note that this mode of operation also provides some protection against the effects of loops at the HVPLS level. 9.2.3. HVPLS spoke node considerations [VPLS] permits spoke nodes to specify Ethernet or Ethernet VLAN when signaling a VC FEC to a core node. This permits [MARTINI-SIG] spokes to attach to an HVPLS. These spoke nodes will normally set a value of 2 for the VC FEC label TTL as specified in [MARTINI-ENC]. With the TTL processing described in Section 9.2.2. , this will cause packets to be erroneously draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 13 discarded inside a HVPLS. Therefore, we require a change in the TTL setting at spoke nodes to be 255 by default. As further discussed in Section 13. , security considerations also require a change to detect OAM packets. 9.2.4. VPLS traceroute format A VPLS traceroute uses the same echo request packet described in Section 9.1. for a VPLS ping. The echo request packet SHOULD contain one or more Downstream Mapping TLVs [LSP-PING]. In this case the Downstream Router ID field contains the IPv4 address of the next HVPLS node that would be used to reach the Target Ethernet MAC in the HVPLS indicated in the L2 VPN Prefix. The Downstream Label field contains the VC FEC label from the targeted LDP session with the downstream peer. The Protocol field indicates a value of 3 for (targeted) LDP. When the indicated Target Ethernet MAC is not known and a packet with this destination MAC would be flooded, the information for all HVPLS peers to which the packet would be flooded is added. For the case where the packet would not be flooded (such as a limit on MAC addresses that has been exceeded for this HVPLS), a new MPLS ping Return code is defined. This new Value 7 is shown in Section X. 9.2.5. VPLS traceroute procedures To create a traceroute, an echo request is created using the destination Ethernet MAC address of the remote spoke or customer device. The TTL in the VC FEC label is set successively to 1, 2, 3, ... As with MPLS traceroute, the echo request SHOULD contain one or more Downstream Mapping TLVs. For TTL=1, all the peer nodes (and corresponding VC FEC labels) for the sender with respect to the remote spoke being pinged SHOULD be sent in the echo request. For n>1, the Downstream Mapping TLVs from the echo reply for TTL=(n-1) are copied to the echo request with TTL=n. As the packet exits a tunnel LSP, the node first checks if the destination MAC address is a MAC belonging to this node. If so, the packet is passed to the nodeÆs control plane and an echo reply is created as described in [LSP-PING]. This includes completing the Downstream Mapping TLV as described in draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 14 Section 9.2.4. The reply is sent based on the value indicated in the Reply Mode field of the echo request. If the MAC is not for this node, it then checks the TTL value of the VC FEC label. If the value is 1, the packet is passed to the node's control plane and an echo reply is created as described in [LSP-PING] and Section 9.1. . Note that the control plane also checks its forwarding database associated with this HVPLS to see if the target MAC is associated with a locally attached customer network. The reply is sent based on the value indicated in the Reply Mode field of the echo request. For security purposes, before replying to a VPLS traceroute, a node SHOULD check the sender information to ensure that the sender is known to be in the specified HVPLS. See Section 13. for further discussion. If the value of the VC FEC label TTL is greater than 1, the TTL is decremented and the result is placed in the VC FEC label TTL for the resulting packet. This packet is encapsulated in the same manner as a customer data packet and then passed to the downstream node that would normally be used to reach the indicated Ethernet MAC. If the packet is flooded to multiple peer nodes, this same TTL value is placed in the VC FEC label TTL of each of the packets. 10. Distribution of VPLS node information A spoke node normally knows the IP address of the core node with which it peers. It typically does not know the IP addresses or MAC addresses of the other spokes in the network. In order to perform the MPLS ping described in Section 9.1. , a spoke must know the MAC addresses of the other spokes. This information could be distributed via the targeted LDP sessions between HVPLS nodes. [VPLS] defines a MAC TLV that can be used with a LDP Address Withdrawal Message [MPLS-LDP]. In a similar manner, a LDP Address Message [MPLS-LDP] can be used to distribute the required information about HVPLS nodes. 10.1. Ethernet VPLS node TLV draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 15 To distribute HVPLS node information, a new TLV is proposed for use with a LDP Address Message. This new Ethernet VPLS node TLV is shown below. Note that the nodeÆs IP address is included to allow remote nodes to correlate the nodeÆs MAC address with its IP address. In this manner an operator can request a VPLS ping by specifying the IP address of the remote spoke. For IPv4: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |U|F| Type | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved | Address Family | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv4 Address of VPLS node #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address of VPLS node #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | 802.1Q Tag #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv4 address of VPLS node #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address of VPLS node #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | 802.1Q Tag #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The scope of the Ethernet VPLS node TLV is the VPLS specified in the FEC TLV in the Address Message. 10.2. Ethernet VPLS node procedures When a spoke and a core node establish a new peer relationship for an HVPLS, they exchange one or more Address Messages with the above information. The spoke node sends information about a single HVPLS node (itself). The IP address should be the address by which the node is known to its peers. The MAC address should be a MAC address of the switch such that an un- encapsulated frame received from an HVPLS peer with this destination MAC address will be delivered to the node's control plane. When two core nodes establish a peer relationship, they also exchange one or more Address Messages with the above draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 16 information. In this case each core sends the MAC address information for all of the spokes to which it is connected, as well as for itself. Returning to the case of a core establishing a peer connection with a spoke, the core node sends to the spoke all of the MAC addresses that it has learned from all of its core peers (as well as for itself). In this manner, each spoke will know the MAC addresses of all of the other spokes in the HVPLS. Note that in large networks multiple Address Messages may be required to ensure that media MTU sizes limitations are not exceeded. When a core node loses the connection to a spoke, it instructs all of its core peers to remove the corresponding spoke MAC address by sending the above information in a Address Withdrawal Message [MPLS-LDP]. The core nodes in turn pass that information along in Address Withdrawal Messages to their spokes. When a core node loses the connection to another core node, it instructs all of its spoke nodes to remove all of the corresponding MAC addresses by sending the above information in an Address Withdrawal Message. Using the above capabilities, a spoke node can test the data plane operation between itself and any other spoke site in the HVPLS. 10.3. Operation of limited HVPLS nodes There may be some implementations that do not want to maintain a list of remote spoke IP and MAC addresses. In such an environment, an operator can still invoke a VPLS ping or VPLS traceroute if the MAC address of the remote spoke or customer device is provided. There may be some cases where an operator knows the IP address of a remote spoke, but not the MAC address. A method to obtain the MAC address when only the IP address is known may be described in a future version. 11. General VPLS testing procedures draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 17 When operating in a background mode, the checks listed in Section 6.1. are normally performed on different time scales. Testing HVPLS nodes and peer sessions requires minimal network impact since this is done on a per machine and a per peer basis. Also, monitoring for these failures can be done with SNMP traps. It is important to note that these tests are expected to catch a significant percentage of the most likely failures. Tunnel LSPs are maintained by their associated MPLS signaling protocols, RSVP-TE [MPLS-RSVP] or LDP. Locally detected failures such as link outages or node failures invoke MPLS recovery actions. These actions can include local recovery as in the case of [RSVP-FRR]. In the case of a successful local recovery of a tunnel LSP, the HVPLS nodes need not be notified. If a local recovery is not possible, then RSVP-TE or LDP notifies the HVPLS node using the tunnel LSP of the failure. This in turn can generate a SNMP trap or an operator error message. In addition to failures and changes detected outside of MPLS, both MPLS signaling protocols have control plane failure detection mechanisms. Both have Hello protocols that can detect link failures as well as MPLS control plane failures. LDP also has a Keep Alive protocol. These Hello and Keep Alive protocols run on the time frame of multiple seconds. They also provide failure notification to the HVPLS node using the tunnel. As above, this can generate a SNMP trap or an operator error message. In current [MARTINI-SIG] or [VPLS] environments, loss of a targeted LDP session to a peer is normally the key operator notification. While a failed tunnel LSP can generate a notification as described above, these failures can be temporary in nature due to routing transients. MPLS ping is designed to catch failures where the control plane believes that a tunnel LSP is operational but it is in fact not functioning correctly. This corrupted LSP is much less likely to occur than a LSP going down "properly." When used as a background check, it should be used in addition to the above tunnel LSP failure detection methods and not as a replacement. When any of these methods detects a tunnel LSP failure, the HVPLS node can switch to another LSP if one is available. draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 18 When the failure is detected by MPLS ping, MPLS traceroute can be used to assist in failure isolation. VPLS ping is designed to detect problems in the "bridging" aspects of the VPLS operation. It detects flooding and/or MAC learning problems in the network which are not checked in the above tests. Note that the number of possible spoke-to-spoke tests to check an entire HVPLS can be significant. Therefore, care should be taken when executing VPLS ping as a background test to avoid overloading the network or the HVPLS nodes. Note also that an individual core node's operation is checked by multiple spoke-to-spoke checks. When a failure is detected by VPLS ping, VPLS traceroute can be used to assist in problem isolation. All of the above tests check the operation of a HVPLS to the edge of the HVPLS. It is also possible to use VPLS ping and traceroute to check for customer device MAC addresses. While not specified by HVPLS, there is normally additional information available to an operator to check for problems between the edge of the HVPLS and the customer. These include: - the state of the local customer VLAN or port - this is the simplest test and will normally catch the most likely failures - the L2 MAC entries for the local customer VLAN or port - the HVPLS transmit/receive statistics As described above, VPLS ping and VPLS traceroute work with previously defined MPLS tests to provide an end-to-end test capability for an HVPLS. In addition, extensions to [VPLS] allow VPLS ping to operate in a background mode with knowledge of the remote sites that need to be checked. 12. Load sharing considerations Some implementations provide for load sharing a tunnel LSP across multiple LSPs. Such implementations have HVPLS test implications. When customer data entering a VPLS at an ingress node is transmitted to another node over multiple (load sharing) tunnel LSPs, each of these LSPs SHOULD be tested. VPLS pings and traceroutes SHOULD be sent over each of these LSPs. draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 19 There may also be multiple load sharing tunnel LSPs between a core node which is not the traffic ingress and a downstream node (which may or may not be the traffic egress). At such a core node, a decision is made as to which load sharing tunnel LSP to use to forward a HVPLS packet. This decision is often based on a hash of some "random" field. There are at least two options. One option is to hash on the label stack of the received HVPLS packet. This forces all packets received on a tunnel LSP for the same HVPLS to use the same load sharing tunnel LSP to the next core node. This method distributes traffic among the load sharing tunnel LSPs on a HVPLS basis. A second option is to hash on fields of the HVPLS packet after it has been un-encapsulated (see Section 9.1.4. ). Such a hash could use the destination and source MAC addresses of the un-encapsulated packet. Thus, traffic received on a tunnel LSP for the same HVPLS may use any of the load sharing tunnel LSPs to the next core node. This method distributes traffic among the load sharing tunnel LSPs on a MAC address pair basis. The second option normally produces a more optimal distribution of packets since MAC addresses should be more random than HVPLS labels. This advantage may be somewhat reduced if customers use a single router at each site to connect to the HVPLS. The first option has an advantage from an HVPLS testing perspective. Since the label stack for a VPLS ping or traceroute is the same as for customer traffic, the first option ensures that VPLS pings and traceroutes are testing all of the LSPs used by customer data. Remember from above that VPLS pings and traceroutes SHOULD be sent using all of the load sharing tunnel LSPs at the ingress node. Load sharing designs and hash algorithms remain implementation options. There are trade-offs between optimal load sharing and testability. Of course, testing using IP ping and traceroute has similar exposures from the effects equal cost multipath. 13. Security Considerations For security purposes, the edge of a provider's HVPLS network is defined to be a spoke node or a PE that has directly attached customers. Some customers and providers may desire that the provider edge node participate in the customer network. This could be the case when the customer is using draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 20 the provider's node as a default gateway. In such a configuration, the provider edge node's IP address and Ethernet MAC address are known in the customer network. However, no other provider network information should be exposed to the customer network. When the provider is not furnishing a default gateway function, no provider network information should be exposed to the customer network. The VPLS ping and VPLS traceroute capabilities described in Sections 9.1. and 9.2. are transported inside the HVPLS in the same manner as customer data. This is required to properly test the HVPLS. However, care must be taken to prevent provider network information contained in these test packets from being exposed to the customer network. A test packet that is forwarded to the customer network exposes provider network information to the customer network. Therefore, spoke nodes SHOULD always check for such test packets as described in Section 9.1.3. . Any detected test packet SHOULD NOT be forwarded to the customer network. Another security concern is the receipt of a VPLS ping or traceroute from a node that is not a member of the HVPLS. Should a HVPLS node respond to a test request from a non-HVPLS member, the response would improperly expose provider network information. To prevent this from happening, the HVPLS node MAY check to ensure that the return Ethernet MAC address is one of the MAC addresses that it has learned using the Ethernet VPLS node TLV described in Section 10. Note that this requires maintaining the MAC information during the entire operation of the HVPLS. 14. Scalability Issues In [VPLS], targeted LDP sessions are used to distribute VPLS labels. Between core nodes, a complete mesh of targeted LDP sessions is required. 15. Intellectual Property Considerations Extreme Networks, Inc. is seeking patent protection on technology described in this Internet-Draft. If technology in this Internet Draft is adopted as a standard, Extreme Networks agrees to license, on reasonable and non-discriminatory terms, any patent rights it obtains covering such technology to the extent necessary to comply with the standard. This document is being submitted for use in IETF standards discussions. draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 21 16. Acknowledgments We would like to acknowledge the contributions of Sunil Khandekar, Kevin Frick, Charles Burton, and Joe Regan in the development of these ideas. 17. References [VPLS] "Virtual Private LAN services over MPLS", draft- lasserre-vkompella-ppvpn-vpls-01.txt (Work In Progress) [LSP-PING] "Detecting Data Plane Liveliness in MPLS", draft- ietf-mpls-lsp-ping-00.txt (Work In Progress) [MARTINI-SIG] "Transport of Layer 2 Frames Over MPLS", draft- martini-l2circuit-trans-mpls-09.txt, April 2002 (Work In Progress) [MARTINI-ENC] "Encapsulation Methods for Transport of Ethernet Frames Over IP and MPLS Networks", draft-martini-ethernet- encap-mpls-00.txt, April 2002 (Work In Progress) [MPLS-LDP] Andersson, et al., "LDP Specification", RFC 3036, January 2001 [MPLS-RSVP] Awduche, et al., "RSVP-TE: Extensions to RSVP for LSP Tunnels", RFC 3209, December 2001 [RSVP-FRR] "Fast Reroute Extensions to RSVP-TE for LSP Tunnels", draft-ietf-mpls-rsvp-lsp-fastreroute-00.txt (Work In Progress) 18. Authors' Addresses Olen Stokes Extreme Networks 630 Davis Drive, Suite 250 Morrisville, NC 27560 Email: ostokes@extremenetworks.com Vach Kompella 274 Ferguson Dr. Mountain View, CA 94034 Email: vkompella@timetra.com Giles Heron draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 22 PacketExchange Ltd. The Truman Brewery 91 Brick Lane LONDON E1 6QL United Kingdom Email: giles@packetexchange.net Yetik Serbest SBC Communications 9505 Arboretum Blvd. Austin TX 78759 serbest@tri.sbc.com draft-stokes-vkompella-ppvpn-hvpls-oam-00.txt June 2002 Page 23