Internet Engineering Task Force N. Akiya Internet-Draft Big Switch Networks Updates: 8029 (if approved) G. Swallow Intended status: Standards Track Cisco Systems Expires: October 6, 2019 S. Litkowski B. Decraene Orange J. Drake Juniper Networks M. Chen Huawei April 04, 2019 Label Switched Path (LSP) Ping/Trace Multipath Support for Link Aggregation Group (LAG) Interfaces draft-ietf-mpls-lsp-ping-lag-multipath-08 Abstract This document defines extensions to the MPLS Label Switched Path (LSP) Ping and Traceroute mechanisms as specified in RFC 8029. The extensions allow the MPLS LSP Ping and Traceroute mechanisms to discover and exercise specific paths of Layer 2 (L2) Equal-Cost Multipath (ECMP) over Link Aggregation Group (LAG) interfaces. Additionally, a mechanism is defined to enable determination of the capabilities of an LSR supported. This document updates RFC8029. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP14 [RFC2119][RFC8174] when, and only when, they appear in all capitals, as shown here. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Akiya, et al. Expires October 6, 2019 [Page 1] Internet-Draft LSP Ping LAG Multipath April 2019 Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on October 6, 2019. Copyright Notice Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 1.2. Background . . . . . . . . . . . . . . . . . . . . . . . 4 2. Overview of Solution . . . . . . . . . . . . . . . . . . . . 4 3. LSR Capability Discovery . . . . . . . . . . . . . . . . . . 6 3.1. Initiator LSR Procedures . . . . . . . . . . . . . . . . 7 3.2. Responder LSR Procedures . . . . . . . . . . . . . . . . 7 4. Mechanism to Discover L2 ECMP Multipath . . . . . . . . . . . 7 4.1. Initiator LSR Procedures . . . . . . . . . . . . . . . . 7 4.2. Responder LSR Procedures . . . . . . . . . . . . . . . . 8 4.3. Additional Initiator LSR Procedures . . . . . . . . . . . 10 5. Mechanism to Validate L2 ECMP Traversal . . . . . . . . . . . 11 5.1. Incoming LAG Member Links Verification . . . . . . . . . 11 5.1.1. Initiator LSR Procedures . . . . . . . . . . . . . . 11 5.1.2. Responder LSR Procedures . . . . . . . . . . . . . . 12 5.1.3. Additional Initiator LSR Procedures . . . . . . . . . 12 5.2. Individual End-to-End Path Verification . . . . . . . . . 14 6. LSR Capability TLV . . . . . . . . . . . . . . . . . . . . . 14 7. LAG Description Indicator Flag: G . . . . . . . . . . . . . . 15 8. Local Interface Index Sub-TLV . . . . . . . . . . . . . . . . 16 9. Remote Interface Index Sub-TLV . . . . . . . . . . . . . . . 16 10. Detailed Interface and Label Stack TLV . . . . . . . . . . . 17 10.1. Sub-TLVs . . . . . . . . . . . . . . . . . . . . . . . . 19 10.1.1. Incoming Label Stack Sub-TLV . . . . . . . . . . . . 19 Akiya, et al. Expires October 6, 2019 [Page 2] Internet-Draft LSP Ping LAG Multipath April 2019 10.1.2. Incoming Interface Index Sub-TLV . . . . . . . . . . 20 11. Rate Limiting On Echo Request/Reply Messages . . . . . . . . 21 12. Security Considerations . . . . . . . . . . . . . . . . . . . 21 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 13.1. LSR Capability TLV . . . . . . . . . . . . . . . . . . . 21 13.1.1. LSR Capability Flags . . . . . . . . . . . . . . . . 22 13.2. Local Interface Index Sub-TLV . . . . . . . . . . . . . 22 13.2.1. Interface Index Flags . . . . . . . . . . . . . . . 22 13.3. Remote Interface Index Sub-TLV . . . . . . . . . . . . . 23 13.4. Detailed Interface and Label Stack TLV . . . . . . . . . 23 13.4.1. Sub-TLVs for TLV Type TBD4 . . . . . . . . . . . . . 23 13.4.2. Interface and Label Stack Address Types . . . . . . 24 13.5. DS Flags . . . . . . . . . . . . . . . . . . . . . . . . 24 14. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 24 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 25 15.1. Normative References . . . . . . . . . . . . . . . . . . 25 15.2. Informative References . . . . . . . . . . . . . . . . . 25 Appendix A. LAG with intermediate L2 Switch Issues . . . . . . . 26 A.1. Equal Numbers of LAG Members . . . . . . . . . . . . . . 26 A.2. Deviating Numbers of LAG Members . . . . . . . . . . . . 26 A.3. LAG Only on Right . . . . . . . . . . . . . . . . . . . . 27 A.4. LAG Only on Left . . . . . . . . . . . . . . . . . . . . 27 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27 1. Introduction 1.1. Terminology The following acronyms/terms are used in this document: o MPLS - Multiprotocol Label Switching. o LSP - Label Switched Path. o LSR - Label Switching Router. o ECMP - Equal-Cost Multipath. o LAG - Link Aggregation Group. o Initiator LSR - The LSR which sends the MPLS echo request message. o Responder LSR - The LSR which receives the MPLS echo request message and sends the MPLS echo reply message. Akiya, et al. Expires October 6, 2019 [Page 3] Internet-Draft LSP Ping LAG Multipath April 2019 1.2. Background The MPLS Label Switched Path (LSP) Ping and Traceroute mechanisms [RFC8029] are powerful tools designed to diagnose all available Layer 3 (L3) paths of LSPs, including diagnostic coverage of L3 Equal-Cost Multipath (ECMP). In many MPLS networks, Link Aggregation Group (LAG) as defined in [IEEE802.1AX], which provides Layer 2 (L2) ECMP, is often used for various reasons. MPLS LSP Ping and Traceroute tools were not designed to discover and exercise specific paths of L2 ECMP. This raises a limitation for the following scenario when an LSP traverses over a LAG: o Label switching over some member links of the LAG is successful, but fails over other member links of the LAG. o MPLS echo request for the LSP over the LAG is load balanced on one of the member links which is label switching successfully. With the above scenario, MPLS LSP Ping and Traceroute will not be able to detect the label switching failure of the problematic member link(s) of the LAG. In other words, lack of L2 ECMP diagnostic coverage can produce an outcome where MPLS LSP Ping and Traceroute can be blind to label switching failures over a problematic LAG interface. It is, thus, desirable to extend the MPLS LSP Ping and Traceroute to have deterministic diagnostic coverage of LAG interfaces. The need for a solution of this problem was motivated by issues encountered in live networks. 2. Overview of Solution This document defines a new TLV to discover the capabilities of a responder LSR and extensions for use with the MPLS LSP Ping and Traceroute mechanisms to describe Multipath Information for individual LAG member links, thus allowing MPLS LSP Ping and Traceroute to discover and exercise specific paths of L2 ECMP over LAG interfaces. The reader is expected to be familiar with mechanics Downstream Detailed Mapping TLV (DDMAP) described in Section 3.4 of [RFC8029]. The solution consists of the MPLS echo request containing a DDMAP TLV and the new LSR Capability TLV to indicate that separate load balancing information for each L2 nexthop over LAG is desired in the MPLS echo reply. The Responder LSR places the same LSR Capability TLV in the MPLS echo reply to provide acknowledgement back to the initiator LSR. It also adds, for each downstream LAG member, load balance information (i.e., multipath information and interface Akiya, et al. Expires October 6, 2019 [Page 4] Internet-Draft LSP Ping LAG Multipath April 2019 index). This mechanism is applicable to all types of LSPs which can traverse over LAG interfaces. Many LAGs are built from p2p links, with router X and router X+1 having direct connectivity and the same number of LAG members. It is possible to build LAGs asymmetrically by using Ethernet switches between two routers. Appendix A lists some use cases for which the mechanisms defined in this document may not be applicable. Note that the mechanisms described in this document do not impose any changes to scenarios where an LSP is pinned down to a particular LAG member (i.e. the LAG is not treated as one logical interface by the LSP). The following figure and description provides an example using an LDP network. <----- LDP Network -----> +-------+ | | A-------B=======C-------E | | +-------D-------+ ---- Non-LAG ==== LAG comprising of two member links Figure 1: Example LDP Network When node A is initiating LSP Traceroute to node E, node B will return to node A load balance information for following entries. 1. Downstream C over Non-LAG (upper path). 2. First Downstream C over LAG (middle path). 3. Second Downstream C over LAG (middle path). 4. Downstream D over Non-LAG (lower path). This document defines: o In Section 3, a mechanism to discover capabilities of responder LSRs; o In Section 4, a mechanism to discover L2 ECMP multipath information; o In Section 5, a mechanism to validate L2 ECMP traversal; Akiya, et al. Expires October 6, 2019 [Page 5] Internet-Draft LSP Ping LAG Multipath April 2019 o In Section 6, the LSR Capability TLV; o In Section 7, the LAG Description Indicator flag; o In Section 8, the Local Interface Index Sub-TLV; o In Section 9, the Remote Interface Index Sub-TLV; o In Section 10, the Detailed Interface and Label Stack TLV; 3. LSR Capability Discovery The MPLS Ping operates by an initiator LSR sending an MPLS echo request message and receiving back a corresponding MPLS echo reply message from a responder LSR. The MPLS Traceroute operates in a similar way except the initiator LSR potentially sends multiple MPLS echo request messages with incrementing TTL values. There have been many extensions to the MPLS Ping and Traceroute mechanisms over the years. Thus it is often useful, and sometimes necessary, for the initiator LSR to deterministically disambiguate the differences between: o The responder LSR sent the MPLS echo reply message with contents C because it has feature X, Y and Z implemented. o The responder LSR sent the MPLS echo reply message with contents C because it has subset of features X, Y and Z implemented but not all. o The responder LSR sent the MPLS echo reply message with contents C because it does not have features X, Y and Z implemented. To allow the initiator LSR to disambiguate the above differences, this document defines the LSR Capability TLV (described in Section 6). When the initiator LSR wishes to discover the capabilities of the responder LSR, the initiator LSR includes the LSR Capability TLV in the MPLS echo request message. When the responder LSR receives an MPLS echo request message with the LSR Capability TLV included, if it knows the LSR Capability TLV, then it MUST include the LSR Capability TLV in the MPLS echo reply message with the LSR Capability TLV describing features and extensions supported by the local LSR. Otherwise, an MPLS echo reply must be sent back to the initiator LSR with the return code set to "One or more of the TLVs was not understood", according to the rules as defined Section 3 of [RFC8029]. Then the initiator LSR can send another MPLS echo request without including the LSR Capability TLV. Akiya, et al. Expires October 6, 2019 [Page 6] Internet-Draft LSP Ping LAG Multipath April 2019 It is RECOMMENDED that implementations supporting the LAG Multipath extensions defined in this document include the LSR Capability TLV in MPLS echo request messages. 3.1. Initiator LSR Procedures If an initiator LSR does not know what capabilities a responder LSR can support, it can send an MPLS each request message and carry the LSR Capability TLV to the responder to discover the capabilities that the responder LSR can support. 3.2. Responder LSR Procedures When a responder LSR received an MPLS echo request message that carries the LSR Capability TLV, the following procedures are used: If the responder knows how to process the LSR Capability TLV, the following procedures are used: o The responder LSR MUST include the LSR Capability TLV in the MPLS echo reply message. o If the responder LSR understands the "LAG Description Indicator flag": * Set the "Downstream LAG Info Accommodation flag" if the responder LSR is capable of describing outgoing LAG member links separately; otherwise, clear the "Downstream LAG Info Accommodation flag". * Set the "Upstream LAG Info Accommodation flag" if responder LSR is capable of describing incoming LAG member links separately; otherwise, clear the "Upstream LAG Info Accommodation flag". 4. Mechanism to Discover L2 ECMP Multipath 4.1. Initiator LSR Procedures Through the "LSR Capability Discovery" as defined in Section 3, the initiator LSR can understand whether the responder LSR can describe incoming/outgoing LAG member links separately in the DDMAP TLV. Once the initiator LSR knows that a responder can support this mechanism, then it sends an MPLS echo request carrying a DDMAP TLV with the "LAG Description Indicator flag" (G) set to the responder LSR. The "LAG Description Indicator flag" (G) indicates that separate load balancing information for each L2 nexthop over a LAG is Akiya, et al. Expires October 6, 2019 [Page 7] Internet-Draft LSP Ping LAG Multipath April 2019 desired in the MPLS echo reply. The new "LAG Description Indicator flag" is described in Section 7. 4.2. Responder LSR Procedures When a responder LSR received an MPLS echo request message with the "LAG Description Indicator flag" set in the DDMAP TLV, if the responder LSR understands the "LAG Description Indicator flag" and is capable of describing outgoing LAG member links separately, the following procedures are used, regardless of whether or not outgoing interfaces include LAG interfaces: o For each downstream that is a LAG interface: * The responder LSR MUST include a DDMAP TLV when sending the MPLS echo reply. There is a single DDMAP TLV for the LAG interface, with member links described using sub-TLVs. * The responder LSR MUST set the "LAG Description Indicator flag" in the DS Flags field of the DDMAP TLV. * In the DDMAP TLV, the Local Interface Index Sub-TLV, Remote Interface Index Sub-TLV and Multipath Data Sub-TLV are used to describe each LAG member link. All other fields of the DDMAP TLV are used to describe the LAG interface. * For each LAG member link of the LAG interface: + The responder LSR MUST add a Local Interface Index Sub-TLV (described in Section 8) with the "LAG Member Link Indicator flag" set in the Interface Index Flags field, describing the interface index of this outgoing LAG member link (the local interface index is assigned by the local LSR). + The responder LSR MAY add a Remote Interface Index Sub-TLV (described in Section 9) with the "LAG Member Link Indicator flag" set in the Interface Index Flags field, describing the interface index of the incoming LAG member link on the downstream LSR (this interface index is assigned by the downstream LSR). How the local LSR obtains the interface index of the LAG member link on the downstream LSR is outside the scope of this document. + The responder LSR MUST add an Multipath Data Sub-TLV for this LAG member link, if the received DDMAP TLV requested multipath information. Akiya, et al. Expires October 6, 2019 [Page 8] Internet-Draft LSP Ping LAG Multipath April 2019 Based on the procedures described above, every LAG member link will have a Local Interface Index Sub-TLV and a Multipath Data Sub-TLV entries in the DDMAP TLV. The order of the Sub-TLVs in the DDMAP TLV for a LAG member link MUST be Local Interface Index Sub-TLV immediately followed by Multipath Data Sub-TLV except as follows. A LAG member link MAY also have a corresponding Remote Interface Index Sub-TLV. When a Local Interface Index Sub-TLV, a Remote Interface Index-Sub-TLV and a Multipath Data Sub-TLV are placed in the DDMAP TLV to describe a LAG member link, they MUST be placed in the order of Local Interface Index Sub-TLV, Remote Interface Index-Sub-TLV and Multipath Data Sub-TLV. The block of local interface index, (optional remote interface index) and multipath data sub-TLVs for each member link MUST appear adjacent to each other in order of increasing local interface index. A responder LSR possessing a LAG interface with two member links would send the following DDMAP for this LAG interface: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ DDMAP fields describing LAG interface with DS Flags G set ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Local Interface Index Sub-TLV of LAG member link #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Remote Interface Index Sub-TLV of LAG member link #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Multipath Data Sub-TLV LAG member link #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Local Interface Index Sub-TLV of LAG member link #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Remote Interface Index Sub-TLV of LAG member link #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Multipath Data Sub-TLV LAG member link #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Label Stack Sub-TLV | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 2: Example of DDMAP in MPLS Echo Reply When none of the received multipath information maps to a particular LAG member link, then the responder LSR MUST still place the Local Interface Index Sub-TLV and the Multipath Data Sub-TLV for that LAG member link in the DDMAP TLV. The value of Multipath Length field of the Multipath Data Sub-TLV is set to zero. Akiya, et al. Expires October 6, 2019 [Page 9] Internet-Draft LSP Ping LAG Multipath April 2019 4.3. Additional Initiator LSR Procedures The procedures above allow an initiator LSR to: o Identify whether or not the responder LSR can describe outgoing LAG member links separately, by looking at the LSR Capability TLV. o Utilize the value of the "LAG Description Indicator flag" in DS Flags to identify whether each received DDMAP TLV describes a LAG interface or a non-LAG interface. o Obtain multipath information which is expected to traverse the specific LAG member link described by corresponding interface index. When an initiator LSR receives a DDMAP containing LAG member information from a downstream LSR with TTL=n, then the subsequent DDMAP sent by the initiator LSR to the downstream LSR with TTL=n+1 through a particular LAG member link MUST be updated with following procedures: o The Local Interface Index Sub-TLVs MUST be removed in the sending DDMAP. o If the Remote Interface Index Sub-TLVs were present and the initiator LSR is traversing over a specific LAG member link, then the Remote Interface Index Sub-TLV corresponding to the LAG member link being traversed SHOULD be included in the sending DDMAP. All other Remote Interface Index Sub-TLVs MUST be removed from the sending DDMAP. o The Multipath Data Sub-TLVs MUST be updated to include just one Multipath Data Sub-TLV. The initiator LSR MAY just keep the Multipath Data Sub-TLV corresponding to the LAG member link being traversed, or combine the Multipath Data Sub-TLVs for all LAG member links into a single Multipath Data Sub-TLV when diagnosing further downstream LSRs. o All other fields of the DDMAP are to comply with procedures described in [RFC8029]. Figure 3 is an example that shows how to use the DDMAP TLV to notify which member link (link #1 in the example) will be chosen to send the MPLS echo request message to the next downstream LSR: Akiya, et al. Expires October 6, 2019 [Page 10] Internet-Draft LSP Ping LAG Multipath April 2019 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ DDMAP fields describing LAG interface with DS Flags G set ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |[OPTIONAL] Remote Interface Index Sub-TLV of LAG member link #1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Multipath Data Sub-TLV LAG member link #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Label Stack Sub-TLV | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 3: Example of DDMAP in MPLS Echo Request 5. Mechanism to Validate L2 ECMP Traversal Section 4 defines the responder LSR procedures to construct a DDMAP for a downstream LAG. The Remote Interface Index Sub-TLVs that describes the incoming LAG member links of the downstream LSR is optional, because this information from the downstream LSR is often not available on the responder LSR. In such case, the traversal of LAG member links can be validated with procedures described in Section 5.1. If LSRs can provide the Remote Interface Index Sub- TLVs, then the validation procedures described in Section 5.2 can be used. 5.1. Incoming LAG Member Links Verification Without downstream LSRs returning remote Interface Index Sub-TLVs in the DDMAP, validation of the LAG member link traversal requires that initiator LSR traverses all available LAG member links and taking the results through a logic. This section provides the mechanism for the initiator LSR to obtain additional information from the downstream LSRs and describes the additional logic in the initiator LSR to validate the L2 ECMP traversal. 5.1.1. Initiator LSR Procedures An MPLS echo request carrying a DDMAP TLV with the "Interface and Label Stack Object Request flag" and "LAG Description Indicator flag" set is sent to indicate the request for Detailed Interface and Label Stack TLV with additional LAG member link information (i.e. interface index) in the MPLS echo reply. Akiya, et al. Expires October 6, 2019 [Page 11] Internet-Draft LSP Ping LAG Multipath April 2019 5.1.2. Responder LSR Procedures When received an echo request with the "LAG Description Indicator flag" set, a responder LSR that understands the "LAG Description Indicator flag" and is capable of describing incoming LAG member link SHOULD use the following procedures, regardless of whether or not incoming interface was a LAG interface: o When the "I" flag ( "Interface and Label Stack Object Request flag") of the DDMAP TLV in the received MPLS echo request is set: * The responder LSR MUST add the Detailed Interface and Label Stack TLV (described in Section 10) in the MPLS echo reply. * If the incoming interface is a LAG, the responder LSR MUST add the Incoming Interface Index Sub-TLV (described in Section 10.1.2) in the Detailed Interface and Label Stack TLV. The "LAG Member Link Indicator flag" MUST be set in the Interface Index Flags field, and the Interface Index field set to the LAG member link which received the MPLS echo request. These procedures allow initiator LSR to: o Utilize the Incoming Interface Index Sub-TLV in the Detailed Interface and Label Stack TLV to derive, if the incoming interface is a LAG, the identity of the incoming LAG member. 5.1.3. Additional Initiator LSR Procedures Along with procedures described in Section 4, the procedures described in this section will allow an initiator LSR to know: o The expected load balance information of every LAG member link, at LSR with TTL=n. o With specific entropy, the expected interface index of the outgoing LAG member link at TTL=n. o With specific entropy, the interface index of the incoming LAG member link at TTL=n+1. Depending on the LAG traffic division algorithm, the messages may or may not traverse different member links. The expectation is that there's a relationship between the interface index of the outgoing LAG member link at TTL=n and the interface index of the incoming LAG member link at TTL=n+1 for all entropies examined. In other words, set of entropies that load balances to outgoing LAG member link X at Akiya, et al. Expires October 6, 2019 [Page 12] Internet-Draft LSP Ping LAG Multipath April 2019 TTL=n should all reach the nexthop on same incoming LAG member link Y at TTL=n+1. With additional logic, the initiator LSR can perform the following checks in a scenario where the initiator LSR knows that there is a LAG, with two LAG members, between TTL=n and TTL=n+1, and has the multipath information to traverse the two LAG member links. The initiator LSR sends two MPLS echo request messages to traverse the two LAG member links at TTL=n+1: o Success case: * One MPLS echo request message reaches TTL=n+1 on an LAG member link 1. * The other MPLS echo request message reaches TTL=n+1 on an LAG member link 2. The two MPLS echo request messages sent by the initiator LSR reach at the immediate downstream LSR from two different LAG member links. o Error case: * One MPLS echo request message reaches TTL=n+1 on an LAG member link 1. * The other MPLS echo request message also reaches TTL=n+1 on an LAG member link 1. * One or both MPLS echo request messages cannot reach the immediate downstream LSR on whichever link. One or two MPLS echo request messages sent by the initiator LSR cannot reach the immediate downstream LSR, or the two MPLS echo request messages reach at the immediate downstream LSR from the same LAG member link. Note that the above defined procedures will provide a deterministic result for LAG interfaces that are back-to-back connected between LSRs (i.e. no L2 switch in between). If there is a L2 switch between the LSR at TTL=n and the LSR at TTL=n+1, there is no guarantee that traversal of every LAG member link at TTL=n will result in reaching from different interface at TTL=n+1. Issues resulting from LAG with L2 switch in between are further described in Appendix A. LAG provisioning models in operated network should be considered when analyzing the output of LSP Traceroute exercising L2 ECMPs. Akiya, et al. Expires October 6, 2019 [Page 13] Internet-Draft LSP Ping LAG Multipath April 2019 5.2. Individual End-to-End Path Verification When the Remote Interface Index Sub-TLVs are available from an LSR with TTL=n, then the validation of LAG member link traversal can be performed by the downstream LSR of TTL=n+1. The initiator LSR follows the procedures described in Section 4.3. The DDMAP validation procedures for the downstream responder LSR are then updated to include the comparison of the incoming LAG member link to the interface index described in the Remote Interface Index Sub-TLV in the DDMAP TLV. Failure of this comparison results in the return code being set to "Downstream Mapping Mismatch (5)". 6. LSR Capability TLV This document defines a new TLV which is referred to as the "LSR Capability TLV. It MAY be included in the MPLS echo request message and the MPLS echo reply message. An MPLS echo request message and an MPLS echo reply message MUST NOT include more than one LSR Capability TLV. The presence of an LSR Capability TLV in an MPLS echo request message is a request that a responder LSR includes an LSR Capability TLV in the MPLS echo reply message, with the LSR Capability TLV describing features and extensions that the responder LSR supports. The format of the LSR Capability TLV is as below: LSR Capability TLV Type is TBD1. Length is 4. The value field of the LSR Capability TLV has following format: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | LSR Capability Flags | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 4: LSR Capability TLV Where: The Type field is 2 octets in length and the value is TBD1. The Length field is 2 octets in length, and the value is 4. The "LSR Capability Flags" field is 4 octets in length, this document defines the following flags: Akiya, et al. Expires October 6, 2019 [Page 14] Internet-Draft LSP Ping LAG Multipath April 2019 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved (Must Be Zero) |U|D| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ This document defines two flags. The unallocated flags MUST be set to zero when sending and ignored on receipt. Both the U and the D flag MUST be cleared in the MPLS echo request message when sending, and ignored on receipt. Neither, either or both the U and the D flag MAY be set in the MPLS echo reply message. Flag Name and Meaning ---- ---------------- U Upstream LAG Info Accommodation An LSR sets this flag when the LSR is capable of describing a LAG member link in the Incoming Interface Index Sub-TLV in the Detailed Interface and Label Stack TLV. D Downstream LAG Info Accommodation An LSR sets this flag when the LSR is capable of describing LAG member links in the Local Interface Index Sub-TLV and the Multipath Data Sub-TLV in the Downstream Detailed Mapping TLV. 7. LAG Description Indicator Flag: G This document defines a new flag, the "G" flag (LAG Description Indicator), in the DS Flags field of the DDMAP TLV. The "G" flag in the MPLS echo request message indicates the request for detailed LAG information from the responder LSR. In the MPLS echo reply message, the "G" flag MUST be set if the DDMAP TLV describes a LAG interface. It MUST be cleared otherwise. The "G" flag is defined as below: The Bit Number is TBD5. 0 1 2 3 4 5 6 7 +-+-+-+-+-+-+-+-+ | MBZ |G|E|L|I|N| +-+-+-+-+-+-+-+-+ Akiya, et al. Expires October 6, 2019 [Page 15] Internet-Draft LSP Ping LAG Multipath April 2019 RFC-Editor-Note: Please update above figure to place the G flag in the bit number TBD5. Flag Name and Meaning ---- ---------------- G LAG Description Indicator When this flag is set in the MPLS echo request, the responder LSR is requested to respond with detailed LAG information. When this flag is set in the MPLS echo reply, the corresponding DDMAP TLV describes a LAG interface. 8. Local Interface Index Sub-TLV The Local Interface Index Sub-TLV describes the interface index assigned by the local LSR to an egress interface. One or more Local Interface Index sub-TLVs MAY appear in a DDMAP TLV. The format of the Local Interface Index Sub-TLV is below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Local Interface Index | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 5: Local Interface Index Sub-TLV Where: o The "Type" field is 2 octets in length, the value is TBD2. o The "Length" filed 2 octets in length, and the value is 4. o The "Local Interface Index" field is 4 octets in length, it is an interface index assigned by a local LSR to an egress interface. It's normally an unsigned integer and in network byte order. 9. Remote Interface Index Sub-TLV The Remote Interface Index Sub-TLV is an optional TLV, it describes the interface index assigned by a downstream LSR to an ingress interface. One or more Remote Interface Index sub-TLVs MAY appear in a DDMAP TLV. Akiya, et al. Expires October 6, 2019 [Page 16] Internet-Draft LSP Ping LAG Multipath April 2019 The format of the Remote Interface Index Sub-TLV is as below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Remote Interface Index | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 6: Remote Interface Index Sub-TLV Where: o The "Type" field is 2 octets in length, and the value is TBD3. o The "Length" field is 2 octets in length, and the value is 4. o The "Remote Interface Index" is 4 octets in length, it is an interface index assigned by a downstream LSR to an ingress interface. It's normally an unsigned integer and in network byte order. 10. Detailed Interface and Label Stack TLV The "Detailed Interface and Label Stack" object is a TLV that MAY be included in an MPLS echo reply message to report the interface on which the MPLS echo request message was received and the label stack that was on the packet when it was received. A responder LSR MUST NOT insert more than one instance of this TLV into the MPLS echo reply message. This TLV allows the initiator LSR to obtain the exact interface and label stack information as it appears at the responder LSR. Detailed Interface and Label Stack TLV Type is TBD4. Length is K + Sub-TLV Length (sum of Sub-TLVs). K is the sum of all fields of this TLV prior to Sub-TLVs, but the length of K depends on the Address Type. Details of this information is described below. The Value field has following format: Akiya, et al. Expires October 6, 2019 [Page 17] Internet-Draft LSP Ping LAG Multipath April 2019 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Address Type | Reserved (Must Be Zero) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP Address (4 or 16 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Interface (4 or 16 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . List of Sub-TLVs . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 7: Detailed Interface and Label Stack TLV The Detailed Interface and Label Stack TLV format is derived from the Interface and Label Stack TLV format (from [RFC8029]). Two changes are introduced. The first is that the label stack is converted into a sub-TLV. The second is that a new sub-TLV is added to describe an interface index. The other fields of Detailed Interface and Label Stack TLV have the same use and meaning as in [RFC8029]. A summary of these fields is as below: Address Type The Address Type indicates if the interface is numbered or unnumbered. It also determines the length of the IP Address and Interface fields. The resulting total length of the initial part of the TLV is listed as "K Octets". The Address Type is set to one of the following values: Type # Address Type K Octets ------ ------------ -------- 1 IPv4 Numbered 16 2 IPv4 Unnumbered 16 3 IPv6 Numbered 40 4 IPv6 Unnumbered 28 IP Address and Interface IPv4 addresses and interface indices are encoded in 4 octets; IPv6 addresses are encoded in 16 octets. If the interface upon which the echo request message was received is numbered, then the Address Type MUST be set to IPv4 Akiya, et al. Expires October 6, 2019 [Page 18] Internet-Draft LSP Ping LAG Multipath April 2019 Numbered or IPv6 Numbered, the IP Address MUST be set to either the LSR's Router ID or the interface address, and the Interface MUST be set to the interface address. If the interface is unnumbered, the Address Type MUST be either IPv4 Unnumbered or IPv6 Unnumbered, the IP Address MUST be the LSR's Router ID, and the Interface MUST be set to the index assigned to the interface. Note: Usage of IPv6 Unnumbered has the same issue as [RFC8029], described in Section 3.4.2 of [RFC7439]. A solution should be considered an applied to both [RFC8029] and this document. 10.1. Sub-TLVs This section defines the sub-TLVs that MAY be included as part of the Detailed Interface and Label Stack TLV. Two sub-TLVs are defined: Sub-Type Sub-TLV Name --------- ------------ 1 Incoming Label stack 2 Incoming Interface Index 10.1.1. Incoming Label Stack Sub-TLV The Incoming Label Stack sub-TLV contains the label stack as received by an LSR. If any TTL values have been changed by this LSR, they SHOULD be restored. Incoming Label Stack Sub-TLV Type is 1. Length is variable, and its format is as below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Label | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Label | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 8: Incoming Label Stack Sub-TLV Akiya, et al. Expires October 6, 2019 [Page 19] Internet-Draft LSP Ping LAG Multipath April 2019 10.1.2. Incoming Interface Index Sub-TLV The Incoming Interface Index object is a Sub-TLV that MAY be included in a Detailed Interface and Label Stack TLV. The Incoming Interface Index Sub-TLV describes the index assigned by a local LSR to the interface which received the MPLS echo request message. Incoming Interface Index Sub-TLV Type is 2. Length is 8, and its format is as below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Interface Index Flags | Reserved (Must Be Zero) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Incoming Interface Index | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 9: Incoming Interface Index Sub-TLV Interface Index Flags Interface Index Flags field is a bit vector with following format. 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved (Must Be Zero) |M| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ One flag is defined: M. The remaining flags MUST be set to zero when sending and ignored on receipt. Flag Name and Meaning ---- ---------------- M LAG Member Link Indicator When this flag is set, interface index described in this sub-TLV is a member of a LAG. Incoming Interface Index An Index assigned by the LSR to this interface. It's normally an unsigned integer and in network byte order. Akiya, et al. Expires October 6, 2019 [Page 20] Internet-Draft LSP Ping LAG Multipath April 2019 11. Rate Limiting On Echo Request/Reply Messages For an LSP path, it may be over several LAGs. Each LAG may have many member links. To exercise all the links, many Echo Request/Reply messages will be sent in a short period. It's possible that those messages may traverse a common path as a burst. Under some circumstances this might cause congestion at the common path. To avoid potential congestion, it is RECOMMENDED that implementations to randomly delay the Echo Request and Reply messages at the Initiating LSRs and Responder LSRs. Rate limiting of ping traffic is further specified in [RFC8029] (Section 5) and [RFC6425] (Section 4.1) which apply to this document as well. 12. Security Considerations This document extends LSP Traceroute mechanism [RFC8029] to discover and exercise L2 ECMP paths to determine problematic member link(s) of a LAG. These on-demand diagnostic mechanisms are used by an operator within an MPLS control domain. [RFC8029] reviews the possible attacks and approaches to mitigate possible threats when using these mechanisms. To prevent leakage of vital information to untrusted users, a responder LSR MUST only accept MPLS echo request messages from designated trusted sources via filtering source IP address field of received MPLS echo request messages. As noted in [RFC8029], spoofing attacks only have a small window of opportunity. If these messages are indeed hijacked (non-delivery) by an intermediate node, the use of these mechanisms will determine the data plane is not working (as it should). Hijacking of a responder node such that it provides a legitimate reply would involve compromising the node itself and the MPLS control domain. [RFC5920] provides additional MPLS network-wide operation recommendations to avoid attacks and recommendations to follow. Please note that source IP address filtering provides only a weak form of access control and is not, in general, a reliable security mechanism. Nonetheless, it is required here in the absence of any more robust mechanisms that might be used. 13. IANA Considerations 13.1. LSR Capability TLV The IANA is requested to assign new value TBD1 (from the range 4-16383) for LSR Capability TLV from the "Multiprotocol Label Switching Architecture (MPLS) Label Switched Paths (LSPs) Ping Parameters - TLVs" registry. Akiya, et al. Expires October 6, 2019 [Page 21] Internet-Draft LSP Ping LAG Multipath April 2019 Value Meaning Reference ----- ------- --------- TBD1 LSR Capability TLV this document 13.1.1. LSR Capability Flags The IANA is requested to create and maintain a registry entitled "LSR Capability Flags" with following registration procedures: Registry Name: LAG Interface Info Flags Bit number Name Reference ---------- ---------------------------------------- --------- 31 D: Downstream LAG Info Accommodation this document 30 U: Upstream LAG Info Accommodation this document 0-29 Unassigned Assignments of LSR Capability Flags are via Standards Action [RFC8126]. 13.2. Local Interface Index Sub-TLV The IANA is requested to assign new value TBD2 (from the range 4-16383) for the Local Interface Index Sub-TLV from the "Multiprotocol Label Switching Architecture (MPLS) Label Switched Paths (LSPs) Ping Parameters - TLVs" registry, "Sub-TLVs for TLV Types 20" sub-registry. Value Meaning Reference ----- ------- --------- TBD2 Local Interface Index Sub-TLV this document 13.2.1. Interface Index Flags The IANA is requested to create and maintain a registry entitled "Interface Index Flags" with following registration procedures: Registry Name: Interface Index Flags Bit number Name Reference ---------- ---------------------------------------- --------- 15 M: LAG Member Link Indicator this document 0-14 Unassigned Assignments of Interface Index Flags are via Standards Action [RFC8126]. Akiya, et al. Expires October 6, 2019 [Page 22] Internet-Draft LSP Ping LAG Multipath April 2019 Note that this registry is used by the Interface Index Flags field of following Sub-TLVs: o The Local Interface Index Sub-TLV which may be present in the "Downstream Detailed Mapping" TLV. o The Remote Interface Index Sub-TLV which may be present in the "Downstream Detailed Mapping" TLV. o The Incoming Interface Index Sub-TLV which may be present in the "Detailed Interface and Label Stack" TLV. 13.3. Remote Interface Index Sub-TLV The IANA is requested to assign new value TBD3 (from the range 32768-49161) for the Remote Interface Index Sub-TLV from the "Multiprotocol Label Switching Architecture (MPLS) Label Switched Paths (LSPs) Ping Parameters - TLVs" registry, "Sub-TLVs for TLV Types 20" sub-registry. Value Meaning Reference ----- ------- --------- TBD3 Remote Interface Index Sub-TLV this document 13.4. Detailed Interface and Label Stack TLV The IANA is requested to assign new value TBD4 (from the range 4-16383) for Detailed Interface and Label Stack TLV from the "Multiprotocol Label Switching Architecture (MPLS) Label Switched Paths (LSPs) Ping Parameters - TLVs" registry ([IANA-MPLS-LSP-PING]). Value Meaning Reference ----- ------- --------- TBD4 Detailed Interface and Label Stack TLV this document 13.4.1. Sub-TLVs for TLV Type TBD4 The IANA is requested to create and maintain a sub-registry entitled "Sub-TLVs for TLV Type TBD4" under "Multiprotocol Label Switching Architecture (MPLS) Label Switched Paths (LSPs) Ping Parameters - TLVs" registry. Initial values for this sub-registry, "Sub-TLVs for TLV Types TBD4", are described below. Akiya, et al. Expires October 6, 2019 [Page 23] Internet-Draft LSP Ping LAG Multipath April 2019 Sub-Type Name Reference ----------- -------------------------------------- --------- 1 Incoming Label Stack this document 2 Incoming Interface Index this document 3-16383 Unassigned (mandatory TLVs) 16384-31743 Specification Required 32768-49161 Unassigned (optional TLVs) 49162-64511 Specification Required Assignments of Sub-Types in the mandatory and optional spaces are via Standards Action [RFC8126]. Assignments of Sub-Types in the Specification Required space is via Specification Required [RFC8126]. 13.4.2. Interface and Label Stack Address Types Since the "Detailed Interface and Label Stack TLV" shares the "Interface and Label Stack Address Types" with the "Interface and Label Stack TLV". IANA is requested to update the "Interface and Label Stack Address Types" registry to reflect this. For example, change the registry name to "Interface and Label Stack and Detailed Interface and Label Stack Address Types", and add a reference to this document. 13.5. DS Flags The IANA is requested to assign a new bit number from the "DS flags" sub-registry from the "Multi-Protocol Label Switching (MPLS) Label Switched Paths (LSPs) Ping Parameters - TLVs" registry ([IANA-MPLS-LSP-PING]). Note: the "DS flags" sub-registry is created by [RFC8029]. Bit number Name Reference ---------- ---------------------------------------- --------- TBD5 G: LAG Description Indicator this document 14. Acknowledgements The authors would like to thank Nagendra Kumar, Sam Aldrin, for providing useful comments and suggestions. The authors would like to thank Loa Andersson for performing a detailed review and providing number of comments. The authors also would like to extend sincere thanks to the MPLS RT review members who took time to review and provide comments. The members are Eric Osborne, Mach Chen and Yimin Shen. The suggestion by Mach Chen to generalize and create the LSR Capability TLV was Akiya, et al. Expires October 6, 2019 [Page 24] Internet-Draft LSP Ping LAG Multipath April 2019 tremendously helpful for this document and likely for future documents extending the MPLS LSP Ping and Traceroute mechanisms. The suggestion by Yimin Shen to create two separate validation procedures had a big impact to the contents of this document. 15. References 15.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC8029] Kompella, K., Swallow, G., Pignataro, C., Ed., Kumar, N., Aldrin, S., and M. Chen, "Detecting Multiprotocol Label Switched (MPLS) Data-Plane Failures", RFC 8029, DOI 10.17487/RFC8029, March 2017, . [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, DOI 10.17487/RFC8126, June 2017, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . 15.2. Informative References [IANA-MPLS-LSP-PING] IANA, "Multi-Protocol Label Switching (MPLS) Label Switched Paths (LSPs) Ping Parameters", . [IEEE802.1AX] IEEE Std. 802.1AX, "IEEE Standard for Local and metropolitan area networks - Link Aggregation", November 2008. [RFC5920] Fang, L., Ed., "Security Framework for MPLS and GMPLS Networks", RFC 5920, DOI 10.17487/RFC5920, July 2010, . Akiya, et al. Expires October 6, 2019 [Page 25] Internet-Draft LSP Ping LAG Multipath April 2019 [RFC6425] Saxena, S., Ed., Swallow, G., Ali, Z., Farrel, A., Yasukawa, S., and T. Nadeau, "Detecting Data-Plane Failures in Point-to-Multipoint MPLS - Extensions to LSP Ping", RFC 6425, DOI 10.17487/RFC6425, November 2011, . [RFC7439] George, W., Ed. and C. Pignataro, Ed., "Gap Analysis for Operating IPv6-Only MPLS Networks", RFC 7439, DOI 10.17487/RFC7439, January 2015, . Appendix A. LAG with intermediate L2 Switch Issues Several flavors of "LAG with L2 switch" provisioning models and the corresponding MPLS data plane ECMP traversal validation issues are described in this section . A.1. Equal Numbers of LAG Members R1 ==== S1 ==== R2 The issue with this LAG provisioning model is that packets traversing a LAG member from Router 1 (R1) to intermediate L2 switch (S1) can get load balanced by S1 towards Router 2 (R2). Therefore, MPLS echo request messages traversing a specific LAG member from R1 to S1 can actually reach R2 via any of the LAG members, and the sender of MPLS echo request messages has no knowledge of this nor no way to control this traversal. In the worst case, MPLS echo request messages with specific entropies to exercise every LAG members from R1 to S1 can all reach R2 via same LAG member. Thus it is impossible for MPLS echo request sender to verify that packets intended to traverse specific LAG member from R1 to S1 did actually traverse that LAG member, and to deterministically exercise "receive" processing of every LAG member on R2. (Notes, AFAICT there's not a better option than "try a bunch of entropy labels and see what responses you can get back" and that's the same remedy in all the described topologies.) A.2. Deviating Numbers of LAG Members ____ R1 ==== S1 ==== R2 There are deviating number of LAG members on the two sides of the L2 switch. The issue with this LAG provisioning model is the same as previous model, sender of MPLS echo request messages have no knowledge of L2 load balance algorithm nor entropy values to control the traversal. Akiya, et al. Expires October 6, 2019 [Page 26] Internet-Draft LSP Ping LAG Multipath April 2019 A.3. LAG Only on Right R1 ---- S1 ==== R2 The issue with this LAG provisioning model is that there is no way for MPLS echo request sender to deterministically exercise both LAG members from S1 to R2. And without such, "receive" processing of R2 on each LAG member cannot be verified. A.4. LAG Only on Left R1 ==== S1 ---- R2 MPLS echo request sender has knowledge of how to traverse both LAG members from R1 to S1. However, both types of packets will terminate on the non-LAG interface at R2. It becomes impossible for MPLS echo request sender to know that MPLS echo request messages intended to traverse a specific LAG member from R1 to S1 did indeed traverse that LAG member. Authors' Addresses Nobo Akiya Big Switch Networks Email: nobo.akiya.dev@gmail.com George Swallow Cisco Systems Email: swallow@cisco.com Stephane Litkowski Orange Email: stephane.litkowski@orange.com Bruno Decraene Orange Email: bruno.decraene@orange.com Akiya, et al. Expires October 6, 2019 [Page 27] Internet-Draft LSP Ping LAG Multipath April 2019 John E. Drake Juniper Networks Email: jdrake@juniper.net Mach(Guoyi) Chen Huawei Email: mach.chen@huawei.com Akiya, et al. Expires October 6, 2019 [Page 28]