NVO3 P. Jain Internet-Draft K. Singh Intended status: Standards Track F. Balus Expires: August 16, 2014 Nuage Networks W. Henderickx Alcatel-Lucent V. Bannai PayPal R. Shekhar A. Lohiya Juniper Networks February 12, 2014 Generic Overlay OAM and Datapath Failure Detection draft-jain-nvo3-overlay-oam-00 Abstract This proposal describes a mechanism that can be used to detect Data Path Failures of various overlay technologies as VXLAN, NVGRE, MPLSoGRE and MPLSoUDP and verifying/sanity of their Control and Data Plane for given Overlay Segment. This document defines the following for each of the above Overlay Technologies: o Encapsulation of OAM Packet, such that it has same Outer and Overlay Header as any End-System's data going over the same Overlay Segment. o The mechanism to trace the Underlay that is exercised by any Overlay Segment. o Procedure to verify presence of any given Tenant VM or End-System within a given Overlay Segment at Overlay End-Point. Even though the present proposal addresses Overlay OAM for VXLAN, NVGRE, MPLSoGRE and MPLSoUDP, but the procedures described are generic enough to accommodate OAM for any other Overlay Technology. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Jain, et al. Expires August 16, 2014 [Page 1] Internet-Draft Detecting Overlay Segment Failure February 2014 Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on August 16, 2014. Copyright Notice Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Jain, et al. Expires August 16, 2014 [Page 2] Internet-Draft Detecting Overlay Segment Failure February 2014 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 6 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 8 3. Motivation for Overlay OAM . . . . . . . . . . . . . . . . . . 9 4. Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 5. Packet Format . . . . . . . . . . . . . . . . . . . . . . . . 11 5.1. Overlay OAM Encapsulation in Layer 2 Context . . . . . . . 11 5.2. Overlay OAM Encapsulation in Layer 3 Context . . . . . . . 11 5.3. Generic Overlay OAM Packet Format . . . . . . . . . . . . 11 5.3.1. TLV Types for various Overlay Ping Models . . . . . . 14 5.3.1.1. TLV for VXLAN Ping . . . . . . . . . . . . . . . . 15 5.3.1.2. TLV for NVGRE Ping . . . . . . . . . . . . . . . . 16 5.3.1.3. TLV for MPLSoGRE Ping . . . . . . . . . . . . . . 17 5.3.1.4. TLV for MPLSoUDP Ping . . . . . . . . . . . . . . 18 6. Return Codes . . . . . . . . . . . . . . . . . . . . . . . . . 19 7. Procedure for Overlay Segment Ping . . . . . . . . . . . . . . 20 7.1. Encoding of Inner Header for Echo Request in Layer 2 Context . . . . . . . . . . . . . . . . . . . . . . . . . 20 7.2. Encoding of Inner Header for Echo Request in Layer 3 Context . . . . . . . . . . . . . . . . . . . . . . . . . 21 7.3. VXLAN Procedures . . . . . . . . . . . . . . . . . . . . . 21 7.3.1. Sending VXLAN Echo Request . . . . . . . . . . . . . . 21 7.3.2. Receiving VXLAN Echo Request . . . . . . . . . . . . . 22 7.3.3. Sending VXLAN Echo Reply . . . . . . . . . . . . . . . 23 7.3.4. Receiving VXLAN Echo Reply . . . . . . . . . . . . . . 23 7.4. NVGRE Procedures . . . . . . . . . . . . . . . . . . . . . 23 7.4.1. Sending NVGRE Echo Request . . . . . . . . . . . . . . 23 7.4.2. Receiving NVGRE Echo Request . . . . . . . . . . . . . 24 7.4.3. Sending NVGRE Echo Reply . . . . . . . . . . . . . . . 25 7.4.4. Receiving NVGRE Echo Reply . . . . . . . . . . . . . . 25 7.5. MPLSoGRE Procedures . . . . . . . . . . . . . . . . . . . 25 7.5.1. Sending MPLSoGRE Echo Request . . . . . . . . . . . . 25 7.5.2. Receiving MPLSoGRE Echo Request . . . . . . . . . . . 25 7.5.3. Sending MPLSoGRE Echo Reply . . . . . . . . . . . . . 26 7.5.4. Receiving MPLSoGRE Echo Reply . . . . . . . . . . . . 26 7.6. MPLSoUDP Procedures . . . . . . . . . . . . . . . . . . . 26 7.6.1. Sending MPLSoUDP Echo Request . . . . . . . . . . . . 26 7.6.2. Receiving MPLSoUDP Echo Request . . . . . . . . . . . 27 7.6.3. Sending MPLSoUDP Echo Reply . . . . . . . . . . . . . 28 7.6.4. Receiving MPLSoUDP Echo Reply . . . . . . . . . . . . 28 Jain, et al. Expires August 16, 2014 [Page 3] Internet-Draft Detecting Overlay Segment Failure February 2014 8. Procedure for Trace . . . . . . . . . . . . . . . . . . . . . 29 9. Procedure for End-System Ping . . . . . . . . . . . . . . . . 30 9.1. Sub-TLV for End-System Ping . . . . . . . . . . . . . . . 30 9.1.1. Sub-TLV for Validating End-System MAC Address . . . . 31 9.1.2. Sub-TLV for Validating End-System IP Address . . . . . 32 9.1.3. Sub-TLV for Validating End-System MAC and IP Address . . . . . . . . . . . . . . . . . . . . . . . 33 9.2. Sending End-System Ping Request . . . . . . . . . . . . . 34 9.3. Receiving End-System Ping Request . . . . . . . . . . . . 34 9.4. Sending End-System Ping Reply . . . . . . . . . . . . . . 35 9.5. Receiving End-System Ping Reply . . . . . . . . . . . . . 35 10. Security Considerations . . . . . . . . . . . . . . . . . . . 37 11. Management Considerations . . . . . . . . . . . . . . . . . . 38 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 39 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 40 14. References . . . . . . . . . . . . . . . . . . . . . . . . . . 41 14.1. Normative References . . . . . . . . . . . . . . . . . . . 41 14.2. Informative References . . . . . . . . . . . . . . . . . . 42 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 43 Jain, et al. Expires August 16, 2014 [Page 4] Internet-Draft Detecting Overlay Segment Failure February 2014 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC2119 [RFC2119]. When used in lower case, these words convey their typical use in common language, and are not to be interpreted as described in RFC2119 [RFC2119]. Jain, et al. Expires August 16, 2014 [Page 5] Internet-Draft Detecting Overlay Segment Failure February 2014 1. Introduction VXLAN [I-D.draft-mahalingam-dutt-dcops-vxlan], NVGRE [I-D.draft-sridharan-virtualization-nvgre], MPLSoGRE [RFC4023] and MPLSoUDP [I-D.draft-ietf-mpls-in-udp] are well known technologies and are used as tunneling mechanism to Overlay either Layer 2 networks or Layer 3 networks on top of Layer 3 Underlay networks. For all above Overlay Models there are two Tunnel End Points for a given Overlay Segment. One End Point is where the Overlay Originates, and other where Overlay Terminates. In most cases the Tunnel End Point is intended to be at the edge of the network, typically connecting an access switch to an IP transport network. The access switch could be a physical or a virtual switch located within the hypervisor on the server which is connected to End System which is a VM. This document describes a mechanism that can be used to detect Data Plane failures and sanity of Overlay Control and Data Plane for a given Overlay Segment, and the method to trace the Underlay path that is exercised by any given Overlay Segment. The document also defines procedures for validating the presence of any given Tenant VM/End-System/End-System or Flow representing the End-System System within a given Overlay Segment. The proposal describes: o The mechanism to verify Overlay Control Plane and Data Plane consistency at the Overlay End Point(s), by encapsulating the OAM Packet in exact the same way as that of any End System Traffic that is transported over the Overlay Segment. o The mechanism to trace the Underlay that is exercised by any Overlay Segment. o The mechanism to verify presence of any "End-System" in a given Overlay Segment. The proposal defines the information to check correct operation of the Data Plane, as well as a mechanism to verify the Data Plane against the Control Plane for a given Overlay Segment. It is important consideration in this proposal to carry Echo Request along same Data Path that any End System's data using the given Overlay Segment takes. The tenants VM(s) or End System(s) are not aware of the Overlays and as such the need for the verification of the Data Path MUST solely rest with the Cloud Provider. The use cases where the Tenant VM(s) Jain, et al. Expires August 16, 2014 [Page 6] Internet-Draft Detecting Overlay Segment Failure February 2014 need to be aware of the Data Plane failures is beyond the scope of this document. Jain, et al. Expires August 16, 2014 [Page 7] Internet-Draft Detecting Overlay Segment Failure February 2014 2. Terminology Terminology used in this document: OAM: Operations, Administration, and Management VXLAN: Virtual eXtensible Local Area Network. NVGRE: Network Virtualization using GRE. MPLSoGRE: Encapsulating MPLS in IP or Generic Routing Encapsulation (GRE) MPLSoUDP: Encapsulating MPLS in UDP. Originating End Point: Overlay Segment's Head End or Starting Point of Overlay Tunnel. Terminating End Point: Overlay Segment's Tail End or Terminating Point of Overlay Tunnel. VM: Virtual Machine. VNI: VXLAN Network Identifier (or VXLAN Segment ID) VSID: Virtual Subnet ID. (for NVGRE) NVE: Network Virtualized Edge End System: Could be Tenant VM, Host, Bridge etc. - System whose data is expected to go over Overlay Segment. Echo Request: Throughout this document, Echo Request packet is expected to be transmitted by Originator Overlay End Point and destined to Overlay Terminating End Point. Echo Reply: Throughout this document, Echo Reply packet is expected to be transmitted by Terminating Overlay End Point and destined to Overlay Originating End Point. Other terminologies are as defined in [I-D.draft-mahalingam-dutt-dcops-vxlan], [I-D.draft-sridharan-virtualization-nvgre], [RFC4023] and [I-D.draft-ietf-mpls-in-udp] Jain, et al. Expires August 16, 2014 [Page 8] Internet-Draft Detecting Overlay Segment Failure February 2014 3. Motivation for Overlay OAM When any Overlay Segment fails to deliver user traffic, there is a need to provide a tool that would enable users, as Cloud Providers to detect such failures, and a mechanism to isolate faults. It may also be desirable to test the data path before mapping End System traffic to the Overlay Segment. The basic idea is to facilitate following verifications:- o End-System's data that are expected to go over a particular Overlay Segment actually ends up using the Data-Path represented by given Overlay Segment between the two End-Points. o To verify the correct value of Overlay Segment Identifier is programmed at Originating and Terminating End Point(s) for a given Overlay Segment. Segment Identifier will be VNI for VXLAN, VSID for NVGRE, MPLS Label for MPLSoGRE and MPLSoUDP. o The facilitate mechanism to trace the Underlay that is exercised by any Overlay Segment. o The mechanism to verify presence of any "End-System" in a given Overlay Segment. To facilitate verification of Overlay Segment or any End-System using the Overlay, this document proposes sending of a Packet (called an "Echo Request") along the same data path as other Packets belonging to this Segment. Echo Request also carries information about the Overlay Segment whose Data Path is to be verified. This Echo Request is forwarded just like any other End System Data Packet belonging to that Overlay Segment, as it contains the same Overlay Encapsulation as regular End System's data. On receiving Echo Request at the end of the Overlay Segment, it is sent to the Control Plane of the Terminating Overlay End Point, which in-turn would respond with Echo Reply. To facilitate tracing of the Underlay used by any given Overlay Segment, the document proposes Echo Request/Reply encapsulation in "trace mode", which would allow the user or Cloud Provider to gather information of the Underlay network. Jain, et al. Expires August 16, 2014 [Page 9] Internet-Draft Detecting Overlay Segment Failure February 2014 4. Approach The proposal aims at validating Data Plane and its view of Control Plane for a particular Overlay Segment. To achieve this aim, the draft proposes creating an Overlay OAM Packet which MUST be encapsulated with the Overlay Header as that of any End-Point data going over the same Overlay Segment. This would guarantee the data- path for OAM Packet follows the same path as that for any End User data going over the same Overlay Segment. The draft outlines procedures to encode Overlay Header and Inner Ethernet or IP Header based on the type of payload that Overlay is expected to carry. Jain, et al. Expires August 16, 2014 [Page 10] Internet-Draft Detecting Overlay Segment Failure February 2014 5. Packet Format Generic Overlay Echo Request/Reply is a UDP Packet identified by well known UDP Port XXXX. The payload carried by Overlay typically could be either be Layer 2 / Ethernet Frame, or it could be Layer 3 / IP Packet. 5.1. Overlay OAM Encapsulation in Layer 2 Context If the encapsulated payload carried by Overlay is of type Ethernet, then the OAM Echo Request packet would have inner Ethernet Header, followed by IP and UDP Header. The payload of inner UDP would be as described in below section "Generic Overlay OAM Packet Format". 5.2. Overlay OAM Encapsulation in Layer 3 Context If the encapsulated payload carried by Overlay is of type IP, then the OAM Echo Request packet would have inner IP Header, followed by UDP Header. The payload of inner UDP would be as described in below section "Generic Overlay OAM Packet Format". 5.3. Generic Overlay OAM Packet Format Following is the format of UDP payload of Generic Overlay OAM Packet: Jain, et al. Expires August 16, 2014 [Page 11] Internet-Draft Detecting Overlay Segment Failure February 2014 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Message Type | Reply mode | Return Code | Return Subcode| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Originator Handle | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TimeStamp Sent (seconds) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TimeStamp Sent (microseconds) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TimeStamp Received (seconds) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TimeStamp Received (microseconds) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TLVs ... | . . . . . . | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Generic Overlay OAM Packet The Message Type is one of the following:- Value What it means ----- ------------- 1 Echo Request 2 Echo Reply Reply Mode Values:- Value What it means ----- --------------------------------- 1 Do not reply 2 Reply via an IPv4/IPv6 UDP Packet 3 Reply via Overlay Segment Echo Request with 1 (Do not reply) in the Reply Mode field may be used for one-way connectivity tests. The receiving node may log gaps in the Sequence Numbers and/or maintain delay/jitter statistics. For normal operation Echo Request would have 2 (Reply via an IPv4 UDP Jain, et al. Expires August 16, 2014 [Page 12] Internet-Draft Detecting Overlay Segment Failure February 2014 Packet) in the Reply Mode field. If it is desired that the reply also comes back via Overlay Segment i.e. encapsulated with the Overlay Header, then the Reply Mode filed needs to be set to 3 (Reply via Overlay Segment). The Originator's Handle is filled in by the Originator, and returned unchanged by the receiver in the Echo Reply (if any). The value used for this field can be implementation dependent, this MAY be used by the Originator for matching up requests with replies. The Sequence Number is assigned by the Originator of Echo Request and can be (for example) used to detect missed replies. The TimeStamp Sent is the time-of-day (in seconds and microseconds, according to the sender's clock) in NTP format [NTP] when the VXLAN Echo Request is sent. The TimeStamp Received in an Echo Reply is the time-of-day (according to the receiver's clock) in NTP format that the corresponding Echo Request was received. TLVs (Type-Length-Value tuples) have the following format: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Value | . . . . . . | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sub-TLV Type | Length | Variable Length Value | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Variable Length Value | | " | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Types are defined below; Length is the length of the Value field in octets. The Value field depends on the Type; it is zero padded to align to a 4-octet boundary. There could be one or many optional Sub-TLV that could be encoded under the TLV. Jain, et al. Expires August 16, 2014 [Page 13] Internet-Draft Detecting Overlay Segment Failure February 2014 5.3.1. TLV Types for various Overlay Ping Models TLV Types:- Value What it means ----- ------------------------------ 1 VXLAN Segment Ping for IPv4 2 VXLAN Segment Ping for IPv6 3 NVGRE Segment Ping for IPv4 4 NVGRE Segment Ping for IPv6 5 MPLSoGRE Segment Ping for IPv4 6 MPLSoGRE Segment Ping for IPv6 7 MPLSoUDP Segment Ping for IPv4 8 MPLSoUDP Segment Ping for IPv6 Jain, et al. Expires August 16, 2014 [Page 14] Internet-Draft Detecting Overlay Segment Failure February 2014 5.3.1.1. TLV for VXLAN Ping 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type = 1(VXLAN ping IPv4)| Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | VXLAN VNI | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv4 Sender Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ TLV if Sender Address is IPv4 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type = 2 (VXLAN ping IPv6)| Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | VXLAN VNI | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | IPv6 Sender Address | | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ TLV if Sender Address is IPv6 Jain, et al. Expires August 16, 2014 [Page 15] Internet-Draft Detecting Overlay Segment Failure February 2014 5.3.1.2. TLV for NVGRE Ping 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type = 3 (NVGRE ping IPv4)| Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | NVGRE VSID | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv4 Sender Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ TLV if Sender Address is IPv4 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type = 4 (NVGRE ping IPv6)| Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | NVGRE VSID | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | IPv6 Sender Address | | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ TLV if Sender Address is IPv6 Jain, et al. Expires August 16, 2014 [Page 16] Internet-Draft Detecting Overlay Segment Failure February 2014 5.3.1.3. TLV for MPLSoGRE Ping 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type = 5 (MPLSoGRE ping IPv4)| Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Route Distinguisher | | (8 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv4 Sender Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ TLV if Sender Address is IPv4 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type = 6 (MPLSoGRE ping IPv6)| Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Route Distinguisher | | (8 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv6 Sender Address | | | | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ TLV if Sender Address is IPv6 Route Distinguisher is defined as part of [RFC4365] Jain, et al. Expires August 16, 2014 [Page 17] Internet-Draft Detecting Overlay Segment Failure February 2014 5.3.1.4. TLV for MPLSoUDP Ping 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type = 7 (MPLSoUDP ping IPv4)| Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Route Distinguisher | | (8 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sender IPv4 Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ TLV if Sender Address is IPv4 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type = 8 (MPLSoUDP ping IPv6)| Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Route Distinguisher | | (8 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sender IPv6 Address | | | | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ TLV if Sender Address is IPv6 Route Distinguisher is defined as part of [RFC4365] Jain, et al. Expires August 16, 2014 [Page 18] Internet-Draft Detecting Overlay Segment Failure February 2014 6. Return Codes Sender MUST always set the Return Code set to zero. The receiver can set it to one of the values listed below when replying back to Echo- Request. Following are the Return Codes (Suggested):- Value What it means ----- ------------------------------- 0 No return code 1 Malformed Echo Request Received 2 Overlay Segment Not Present 3 Overlay Segment Not Operational 4 Return-Code-OK Jain, et al. Expires August 16, 2014 [Page 19] Internet-Draft Detecting Overlay Segment Failure February 2014 7. Procedure for Overlay Segment Ping Echo Request is used to test Data Plane and its view of Control Plane for particular Overlay Segment. The Overlay Segment to be verified is identified differently for various Overlay Technologies. For VXLAN, VNI is used to identify given Overlay Segment. For NVGRE, VSID is used. For MPLSoGRE and MPLSoUDP the MPLS Stack is used to identify a given Overlay Segment. For the Data Plane verification, the Overlay Echo Request Packet MUST be encapsulated within the Overlay Header, which is same as that of any End-Point data going over the same Overlay Segment. This would guarantee the data-path for OAM Packet follows the same path as that for any End User data going over the same Overlay Segment. The payload carried by Overlay typically could be either be Layer 2 or Ethernet Frame, or it could be Layer 3 or IP Packet. Based on the type of payload following is the way inner Header(s) of Echo Request would be encoded. 7.1. Encoding of Inner Header for Echo Request in Layer 2 Context If the encapsulated payload carried by Overlay is of type Ethernet, then the OAM Echo Request packet would have inner Ethernet Header, followed by IP and UDP Header. The payload of inner UDP would be as described in below section "Generic Overlay OAM Packet Format". Inner Ethernet Header for the Echo Request Packet MUST have the Destination Mac set to 00-00-5E-90-XX-XX (to be assigned IANA). The Source Mac should be set to Mac Address of the Originating VTEP. However, it is desired that the Inner Source Mac SHOULD not be learnt in the MAC-Table as this represent Control Packet in context of Overlay OAM. Inner IP header is set with the Source IP Address which is a routable Address of the sender; the Destination IP Address is a (randomly chosen) IPv4 Address from the range 127/8, IPv6 addresses are chosen from the range 0:0:0:0:0:FFFF:127/104. The IP TTL is set to 255. The inner Destination UDP port is set to xxxx (assigned by IANA for Overlay OAM). The "Generic Overlay OAM Packet" will now be encoded, with following information. The sender chooses a Originator's Handle and a Sequence Number. When sending subsequent Overlay Echo Requests, the sender SHOULD increment the Sequence Number by 1. Jain, et al. Expires August 16, 2014 [Page 20] Internet-Draft Detecting Overlay Segment Failure February 2014 The TimeStamp Sent is set to the time-of-day (in seconds and microseconds) that the Echo Request is sent. The TimeStamp Received is set to zero. Also, the Reply Mode must be set to the desired reply mode. The Return Code and Subcode are set to zero. Next, the TLV is Encoded for desired Overlay Type, as per Section "Types of TLVs defined for various Overlay Ping Models" 7.2. Encoding of Inner Header for Echo Request in Layer 3 Context If the encapsulated payload carried by Overlay is of type IP, then the Encoding of the Echo Request would be same as above Section "Encoding of Inner Header for Echo Request in Layer 2 Context", but without the presence of Inner Ethernet Header. 7.3. VXLAN Procedures 7.3.1. Sending VXLAN Echo Request The Outer VxLAN header for the Echo Request packet follows the encapsulation as defined in [I-D.draft-mahalingam-dutt-dcops-vxlan]. The VNI is same as that of the VXLAN Segment that is being verified. This would make sure that OAM Packet takes the same datapath as any other End System data going over this VXLAN Segment. The VXLAN Router Alert option [I-D.draft-singh-nvo3-vxlan-router-alert] MUST be set in the VXLAN header as shown below. VXLAN Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |R|R|R|R|I|R|R|RA| Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | VXLAN Network Identifier (VNI) | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ RA: Router Alter Bit (Proposed) Originating VTEP MAY set the I Bit to 1 in VXLAN Header when sending OAM Frame. This would cause dropping of such VXLAN frames on any Terminating VTEP that does not understand Overlay OAM framework, and prevent sending those frames to End-Systems or VMs. It is desired to choose the Source UDP port (in the outer header), so as to exercise the same Data-Path as that of the traffic carried over the VXLAN Segment and is left to the implementation. Jain, et al. Expires August 16, 2014 [Page 21] Internet-Draft Detecting Overlay Segment Failure February 2014 The Encoding of Inner Header(s) and UDP payload of Generic Overlay OAM Packet is as described in above Sub-Section i.e. "Encoding of Inner Header for Echo Request in Layer 2/Layer 3 Context". 7.3.2. Receiving VXLAN Echo Request At the Terminating Overlay End Point or VTEP, since the Overlay OAM Packet is exactly same as that of End-System Packet(s). It is important to send OAM packet to Control Plane and prevent it from sending to the End System. The trapping and sending VXLAN Echo Request to the Control Plane is triggered by one of the following Packet processing exceptions: VXLAN Router Alert option, [I-D.draft-singh-nvo3-vxlan-router-alert] the Inner Destination MAC Address of 00-00-5E-90-XX-XX as defined in above section, and the Destination IP Address in the 127/8 Address range for IPv4 Address, or 0:0:0:0:0:FFFF:127/104 for IPv6 Address. The Control Plane further identifies the Overlay OAM Application by UDP well know destination port xxxx. Since the VxLAN Router Alert bit is set in VxLAN Header, which signifies the presence of Control Packet. The terminating VTEP SHOULD not learn the Mac address set in the Inner Mac Header of VxLAN Echo Request Packet. Once the VXLAN Echo Request Packet is identified at Control Plane, it is processed as follows:- o General Packet sanity is verified. If the Packet is not well- formed, VTEP SHOULD send VXLAN Echo Reply with the Return Code set to "Malformed Echo Request received" and the Subcode to zero. The header fields Originator's Handle, Sequence Number, and Timestamp Sent are not examined, but are included in the VXLAN Echo Reply message o VNI Validation: If there is no entry for VNI, it indicates that there could be a transient or permanent disconnect between Control Plane and data Plane and VTEP needs to report an error with Return Code of "Overlay Segment Not Present" and a Return Subcode of Zero. If the mapping for VNI Exists, but the state is not Operational, VTEP needs to report an error with Return Code of "Overlay Segment Not Operational" If the mapping exists then send VXLAN Echo Reply with a Return Code of "Return-Code-OK", and a Return Subcode of Zero. The procedures for sending the Echo Reply are found in subsection below section. Jain, et al. Expires August 16, 2014 [Page 22] Internet-Draft Detecting Overlay Segment Failure February 2014 7.3.3. Sending VXLAN Echo Reply If the Reply Mode is set to "Reply via an IPv4/IPv6 UDP Packet", the Echo Reply is a UDP Packet. It MUST ONLY be sent in response to Echo Request. The Source IP Address in the Header should be Routable Address of the replier; The Destination IP Address should be IP Address of the Echo Request's Originating End Point or the requester. The destination UDP Port is set to XXXX (assigned by IANA for identifying VXLAN OAM application). The IP TTL is set to 255. The format of the Echo Reply is the same as the Echo Request. The Originator Handle, the Sequence Number, and TimeStamp Sent are copied from the Echo Request; the TimeStamp Received is set to the time-of- day that the Echo Request is received (note that this information is most useful if the time-of-day clocks on the requester and the replier are synchronized). The replier MUST fill in the Return Code and Subcode, as determined in the previous subsection. If the Reply Mode is set to "Reply via Overlay Segment", then the Replying Overlay End Point is expected to place Echo Reply packet in- band in the Overlay Segment destined to the Originating Overlay End Point. The detailed encapsulation for this would be covered in next revision of the draft. 7.3.4. Receiving VXLAN Echo Reply An Originating Overlay End Point should only receive Echo Reply in response to an Echo Request that it sent. When the Reply Mode is "Reply via an IPv4/IPv6 UDP Packet", the Echo Reply would be and IP Packet/UDP Packet, and is identified by the destination UDP Port XXXX. The Originating Overlay End Point should parse the Packet to ensure that it is well-formed, then attempt to match up the Echo Reply with an Echo Request that it had previously sent, and the Originator Handle. If no match is found, then it should drop the Echo Reply Packet; otherwise, it checks the Sequence Number to see if it matches. 7.4. NVGRE Procedures 7.4.1. Sending NVGRE Echo Request The Outer NVGRE header for the Echo Request packet follows the encapsulation as defined in [I-D.draft-sridharan-virtualization-nvgre]. The VSID is same as that of the NVGRE Segment that is being verified. This would make sure that OAM Packet takes the same datapath as any other End System data going over this NVGRE Segment. Jain, et al. Expires August 16, 2014 [Page 23] Internet-Draft Detecting Overlay Segment Failure February 2014 The NVGRE Router Alert option [I-D.draft-singh-nvo3-nvgre-router-alert] MUST be set in the NVGRE header as shown below. GRE Header: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| |1|0| Reserved0 RA| Ver | Protocol Type 0x6558 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Virtual Subnet ID (VSID) | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ RA: Router Alter Bit (Proposed) The Encoding of Inner Header(s) and UDP payload of Generic Overlay OAM Packet is as described in above Sub-Section i.e. "Encoding of Inner Header for Echo Request in Layer 2/Layer 3 Context". 7.4.2. Receiving NVGRE Echo Request At the Terminating Overlay End Point, since the Overlay OAM Packet is exactly same as that of End-System Packet(s). It is important to send OAM packet to Control Plane and prevent it from sending to the End System. The trapping and sending NVGRE Echo Request to the Control Plane is triggered by one of the following Packet processing exceptions: NVGRE Router Alert option, [I-D.draft-singh-nvo3-nvgre-router-alert] the Inner Destination MAC Address of 00-00-5E-90-XX-XX as defined in above section, and the Destination IP Address in the 127/8 Address range for IPv4 Address, or 0:0:0:0:0:FFFF:127/104 for IPv6 Address. The Control Plane further identifies the Overlay OAM Application by UDP well know destination port xxxx. Since the NVGRE Router Alert bit is set in NVGRE Header, which signifies the presence of Control Packet. The Terminating Overlay End Point SHOULD not learn the Mac address set in the Inner Mac Header of NVGRE Echo Request Packet. Once the NVGRE Echo Request Packet is identified at Control Plane, it is processed as follows:- o General Packet sanity is verified. If the Packet is not well- formed, NVGRE End Point SHOULD send NVGRE Echo Reply with the Return Code set to "Malformed Echo Request received" and the Subcode to zero. The header fields Originator's Handle, Sequence Number, and Timestamp Sent are not examined, but are included in the NVGRE Echo Reply message Jain, et al. Expires August 16, 2014 [Page 24] Internet-Draft Detecting Overlay Segment Failure February 2014 o VSID Validation: If there is no entry for VSID, it indicates that there could be a transient or permanent disconnect between Control Plane and data Plane and NVGRE End Point needs to report an error with Return Code of "Overlay Segment Not Present" and a Return Subcode of Zero. If the mapping for VSID Exists, but the state is not Operational, NVGRE End Point needs to report an error with Return Code of "Overlay Segment Not Operational" If the mapping exists then send NVGRE Echo Reply with a Return Code of "Return- Code-OK", and a Return Subcode of Zero. The procedures for sending the Echo Reply are found in subsection below section. 7.4.3. Sending NVGRE Echo Reply The procedure for sending NVGRE Echo Reply are exactly same as defined in above section "Sending VXLAN Echo Reply". 7.4.4. Receiving NVGRE Echo Reply The procedure for Receiving NVGRE Echo Reply are exactly same as defined in above section "Receiving VXLAN Echo Reply". 7.5. MPLSoGRE Procedures 7.5.1. Sending MPLSoGRE Echo Request The Outer header of MPLSoGRE for the Echo Request packet follows the encapsulation as defined in [RFC4023]. The MPLS Stack is same as that of the MPLSoGRE Segment that is being verified. This would make sure that OAM Packet takes the same datapath as any other End System data going over this MPLSoGRE Segment. However, the bottommost Label in MPLS Stack MUST be MPLS Router Alert Label [RFC3032]. This would indicate the Overlay Terminating End Point that the payload is a Control Packet and needs to be delivered to Control Plane. The Encoding of Inner Header(s) and UDP payload of Generic Overlay OAM Packet is as described in above Sub-Section i.e. "Encoding of Inner Header for Echo Request in Layer 2/Layer 3 Context". 7.5.2. Receiving MPLSoGRE Echo Request At the Terminating Overlay End Point, since the Overlay OAM Packet is exactly same as that of End-System Packet(s). It is important to send OAM packet to Control Plane and prevent it from sending to the End System. The trapping and sending MPLSoGRE Echo Request to the Control Plane is triggered by one of the following Packet processing exceptions: MPLS Router Alert Label, and the Destination IP Address Jain, et al. Expires August 16, 2014 [Page 25] Internet-Draft Detecting Overlay Segment Failure February 2014 in the 127/8 Address range for IPv4 Address, or 0:0:0:0:0:FFFF:127/ 104 for IPv6 Address. The Control Plane further identifies the Overlay OAM Application by UDP well know destination port xxxx. Once the MPLSoGRE Echo Request Packet is identified at Control Plane, it is processed as follows:- o General Packet sanity is verified. If the Packet is not well- formed, MPLSoGRE End Point SHOULD send MPLSoGRE Echo Reply with the Return Code set to "Malformed Echo Request received" and the Subcode to zero. The header fields Originator's Handle, Sequence Number, and Timestamp Sent are not examined, but are included in the MPLSoGRE Echo Reply message o Segment Validation: If there is no entry for service represented by given Route Distinguisher for the MPLSoGRE Segment, it indicates that there could be a transient or permanent disconnect between Control Plane and Data Plane and MPLSoGRE End Point needs to report an error with Return Code of "Overlay Segment Not Present" and a Return Subcode of Zero. If the entry for service represented by given Route Distinguisher for the MPLSoGRE Segment is present, but is Operationally Down. The End Point needs to report an error with Return Code of "Overlay Segment Not Operational" If the mapping of service represented by given Route Distinguisher for the MPLSoGRE Segment is present and Active, then send MPLSoGRE Echo Reply with a Return Code of "Return-Code-OK". 7.5.3. Sending MPLSoGRE Echo Reply The procedure for sending MPLSoGRE Echo Reply are exactly same as defined in above section "Sending VXLAN Echo Reply". 7.5.4. Receiving MPLSoGRE Echo Reply The procedure for Receiving MPLSoGRE Echo Reply are exactly same as defined in above section "Receiving VXLAN Echo Reply". 7.6. MPLSoUDP Procedures 7.6.1. Sending MPLSoUDP Echo Request The Outer header of MPLSoUDP for the Echo Request packet follows the encapsulation as defined in [I-D.draft-ietf-mpls-in-udp]. The MPLS Stack is same as that of the MPLSoUDP Segment that is being verified. This would make sure that OAM Packet takes the same datapath as any other End System data going over this MPLSoUDP Segment. Jain, et al. Expires August 16, 2014 [Page 26] Internet-Draft Detecting Overlay Segment Failure February 2014 However, the bottommost Label in MPLS Stack MUST be MPLS Router Alert Label [RFC3032]. This would indicate the Overlay Terminating End Point that the payload is a Control Packet and needs to be delivered to Control Plane. It is desired to choose the Source UDP port (in the outer header), so as to exercise the same Data-Path as that of the traffic carried over the MPLSoUDP Segment and is left to the implementation. The Encoding of Inner Header(s) and UDP payload of Generic Overlay OAM Packet is as described in above Sub-Section i.e. "Encoding of Inner Header for Echo Request in Layer 2/Layer 3 Context". 7.6.2. Receiving MPLSoUDP Echo Request At the Terminating Overlay End Point, since the Overlay OAM Packet is exactly same as that of End-System Packet(s). It is important to send OAM packet to Control Plane and prevent it from sending to the End System. The trapping and sending MPLSoGRE Echo Request to the Control Plane is triggered by one of the following Packet processing exceptions: MPLS Router Alert Label, and the Destination IP Address in the 127/8 Address range for IPv4 Address, or 0:0:0:0:0:FFFF:127/ 104 for IPv6 Address. The Control Plane further identifies the Overlay OAM Application by UDP well know destination port xxxx. Once the MPLSoUDP Echo Request Packet is identified at Control Plane, it is processed as follows:- o General Packet sanity is verified. If the Packet is not well- formed, MPLSoUDP End Point SHOULD send MPLSoUDP Echo Reply with the Return Code set to "Malformed Echo Request received" and the Subcode to zero. The header fields Originator's Handle, Sequence Number, and Timestamp Sent are not examined, but are included in the MPLSoUDP Echo Reply message o Segment Validation: If there is no entry for service represented by given Route Distinguisher for the MPLSoUDP Segment, it indicates that there could be a transient or permanent disconnect between Control Plane and data Plane and MPLSoUDP End Point needs to report an error with Return Code of "Overlay Segment Not Present" and a Return Subcode of Zero. If the entry for service represented by given Route Distinguisher for the MPLSoUDP Segment is present, but is Operationally Down. The End Point needs to report an error with Return Code of "Overlay Segment Not Operational" If the mapping of service represented by given Route Distinguisher for the MPLSoUDP Segment is present and Active, then Jain, et al. Expires August 16, 2014 [Page 27] Internet-Draft Detecting Overlay Segment Failure February 2014 send MPLSoUDP Echo Reply with a Return Code of "Return-Code-OK". 7.6.3. Sending MPLSoUDP Echo Reply The procedure for sending MPLSoGRE Echo Reply are exactly same as defined in above section "Sending VXLAN Echo Reply". 7.6.4. Receiving MPLSoUDP Echo Reply The procedure for Receiving MPLSoGRE Echo Reply are exactly same as defined in above section "Receiving VXLAN Echo Reply". Jain, et al. Expires August 16, 2014 [Page 28] Internet-Draft Detecting Overlay Segment Failure February 2014 8. Procedure for Trace In order to be able to trace the Path that a particular flow in the Overlay takes through the Underlay Network, following mechanism can be used - An overlay Echo Request packet is built and sent using the mechanisms described in the Section "Procedure for Overlay Segment Ping" so that the overlay traceroute follows the same path as the data packet for the overlay segment being traced. The Echo Request packet in the traceroute mode is sent with the initial TTL set to 1 in the Outer IP header and thereafter incremented by 1 in each successive request. At each transit hop where the TTL expires, an exception is created. Because of this exception, the packet gets delivered to the Control Plane. Control plane can further deliver the packet to the OAM application based on the TTL exception and the specific UDP port XXXX in the incoming overlay echo request packet. If the transit node has the IP reachability to the destination IP address in the outer IP header, it sends back an overlay echo reply response otherwise the Overlay Echo Request is discarded by the Overlay OAM module on the transit nodes. If the transit node does not support overlay OAM functionality, it will simply generate a regular ICMP TTL exceeded response. This could result into "false negatives". The originating Overlay node that generated the OAM echo request SHOULD try sending the echo request with TTL=n+1, n+2, ... to probe the nodes further down the path to the terminating overlay End-point. At the originating node, when the Echo Reply from the transit node corresponding to the traceroute query is received, it can correlate the incoming Echo Reply with the traceroute query by matching on the sequence numbers in the Overlay Echo Request/Reply packets. Current revision of this draft limits overlay traceroute capability to fault isolation only. A subsequent version of the draft will include mechanisms to trace all possible paths in the underlay that can be used to carry overlay tunnel traffic. Jain, et al. Expires August 16, 2014 [Page 29] Internet-Draft Detecting Overlay Segment Failure February 2014 9. Procedure for End-System Ping In typical Overlay deployment scenarios there is a desired to check the presence of any given Tenant VM/End-System or Flow representing the End-System System within a given Overlay Segment. This draft proposes the way to achieve it via End-System Ping. The End-System can be identified at Overlay End Point by either its IP Address, Ethernet MAC Address or combination of IP/MAC Address. In that case, it would be important to verify the End-System connectivity by procedure which goes over the Overlay Segment from Originating Overlay End-Point and verifies the presence of the End- System at the Terminating Overlay End-Point. The scope of End-System Ping is solely with the Cloud Provider which owns control of the Overlay End Point(s). It is expected that the Overlay End Point traps this request and checks the Presence of the End-System via its MAC Address, Route or Flow information and replies back. There SHOULD not be a case where the End-System Ping is delivered to the actual End-Point. 9.1. Sub-TLV for End-System Ping Sub-TLV Types:- Value What it means ----- --------------------------- 1 End-System MAC Sub-TLV 2 End-System IPv4 Sub-TLV 3 End-System IPv6 Sub-TLV 4 End-System MAC/IPv4 Sub-TLV 6 End-System MAC/IPv6 Sub-TLV End-System Return Code:- Value What it means ----- ---------------------- 1 End-System Present 2 End-System Not Present Jain, et al. Expires August 16, 2014 [Page 30] Internet-Draft Detecting Overlay Segment Failure February 2014 9.1.1. Sub-TLV for Validating End-System MAC Address 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | End-System MAC Sub-TLV (1) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #1 | Return Code#1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #2 | Return Code#2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ ... ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #n | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #n | Return Code #n | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ MAC Address: MAC Address of the End-System, that user is interested to validate. Return Code: Return Code specifying status of End-System at Overlay End Point Jain, et al. Expires August 16, 2014 [Page 31] Internet-Draft Detecting Overlay Segment Failure February 2014 9.1.2. Sub-TLV for Validating End-System IP Address 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | End-System IPv4 Sub-TLV (2) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP address #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Return Code #1 | IP address #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP address #2 | Return Code #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ ... ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP address #n | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Return Code #n | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | End-System IPv6 Sub-TLV (3) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | IPv6 Address #1 | | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Return Code #1 | IPv6 Address #2... ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ ... ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | IPv6 Address #n | | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Return Code #n | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ IP Address : IP Address of the End-System, that user is interested to validate. Return Code: Return Code specifying status of End-System at Overlay End Point Jain, et al. Expires August 16, 2014 [Page 32] Internet-Draft Detecting Overlay Segment Failure February 2014 9.1.3. Sub-TLV for Validating End-System MAC and IP Address 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | End-System IPv4/MAC Sub-TLV (4)| Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #1 | IP address #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP address #1 | Return Code #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #2 | IP address #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP address #2 | Return Code #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ ... ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #n | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #n | IP address #n | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP address #n | Return Code #n | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | End-System IPv6/MAC Sub-TLV(5) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #1 | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | | + + | IPv6 address #1 | + + | | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | Return Code #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ ... ~ Jain, et al. Expires August 16, 2014 [Page 33] Internet-Draft Detecting Overlay Segment Failure February 2014 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #n | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address #n | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | | + | IPv6 address #1 | + + | | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | Return Code #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ IP Address : IP Address of the End-System, that user is interested to validate. MAC Address: MAC Address of the End-System, that user is interested to validate. Return Code: Return Code specifying status of End-System at Overlay End Point 9.2. Sending End-System Ping Request When it is desired to check presence of a given End-System, the Echo Request Message is prepared as described in above Section "Procedure for Overlay Segment Ping". This packet should compose of Outer Header, Overlay Header, Inner Header, Generic Overlay Header with TLV representing desired Overlay Type (VXLAN, NVGRE, MPLSoGRE or MPLSoUDP). Apart form this the packet should also have one of the Sub-TLV's as defined in above section "Sub-TLV for End-System Ping" to identify the type of End-System Ping that user is interested in. Because of the above mentioned encapsulation, it would be guaranteed that the packet follows the same Data Path as that of any End-User data going over the given Overlay Segment. User need to fill in MAC, IP or MAC/IP combination for the End- System(s) that needs to be validated at the Overlay End Point in the respective Sub-TLV for End-System Ping. 9.3. Receiving End-System Ping Request On receiving the End-System Ping Request the processing to trap this Packet, and sent it to Control Plane is done by Overlay Terminating End-System as define in above Section "Procedure for Overlay Segment Ping". Once the OAM Packet reaches OAM Application, it is identified as End-System Ping Request by virtue of presence any of the Sub-TLV's as defined in Section "Sub-TLV for End-System Ping". Jain, et al. Expires August 16, 2014 [Page 34] Internet-Draft Detecting Overlay Segment Failure February 2014 If the Sub-TLV is of Type "End-System MAC Sub-TLV", the Overlay End Point should iterate through the list of MAC Addresses and verify the presence of individual MAC Address in its Flow Table or MAC Table for the given Overlay Segment. If the MAC Address is present, it should set the respective End- System's Return Code field in the Sub-TLV to 1 "End-System-Present". If the MAC Address is not present, it should set respective the End- System's Return Code filed in the Sub-TLV to 2 "End-System-Not- Present". If the Sub-TLV is of Type "End-System IP Sub-TLV", the Overlay End Point should iterate through the list of IP Addresses and verify the presence of individual IP Address in its Flow Table or Route Table for the given Overlay Segment. If the IP Address is present, it should set the respective End- System's Return Code field in the Sub-TLV to 1 "End-System-Present". If the IPC Address is not present, it should set respective the End- System's Return Code filed in the Sub-TLV to 2 "End-System-Not- Present". If the Sub-TLV is of Type "End-System MAC and IP Sub-TLV", the Overlay End Point should iterate through the list of MAC/IP Addresses and verify the presence of individual MAC/IP Combination in its Flow Table or MAC and IP Table for the given Overlay Segment. If the IP and MAC Address is present, it should set the respective End-System's Return Code field in the Sub-TLV to 1 "End-System- Present". If the IP and MAC Address is not present, it should set respective the End-System's Return Code filed in the Sub-TLV to 2 "End-System- Not-Present". 9.4. Sending End-System Ping Reply The procedure for sending End-System Echo Reply is same as defined in above section "Sending VXLAN Echo Reply". The replier MUST fill Sub- TLV with proper Return Code for each element in the End-System Sub- TLV. 9.5. Receiving End-System Ping Reply An Originating Overlay End Point should only receive Echo Reply for End-System Ping, in response to an Echo Request that it sent. By Jain, et al. Expires August 16, 2014 [Page 35] Internet-Draft Detecting Overlay Segment Failure February 2014 virtue of presence of End-System Sub-TLV it would identify the status of respective End-System, and report it to the user. The other part of the handling is similar to section "Receiving VXLAN Echo Reply" Jain, et al. Expires August 16, 2014 [Page 36] Internet-Draft Detecting Overlay Segment Failure February 2014 10. Security Considerations TBD Jain, et al. Expires August 16, 2014 [Page 37] Internet-Draft Detecting Overlay Segment Failure February 2014 11. Management Considerations None Jain, et al. Expires August 16, 2014 [Page 38] Internet-Draft Detecting Overlay Segment Failure February 2014 12. Acknowledgements This document is the outcome of many discussions among many people, including Saurabh Shrivastava, Krishna Ram Kuttuva Jeyaram and Suresh Boddapati of Nuage Networks, Jorge Rabadan of Alcatel-Lucent, Inc and Rahul Kasralikar of Juniper Networks, Inc. Jain, et al. Expires August 16, 2014 [Page 39] Internet-Draft Detecting Overlay Segment Failure February 2014 13. IANA Considerations Action-1: This specification reserves a IANA UDP Port Number to be used when sending the Overlay OAM Packet Action-2: This specification reserves a IANA Ethernet unicast Address for VXLAN/NVGRE Exception handling. This Address needs to be reserved from the block. "IANA Ethernet Address block - Unicast Use" Jain, et al. Expires August 16, 2014 [Page 40] Internet-Draft Detecting Overlay Segment Failure February 2014 14. References 14.1. Normative References [I-D.draft-ietf-mpls-in-udp] Xu, Sheth, Yong, Pignataro, Yongbing , and Li, "Encapsulating MPLS in UDP", May 2013. [I-D.draft-lasserre-nvo3-framework] Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y. Rekhter, "Framework for DC Network Virtualization", September 2011. [I-D.draft-mahalingam-dutt-dcops-vxlan] Mahalingam, M., Dutt, D., Agarwal, P., Kreeger, L., Sridhar, T., Bursell, M., and C. Wright, "VXLAN: A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks", May 2013. [I-D.draft-singh-nvo3-nvgre-router-alert] Singh, K., Jain, P., Balus, F., and W. Henderickx, "NVGRE Router Alert Option", May 2013. [I-D.draft-singh-nvo3-vxlan-router-alert] Singh, K., Jain, P., Balus, F., and W. Henderickx, "VxLAN Router Alert Option", May 2013. [I-D.draft-sridharan-virtualization-nvgre] Sridharan, M., Duda, K., Greenberg, A., Lin, G., Pearson, M., Thaler, P., Tumuluri, C., Venkataramiah, N., and Y. Wang, "NVGRE: Network Virtualization using Generic Routing Encapsulation", February 2013. [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack Encoding", RFC 3032, January 2001. [RFC4023] Worster, T., Rekhter, Y., and E. Rosen, "Encapsulating MPLS in IP or Generic Routing Encapsulation (GRE)", RFC 4023, March 2005. [RFC4365] Rosen, E., "Applicability Statement for BGP/MPLS IP Virtual Private Networks (VPNs)", RFC 4365, February 2006. [RFC4379] Kompella, K. and G. Swallow, "Detecting Multi-Protocol Label Switched (MPLS) Data Plane Failures", RFC 4379, February 2006. Jain, et al. Expires August 16, 2014 [Page 41] Internet-Draft Detecting Overlay Segment Failure February 2014 14.2. Informative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC4330] Mills, D., "Simple Network Time Protocol (SNTP) Version 4 for IPv4, IPv6 and OSI", RFC 4330, January 2006. Jain, et al. Expires August 16, 2014 [Page 42] Internet-Draft Detecting Overlay Segment Failure February 2014 Authors' Addresses Pradeep Jain Nuage Networks 755 Ravendale Drive Mountain View, CA 94043 USA Email: pradeep@nuagenetworks.net Kanwar Singh Nuage Networks 755 Ravendale Drive Mountain View, CA 94043 USA Email: kanwar@nuagenetworks.net Florin Balus Nuage Networks 755 Ravendale Drive Mountain View, CA 94043 USA Email: florin@nuagenetworks.net Wim Henderickx Alcatel-Lucent Copernicuslaan 50 Antwerp 2018 Belgium Email: wim.henderickx@alcatel-lucent.be Vinay Bannai PayPal 2211 N. First St, San Jose 95131 USA Email: vbannai@paypal.com Jain, et al. Expires August 16, 2014 [Page 43] Internet-Draft Detecting Overlay Segment Failure February 2014 Ravi Shekhar Juniper Networks 1194 North Mathilda Ave. Sunnyvale, CA 94089 USA Email: rshekhar@juniper.net Anil Lohiya Juniper Networks 1194 North Mathilda Ave. Sunnyvale, CA 94089 USA Email: alohiya@juniper.net Jain, et al. Expires August 16, 2014 [Page 44]