SPRING Working Group R. Gandhi, Ed. Internet-Draft C. Filsfils Intended status: Informational Cisco Systems, Inc. Expires: 6 October 2024 D. Voyer Bell Canada M. Chen Huawei R. Foote Nokia 4 April 2024 Performance Measurement Using Simple Two-Way Active Measurement Protocol (STAMP) for Segment Routing Networks draft-ietf-spring-stamp-srpm-14 Abstract Segment Routing (SR) leverages the source routing paradigm. SR is applicable to both Multiprotocol Label Switching (SR-MPLS) and IPv6 (SRv6) data planes. This document describes procedures for Performance Measurement in SR networks using Simple Two-Way Active Measurement Protocol (STAMP) defined in RFC 8762 and its optional extensions defined in RFC 8972 and further augmented in RFC 9503. The procedure described is used for links, SR paths (including SR Policies and SR Flexible Algorithm IGP paths) as well as Layer-3 and Layer-2 services in SR networks, and is applicable to both SR-MPLS and SRv6 data planes. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 6 October 2024. Gandhi, et al. Expires 6 October 2024 [Page 1] Internet-Draft Using STAMP for Segment Routing April 2024 Copyright Notice Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Conventions Used in This Document . . . . . . . . . . . . . . 4 2.1. Requirements Language . . . . . . . . . . . . . . . . . . 4 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 4 3. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4. One-Way and Two-Way Measurement Modes in SR Networks . . . . 7 4.1. Example STAMP Reference Model . . . . . . . . . . . . . . 7 4.2. Session-Sender Test Packet . . . . . . . . . . . . . . . 9 4.3. Session-Sender Test Packet for Links . . . . . . . . . . 10 4.4. Session-Sender Test Packet for SR-MPLS Data Plane . . . . 10 4.4.1. Session-Sender Test Packet for SR-MPLS Paths . . . . 10 4.4.2. Session-Sender Test Packet for Layer-3 Services over SR-MPLS Path . . . . . . . . . . . . . . . . . . . . 12 4.4.3. Session-Sender Test Packet for Layer-2 Services over SR-MPLS Path . . . . . . . . . . . . . . . . . . . . 12 4.5. Session-Sender Test Packet for SRv6 Data Plane . . . . . 13 4.5.1. Session-Sender Test Packet for SRv6 Paths . . . . . . 13 4.5.2. Session-Sender Test Packet for Layer-3 Services over SRv6 Path . . . . . . . . . . . . . . . . . . . . . . 16 4.5.3. Session-Sender Test Packet for Layer-2 Services over SRv6 Path . . . . . . . . . . . . . . . . . . . . . . 18 4.6. Session-Sender Test Packet for P2MP SR Paths . . . . . . 20 4.7. Session-Reflector Test Packet . . . . . . . . . . . . . . 20 4.7.1. One-Way Measurement Mode . . . . . . . . . . . . . . 21 4.7.2. Two-Way Measurement Mode . . . . . . . . . . . . . . 22 5. Loopback Measurement Mode in SR Networks . . . . . . . . . . 23 5.1. Loopback Measurement Mode STAMP Packet Processing . . . . 24 5.2. Loopback Measurement Mode for Links . . . . . . . . . . . 25 5.3. Loopback Measurement Mode for SR-MPLS Data Plane . . . . 26 5.3.1. Loopback Measurement Mode for SR-MPLS Paths . . . . . 26 5.3.2. Loopback Measurement Mode for Layer-3 Services over SR-MPLS Path . . . . . . . . . . . . . . . . . . . . 27 Gandhi, et al. Expires 6 October 2024 [Page 2] Internet-Draft Using STAMP for Segment Routing April 2024 5.3.3. Loopback Measurement Mode for Layer-2 Services over SR-MPLS Path . . . . . . . . . . . . . . . . . . . . 29 5.4. Loopback Measurement Mode for SRv6 Data Plane . . . . . . 30 5.4.1. Loopback Measurement Mode for SRv6 Paths . . . . . . 30 5.4.2. Loopback Measurement Mode for Layer-3 Services over SRv6 Path . . . . . . . . . . . . . . . . . . . . . . 32 5.4.3. Loopback Measurement Mode for Layer-2 Services over SRv6 Path . . . . . . . . . . . . . . . . . . . . . . 35 6. Loopback Measurement Mode with Timestamp and Forward Function in SR Networks . . . . . . . . . . . . . . . . . . . . . 36 6.1. Loopback Measurement Mode with Timestamp and Forward Function for SR-MPLS Data Plane . . . . . . . . . . . . . 37 6.1.1. Timestamp and Forward Network Action Assignment . . . 38 6.1.2. Node Capability for MNA Sub-Stack with Opcode MNA.TSF . . . . . . . . . . . . . . . . . . . . . . . 38 6.2. Loopback Measurement Mode with Timestamp and Forward Function for SRv6 Data Plane . . . . . . . . . . . . . . 39 6.2.1. Timestamp and Forward Endpoint Function Assignment . 41 6.2.2. Node Capability for Timestamp and Forward Endpoint Function . . . . . . . . . . . . . . . . . . . . . . 41 7. Packet Loss Measurement in SR Networks . . . . . . . . . . . 41 8. Direct Measurement in SR Networks . . . . . . . . . . . . . . 42 9. ECMP Measurement in SR Networks . . . . . . . . . . . . . . . 42 10. STAMP Session State . . . . . . . . . . . . . . . . . . . . . 43 11. Additional STAMP Test Packet Processing Rules . . . . . . . . 43 11.1. TTL . . . . . . . . . . . . . . . . . . . . . . . . . . 43 11.2. IPv6 Hop Limit . . . . . . . . . . . . . . . . . . . . . 43 11.3. Router Alert Option . . . . . . . . . . . . . . . . . . 43 11.4. IPv6 Flow Label . . . . . . . . . . . . . . . . . . . . 44 11.5. UDP Checksum . . . . . . . . . . . . . . . . . . . . . . 44 12. Implementation Status . . . . . . . . . . . . . . . . . . . . 44 13. Security Considerations . . . . . . . . . . . . . . . . . . . 44 14. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 45 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 45 15.1. Normative References . . . . . . . . . . . . . . . . . . 45 15.2. Informative References . . . . . . . . . . . . . . . . . 46 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 50 Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 50 1. Introduction Segment Routing (SR) leverages the source routing paradigm and greatly simplifies network operations for Software Defined Networks (SDNs). SR is applicable to both Multiprotocol Label Switching (SR- MPLS) and IPv6 (SRv6) data planes [RFC8402]. SR takes advantage of the Equal-Cost Multipaths (ECMPs) between source and transit nodes, between transit nodes and between transit and destination nodes. SR Gandhi, et al. Expires 6 October 2024 [Page 3] Internet-Draft Using STAMP for Segment Routing April 2024 Policies as defined in [RFC9256] are used to steer traffic through a specific, user-defined paths using a stack of Segments. A comprehensive SR Performance Measurement (PM) toolset is one of the essential requirements to measure network performance to provide Service Level Agreements (SLAs). The Simple Two-Way Active Measurement Protocol (STAMP) provides capabilities for the measurement of various performance metrics in IP networks [RFC8762] without the use of a control channel to pre-signal session parameters. [RFC8972] defines optional extensions, in the form of TLVs, for STAMP. [RFC9503] augments that framework to define STAMP extensions for SR networks. This document describes procedures for Performance Measurement in SR networks using STAMP defined in [RFC8762] and its optional extensions defined in [RFC8972] and further augmented in [RFC9503]. The procedure described is used for links, SR paths [RFC8402] (including SR Policies [RFC9256] and SR Flexible Algorithm (Flex-Algo) IGP paths [RFC9350]) as well as Layer-3 (L3) and Layer-2 (L2) services in SR networks, and is applicable to both SR-MPLS and SRv6 data planes. STAMP requires protocol support on the Session-Reflector to process the received test packets, and hence the received test packets need to be punted from the fast path in data plane and return test packets need to be generated. This limits the scale for number test sessions and the ability to provide faster measurement interval. This document enhances the procedure for Performance Measurement using STAMP to improve the scale for number of sessions and the interval for measurement of SR paths, for both SR-MPLS and SRv6 data planes by defining new loopback measurement mode with "timestamp and forward network programming function". 2. Conventions Used in This Document 2.1. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. 2.2. Abbreviations ECMP: Equal Cost Multi-Path. HMAC: Hashed Message Authentication Code. Gandhi, et al. Expires 6 October 2024 [Page 4] Internet-Draft Using STAMP for Segment Routing April 2024 I2E: Ingress-To-Egress. IHS: Ingress-To-Egress, Hop-By-Hop or Select Scope. L2: Layer-2. L3: Layer-3. MBZ: Must be Zero. MNA: MPLS Network Action. MPLS: Multiprotocol Label Switching. PSID: Path Segment Identifier. SHA: Secure Hash Algorithm. SID: Segment ID. SR: Segment Routing. SRH: Segment Routing Header. SR-MPLS: Segment Routing with MPLS data plane. SRv6: Segment Routing with IPv6 data plane. SSID: STAMP Session Identifier. STAMP: Simple Two-Way Active Measurement Protocol. TC: Traffic Class. TSF: Timestamp and Forward. TTL: Time To Live. VPN: Virtual Private Network. 3. Overview For performance measurement in SR networks, the STAMP Session-Sender and Session-Reflector can use the base test packets defined [RFC8762]. However, the STAMP test packets defined in [RFC8972] are preferred in SR environment because of the optional extensions. The STAMP test packets are encapsulated using IP/UDP header and use the Destination UDP port 862 [RFC8762], by default. In this document, Gandhi, et al. Expires 6 October 2024 [Page 5] Internet-Draft Using STAMP for Segment Routing April 2024 the STAMP test packets using IP/UDP header are considered for SR networks, where the STAMP test packets are further encapsulated with an SR-MPLS or SRv6 header. The STAMP test packets carry the same SR- MPLS and SRv6 encapsulation as used by the data packets transmitted on the SR path as well as L3 and L2 service under measurement. The STAMP test packets are transmitted in a performance measurement mode of either one-way, two-way, loopback, or loopback with "timestamp and forward network programming function" in SR networks. Note that one-way and two-way performance measurement modes are referred to in [RFC8762] and are further described for SR networks in this document. The procedure defined in [RFC8762] is used to measure packet loss based on the transmission and reception of the STAMP test packets. The optional STAMP extensions defined in [RFC8972] are used for direct measurement of packet loss in SR networks. The performance measurement modes defined in this document are also applicable to measure packet loss in SR networks. The STAMP test packets are transmitted on the same path as the data traffic flow under measurement to measure the delay and packet loss experienced by the data traffic flow. Typically, the STAMP test packets are transmitted along an IP path between a Session-Sender and a Session-Reflector to measure delay and packet loss along that IP path. Matching forward direction path and return path for STAMP test packets, even for directly connected nodes are not guaranteed. It may be desired in SR networks that the same path (same set of links and nodes) between the Session-Sender and Session-Reflector be used for the STAMP test packets in both directions, for example, in an ECMP environment. This is achieved by using the optional STAMP extensions for SR-MPLS and SRv6 networks specified in [RFC9503] in one-way and two-way performance measurement modes. The STAMP Session-Reflector uses the return path parameters for the reply test packet from the STAMP extensions in the received Session-Sender test packet, as described in [RFC9503]. In case of loopback measurement mode, this is achieved by adding both forward direction path and return path in the SR-MPLS and SRv6 encapsulation of the STAMP test packets. Gandhi, et al. Expires 6 October 2024 [Page 6] Internet-Draft Using STAMP for Segment Routing April 2024 4. One-Way and Two-Way Measurement Modes in SR Networks As shown in Figure 1, Reference Topology, the STAMP Session-Sender S1 initiates a STAMP Session-Sender test packet and the STAMP Session- Reflector R1 transmits a reply test packet. The reply test packet may be transmitted to the STAMP Session-Sender S1 on the same path (same set of links and nodes) or a different path in the reverse direction from the path taken towards the Session-Reflector R1. T1 T2 / \ +-------+ Test Packet +-------+ | | - - - - - - - - - ->| | | S1 |=====================| R1 | | |<- - - - - - - - - - | | +-------+ Reply Test Packet +-------+ \ / T4 T3 STAMP Session-Sender STAMP Session-Reflector Figure 1: Reference Topology for One-Way and Two-Way Measurement Modes The T1 is a transmit timestamp, and T4 is a receive timestamp added by node S1. The T2 is a receive timestamp, and T3 is a transmit timestamp added by node R1. The nodes S1 and R1 may be connected via a link or an SR path [RFC8402]. The link may be a physical interface, virtual link, or Link Aggregation Group (LAG) [IEEE802.1AX], or LAG member. The SR path may be an SR Policy [RFC9256] on node S1 (called "head-end") with destination to node R1 (called "tail-end") or SR Flex-Algo IGP path [RFC9350]. A Layer-3 (L3) and Layer-2 (L2) VPN service may be carried over the SR path. 4.1. Example STAMP Reference Model An example STAMP Reference Model with some of the typical measurement parameters for STAMP test sessions for performance measurement mode of one-way and two-way is shown in Figure 2. Gandhi, et al. Expires 6 October 2024 [Page 7] Internet-Draft Using STAMP for Segment Routing April 2024 +------------+ | Network | | Controller | +------------+ / \ Performance Measurement Mode / \ Stateful or Stateless Destination UDP Port / \ Destination UDP Port Authentication Mode / \ Authentication Mode Keychain / \ Keychain Timestamp Format / \ Timestamp Format Protocol Mode / \ Protocol Mode Metric Type / \ v v +-------+ +-------+ | | | | | S1 |==========| R1 | | | | | +-------+ +-------+ STAMP Session-Sender STAMP Session-Reflector Figure 2: Example STAMP Reference Model for One-Way and Two-Way Measurement Modes A Destination UDP port number is selected for STAMP function as described in [RFC8762]. The same Destination UDP port can be used for STAMP test sessions for links, SR paths, and L3 and L2 services in SR networks. In this case, the Destination UDP port does not distinguish between the link, SR path, or L3 and L2 service STAMP test sessions. The Source UDP port is chosen by the Session-Sender. The same or different UDP Source port can be used for different STAMP test sessions and for STAMP test sessions for links, SR paths, and L3 and L2 services in SR networks. When using the authentication mode for the STAMP test sessions, the matching Authentication Type (e.g., HMAC-SHA-256) and Keychain is configured on STAMP Session-Sender and STAMP Session-Reflector [RFC8762]. Examples of the Timestamp Format is 64-bit truncated Precision Time Protocol (PTPv2) [IEEE.1588] and 64-bit Network Time Protocol (NTP) [RFC5905]. By default, the Session-Reflector replies in kind to the timestamp format received in the received Session-Sender test packet, as indicated by the "Z" flag in the Error Estimate field as described in [RFC8762] and it can be based on the Session-Reflector capability. In case of One-Way Protocol Mode (default Protocol Mode is Two-Way), Session-Reflector does not transmit reply test packets. Gandhi, et al. Expires 6 October 2024 [Page 8] Internet-Draft Using STAMP for Segment Routing April 2024 The Session-Reflector mode can be Stateful or Stateless as described in Section 4 of [RFC8762]. In One-Way Protocol Mode, Stateful Session-Reflector may be desired whereas in Two-Way Protocol Mode Stateless Session-Reflector may be desired. Examples of Delay Metric Type are one-way delay, round-trip delay, near-end (forward direction) and far-end (backward direction) delay as defined in [RFC8762], and loopback delay as defined in this document. Examples of Packet Loss Metric Type are round-trip, near-end (forward direction) and far-end (backward direction) packet loss as defined in [RFC8762]. The network controller shown in Figure 2 can be used to provision STAMP test sessions. The YANG data model for STAMP defined in [I-D.ietf-ippm-stamp-yang] can be used to provision Session-Sender and Session-Reflector and can also be used for streaming telemetry of the operational data. 4.2. Session-Sender Test Packet The content of an example Session-Sender test packet using an IP and UDP header [RFC0768] is shown in Figure 3. The payload containing the Session-Sender test packet, as defined in Section 3 of [RFC8972], is transmitted in an IP network. Note that [RFC8972] updates the Session-Sender test packet defined in [RFC8762] with optional STAMP Session Identifier (SSID). The SR encapsulation of the STAMP test packet is further described later in this document. +---------------------------------------------------------------+ | IP Header | . Source IP Address = Session-Sender IPv4 or IPv6 Address . . Destination IP Address=Session-Reflector IPv4 or IPv6 Address. . IPv4 Protocol or IPv6 Next-header = UDP (17) . . . +---------------------------------------------------------------+ | UDP Header | . Source Port = Chosen by Session-Sender . . Destination Port = User-configured Destination Port | 862 . . . +---------------------------------------------------------------+ | Payload = Test Packet as specified in Section 3 of RFC 8972 | . in Figure 1 and Figure 3 . . . +---------------------------------------------------------------+ Figure 3: Example Session-Sender Test Packet Gandhi, et al. Expires 6 October 2024 [Page 9] Internet-Draft Using STAMP for Segment Routing April 2024 4.3. Session-Sender Test Packet for Links The Session-Sender test packet as shown in Figure 3 is transmitted over the link for delay measurement. The local and remote IP addresses of the link MUST be used as Source and Destination Addresses in the IP header of the Session-Sender test packets, respectively. For IPv6 links, the link local addresses [RFC7404] can be used in the IPv6 header. An SR encapsulation (e.g., containing local adjacency SID of the link) can also be added for transmitting the Session-Sender test packets for links. The Session-Sender can use the local Address Resolution Protocol (ARP) table or any other similar method to obtain the IP and MAC addresses for the links for transmitting STAMP packets. Note that the Session-Sender test packet is further encapsulated with a Layer-2 header containing Session-Reflector MAC address as the Destination MAC address and Session-Sender MAC address as the Source MAC address for Ethernet links. For LAG member links, the STAMP extension for the Micro-Session ID TLV defined in [RFC9534] can be used to identify the member link. 4.4. Session-Sender Test Packet for SR-MPLS Data Plane 4.4.1. Session-Sender Test Packet for SR-MPLS Paths A Candidate-Path of an SR-MPLS Policy can contain one or more Segment Lists. Each SR-MPLS Segment List contains a list of 32-bit Label Stack Entry (LSE) that includes a 20-bit label value, 8-bit Time-To- Live (TTL) value, 3-bit Traffic-Class (TC) value and 1-bit End-Of- Stack (S) field. The Session-Sender test packets MUST be transmitted using each Segment List of the Candidate-Path of the SR-MPLS Policy for delay measurement. The content of an example Session-Sender test packet for an SR-MPLS path using the same SR-MPLS encapsulation as the data traffic transmitted over the path is shown in Figure 4. Gandhi, et al. Expires 6 October 2024 [Page 10] Internet-Draft Using STAMP for Segment Routing April 2024 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Segment[1] (top of stack) | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Segment[n] | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PSID (optional) | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Test Packet as shown in Figure 3 | . . +---------------------------------------------------------------+ Figure 4: Example Session-Sender Test Packet for SR-MPLS Path The head-end node address of the SR-MPLS Policy MUST be used as the Source Address in the IP header of the Session-Sender test packet. The endpoint address of the SR-MPLS Policy MUST be used as the Destination Address in the IP header of the Session-Sender test packet. The Session-Sender test packets may not carry MPLS header, for example, in case of a single-hop SR-MPLS path with Implicit NULL label or when using Penultimate Hop Popping (PHP). In this case, the Destination Address in the IP header ensures that the test packets reach the Session-Reflector on the SR-MPLS Policy endpoint. In case of SR-MPLS Policy with Color-Only Destination Steering, with endpoint as unspecified address (the null endpoint is 0.0.0.0 for IPv4 or :: for IPv6 (all bits set to the 0 value)) as defined in Section 8.8.1 of [RFC9256], the loopback address from the range 127/8 for IPv4, or the loopback address ::1/128 for IPv6 [RFC4291] can be used as the Destination Address in the IP header of the Session- Sender test packets, respectively. In this case, the SR-MPLS encapsulation MUST ensure the Session-Sender test packets reach the endpoint of the SR Policy (for example, by adding the Prefix SID label of the SR-MPLS Policy endpoint node in the Segment List). The Path Segment Identifier (PSID) [RFC9545] of an SR-MPLS Policy (either for Segment List or for Candidate-Path) can be added in the Segment List of the STAMP test packets, and can be used for direct measurement as described in Section 8, "Direct Measurement in SR Networks". Gandhi, et al. Expires 6 October 2024 [Page 11] Internet-Draft Using STAMP for Segment Routing April 2024 Each Flex-Algo in IGP in SR-MPLS networks [RFC9350] has Prefix SID labels advertised by the nodes. For delay measurement of SR-MPLS Flex-Algo IGP paths, the Session-Sender test packets carry the Flex- Algo Prefix SID label(s) of the Session-Sender and Session-Reflector in the MPLS header for that Flex-Algo IGP path under measurement. 4.4.2. Session-Sender Test Packet for Layer-3 Services over SR-MPLS Path For delay measurement of L3 service over SR-MPLS path, the same SR- MPLS label stack as the data packets transmitted over the L3 service including the L3VPN label (advertised by the Session-Reflector) is used to transmit Session-Sender test packets as shown in Figure 5. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Segment[1] (top of stack) | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | L3VPN Segment | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Test Packet as shown in Figure 3 | . Destination IP Address in L3VPN table . . Source IP Address in L3VPN table (return direction). . . +---------------------------------------------------------------+ Figure 5: Example Session-Sender Test Packet for L3 Service over SR-MPLS Path An IP header (as shown in Figure 3) is added in the Session-Sender test packets after the SR-MPLS encapsulation. The Destination Address added in the IP header MUST be reachable via the IP table lookup associated with the L3VPN label added for the L3 service on the Session-Reflector. The Source Address added in the IP header of the Session-Sender test packets MUST be reachable via the IP table lookup associated with the L3 service in the reverse direction. 4.4.3. Session-Sender Test Packet for Layer-2 Services over SR-MPLS Path For delay measurement of L2 service over SR-MPLS path, the same SR- MPLS label stack as the data packets transmitted over the L2 service including the L2VPN label (advertised by the Session-Reflector) is used to transmit Session-Sender test packets as shown in Figure 6. Gandhi, et al. Expires 6 October 2024 [Page 12] Internet-Draft Using STAMP for Segment Routing April 2024 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Segment[1] (top of stack) | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | L2VPN Segment | TC |1| TTL=1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Test Packet as shown in Figure 3 | . . +---------------------------------------------------------------+ Figure 6: Example Session-Sender Test Packet for L2 Service over SR-MPLS Path The L2VPN label is added with a TTL value of 1 in order to punt the Session-Sender test packet from data plane to CPU or slow path on Session-Reflector for STAMP processing. An IP header as shown in Figure 3 is added in the Session-Sender test packets after the MPLS header. It contains the Session-Sender Address as the Source Address and the Session-Reflector Address as the Destination Address. 4.5. Session-Sender Test Packet for SRv6 Data Plane 4.5.1. Session-Sender Test Packet for SRv6 Paths A Candidate-Path of an SRv6 Policy can contain one or more Segment Lists. Each Segment List can contain a number of SRv6 SIDs as defined in [RFC8986]. The Session-Sender test packets MUST be transmitted using each Segment List of the Candidate-Path of the SRv6 Policy for delay measurement. The test packet can contain an IPv6 header and SRv6 Segment Routing Header (SRH) carrying a Segment List as described in [RFC8754]. The content of an example Session-Sender test packet for an SRv6 path using the same IPv6/SRH encapsulation as the data traffic transmitted over the path can be encoded in Insert-Mode or Encap-Mode as shown in Figure 7. In Insert-Mode, an SRH is inserted after the IPv6 header of the STAMP test packets as shown in Example 1 of Figure 7. In Encap-Mode, the STAMP test packets are encapsulated in an outer IPv6 header with an SRH as shown in Example 2 of Figure 7. Gandhi, et al. Expires 6 October 2024 [Page 13] Internet-Draft Using STAMP for Segment Routing April 2024 +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ | SRH | . Segment List[0] = Session-Reflector IPv6 Address or . . Last Segment of Segment List or . . Optional PSID . . . . Next-Header = UDP (17) . . . +---------------------------------------------------------------+ | UDP Header as shown in Figure 3 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 3 | . . +---------------------------------------------------------------+ Example 1: Using Insert-Mode Encoding +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ | SRH | . Segment List[0] = Session-Reflector IPv6 Address or . . Last Segment of Segment List or . . Optional PSID . . . . Next-Header = 43 (IPv6) . . . +---------------------------------------------------------------+ | IPv6 Header as shown in Figure 3 | . . +---------------------------------------------------------------+ | UDP Header as shown in Figure 3 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 3 | . . +---------------------------------------------------------------+ Gandhi, et al. Expires 6 October 2024 [Page 14] Internet-Draft Using STAMP for Segment Routing April 2024 Example 2: Using Encap-Mode Encoding Figure 7: Example Session-Sender Test Packet for SRv6 Path The head-end node address of the SRv6 Policy MUST be used as the Source Address in the outer IPv6 header. The next Segment in the Segment List is used as the Destination Address in the outer header. A Segment List of the Candidate-Path of the SRv6 Policy can be empty. In this case, the endpoint address of the SRv6 Policy is added as the Destination Address in the outer IPv6 header. In Encap-Mode, an inner IPv6 header added MUST contain the endpoint address of the SRv6 Policy as the Destination Address and the head- end node address of the SRv6 Policy as the Source Address. In case of SRv6 Policy with Color-Only Destination Steering, with endpoint as unspecified address (the null endpoint :: for IPv6 (all bits set to the 0 value)) as defined in Section 8.8.1 of [RFC9256], the loopback address ::1/128 for IPv6 [RFC4291] can be used as the Destination Address in the inner IP header of the Session-Sender test packets. In this case, the IPv6/SRv6 encapsulation MUST ensure the Session- Sender test packets reach the endpoint of the SRv6 Policy (for example, by adding the Prefix SID or the IPv6 address of the SRv6 Policy endpoint node in the Segment List). In case of Penultimate Segment Popping (PSP), the IPv6/SRH encapsulation is removed by the penultimate node. In Insert-Mode, the Session-Sender MUST ensure that the Session-Sender test packets using the Segment List reach the Session-Reflector on the endpoint. In Encap-Mode, the Segment List or the inner IPv6 header MUST ensure that the Session-Sender test packets reach the Session-Reflector on the endpoint. The SRv6 network programming is described in [RFC8986]. The procedure defined for Upper-Layer (UL) Header processing for SRv6 End SIDs in Section 4.1.1 of [RFC8986] is used to process the IPv6/UDP header in the received Session-Sender test packets on the Session- Reflector. The Path Segment Identifier (PSID) [I-D.ietf-spring-srv6-path-segment] of the SRv6 Policy (either for Segment List or for Candidate-Path) can be added in the Segment List of the STAMP test packets and can be used for direct measurement as described in Section 8, "Direct Measurement in SR Networks". Gandhi, et al. Expires 6 October 2024 [Page 15] Internet-Draft Using STAMP for Segment Routing April 2024 Each Flex-Algo in IGP in SRv6 networks [RFC9350] has Prefix SIDs advertised by the nodes. For delay measurement of SRv6 Flex-Algo IGP paths, the Session-Sender test packets carry the Flex-Algo Prefix SIDs of the Session-Sender and Session-Reflector as the Source Address and Destination Address in the IPv6 header, respectively, for that SRv6 Flex-Algo IGP path under measurement. 4.5.2. Session-Sender Test Packet for Layer-3 Services over SRv6 Path For delay measurement of L3 service over SRv6 path, the same IPv6/SRH encapsulation as the data packets transmitted over the L3 service including the L3VPN SRv6 SID instantiated on the Session-Reflector (for example, End.DT6 SID instance, End.DT4 SID instance, End.DT46 instance, defined in [RFC8986]) is used to transmit Session-Sender test packets as shown in Figure 8 for both encoding modes, Insert- Mode and Encap-Mode. +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . in L3VPN table (return direction) . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ | SRH | . Segment List[0] = End.DT6/End.DT46 SID . . . . Next-Header = UDP (17) . . . +---------------------------------------------------------------+ | UDP Header as shown in Figure 3 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 3 | . . +---------------------------------------------------------------+ Example 1: Using Insert-Mode Encoding +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ | SRH | Gandhi, et al. Expires 6 October 2024 [Page 16] Internet-Draft Using STAMP for Segment Routing April 2024 . Segment List[0] = End.DT4/End.DT46 SID . . . . Next-Header = 4 (IPv4) . . . +---------------------------------------------------------------+ | IPv4 Header as shown in Figure 3 | . Destination IPv4 Address in L3VPN table . . Source IPv4 Address in L3VPN table (return direction) . . . +---------------------------------------------------------------+ | UDP Header as shown in Figure 3 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 3 | . . +---------------------------------------------------------------+ Example 2: Using Encap-Mode Encoding for IPv4 +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ | SRH | . Segment List[0] = End.DT6/End.DT46 SID . . . . Next-Header = 43 (IPv6) . . . +---------------------------------------------------------------+ | IPv6 Header as shown in Figure 3 | . Destination IPv6 Address in L3VPN table . . Source IPv6 Address in L3VPN table (return direction) . . . +---------------------------------------------------------------+ | UDP Header as shown in Figure 3 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 3 | . . +---------------------------------------------------------------+ Example 3: Using Encap-Mode Encoding for IPv6 Figure 8: Example Session-Sender Test Packet for L3 Service over SRv6 Path Gandhi, et al. Expires 6 October 2024 [Page 17] Internet-Draft Using STAMP for Segment Routing April 2024 In Insert-Mode, an SRH is inserted after the IPv6 header of the STAMP test packets as shown in Example 1 of Figure 8. The IPv6 Source Address added in the IPv6 header MUST be reachable via the IPv6 table lookup for the L3 service in the reverse direction in case of End.DT6 and End.DT46 SIDs to return the reply test packets over that L3 service. In Encap-Mode, the STAMP test packets are encapsulated in outer IPv6 header with an SRH as shown in Examples 2 and 3 of Figure 8. An inner IP header is added in the Session-Sender test packets after the IPv6/SRH encapsulation. The IPv6 Destination Address added in the inner IPv6 header MUST be reachable via the IPv6 table lookup associated with the L3VPN SRv6 SID added. Similarly, the IPv4 Destination Address added in the inner IPv4 header MUST be reachable via the IPv4 table lookup associated with the L3VPN SRv6 SID added. The IPv6 Source Address added in the inner IPv6 header MUST be reachable via the IPv6 table lookup for the L3 service in the reverse direction to return the reply test packets over that L3 service. Similarly, the IPv4 Source Address added in the inner IPv4 header MUST be reachable via the IPv4 table lookup for the L3 service in the reverse direction. 4.5.3. Session-Sender Test Packet for Layer-2 Services over SRv6 Path For delay measurement of L2 service over SRv6 path, the same IPv6/SRH encapsulation as the data packets transmitted over the L2 service including the L2VPN SRv6 SID instantiated on the Session-Reflector (for example, End.DT2U SID instance defined in [RFC8986]) is used to transmit Session-Sender test packets as shown in Figure 9 for both encoding modes, Insert-Mode and Encap-Mode. Gandhi, et al. Expires 6 October 2024 [Page 18] Internet-Draft Using STAMP for Segment Routing April 2024 +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ | SRH | . Segment List[0] = End.DT2U SID . . . . Next-Header = UDP (17) . . . +---------------------------------------------------------------+ | UDP Header as shown in Figure 3 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 3 | . . +---------------------------------------------------------------+ Example 1: Using Insert-Mode Encoding +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ | SRH | . Segment List[0] = End.DT2U SID . . . . Next-Header = 43 (IPv6) . . . +---------------------------------------------------------------+ | IPv6 Header as shown in Figure 3 | . Hop Limit = 1 . . . +---------------------------------------------------------------+ | UDP Header as shown in Figure 3 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 3 | . . +---------------------------------------------------------------+ Example 2: Using Encap-Mode Encoding Gandhi, et al. Expires 6 October 2024 [Page 19] Internet-Draft Using STAMP for Segment Routing April 2024 Figure 9: Example Session-Sender Test Packet for L2 Service over SRv6 Path In both encoding modes, the Session-Sender address is added as the Source Address and Session-Reflector address is added as the Destination Address in the outer IPv6 header. In Insert-Mode, an SRH is inserted after the IPv6 header of the STAMP test packets as shown in Example 1 of Figure 9. In Encap-Mode, in addition to the outer IPv6/SRH encapsulation, an inner IPv6 header is added as shown in Example 2 of Figure 9, with Hop Limit value of 1 in order to punt the Session-Sender test packets from data plane to CPU or slow path on Session-Reflector for STAMP processing. The inner IPv6 header contains the Session-Sender Address as the Source Address and the Session-Reflector Address as the Destination Address. 4.6. Session-Sender Test Packet for P2MP SR Paths The delay measurement procedure for SR-MPLS and SRv6 paths is equally applicable to the P2MP SR-MPLS and SRv6 paths. The Point-to-Multipoint (P2MP) SR policy that originates from a root node terminates on multiple destinations called leaf nodes (e.g., P2MP SR Policy [I-D.ietf-pim-sr-p2mp-policy] Candidate-Path). The Session-Sender root node MUST transmit the Session-Sender test packets for delay measurement using each Segment List of the Candidate-Path. These Segment Lists may contain replication SIDs [RFC9524]. The [I-D.mirsky-ippm-asymmetrical-pkts] defines extensions for using STAMP for performance measurement in multicast environment. Those extensions also apply to the performance measurement for P2MP SR paths. The Session-Reflector MUST return the leaf node address in Source Address of the Session-Reflector test packets. This is used by the P2MP root node to measure the delay for each leaf node independently. 4.7. Session-Reflector Test Packet The Session-Reflector decapsulates the SR header (SR-MPLS header or IPv6/SRH) from the received Session-Sender test packets. The Session-Reflector test packet is generated using the information from the IP/UDP header of the received Session-Sender test packet as shown in Figure 10. Gandhi, et al. Expires 6 October 2024 [Page 20] Internet-Draft Using STAMP for Segment Routing April 2024 +---------------------------------------------------------------+ | IP Header | . Source IP Address . . = Destination IP Address from Session-Sender Test Packet . . Destination IP Address . . = Source IP Address from Session-Sender Test Packet . . IPv4 Protocol or IPv6 Next-header = UDP (17) . . . +---------------------------------------------------------------+ | UDP Header | . Source Port . . = Destination Port from Session-Sender Test Packet . . Destination Port . . = Source Port from Session-Sender Test Packet . . . +---------------------------------------------------------------+ | Payload = Test Packet as specified in Section 3 of RFC 8972 | . in Figure 2 and Figure 4 . . . +---------------------------------------------------------------+ Figure 10: Example Session-Reflector Test Packet The payload contains the Session-Reflector test packet defined in Section 3 of [RFC8972]. 4.7.1. One-Way Measurement Mode In one-way measurement mode, a reply test packet with the contents as shown in Figure 10 is transmitted by the Session-Reflector, for links, SR paths and L3 and L2 services in SR networks. The Session- Reflector test packet can be transmitted in the reverse direction on the same path as the forward direction or a different path than the forward direction to the Session-Sender. In this mode, as per Reference Topology in Figure 1, all timestamps T1, T2, T3, and T4 are collected by the STAMP test packets. However, only timestamps T1 and T2 are used to measure one-way delay as (T2 - T1). Note that the delay value (T2 - T1) is referred to as near-end (forward direction) one-way delay and the delay value (T4 - T3) is referred to as far-end (backward direction) one-way delay. The one- way measurement mode requires the clocks on the Session-Sender and Session-Reflector to be synchronized. Gandhi, et al. Expires 6 October 2024 [Page 21] Internet-Draft Using STAMP for Segment Routing April 2024 4.7.1.1. One-Way Protocol Mode In one-way Protocol Mode, Session-Reflector does not generate and transmit reply test packets upon receiving Session-Sender test packets. The Session-Sender can request in the test packet to the Session-Reflector to not transmit the reply test packet using the "No Reply Requested" flag in the Control Code Sub-TLV in the Return Path TLV defined in [RFC9503]. Alternatively, Session-Reflector can be provisioned with Protocol Mode as one-way to not generate and transmit reply test packets. A different Destination UDP port (other than port 862) can be provisioned for Protocol Mode one-way than the two-way mode. In this mode, as per Reference Topology in Figure 1, only timestamps T1 and T2 are collected by the STAMP Session-Sender test packets and one-way delay value (T2 - T1) is measured by the Session-Reflector. 4.7.2. Two-Way Measurement Mode In two-way measurement mode, it may be desired that the Session- Reflector test packets, shown in Figure 10, are transmitted on the same return path as the forward direction path by the Session- Reflector. For links, the Session-Sender may request in the test packet to the Session-Reflector to transmit the reply test packet on the same link in the reverse direction. It can use the "Reply Requested on the Same Link" flag in the Control Code Sub-TLV in the Return Path TLV defined in [RFC9503] for this request. For SR paths, the Session-Sender may request in the test packet to the Session-Reflector to transmit the reply test packet on a specific SR return path. For example, reverse SR path associated with the forward direction SR path [I-D.ietf-pce-sr-bidir-path] or the Binding SID of the reverse SR Policy or the Prefix SID of the Session-Sender. It can use Segment List sub-TLV in the Return Path TLV defined in [RFC9503] for this request. For SR Flex-Algo IGP paths, the Session-Sender may request in the test packet to the Session-Reflector to transmit the reply test packet on the SR path for the same Flex-Algo in the reverse direction using Segment List sub-TLV in the Return Path TLV defined in [RFC9503]. Gandhi, et al. Expires 6 October 2024 [Page 22] Internet-Draft Using STAMP for Segment Routing April 2024 For L3 services, the Session-Reflector can derive the L3 service in the reverse direction using the L3VPN SID received in the Session- Sender test packets to transmit the Session-Reflector test packets. For L2 services, the Session-Reflector can derive the L2 service in the reverse direction using the L2VPN SID received in the Session- Sender test packets to transmit the Session-Reflector test packets. In this mode, as per Reference Topology in Figure 1, all timestamps T1, T2, T3, and T4 are collected by the STAMP test packets. All four timestamps are used to measure round-trip delay as ((T4 - T1) - (T3 - T2)). 5. Loopback Measurement Mode in SR Networks The Session-Sender test packets are transmitted in loopback measurement mode to measure loopback delay of a bidirectional circular path. In this mode, the received Session-Sender test packets MUST NOT be punted out of the fast path in data plane (i.e., to slow path or control-plane) at the Session-Reflector. In other words, the Session-Reflector does not process them and generate Session-Reflector test packets. This is a new measurement mode, not defined by the STAMP process in [RFC8762]. T1 / +-------+ Test Packet +-------+ | | - - - - - - - - - - | | | S1 |====================|| R1 | | |<- - - - - - - - - - | | +-------+ Return Test Packet +-------+ \ Loopback T4 STAMP Session-Sender Figure 11: Reference Topology for Loopback Measurement Mode Gandhi, et al. Expires 6 October 2024 [Page 23] Internet-Draft Using STAMP for Segment Routing April 2024 In this mode, as shown in Figure 11, Reference Topology for Loopback Measurement Mode, the Session-Sender test packet received back at the Session-Sender retrieves the timestamp T1 from the test packet and collects the receive timestamp T4 locally. Both these timestamps are used to measure the loopback delay as (T4 - T1). The loopback delay includes the STAMP test packet processing delay on the Session- Reflector component. The Session-Reflector processing delay component includes only the time required to loop the STAMP test packet from the incoming interface to the outgoing interface in the data plane. The Session-Reflector does not timestamp the test packets and hence does not need timestamping capability. 5.1. Loopback Measurement Mode STAMP Packet Processing The Session-Sender MUST set the Destination UDP port to the UDP port it uses to receive the return Session-Reflector test packets (other than the Destination UDP port 862 which is used by the STAMP Session- Reflector). The same UDP port can be used as the Destination and Source UDP port in the Session-Sender test packet as shown in Figure 12. The IP header for the return path in the Session-Sender test packets MUST set the Destination Address equal to the Session-Sender address as shown in Figure 12 to return the packet to the Session-Sender. The test packet is encapsulated with the forward direction path to transmit to the Session-Reflector. +---------------------------------------------------------------+ | IP Header (Return Path) | . Source IP Address = Session-Sender IP Address . . Destination IP Address = Session-Sender IP Address . . IPv4 Protocol or IPv6 Next-header = UDP (17) . . . +---------------------------------------------------------------+ | UDP Header | . Source Port = Chosen by Session-Sender . . Destination Port = Source Port . . . +---------------------------------------------------------------+ | Payload = Test Packet as specified in Section 3 of RFC 8972 | . in Figure 1 and Figure 3 . . . +---------------------------------------------------------------+ Figure 12: Example Session-Sender Return Test Packet in Loopback Measurement Mode Gandhi, et al. Expires 6 October 2024 [Page 24] Internet-Draft Using STAMP for Segment Routing April 2024 The Session-Reflector does not perform the STAMP process, hence the loopback function simply processes the encapsulation including IP and SR headers (but does not process the UDP header) to forward the received Session-Sender test packet to the Session-Sender without STAMP modifications defined in [RFC8762]. The Session-Sender can use the STAMP Session ID (SSID) field in the received test packet and local configuration to identify its STAMP test session that uses the loopback measurement mode. In this mode, at the Session-Sender, the 'Session-Sender Sequence Number', 'Session-Sender Timestamp', 'Session-Sender Error Estimate', and 'Session-Sender TTL' fields MUST be set to zero in the transmitted Session-Sender test packets and MUST be ignored in the received test packets. 5.2. Loopback Measurement Mode for Links The Session-Sender test packets in loopback measurement mode may be transmitted with a Layer-2 header for the forward direction path as shown in Figure 13, containing the link MAC address on the Session- Reflector as the Destination Address and the link MAC address on the Session-Sender as the Source MAC address for Ethernet links. +---------------------------------------------------------------+ | L2 MAC Header (Forward Path) | . Source Address = Link MAC Address on Session-Sender . . Destination Address = Link MAC Address on Session-Reflector . . Ether-Type = 0x0800 (IPv4) Or 0x86DD (IPv6) . . . +---------------------------------------------------------------+ | Test Packet as shown in Figure 12 (Return Path) | . . +---------------------------------------------------------------+ Figure 13: Example Session-Sender Test Packet in Loopback Measurement Mode for Ethernet Link An SR encapsulation (e.g., containing adjacency SID of the link) for the forward direction path can also be added after the Layer-2 header. The IP header for the return path of the Session-Sender test packets is added as shown in Figure 12 and it MUST set the Source and Destination Address equal to the link address on the Session-Sender to return the packet to the Session-Sender. The Session-Reflector decapsulates the L2 header and forwards the packet using the IP header for the return path to the Session-Sender. Gandhi, et al. Expires 6 October 2024 [Page 25] Internet-Draft Using STAMP for Segment Routing April 2024 5.3. Loopback Measurement Mode for SR-MPLS Data Plane 5.3.1. Loopback Measurement Mode for SR-MPLS Paths In loopback measurement mode for SR-MPLS paths, the Session-Sender test packet can carry either the Segment List of the forward direction path only or both the forward direction and the return paths in MPLS header as shown in Figure 14. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Segment[1] (top of stack) | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Segment[n] | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PSID (optional) | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Test Packet as shown in Figure 12 (Return Path) | . . +---------------------------------------------------------------+ Figure 14: Example Session-Sender Test Packet in Loopback Measurement Mode for SR-MPLS Path In case of SR-MPLS Policy using Penultimate Hop Popping (PHP), the Session-Sender MUST ensure that the STAMP test packets reach the SR- MPLS Policy endpoint (for example, by adding the Prefix SID label of the SR-MPLS Policy endpoint in the Segment List of the forward direction path). The IP header for the return path of the Session-Sender test packets is added and it MUST set the Destination Address equal to the Session-Sender address as shown in Figure 12. 5.3.1.1. SR-MPLS Return Path The Session-Sender test packets, in SR-MPLS label stack, carry return path, in addition to forward direction path. For example, they can carry the SR-MPLS label stack of the Segment List of the associated reverse Candidate-Path [I-D.ietf-pce-sr-bidir-path] or the Binding SID label of the reverse SR-MPLS Policy or the SR-MPLS Prefix SID label of the Session-Sender. Gandhi, et al. Expires 6 October 2024 [Page 26] Internet-Draft Using STAMP for Segment Routing April 2024 For SR-MPLS Flex-Algo IGP paths, Session-Sender test packets can carry the SR-MPLS Prefix SID label of the Session-Sender for the same Flex-Algo in the reverse direction. In this case, the optional PSID added in the Session-Sender test packet is for the SR-MPLS return path and is allocated by the Session-Sender. 5.3.1.2. IP Return Path The Session-Sender test packets in MPLS header carry the SR-MPLS label stack of the forward direction path only. The Session-Reflector decapsulates the MPLS header and forwards the packet using the IP header for the return path. In this case, the optional PSID added in the Session-Sender test packet is for the SR-MPLS forward direction path and is allocated by the Session-Reflector. 5.3.2. Loopback Measurement Mode for Layer-3 Services over SR-MPLS Path In loopback measurement mode for L3 service over SR-MPLS path, the SR-MPLS label stack of the data packets transmitted over the L3 service is used to transmit Session-Sender test packets as shown in Figure 15. Gandhi, et al. Expires 6 October 2024 [Page 27] Internet-Draft Using STAMP for Segment Routing April 2024 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Segment[1] (top of stack) | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | L3VPN Segment (Return Path)| TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Test Packet as shown in Figure 12 (Return Path) | . Destination IP Address in L3VPN table . . . +---------------------------------------------------------------+ Example 1: Using SR-MPLS Return Path 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Segment[1] (top of stack) | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | L3VPN Segment(Forward Path)| TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Test Packet as shown in Figure 12 (Return Path) | . Destination IP Address in L3VPN table . . . +---------------------------------------------------------------+ Example 2: Using IP Return Path Figure 15: Example Session-Sender Test Packet in Loopback Measurement Mode for L3 Service over SR-MPLS Path The IP header for the return path of the Session-Sender test packets is added and it MUST set the Destination Address equal to the Session-Sender address as shown in Figure 12. The Destination Address added in the IP header for the return path MUST be reachable via the IP table lookup associated with the L3VPN label added in the test packets. Gandhi, et al. Expires 6 October 2024 [Page 28] Internet-Draft Using STAMP for Segment Routing April 2024 5.3.2.1. SR-MPLS Return Path The SR-MPLS label stack except the L3VPN label (advertised by the Session-Reflector) of the forward direction L3 service is added in the test packets. In addition, the SR-MPLS label stack including the L3VPN label for the reverse direction L3 service is also added in the test packets. 5.3.2.2. IP Return Path The SR-MPLS label stack including the L3VPN label (advertised by the Session-Reflector) of the forward direction L3 service is added in the test packets. The Session-Reflector decapsulates the MPLS header and forwards the packet using the IP header for the return path (after adding SR-MPLS encapsulation for the reverse direction L3 service). 5.3.3. Loopback Measurement Mode for Layer-2 Services over SR-MPLS Path In loopback measurement mode for L2 service over SR-MPLS path, the SR-MPLS label stack of the data packets transmitted over the L2 service is used to transmit Session-Sender test packets as shown in Figure 16. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Segment[1] (top of stack) | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | L2VPN Segment (Return Path)| TC |1| TTL=1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Test Packet as shown in Figure 12 (Return Path) | . . +---------------------------------------------------------------+ Example: Using SR-MPLS Return Path Figure 16: Example Session-Sender Test Packet in Loopback Measurement Mode for L2 Service over SR-MPLS Path Gandhi, et al. Expires 6 October 2024 [Page 29] Internet-Draft Using STAMP for Segment Routing April 2024 The IP header for the return path MUST be added in the Session-Sender test packets that has the Destination Address equal to the Session- Sender address as shown in Figure 12. 5.3.3.1. SR-MPLS Return Path The SR-MPLS label stack except the L2VPN label (advertised by the Session-Reflector) of the forward direction L2 service is added in the test packets. In addition, the SR-MPLS label stack including the L2VPN label for the reverse direction L2 service is added the test packet with a TTL value of 1 in order to punt the test packet from data plane to CPU or slow path on Session-Sender for STAMP processing. 5.3.3.2. IP Return Path The STAMP test packets without using SR-MPLS return path is outside the scope of this document. 5.4. Loopback Measurement Mode for SRv6 Data Plane 5.4.1. Loopback Measurement Mode for SRv6 Paths In loopback measurement mode for SRv6 paths, the Session-Sender test packet can carry either the Segment List of the forward direction path only using Encap-Mode encoding or both the forward direction and the return paths in IPv6/SRH using Insert-Mode encoding as shown in Figure 17. +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ | SRH | . Segment List[0] = Session-Sender IPv6 Address or . . Last Segment of Segment List of Return Path. . . . . . Next-Header = UDP (17) . . . +---------------------------------------------------------------+ | UDP Header as shown in Figure 12 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 12 | Gandhi, et al. Expires 6 October 2024 [Page 30] Internet-Draft Using STAMP for Segment Routing April 2024 . . +---------------------------------------------------------------+ Example 1: Using Insert-Mode Encoding with SRv6 Return Path +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ | SRH | . Segment List[0] = Session-Reflector IPv6 Address or . . Last Segment of Segment List or . . Optional PSID . . . . Next-Header = 43 (IPv6) or 4 (IPv4) . . . +---------------------------------------------------------------+ | IP Header as shown in Figure 12 (Return Path) | . . +---------------------------------------------------------------+ | UDP Header as shown in Figure 12 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 12 | . . +---------------------------------------------------------------+ Example 2: Using Encap-Mode Encoding with IP Return Path Figure 17: Example Session-Sender Test Packet in Loopback Measurement Mode for SRv6 Path The Session-Sender MUST ensure that the Session-Sender test packets using the Segment List reach the SRv6 Policy endpoint (for example, by adding the Prefix SID or IPv6 address of the SRv6 Policy endpoint in the Segment List) in both encoding modes. 5.4.1.1. SRv6 Return Path For SRv6 return path, the STAMP test packets are encoded in Insert- Mode as shown in Example 1 in Figure 17. Gandhi, et al. Expires 6 October 2024 [Page 31] Internet-Draft Using STAMP for Segment Routing April 2024 The Session-Sender test packets, in SRv6 Segment List, carry return path, in addition to forward direction path. For example, they can carry the Segment List of the associated reverse Candidate-Path [I-D.ietf-pce-sr-bidir-path] or the Binding SID of the reverse SRv6 Policy or the SRv6 Prefix SID of the Session-Sender. For SRv6 Flex-Algo IGP paths, the Session-Sender test packets can carry the SRv6 Prefix SID of the Session-Sender for the same Flex- Algo in the reverse direction. In this case, the optional PSID added in the Session-Sender test packet is for the SRv6 return path and is allocated by the Session- Sender. 5.4.1.2. IP Return Path For IP return path, the STAMP test packets are encoded in Encap-Mode as shown in Example 2 in Figure 17. The Session-Sender test packets carry the Segment List of the SRv6 forward direction path only. An inner IP header for return path MUST be added in the Session- Sender test packets that has the Destination Address equal to the Session-Sender address to return the packet to the Session-Sender. The Session-Reflector decapsulates the IPv6/SRH headers and forwards the packet using the inner IP header for the return path. In this case, the optional PSID added in the Session-Sender test packet is for the SRv6 forward direction path and is allocated by the Session-Reflector. 5.4.2. Loopback Measurement Mode for Layer-3 Services over SRv6 Path In loopback measurement mode for L3 service over SRv6 path, the IPv6/ SRH encapsulation of the data packets transmitted over the L3 service including the L3VPN SRv6 SID (for example, End.DT6 SID instance, End.DT4 SID instance, etc. defined in [RFC8986]) is used to transmit Session-Sender test packets as shown in Figure 18. +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ Gandhi, et al. Expires 6 October 2024 [Page 32] Internet-Draft Using STAMP for Segment Routing April 2024 | SRH | . Segment List[0] = End.DT4/DT6/DT46 SID of Return Path . . . . . . Next-Header = UDP (17) . . . +---------------------------------------------------------------+ | UDP Header as shown in Figure 12 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 12 | . . +---------------------------------------------------------------+ Example 1: Using Insert-Mode Encoding with SRv6 Return Path +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ | SRH | . Segment List[0] = End.DT4/DT46 SID of Forward Path . . . . Next-Header = 4 (IPv4) . . . +---------------------------------------------------------------+ | IPv4 Header as shown in Figure 12 (Return Path) | . Destination IPv4 Address in L3VPN table . +---------------------------------------------------------------+ | UDP Header as shown in Figure 12 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 12 | . . +---------------------------------------------------------------+ Example 2: Using Encap-Mode Encoding with IPv4 Return Path +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ Gandhi, et al. Expires 6 October 2024 [Page 33] Internet-Draft Using STAMP for Segment Routing April 2024 | SRH | . Segment List[0] = End.DT6/DT46 SID of Forward Path . . . . Next-Header = 43 (IPv6) . . . +---------------------------------------------------------------+ | IPv6 Header as shown in Figure 12 (Return Path) | . Destination IPv6 Address in L3VPN table . +---------------------------------------------------------------+ | UDP Header as shown in Figure 12 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 12 | . . +---------------------------------------------------------------+ Example 3: Using Encap-Mode Encoding with IPv6 Return Path Figure 18: Example Session-Sender Test Packet in Loopback Measurement Mode for L3 Service over SRv6 Path 5.4.2.1. SRv6 Return Path For SRv6 return path, the STAMP test packets are encoded in Insert- Mode as shown in Example 1 in Figure 18. The SRv6 Segment List except the L3VPN SRv6 SID instantiated on the Session-Reflector of the forward direction L3 service is added in the IPv6/SRH encapsulation of the test packet. In addition, SRv6 Segment List including the L3VPN SRv6 SID instantiated on the Session-Sender for the reverse direction L3 service is also added in the IPv6/SRH encapsulation to return the test packet to the Session-Sender from the Session-Reflector. 5.4.2.2. IP Return Path For IP return path, the STAMP test packets are encoded in Encap-Mode as shown in Example 2 and 3 in Figure 18. The SRv6 Segment List including the L3VPN SRv6 SID instantiated on the Session-Reflector for the forward direction L3 service is added in the IPv6/SRH encapsulation to transmit the test packet to the Session-Reflector. An inner IP header for return path MUST also be added in the Session- Sender test packets that has the Destination Address equal to the Session-Sender address to forward the test packet to the Session- Sender from the Session-Reflector. In this case, the Destination Gandhi, et al. Expires 6 October 2024 [Page 34] Internet-Draft Using STAMP for Segment Routing April 2024 Address added in the inner IP header for the return path MUST be reachable via the IPv4 or IPv6 table lookup associated with the L3VPN SRv6 SID on the Session-Reflector. The Session-Reflector decapsulates the IPv6/SRH and forwards the packet using the inner IP header for the return path (after adding IPv6/SRv6 encapsulation for the reverse direction L3 service). 5.4.3. Loopback Measurement Mode for Layer-2 Services over SRv6 Path In loopback measurement mode for L2 service over SRv6 path, the IPv6/ SRH encapsulation of the data packets transmitted over the L2 service including the L2VPN SRv6 SID (for example, End.DT2U SID instance defined in [RFC8986]) is used to transmit Session-Sender test packets as shown in Figure 19. +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ | SRH | . Segment List[0] = End.DT2U SID of Return Path . . . . . . Next-Header = UDP (17) . . . +---------------------------------------------------------------+ | UDP Header as shown in Figure 12 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 12 | . . +---------------------------------------------------------------+ Example: Using Insert-Mode Encoding with SRv6 Return Path Figure 19: Example Session-Sender Test Packet in Loopback Mode for L2 Service over SRv6 Path 5.4.3.1. SRv6 Return Path For SRv6 return path, the STAMP test packets are encoded in Insert- Mode as shown in Figure 19. Gandhi, et al. Expires 6 October 2024 [Page 35] Internet-Draft Using STAMP for Segment Routing April 2024 The SRv6 Segment List except the L2VPN SRv6 SID instantiated on the Session-Reflector of the forward direction L2 service is added in the IPv6/SRH encapsulation of the test packet. In addition, SRv6 Segment List including the L2VPN SRv6 SID instantiated on the Session-Sender for the reverse direction L2 service is also added in the IPv6/SRH encapsulation to return the test packet to the Session-Sender from the Session-Reflector. 5.4.3.2. IP Return Path For IP return path, the STAMP test packets are encoded in Encap-Mode. However, this mode is outside the scope of this document. 6. Loopback Measurement Mode with Timestamp and Forward Function in SR Networks This document defines a new STAMP measurement mode, called "loopback measurement mode with timestamp and forward" that uses network programming function. In this mode, the timestamps T1, T2, and T4 are collected by the Session-Sender test packet as shown in Figure 20. The network programming function is used to optimize the "operations of punt test packet and generate return test packet" on the Session-Reflector, as timestamping is implemented in fast path in data plane. This helps to achieve higher number of STAMP test session scale and faster measurement interval. T1 T2 / \ +-------+ Test Packet +-------+ | | - - - - - - - - - - - | | | S1 |======================|| R1 | | |<- - - - - - - - - - - | | +-------+ Return Test Packet +-------+ \ Loopback T4 STAMP Session-Sender STAMP Session-Reflector (Timestamp, and Forward) Figure 20: Reference Topology for Loopback Measurement Mode with Timestamp and Forward Function The Session-Sender adds transmit timestamp (T1) in the payload of the Session-Sender test packet. The Session-Reflector adds the receive timestamp (T2) in the payload of the received test packet in fast path in data plane without punting the test packet (e.g., to slow path or control-plane) for STAMP packet processing. The network Gandhi, et al. Expires 6 October 2024 [Page 36] Internet-Draft Using STAMP for Segment Routing April 2024 programming function carried by the test packet enables the Session- Reflector to add the "receive timestamp" (T2) at specific offset in the payload of the test packet. 6.1. Loopback Measurement Mode with Timestamp and Forward Function for SR-MPLS Data Plane The MPLS Network Action (MNA) Sub-Stack defined in [I-D.ietf-mpls-mna-hdr] is used for SR-MPLS paths for "timestamp and forward network programming function" for STAMP test packets. The MNA Sub-Stack carries the MNA Label (bSPL value TBA1) as defined in [I-D.ietf-mpls-mna-hdr]. A new MNA Opcode (value MNA.TSF) is defined for the network action for "Timestamp and Forward network programming function". In the Session-Sender test packets for SR-MPLS paths, the MNA Sub- Stack with Opcode MNA.TSF is added in the MPLS header as shown in Figure 21, to collect timestamp in the "Receive Timestamp" field in the payload of the test packet from Session-Reflector. The Ingress- to-Egress (I2E), Hop-By-Hop (HBH), Select scope (IHS) is set to "I2E" when return path is IP/UDP. The Network Action Sub-Stack Length (NASL) is set to 0 when there is no Label Stack Entry (LSE) after the MNA.TSF Opcode in the MNA Sub-Stack. The U flag is set to skip the network action and forward the packet (and not drop the packet). The Label Stack for the SR-MPLS return path can be added after the MNA Sub-Stack to receive the return test packet on a specific path as described in loopback measurement for SR-MPLS path this document. The Ingress-to-Egress (I2E), Hop-By-Hop (HBH), Select scope (IHS) is set to "Select" in this case. Gandhi, et al. Expires 6 October 2024 [Page 37] Internet-Draft Using STAMP for Segment Routing April 2024 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Segment[1] (top of stack) | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Segment[n] | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MNA Label (value TBA1) | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |7-bit MNA.TSF| 0x0 |R|IHS|S| RES |U|NASL=0 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Test Packet as shown in Figure 12 (Return Path) | . . +---------------------------------------------------------------+ Figure 21: Example Session-Sender Test Packet in Loopback Measurement Mode with TSF for SR-MPLS Paths When a Session-Reflector receives a packet with MNA Sub-Stack with Opcode MNA.TSF, after timestamping the packet in STAMP payload at specific offset, the Session-Reflector pops the MNA Sub-Stack (after completing any other network actions) and forwards the packet as defined in loopback measurement mode for SR-MPLS path in this document. 6.1.1. Timestamp and Forward Network Action Assignment New MPLS Network Action Opcode is defined called "Timestamp and Forward Network Action, MNA.TSF". The MNA.TSF Opcode is statically configured on the STAMP Session-Reflector node with a value from "Private Use from Range 111-126". The timestamp format for 64-bit PTPv2 and NTP to be added in the STAMP payload is statically configured for MNA.TSF. The offset in the STAMP payload (e.g., for unauthenticated mode with offset 16 bytes) is also statically configured for MNA.TSF. 6.1.2. Node Capability for MNA Sub-Stack with Opcode MNA.TSF The STAMP Session-Sender needs to know if the Session-Reflector can process the MNA Sub-Stack with Opcode MNA.TSF to avoid dropping the test packets. The signaling extension for this capability exchange or local configuration are outside the scope of this document. Gandhi, et al. Expires 6 October 2024 [Page 38] Internet-Draft Using STAMP for Segment Routing April 2024 6.2. Loopback Measurement Mode with Timestamp and Forward Function for SRv6 Data Plane The [RFC8986] defines SRv6 Endpoint Behaviours for SRv6 nodes. A new SRv6 Endpoint Behaviour is defined for "Timestamp and Forward (TSF) network programming function" for STAMP test packets. In the Session-Sender test packets for SRv6 paths, Timestamp and Forward Endpoint Function (End.TSF) is carried with the target Segment Identifier (SID) in SRH [RFC8754] as shown in Figure 22, for both Insert-Mode and Encap-Mode encoding, to collect timestamp in the "Receive Timestamp" field in the payload of the test packet from Session-Reflector. Gandhi, et al. Expires 6 October 2024 [Page 39] Internet-Draft Using STAMP for Segment Routing April 2024 +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ | SRH | . . . . . Next-Header = UDP (17) . . . +---------------------------------------------------------------+ | UDP Header as shown in Figure 12 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 12 | . . +---------------------------------------------------------------+ Example 1: Using Insert-Mode Encoding with SRv6 Return Path +---------------------------------------------------------------+ | IPv6 Header | . Source IP Address = Session-Sender IPv6 Address . . Destination IP Address = Segment List[Segments Left] . . Next-Header = 43, Routing Type = SRH (4) . . . +---------------------------------------------------------------+ | SRH | . Segment List[0] = End.TSF SID . . . . Next-Header = 43 (IPv6) or 4 (IPv4) . . . +---------------------------------------------------------------+ | IP Header as shown in Figure 12 (Return Path) | . . +---------------------------------------------------------------+ | UDP Header as shown in Figure 12 | . . +---------------------------------------------------------------+ | Payload as shown in Figure 12 | . . +---------------------------------------------------------------+ Example 2: Using Encap-Mode Encoding with IP Return Path Gandhi, et al. Expires 6 October 2024 [Page 40] Internet-Draft Using STAMP for Segment Routing April 2024 Figure 22: Example Session-Sender Test Packet in Loopback Measurement Mode with TSF for SRv6 Paths The STAMP test packets are encoded in Insert-Mode for SRv6 return path and in Encap-Mode for IP return path as defined in loopback measurement mode for SRv6 path in this document. When a Session-Reflector receives a packet with Timestamp and Forward Endpoint (End.TSF) for the target SID, which is local, after timestamping the test packet at specific offset, the Session- Reflector forwards the packet as defined in the loopback measurement mode for SRv6 paths. 6.2.1. Timestamp and Forward Endpoint Function Assignment New SRv6 Endpoint Behavior is defined called "Endpoint Behavior bound to SID with Timestamp and Forward (End.TSF)". The End.TSF is a node SID instantiated at STAMP Session-Reflector node. The End.TSF is statically configured on the STAMP Session-Reflector node and not advertised into the routing protocols. The timestamp format for 64-bit PTPv2 and NTP to be added in the STAMP payload is statically configured for End.TSF. The offset in the STAMP payload (e.g., for unauthenticated mode with offset 16 bytes) is also statically configured for End.TSF. 6.2.2. Node Capability for Timestamp and Forward Endpoint Function The STAMP Session-Sender needs to know if the Session-Reflector can process the Timestamp and Forward Endpoint Function to avoid dropping test packets. The signaling extension for this capability exchange or local configuration are outside the scope of this document. 7. Packet Loss Measurement in SR Networks The procedure described in Section 4 for delay measurement in SR networks using STAMP test packets also allows for round-trip, near- end (forward direction) and far-end (backward direction) inferred packet loss measurement in SR networks. This, however, provides only an approximate view of the data packet loss. The loopback measurement mode and loopback measurement mode with timestamp and forward network programming function allow only the round-trip packet loss measurement. Gandhi, et al. Expires 6 October 2024 [Page 41] Internet-Draft Using STAMP for Segment Routing April 2024 8. Direct Measurement in SR Networks The STAMP "Direct Measurement" TLV (Type 5) defined in [RFC8972] can be used in SR networks for data packet loss measurement. The STAMP test packets with this TLV are transmitted using the procedure described in Section 4 for delay measurement in SR networks using STAMP test packets to collect the Session-Sender transmit counters and Session-Reflector receive and transmit counters of the data packet flows for direct measurement. The PSID carried in the received data packet for the traffic flow under measurement can be used to measure receive data packets (for receive traffic counter) for an SR path on the Session-Reflector. In case of L3 and L2 services in SR networks, the associated SR-MPLS service labels and SRv6 service SIDs can be used to measure receive data packets (for receive traffic counters) on the Session-Reflector. In loopback measurement mode and loopback measurement mode with timestamp and forward network programming function, the direct measurement is not applicable. 9. ECMP Measurement in SR Networks An SR path can have ECMPs between the source and transit nodes, between transit nodes and between transit and destination nodes. Usage of Anycast SID [RFC8402] by an SR path can result in ECMP paths via transit nodes, part of that Anycast group. The STAMP test packets need to be transmitted to traverse different ECMP paths to measure delay of an SR path. Forwarding plane has various hashing functions available to forward packets on specific ECMP paths. The mechanisms described in [RFC8029] and [RFC5884] for handling ECMPs are also applicable to delay measurement. For SR-MPLS paths, sweeping of MPLS entropy label [RFC6790] values can be used in Session-Sender test packets and Session-Reflector test packets to take advantage of the hashing function in data plane to influence the ECMP path taken by them. In IPv4 header of the Session-Sender test packets and Session- Reflector test packets sweeping of Destination Address from the range 127/8 can be used to exercise ECMP paths taken by them when using MPLS header. As specified in [RFC6437], Flow Label field in the outer IPv6 header can also be used for sweeping to exercise different IPv6 ECMP paths. Gandhi, et al. Expires 6 October 2024 [Page 42] Internet-Draft Using STAMP for Segment Routing April 2024 10. STAMP Session State The threshold-based notification for delay and packet loss metrics may not be generated if the delay and packet loss metrics are not changing significantly. For an unambiguous monitoring, the controller may need to distinguish the cases whether the session is active, but delay and packet loss metrics are not changing significantly crossing the threshold or the session has failed. The STAMP test session state monitoring allows to know if the performance measurement test is active, idle or failed. The STAMP test session state is notified as idle when Session-Sender is not transmitting test packets. The STAMP test session state is initially notified as active when Session-Sender is transmitting test packets and as soon as one or more reply test packets are received at the Session-Sender. The STAMP test session state is notified as failed when consecutive N number of reply test packets are not received at the Session-Sender after the session state is notified as active, where N (consecutive packet loss count) is a locally provisioned value. In this case, the failed state of the STAMP test session on the Session-Sender also indicates the connectivity (i.e., liveness) failure of the link, SR path or the L3/L2 service where the STAMP session was active. 11. Additional STAMP Test Packet Processing Rules The processing rules described in this section are applicable to the STAMP test packets for links, SR paths, and L3 and L2 services in SR networks. 11.1. TTL The TTL field in the IPv4 and MPLS headers of the Session-Sender and Session-Reflector test packets MUST be set to 255 as per Generalized TTL Security Mechanism (GTSM) [RFC5082]. 11.2. IPv6 Hop Limit The Hop Limit (HL) field in all IPv6 headers of the Session-Sender and Session-Reflector test packets MUST be set to 255 as per Generalized TTL Security Mechanism (GTSM) [RFC5082]. 11.3. Router Alert Option The Router Alert IP option (RAO) [RFC2113] MUST NOT be set in the STAMP test packets to be able to punt the test packets using the UDP ports for STAMP. Gandhi, et al. Expires 6 October 2024 [Page 43] Internet-Draft Using STAMP for Segment Routing April 2024 11.4. IPv6 Flow Label The Flow Label field in the IPv6 header of the STAMP test packet is set to the value that is used by the data packets for the traffic flow on the SR path being measured by the Session-Sender. The Session-Reflector SHOULD use the Flow Label value it received in the IPv6 header of the Session-Sender test packet in the reply test packet, and it can be based on the local configuration on the Session-Reflector. 11.5. UDP Checksum For IPv4 STAMP test packets, where the local processor after adding the timestamp, is not capable of re-computing the UDP checksum or adding checksum complement [RFC7820], the Session-Sender and Session- Reflector can set the UDP checksum value to 0 [RFC8085]. For IPv6 STAMP test packets, where the local processor after adding the timestamp, is not capable of re-computing the UDP checksum or adding checksum complement [RFC7820], the Session-Sender and Session- Reflector can use the procedure defined in [RFC6936] for the UDP checksum (with value set to 0) for the UDP ports used for the STAMP sessions, and it can be based on the local policy. 12. Implementation Status Editorial note: Please remove this section prior to publication. The following Cisco routing platforms running IOS-XR operating system have participated in an interop testing for one-way, two-way and loopback measurement modes for SR-MPLS and SRv6: * Cisco 8802 (based Cisco Silicon One Q200) * Cisco ASR9904 with Lightspeed linecard and Tomahawk linecard * Cisco NCS5500 (based on Broadcom Jericho1 platform) * Cisco NCS5700 (based on Broadcom Jericho2 platform) 13. Security Considerations The security considerations specified in [RFC8762], [RFC8972], and [RFC9503] also apply to the procedures described in this document. Gandhi, et al. Expires 6 October 2024 [Page 44] Internet-Draft Using STAMP for Segment Routing April 2024 Use of HMAC-SHA-256 in the authenticated mode protects the data integrity of the STAMP test packets. The message integrity protection using HMAC defined in Section 4.4 of [RFC8762] can be used with the procedure described in this document. STAMP uses the well-known UDP port number that could become a target of denial of service (DoS) or could be used to aid on-path attacks. Thus, the security considerations and measures to mitigate the risk of the attack documented in Section 6 of [RFC8545] equally apply to the procedures described in this document. The procedures defined in this document is intended for deployment in a single network administrative domain. As such, the Session-Sender address, Session-Reflector address, and forward direction and return paths are provisioned by the operator for the STAMP session. It is assumed that the operator has verified the integrity of the forward direction and return paths of the STAMP test packets. When using the procedures defined in [RFC6936], the security considerations specified in [RFC6936] also apply. The security considerations specified in [I-D.ietf-mpls-mna-hdr] are also applicable to the procedures for the SR-MPLS data plane defined in this document. SRv6 STAMP test packets can use the HMAC protection authentication defined for SRH in [RFC8754]. The security considerations specified in [RFC8986] are also applicable to the procedures for the SRv6 data plane defined in this document. 14. IANA Considerations This document does not require any IANA action. 15. References 15.1. Normative References [RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768, DOI 10.17487/RFC0768, August 1980, . [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . Gandhi, et al. Expires 6 October 2024 [Page 45] Internet-Draft Using STAMP for Segment Routing April 2024 [RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and L. Yong, "The Use of Entropy Labels in MPLS Forwarding", RFC 6790, DOI 10.17487/RFC6790, November 2012, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . [RFC8762] Mirsky, G., Jun, G., Nydell, H., and R. Foote, "Simple Two-Way Active Measurement Protocol", RFC 8762, DOI 10.17487/RFC8762, March 2020, . [RFC8972] Mirsky, G., Min, X., Nydell, H., Foote, R., Masputra, A., and E. Ruffini, "Simple Two-Way Active Measurement Protocol Optional Extensions", RFC 8972, DOI 10.17487/RFC8972, January 2021, . [RFC8986] Filsfils, C., Ed., Camarillo, P., Ed., Leddy, J., Voyer, D., Matsushima, S., and Z. Li, "Segment Routing over IPv6 (SRv6) Network Programming", RFC 8986, DOI 10.17487/RFC8986, February 2021, . [RFC9503] Gandhi, R., Filsfils, C., Chen, M., Janssens, B., and R. Foote, "Simple Two-Way Active Measurement Protocol (STAMP) Extensions for Segment Routing Networks", RFC 9503, October 2023, . [RFC9534] Li, Z., Zhou, T., Guo, J., Mirsky, G., and R. Gandhi, "Simple Two-Way Active Measurement Protocol Extensions for Performance Measurement on a Link Aggregation Group", RFC 9534, January 2024, . [I-D.ietf-mpls-mna-hdr] Rajamanickam, J., Ed., Gandhi, R., Ed., Zigler, R., Song, H., and K. Kompella, "MPLS Network Action Sub-Stack Solution", Work in Progress, Internet-Draft, draft-ietf- mpls-mna-hdr-04, October 2023, . 15.2. Informative References Gandhi, et al. Expires 6 October 2024 [Page 46] Internet-Draft Using STAMP for Segment Routing April 2024 [IEEE.1588] IEEE, "1588-2008 IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems", March 2008. [IEEE802.1AX] IEEE Std. 802.1AX, "IEEE Standard for Local and metropolitan area networks - Link Aggregation", November 2008. [RFC2113] Katz, D., "IP Router Alert Option", RFC 2113, DOI 10.17487/RFC2113, February 1997, . [RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing Architecture", RFC 4291, DOI 10.17487/RFC4291, February 2006, . [RFC5082] Gill, V., Heasley, J., Meyer, D., Savola, P., Ed., and C. Pignataro, "The Generalized TTL Security Mechanism (GTSM)", RFC 5082, DOI 10.17487/RFC5082, October 2007, . [RFC5884] Aggarwal, R., Kompella, K., Nadeau, T., and G. Swallow, "Bidirectional Forwarding Detection (BFD) for MPLS Label Switched Paths (LSPs)", RFC 5884, DOI 10.17487/RFC5884, June 2010, . [RFC5905] Mills, D., Martin, J., Burbank, J., and W. Kasch, "Network Time Protocol Version 4: Protocol and Algorithms Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, . [RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme, "IPv6 Flow Label Specification", RFC 6437, DOI 10.17487/RFC6437, November 2011, . [RFC6936] Fairhurst, G. and M. Westerlund, "Applicability Statement for the Use of IPv6 UDP Datagrams with Zero Checksums", RFC 6936, DOI 10.17487/RFC6936, April 2013, . [RFC7404] Behringer, M. and E. Vyncke, "Using Only Link-Local Addressing inside an IPv6 Network", RFC 7404, DOI 10.17487/RFC7404, November 2014, . Gandhi, et al. Expires 6 October 2024 [Page 47] Internet-Draft Using STAMP for Segment Routing April 2024 [RFC7820] Mizrahi, T., "UDP Checksum Complement in the One-Way Active Measurement Protocol (OWAMP) and Two-Way Active Measurement Protocol (TWAMP)", RFC 7820, DOI 10.17487/RFC7820, March 2016, . [RFC8029] Kompella, K., Swallow, G., Pignataro, C., Ed., Kumar, N., Aldrin, S., and M. Chen, "Detecting Multiprotocol Label Switched (MPLS) Data-Plane Failures", RFC 8029, DOI 10.17487/RFC8029, March 2017, . [RFC8085] Eggert, L., Fairhurst, G., and G. Shepherd, "UDP Usage Guidelines", BCP 145, RFC 8085, DOI 10.17487/RFC8085, March 2017, . [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., Decraene, B., Litkowski, S., and R. Shakir, "Segment Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, July 2018, . [RFC8545] Morton, A., Ed. and G. Mirsky, Ed., "Well-Known Port Assignments for the One-Way Active Measurement Protocol (OWAMP) and the Two-Way Active Measurement Protocol (TWAMP)", RFC 8545, DOI 10.17487/RFC8545, March 2019, . [RFC8754] Filsfils, C., Ed., Dukes, D., Ed., Previdi, S., Leddy, J., Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header (SRH)", RFC 8754, DOI 10.17487/RFC8754, March 2020, . [RFC9256] Filsfils, C., Talaulikar, K., Voyer, D., Bogdanov, A., and P. Mattes, "Segment Routing Policy Architecture", RFC 9256, July 2022, . [RFC9350] Psenak, P., Ed., Hegde, S., Filsfils, C., Talaulikar, K., and A. Gulko, "IGP Flexible Algorithm", RFC 9350, February 2023, . [RFC9524] Voyer, D., Ed., Filsfils, C., Parekh, R., Bidgoli, H., and Z. Zhang, "Segment Routing Replication for Multipoint Service Delivery", RFC 9524, February 2024, . Gandhi, et al. Expires 6 October 2024 [Page 48] Internet-Draft Using STAMP for Segment Routing April 2024 [RFC9545] Cheng, W., Li, H., Li, C., Gandhi, R., and R. Zigler, "Path Segment in MPLS-Based Segment Routing Network", RFC 9545, February 2024, . [I-D.ietf-pim-sr-p2mp-policy] Voyer, D., Ed., Filsfils, C., Parekh, R., Bidgoli, H., and Z. Zhang, "Segment Routing Point-to-Multipoint Policy", Work in Progress, Internet-Draft, draft-ietf-pim-sr-p2mp- policy-07, 11 October 2023, . [I-D.ietf-spring-srv6-path-segment] Li, C., Cheng, W., Chen, M., Dhody, D., and Y. Zhu, "Path Segment for SRv6 (Segment Routing in IPv6)", Work in Progress, Internet-Draft, draft-ietf-spring-srv6-path- segment-07, 19 October 2023, . [I-D.ietf-pce-sr-bidir-path] Li, C., Chen, M., Cheng, W., Gandhi, R., and Q. Xiong, "Path Computation Element Communication Protocol (PCEP) Extensions for Associated Bidirectional Segment Routing (SR) Paths", Work in Progress, Internet-Draft, draft-ietf- pce-sr-bidir-path-13, 13 February 2024, . [I-D.ietf-ippm-stamp-yang] Mirsky, G., Min, X., Luo, W. S., and R. Gandhi, "Simple Two-way Active Measurement Protocol (STAMP) Data Model", Work in Progress, Internet-Draft, draft-ietf-ippm-stamp- yang-12, 5 November 2023, . [I-D.mirsky-ippm-asymmetrical-pkts] Mirsky, G., Ruffini, E., Nydell, H., and R. Foote, "Performance Measurement with Asymmetrical Packets in STAMP", Work in Progress, Internet-Draft, draft-mirsky- ippm-asymmetrical-pkts-04, 20 February 2024, . Gandhi, et al. Expires 6 October 2024 [Page 49] Internet-Draft Using STAMP for Segment Routing April 2024 Acknowledgments The authors would like to thank Thierry Couture and Ianik Semco for the discussions on the use-cases for Performance Measurement in Segment Routing. The authors would also like to thank Greg Mirsky, Gyan Mishra, Xie Jingrong, and Mike Koldychev for reviewing this document and providing useful comments and suggestions. Patrick Khordoc, Haowei Shi, Amila Tharaperiya Gamage, Pengyan Zhang, Ruby Lin and Radu Valceanu have helped improve the mechanisms described in this document. Contributors The following people have substantially contributed to this document: Bart Janssens Colt Email: Bart.Janssens@colt.net Navin Vaghamshi Reliance Email: Navin.Vaghamshi@ril.com Moses Nagarajah Telstra Email: Moses.Nagarajah@team.telstra.com Amit Dhamija Arrcus India Email: amitd@arrcus.com Authors' Addresses Rakesh Gandhi (editor) Cisco Systems, Inc. Canada Email: rgandhi@cisco.com Clarence Filsfils Cisco Systems, Inc. Email: cfilsfil@cisco.com Daniel Voyer Bell Canada Email: daniel.voyer@bell.ca Gandhi, et al. Expires 6 October 2024 [Page 50] Internet-Draft Using STAMP for Segment Routing April 2024 Mach(Guoyi) Chen Huawei Email: mach.chen@huawei.com Richard Foote Nokia Email: footer.foote@nokia.com Gandhi, et al. Expires 6 October 2024 [Page 51]