Internet DRAFT - draft-turaga-mpls-test-labels
draft-turaga-mpls-test-labels
Network Working Group P. Turaga
Internet-Draft R. Raszuk
Intended status: Standards Track Bloomberg LP
Expires: March 12, 2017 September 8, 2016
MPLS Test Labels
draft-turaga-mpls-test-labels-01
Abstract
This document describes an underlying mechanism for automatic
diagnostics of link quality between any two devices connected
together by standard point to point link.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on March 12, 2017.
Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
Turaga & Raszuk Expires March 12, 2017 [Page 1]
Internet-Draft draft-turaga-mpls-test-labels September 2016
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
3. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 3
4. MPLS Special Purpose Loop Label . . . . . . . . . . . . . . . 4
4.1. Operation of MPLS Special Purpose Loop Label . . . . . . 6
4.2. Comparison with stated test requirements . . . . . . . . 8
4.3. Probe size and rate calculation . . . . . . . . . . . . . 8
5. MPLS Special Purpose Swap-to-Drop and Drop Labels . . . . . . 9
6. Probe's QOS marking . . . . . . . . . . . . . . . . . . . . . 9
7. Bandwidth Considerations for link under test . . . . . . . . 10
8. I2RS and YANG modelling . . . . . . . . . . . . . . . . . . . 10
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10
10. Security Considerations . . . . . . . . . . . . . . . . . . . 10
11. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 10
12. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 10
13. References . . . . . . . . . . . . . . . . . . . . . . . . . 11
13.1. Normative References . . . . . . . . . . . . . . . . . . 11
13.2. Informative References . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11
1. Terminology
o RTT - Round Trip Time
o TTL - Time to Live
o BFD - BiDirectional Failure Detection
o LFM - Link Fault Management
o ICMP - Internet Control Message Protocol
2. Introduction
Real time monitoring of WAN or MAN link quality presents a real
operational challenge. The common use of circuit emulation
techniques by carriers makes detection of the circuits degradation
difficult. Very often such reduced link quality results in increased
queuing times or packet drops beyond SLA guarantees. Furthermore,
the characteristics of link degradation is different from link to
link.
The problem space described above is further complicated due to the
following reasons:
Turaga & Raszuk Expires March 12, 2017 [Page 2]
Internet-Draft draft-turaga-mpls-test-labels September 2016
o Link anomalies may not occur at the same uniform rate or be of the
same constant and continuous pattern. This transient
characteristic maybe a function of load or other temporary
problems for example transport network over-subscription.
o Encountered degradated service behavior may not translate to link
errors or packet discards on either end of the suspected link
because the emulated link consisting of multiple independent L2
segments in the carrier's network.
Currently available tools on the circuit endpoints (usually routers)
do not allow easy way to diagnose circuit health. Tools used today
to detect link issues include:
o Creating hardware or software loops manually - this results in the
actual link under test to be taken out of service. Test traffic
is then sent through the link and based on the results of the
test, link quality issues are detected.
o Regular pings/probes on directly connected links between routers/
network devices - Depending on the size of the probe packets and
the rate at which they are sent between the network devices and
the loss, the link issues are detected. The issue with this
approach is that network processor on the router has to process
all these packets. This causes an additional processing load on
the routers.
o BFD, IP protocol hellos etc are based on detecting neighbor state
based on tiny and lightweight hellos. Such probes were designed
for fast detection of end-to-end link state events .. not to
evaluate link quality. If say N hellos send in T interval are
lost it is an indication about link or peer down event.
o The layer 2 OAM tools are not capable of addressing the
requirements since by definition an emulated link consists of
number of different L2 links hidden by the emulation layer and its
encapsulation. L2 OAM could only indicate potential problems
within single layer 2 link. They are light weight and some of
these issues can only be detected at various levels of data rates
(within agreed SLAs) transiting via such links.
3. Requirements
The following are some of the key considerations required to be
addressed in an alternative diagnostics solutions:
o The testing should be atomic in nature - the UUT in this document
is a single p2p link.
o The test should not be subject to any alterations by externally
injected packets
o The probe packets should never be able to transit L3 node to any
other L3 node
Turaga & Raszuk Expires March 12, 2017 [Page 3]
Internet-Draft draft-turaga-mpls-test-labels September 2016
o The level of diagnostics data should be configurable such that
operator is able to inject anywhere from 0.1% to 100% of test load
of a given max link capacity with build in automatic consideration
of existing average of production traffic load (unless link is
considered as taken out-of-service).
o The duration of the test traffic should be either configurable by
the operator or controlled by built-in detection heuristics.
o The frequency of the test traffic should be either configurable by
the operator or controlled by built-in detection heuristics.
o Probes should not be subject to process switching by the route
processors on either end of the link during the burst.
o The solution should strive to minimize amount of required protocol
extensions for as easy as possible inter-operability
characteristics.
o In the topologies where Link Aggregation is used, the aggregated
bandwidth of the link should be considered instead of the
individual links. The probe accounting should be recorded as
total of all link members. Probe's hashing should follow normal
data plane load balancing rules as configured on the directly
connected peering routers.
4. MPLS Special Purpose Loop Label
The mechanism for the set of proposed requirements can be constructed
by combining two standards based protocol elements: TTL field
processing and mpls label lookup.
Special purpose mpls label will allow to setup a scoped link based
loop and TTL field can be used to limit the loop duration.
The MPLS local loop label can be either special purpose MPLS label
(value 4-6 or 8-12) or Extended special purpose MPLS label (values
16-239). The use of the extended special purpose label would
inherently increase the label stack to two for the probe packets with
top most label to be of value 15 (extension label per [RFC7274]).
By injecting N number of such probe packets (with max payload up to
given MTU value) it is possible to control test load. The
observation of the difference between number of injected probes and
number of MPLS TTL expiration for a given test bundle would be equal
to the number of packet drops observed. Similarly by calculating the
time difference of time stamp from the moment of injection of the
probe packet to its TTL expiration a good average of real RTT of a
link can be calculated. The observation of such RTT times across
number of test sequences will allow to model the aggregated queuing
delays possibly allowing prediction of upcoming drops.
Turaga & Raszuk Expires March 12, 2017 [Page 4]
Internet-Draft draft-turaga-mpls-test-labels September 2016
It needs to be also observed that the above tests are completely
orthogonal and would co-exit with current mechanism like LSP Ping
[RFC4379] or MPLS BFD [RFC5884]. The proposed in this document
procedures do not intend to verify control plane or control plane to
data plane correctness.
As stated already TTL handling or label lookup are both standards
based and do not require any changes to the underlying hardware.
However choice of MPLS special purpose label is proposed to simplify
the test operation and remove significant new data plane
requirements. The semantics of such label would be following:
When packet is received with MPLS Local Loop Label TTL must be
decremented and if > 0 the entire packet must be returned over the
same interface over which it was received. If after decrementing TTL
the resulting register value is 0 the probe packet should be dropped
and local TTL-equal-zero error should be generated. If the probes
header contain additional information (for example as described in
LSP ping RFC) such header should be copied into the error message
punted the the local control plane CPU for further processing.
If implementation allows such local error could be optionally logged
and handled in data plane only reporting the aggregated results to
the local RE/RP.
Such solutions has following advantages over possible alternative
which would be use of regular IP packets:
o The new mechanism with new type of MPLS label allows for defining
a special handling which will not overlap with any of the possible
interference with existing protocols ... for example ICMP
traceroute
o The new non transitive specification of mpls special purpose label
allows for much stricter security and safer operation of the
proposed testing model
o The separate new label TTL expiration may be easily handled
differently then general TTL expiration thus resulting in no data
plane rate limiting or pacing
o Use of signalling less special purpose MPLS LLL relaxes the
solution from either additional control plane extensions or
requirements to either extend opex with new static routes or to
introduce new extensions to already established link local
addresses.
o The main principle of the tests are to be congruent with switching
vectors of production traffic. Therefor it is important that
probe packets use the same lookup LFIB structure as regular user
data packets as far as egress and ingress line cards are
concerned.
Turaga & Raszuk Expires March 12, 2017 [Page 5]
Internet-Draft draft-turaga-mpls-test-labels September 2016
The MPLS LLL label is to be auto assigned to FECs by either automated
association with numbered IP addresses for the given link or with
link local addresses. The resolution to MAC address of L2 rewrites
(with 8847 ethertype) would be resolved locally through corresponding
L3 adjacency addresses.
4.1. Operation of MPLS Special Purpose Loop Label
The following is considered as a high level description of proposed
solution:
o Two routers R1 and R2 connected together by link L1
o The RTT between R1 and R2 on link L1 is 5ms
o R1 and R2 have IP connectivity with each other on 10.10.10.0/30
numbered link. R1 has been configured with IP an address of
10.10.10.1 and R2 has been configured with an IP address of
10.10.10.2
o A MPLS local loop label is allocated to match the corresponding
FECs of the link addresses (or link local when applicable).
The following IPv4 probe packet has been injected from R1 towards the
FEC of R2:
o Source IP address: 10.10.10.1
o Destination IP address: 10.10.10.100
o TTL = 254
o payload optional ... (to be discussed by WG)
o Proposed format:
o R1 sends probe of the following format:
0 1 2 3
+-------------+-------------+-------------+-------------+
| LLL Label |EXP |S| TTL |
+-------------+-------------+-------------+-------------+
| |
| Packet payload (TBD) | |
| Header - IP v4/v6 or perhaps lift from MPLS echo RFC) |
| Data - filled with data up to MTU if needed |
+-------------+-------------+-------------+-------------+
Figure 1: MPLS Label Stack Object
Label: 20 bits
Exp: Experimental Use, 3 bits
S: Bottom of Stack, 1 bit
TTL: Time to Live, 8 bits
Figure 1: MPLS LLL probe format
Turaga & Raszuk Expires March 12, 2017 [Page 6]
Internet-Draft draft-turaga-mpls-test-labels September 2016
Such test packet would be resolve and encapsulated into MPLS LLL
label and directed towards R2 with even TTL value.
Test sequence:
o Packet arrives at R2 and TTL is decremented following MPLS LLL
label lookup and reinjection towards R1
o Packet keeps looping till TTL expires on R1.
o Upon TTL expiration an ICMP TTL-eq-zero error message is being
logged on R1 (originator of probe sequence). The ICMP message
contains the header information of the original packet is being
send to local control plane processor.
o The local implementation may optionally optimize the accounting
for the received vs missed in flight probe packets in the data
plane layer and only report the aggregated sequence history
o The analysis of the packets header would be logged in local or
remote database and become very valuable source of the link's
behavioral metrics.
Observations:
o A test probe packet has potential to be amplified up to 254 times
o An ICMP TTL expired message is indicative of successful test -
healthy link
o No ICMP TTL message implies that the original packet was lost
while it was getting looped between two routers. So, No ICMP TTL
message means that test for a specific probe has failed. Let's
note that this would have no bearing on the local control plane of
either end of the test. Any reaction associated and triggered by
the test results would be driven by controller (residing together
or separate from participating routers.
o Ability to send multiple packets of different sizes on the link
with inherently controlled TTL loop can results in expected burst
of control/probe traffic on the link under test
o Such probe burst can be programmed to get to a certain % of the
link speed for a short time
Based on fine tuned testing scenario allowing to fill the bandwidth
up to a certain % of link capacity the count of packets originally
sent by router R1 should be the same as the number of ICMP TTL
expired messages. If the count of packets originally sent by router
is the same as the number of ICMP TTL expired messages then the test
is successful. If however the number of ICMP TTL expired messages is
less than the count of packets originally sent by the router then the
test is unsuccessful proving potential problems with the link.
A test probe packet with even initial TTL value will generate a TTL
time expired ICMP message on the originating router. A test probe
Turaga & Raszuk Expires March 12, 2017 [Page 7]
Internet-Draft draft-turaga-mpls-test-labels September 2016
packet with odd initial TTL value will generate a TTL time expired
ICMP message on the neighboring router. It is RECOMMENDED that the
test probe is sent with even initial TTL value. So, ICMP messages
are not traversing the link under test.
It is RECOMMENDED that a special payload structure is built for these
test probes with sequence numbers. When the TTL expires and an ICMP
message is generated, the IP header + 64 bits from original packet
gets copied to ICMP message. [ RFC 792 ]. This can be used for
associating the ICMP message and the test.
4.2. Comparison with stated test requirements
Analysis of the proposed solution against the actual new test
methodology requirements:
o Provides means to potentially fill up the part of link bandwidth
very rapidly because of inherent amplification due to high initial
TTL value. The fill level of the test traffic is a function of:
Initial packet size (higher the packet size the higher the fill
level), Initial TTL value (higher the TTL value, higher the
multiplicative factor for packets and hence higher the fill
level), Initial number of packets sent (the more the packets sent
the more the fill level).
o Test can be run together with production traffic. There is no
impact on production traffic neither there is any requirement to
stop production traffic in order to perform the test.
o The amplification of the packets and looping happens as a part of
inherent forwarding in the routers. This solution does not
require a special process in software or hardware to send the test
probes between the two routers.
o The link is not required to be taken offline for testing. This
testing can co-exist with production traffic.
o This mechanism is light-weight and does not require a lot of
protocol programming or significant enhancements.
4.3. Probe size and rate calculation
Initial packet size and rate are important to determine the test fill
level for the link. The test packet loops the same number of times
as the original TTL value of the original packet. The time it takes
for the original packet to come back to the original router is the
RTT (Round Trip Time) value between two routers.
Under the assumptions that: RTT of link under test is 1ms, link speed
1 Gb/s, packet size of test packet is 1536 bytes, TTL on original
packet is 254, then the test packets would loop on the link for next
254ms.
Turaga & Raszuk Expires March 12, 2017 [Page 8]
Internet-Draft draft-turaga-mpls-test-labels September 2016
Under the above assumptions it is easy to calculate that in order
fill the 1 Gb/s link to 100% 81 such probe packets need to be
injected into any link under test. Likewise in order to fill the
link to 20% of its capacity 16 probe packets are required.
Link Simmering - To be able to set non user impacting graceful link
removal from IGP topology and conduct full bandwidth test then return
the link to the topology.
5. MPLS Special Purpose Swap-to-Drop and Drop Labels
While tests described in the former sections are addressing most of
the link health monitoring cases there is additional class of tests
which may require quite different data plane pattern characteristics.
Specifically there is requirement for injection of large number of
full MTU echo probes which will only be able to be received by peer's
ingress line card and returned once to the sender without even
processing the TTL field.
For such purpose this document also defines two additional types of
Special Purpose MPLS labels: Swap-to-Drop and Drop Labels.
The semantics of Swap-to-Drop label requires the network device which
received packets with Swap-to-Drop label to decrement TTL and swap
the label with Drop label and return it over the same interface over
which it arrived.
The semantics of Drop Label means to drop the packet received with
such special purpose label while incrementing either global interface
drop counters or defining the new counters dedicated to logging the
received packets with drop label.
Similar to MPLS Special purpose loop label the swap-to-drop as well
as drop labels are local to the link. Packets containing such labels
should be never switched via a network node.
6. Probe's QOS marking
Since injected test packets are regular IP packets encapsulated in
MPLS they can be marked with any class of service inserted into EXP
field. As a result the test probes similar to actual data will be
processed based on the real QoS configuration and will be subject to
treatment defined for a given packet class.
That allows both prioritization as well as de-prioritization of a
given set of test probes.
Turaga & Raszuk Expires March 12, 2017 [Page 9]
Internet-Draft draft-turaga-mpls-test-labels September 2016
7. Bandwidth Considerations for link under test
The payload of the test packets can be of any IP protocols.
The link fill levels is also a function of Inter-packet gap of the
test and the RTT of that link. Deterministic fill levels can only be
derived by accounting for RTT of the link under test.
8. I2RS and YANG modelling
It is expected that link testing methodology described in this
document will be accessible by I2RS channel as well as YANG models
will be defined for both setting and retrieval of the data.
9. IANA Considerations
This document requires IANA to define new Special purpose mpls label
or extended special purpose MPLS label values subject to WG
recommendation.
The following special purpose labels are to be allocated:
o Local Loop Label
o Swap-to-Drop Label
o Drop Label
10. Security Considerations
While the proposed mechanism does not define any new protocols nor
protocol extensions of already existing specifications it does relay
on the TTL-expiry notifications.
Such notifications must be enabled and must not be limited in any way
for the specific class of probe packets.
It is highly recommended that test destinations addresses to be not
routeable beyond their locally attached devices.
11. Contributors
Authors would like to thank Truman Boyes and Leo Pang for their
valuable input.
12. Acknowledgments
Turaga & Raszuk Expires March 12, 2017 [Page 10]
Internet-Draft draft-turaga-mpls-test-labels September 2016
13. References
13.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>.
13.2. Informative References
[RFC0792] Postel, J., "Internet Control Message Protocol", STD 5,
RFC 792, DOI 10.17487/RFC0792, September 1981,
<http://www.rfc-editor.org/info/rfc792>.
[RFC3927] Cheshire, S., Aboba, B., and E. Guttman, "Dynamic
Configuration of IPv4 Link-Local Addresses", RFC 3927,
DOI 10.17487/RFC3927, May 2005,
<http://www.rfc-editor.org/info/rfc3927>.
[RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing
Architecture", RFC 4291, DOI 10.17487/RFC4291, February
2006, <http://www.rfc-editor.org/info/rfc4291>.
[RFC4379] Kompella, K. and G. Swallow, "Detecting Multi-Protocol
Label Switched (MPLS) Data Plane Failures", RFC 4379,
DOI 10.17487/RFC4379, February 2006,
<http://www.rfc-editor.org/info/rfc4379>.
[RFC5884] Aggarwal, R., Kompella, K., Nadeau, T., and G. Swallow,
"Bidirectional Forwarding Detection (BFD) for MPLS Label
Switched Paths (LSPs)", RFC 5884, DOI 10.17487/RFC5884,
June 2010, <http://www.rfc-editor.org/info/rfc5884>.
[RFC7274] Kompella, K., Andersson, L., and A. Farrel, "Allocating
and Retiring Special-Purpose MPLS Labels", RFC 7274,
DOI 10.17487/RFC7274, June 2014,
<http://www.rfc-editor.org/info/rfc7274>.
Authors' Addresses
Partha Turaga
Bloomberg LP
731 Lexington Ave
New York City, NY 10022
USA
Email: pturaga@bloomberg.net
Turaga & Raszuk Expires March 12, 2017 [Page 11]
Internet-Draft draft-turaga-mpls-test-labels September 2016
Robert Raszuk
Bloomberg LP
731 Lexington Ave
New York City, NY 10022
USA
Email: robert@raszuk.net
Turaga & Raszuk Expires March 12, 2017 [Page 12]
