Internet DRAFT - draft-elkins-ippm-6man-pdm-option

draft-elkins-ippm-6man-pdm-option



 



INTERNET-DRAFT                                                 N. Elkins
                                                         Inside Products
                                                             R. Hamilton
                                              Chemical Abstracts Service
                                                            M. Ackermann
Intended Status: Proposed Standard                         BCBS Michigan
Expires: October 2, 2015                                  March 31, 2015




    IPv6 Performance and Diagnostic Metrics (PDM) Destination Option
                  draft-elkins-ippm-6man-pdm-option-00

Table of Contents

   1  Background  . . . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.1 Terminology  . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.2 End User Quality of Service (QoS)  . . . . . . . . . . . . .  4
     1.3 Need for a Packet Sequence Number  . . . . . . . . . . . . .  5
     1.4 Rationale for proposed solution  . . . . . . . . . . . . . .  5
     1.5 PDM Works in Collaboration with Other Headers  . . . . . . .  6
   2 Measurement Information Derived from PDM . . . . . . . . . . . .  6
     2.1 Round-Trip Delay . . . . . . . . . . . . . . . . . . . . . .  6
     2.2 Server Delay . . . . . . . . . . . . . . . . . . . . . . . .  7
   3 Performance and Diagnostic Metrics Destination Option Layout . .  7
     3.1 Destination Options Header . . . . . . . . . . . . . . . . .  7
     3.2 Performance and Diagnostic Metrics Destination Option  . . .  7
     3.3 Header Placement . . . . . . . . . . . . . . . . . . . . . . 10
     3.4 Implementation Considerations  . . . . . . . . . . . . . . . 11
     3.5 Dynamic Configuration Options  . . . . . . . . . . . . . . . 11
     3.6 5-tuple Aging  . . . . . . . . . . . . . . . . . . . . . . . 12
   4 Considerations of Timing Representation  . . . . . . . . . . . . 12
     4.1 Encoding the Delta-Time Values . . . . . . . . . . . . . . . 12
     4.2 Timer registers are different on different hardware  . . . . 12
     4.3 Timer Units on Other Systems . . . . . . . . . . . . . . . . 13
     4.4 Time Base  . . . . . . . . . . . . . . . . . . . . . . . . . 13
     4.5 Timer-value scaling  . . . . . . . . . . . . . . . . . . . . 14
     4.6 Limitations with this encoding method  . . . . . . . . . . . 15
     4.7 Lack of precision induced by timer value truncation  . . . . 15
   5 PDM Flow - Simple Client Server  . . . . . . . . . . . . . . . . 16
     5.1 Step 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
     5.2 Step 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
     5.3 Step 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
     5.4 Step 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
     5.5 Step 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
   6 Other Flows  . . . . . . . . . . . . . . . . . . . . . . . . . . 21
     6.1 PDM Flow - One Way Traffic . . . . . . . . . . . . . . . . . 22
 


Elkins                  Expires October 2, 2015                 [Page 1]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


     6.2 PDM Flow - Multiple Send Traffic . . . . . . . . . . . . . . 23
     6.3 PDM Flow - Multiple Send with Errors . . . . . . . . . . . . 24
   7 Potential Overhead Considerations  . . . . . . . . . . . . . . . 25
   8 Security Considerations  . . . . . . . . . . . . . . . . . . . . 26
   9 IANA Considerations  . . . . . . . . . . . . . . . . . . . . . . 27
   10 References  . . . . . . . . . . . . . . . . . . . . . . . . . . 27
     10.1 Normative References  . . . . . . . . . . . . . . . . . . . 27
     10.2 Informative References  . . . . . . . . . . . . . . . . . . 27
   11 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 27
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 27


Abstract

   To assess performance problems,  measurements based on optional
   sequence numbers and timing may be embedded in each packet.  Such
   measurements may be interpreted in real-time or after the fact. An
   implementation of the existing IPv6 Destination Options extension
   header, the Performance and Diagnostic Metrics (PDM) Destination
   Options extension header as well as the field limits, calculations,
   and usage of the PDM in measurement are included in this document.

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as
   Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html






 


Elkins                  Expires October 2, 2015                 [Page 2]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


Copyright and License Notice

   Copyright (c) 2015 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document. Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License


































 


Elkins                  Expires October 2, 2015                 [Page 3]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


1  Background

   To assess performance problems,  measurements based on optional
   sequence numbers and timing may be embedded in each packet.  Such
   measurements may be interpreted in real-time or after the fact. An
   implementation of the existing IPv6 Destination Options extension
   header, the Performance and Diagnostic Metrics (PDM) Destination
   Options extension header has been proposed in a companion document.
   This document specifies the layout, field limits, calculations, and
   usage of the PDM in measurement.

   As defined in RFC2460 [RFC2460], destination options are carried by
   the IPv6 Destination Options extension header.  Destination options
   include optional information that need be examined only by the IPv6
   node given as the destination address in the IPv6 header, not by
   routers or other "middle boxes".  This document specifies a new
   destination option, the Performance and Diagnostic Metrics (PDM)
   destination option.

1.1 Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

1.2 End User Quality of Service (QoS)

   The difference between timing values in the PDM traveling along with
   the packet will be used to estimate QoS as experienced by an end user
   device.

   For many applications, the key user performance indicator is response
   time.  When the end user is an individual, he is generally
   indifferent to what is happening along the network; what he really
   cares about is how long it takes to get a response back.  But this is
   not just a matter of individuals' personal convenience.  In many
   cases, rapid response is critical to the business being conducted.

   When the end user is a device (e.g. with the Internet of Things),
   what matters is the speed with which requested data can be
   transferred -- specifically, whether the requested data can be
   transferred in time to accomplish the desired actions.  This can be
   important when the relevant external conditions are subject to rapid
   change.

   Response time and consistency are not just "nice to have".  On many
   networks, the impact can be financial hardship or endanger human
   life.  In some cities, the emergency police contact system operates
 


Elkins                  Expires October 2, 2015                 [Page 4]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   over IP, law enforcement uses TCP/IP networks, transactions on our
   stock exchanges are settled using IP networks.  The critical nature
   of such activities to our daily lives and financial well-being demand
   a simple solution to support measurements.   

1.3 Need for a Packet Sequence Number

   While performing network diagnostics of an end-to-end connection, it
   often becomes necessary to find the device along the network path
   creating problems.  Diagnostic data may be collected at multiple
   places along the path (if possible), or at the source and
   destination.  Then, in post-collection processing, the diagnostic
   data corresponding to each packet at different observation points
   must be matched for proper measurements. A sequence number in each
   packet provides sufficient basis for the matching process.  If need
   be, the timing fields may be used along with the sequence number to
   ensure uniqueness.

   This method of data collection along the path is of special use to
   determine where packet loss or packet corruption is happening.    

   The packet sequence number needs to be unique in the context of the
   session (5-tuple).  See section 2 for a definition of 5-tuple.

1.4 Rationale for proposed solution

   The current IPv6 specification does not provide timing nor a similar
   field in the IPv6 main header or in any extension header. So, we
   propose the IPv6 Performance and Diagnostic Metrics destination
   option (PDM).

   Advantages include:


   1.  Real measure of actual transactions.
   2.  Independence from transport layer protocols.
   3.  Ability to span organizational boundaries with consistent 
       instrumentation
   4.  No time synchronization needed between session partners


   The PDM provides the ability to quickly determine if the (latency)
   problem is in the network or in the server (application).  More
   intermediate measurements may be needed if the host or network
   discrimination is not sufficient.  At the client, TCP/IP stack time
   vs. applications time may still need to be broken out by client
   software.  

 


Elkins                  Expires October 2, 2015                 [Page 5]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


1.5 PDM Works in Collaboration with Other Headers

   The purpose of the PDM is not to supplant all the variables present
   in all other headers but to provide data which is not available or
   very difficult to get.   The way PDM would be used is by a technician
   (or tool) looking at a packet capture.   Within the packet capture,
   they would have available to them the layer 2 header, IP header (v6
   or v4), TCP, UCP, ICMP, SCTP or other headers.   All information
   would be looked at together to make sense of the packet flow.   The
   technician or processing tool could analyze, report or ignore the
   data from PDM, as necessary.

   For an example of how PDM can help with TCP retransmit problems,
   please look at section 8.

2 Measurement Information Derived from PDM

   Each packet contains information about the sender and receiver. In IP
   protocol,  the identifying information is called a "5-tuple".  

   The 5-tuple consists of:

      SADDR : IP address of the sender
      SPORT : Port for sender
      DADDR : IP address of the destination
      DPORT : Port for destination
      PROTC : Protocol for upper layer (ex. TCP, UDP, ICMP, etc.)

   The PDM contains the following base fields:


      PSNTP    : Packet Sequence Number This Packet
      PSNLR    : Packet Sequence Number Last Received
      DELTATLR : Delta Time Last Received
      DELTATLS : Delta Time Last Sent

   Other fields for scaling and time base are also in the PDM and will
   be described in section 3.

   This information, combined with the 5-tuple, allows the measurement
   of the following metrics:

      1.  Round-trip delay
      2.  Server delay

2.1 Round-Trip Delay

   Round-trip *Network* delay is the delay for packet transfer from a
 


Elkins                  Expires October 2, 2015                 [Page 6]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   source host to a destination host and then back to the source host.
   This measurement has been defined, and the advantages and
   disadvantages discussed in "A Round-trip Delay Metric for IPPM"
   [RFC2681].

2.2 Server Delay

   Server delay is the interval between when a packet is received by a
   device and the first corresponding packet is sent back in response. 
   This may be "Server Processing Time".  It may also be a delay caused
   by acknowledgements.  Server processing time includes the time taken
   by the combination of the stack and application to return the
   response. The stack delay may be related to network performance.   If
   this aggregate time is seen as a problem, and there is a need to make
   a clear distinction between application processing time and stack
   delay, including that caused by the network, then more client based
   measurements are needed.

3 Performance and Diagnostic Metrics Destination Option Layout

3.1 Destination Options Header

   The IPv6 Destination Options Header is used to carry optional
   information that need be examined only by a packet's destination
   node(s). The Destination Options Header is identified by a Next
   Header value of 60 in the immediately preceding header and is defined
   in RFC2460 [RFC2460].  The IPv6 Performance and Diagnostic Metrics
   Destination Option (PDM) is an implementation of the Destination
   Options Header (Next Header value = 60).  The PDM does not require
   time synchronization.

3.2 Performance and Diagnostic Metrics Destination Option

   The IPv6 Performance and Diagnostic Metrics Destination Option (PDM)
   contains the following fields:

      TIMEBASE : Base timer unit
      SCALEDTLR: Scale for Delta Time Last Received
      SCALEDTLS: Scale for Delta Time Last Sent
      PSNTP    : Packet Sequence Number This Packet
      PSNLR    : Packet Sequence Number Last Received
      DELTATLR : Delta Time Last Received
      DELTATLS : Delta Time Last Sent





 


Elkins                  Expires October 2, 2015                 [Page 7]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   The PDM destination option is encoded in type-length-value (TLV) 
   format as follows:

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  Option Type  | Option Length |TB |ScaleDTLR    |   ScaleDTLS |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |   PSN This Packet             |  PSN Last Received            |
      |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |   Delta Time Last Received    |  Delta Time Last Sent         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Option Type

   TBD = 0xXX (TBD)  [To be assigned by IANA] [RFC2780]

   Option Length

   8-bit unsigned integer. Length of the option, in octets, excluding
   the Option Type and Option Length fields. This field MUST be set to
   16.

   Time Base

   2-bit unsigned integer.  It will indicate the lowest granularity
   possible for this device.  That is, for a value of 00 in the Time
   Base field, a value of 1 in the DELTA fields indicates 1
   microsecond.

   This field is being included so that a device may choose the
   granularity which most suits its timer ticks.   That is, so that it
   does not have to do more work than needed to convert values required
   for the PDM.

   The possible values of Time Base are as follows:

           00 - milliseconds 
           01 - microseconds
           10 - nanoseconds
           11 - picoseconds

   Scale Delta Time Last Received (SCALEDTLR)

   7-bit signed integer.  This is the scaling value for the Delta Time
   Last Received (DELTATLR) field.  The possible values are from -128 to
   +127.  See Section 4 for further discussion on Timing Considerations
   and formatting of the scaling values.
 


Elkins                  Expires October 2, 2015                 [Page 8]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   Scale Delta Time Last Sent (SCALEDTLS)

   7-bit signed integer.  This is the scaling value for the Delta Time
   Last Sent (DELTATLS) field.  The possible values are from -128 to
   +127.


   Packet Sequence Number This Packet (PSNTP)

   16-bit unsigned integer.  This field will wrap. It is intended for
   human use.   That is, while to be used while analyzing packet traces.

   Initialized at a random number and monotonically incremented for each
   packet on the 5-tuple.  The 5-tuple consists of the source and
   destination IP addresses, the source and destination ports, and the
   upper layer protocol (ex. TCP, ICMP, etc).   The random number
   initialization is to make it harder to spoof and insert such packets.

   Operating systems MUST implement a separate packet sequence number
   counter per 5-tuple. Operating systems MUST NOT implement a single
   counter for all connections.


   Packet Sequence Number Last Received (PSNLR)

   16-bit unsigned integer.  This is the PSN of the packet last received
   on the 5-tuple.


   Delta Time Last Received (DELTATLR)

   A 16-bit unsigned integer field.  The value is according to the scale
   in SCALEDTLR.

   DELTATLR = Send time packet 2 - Receive time packet 1


   Delta TimeLast Sent (DELTATLS)

   A 16-bit unsigned integer field.   The value is according to the
   scale in SCALEDTLS.

   Delta Time Last Sent = Receive time packet 2 - Send time packet 1


   Option Type

   The two highest-order bits of the Option Type field are encoded to
 


Elkins                  Expires October 2, 2015                 [Page 9]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   indicate specific processing of the option; for the PDM destination
   option, these two bits MUST be set to 00. This indicates the
   following processing requirements:

   00 - skip over this option and continue processing the header.

   RFC2460 [RFC2460] defines other values for the Option Type field.
   These MUST NOT be used in the PDM.  The other values are as follows:

   01 - discard the packet.

   10 - discard the packet and, regardless of whether or not the
   packet's Destination Address was a multicast address, send an ICMP
   Parameter Problem, Code 2, message to the packet's Source Address,
   pointing to the unrecognized Option Type.

   11 - discard the packet and, only if the packet's Destination Address
   was not a multicast address, send an ICMP Parameter Problem, Code 2,
   message to the packet's Source Address, pointing to the unrecognized
   Option Type.

   In keeping with RFC2460 [RFC2460], the third-highest-order bit of the
   Option Type specifies whether or not the Option Data of that option
   can change en-route to the packet's final destination.

   In the PDM, the value of the third-highest-order bit MUST be 0.  The
   possible values are as follows:

   0 - Option Data does not change en-route

   1 - Option Data may change en-route

   The three high-order bits described above are to be treated as part
   of the Option Type, not independent of the Option Type.  That is, a
   particular option is identified by a full 8-bit Option Type, not just
   the low-order 5 bits of an Option Type.

3.3 Header Placement

   The PDM destination option MUST be placed as follows:

         - Before the upper-layer header.  That is, this is the last
   extension header.

   This follows the order defined in RFC2460 [RFC2460]

                 IPv6 header

 


Elkins                  Expires October 2, 2015                [Page 10]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


                 Hop-by-Hop Options header

                 Destination Options header

                 Routing header

                 Fragment header

                 Authentication header

                 Encapsulating Security Payload header

                 Destination Options header

                 upper-layer header

   For each IPv6 packet header, the PDM MUST NOT appear more than once.
   However, an encapsulated packet MAY contain a separate PDM associated
   with each encapsulated IPv6 header.

3.4 Implementation Considerations

   The PDM destination options extension header SHOULD be turned on by
   each stack on a host node. It MAY also be turned on only in case of
   diagnostics needed for problem resolution.

3.5 Dynamic Configuration Options

   If implemented, each operating system MUST have a default
   configuration parameter, e.g. diag_header_sys_default_value=yes/no.
   The operating system MAY also have a dynamic configuration option to
   change the configuration setting as needed.

   If the PDM destination options extension header is used, then it MAY
   be turned on for all packets flowing through the host, applied to an
   upper-layer protocol (TCP, UDP, SCTP, etc), a local port, or IP
   address only.  These are at the discretion of the implementation.

   The PDM MUST NOT be changed dynamically via packet flow as this may
   create potential security violation or DoS attack by numerous packets
   turning the header on and off.

   As with all other destination options extension headers, the PDM is
   for destination nodes only. As specified above, intermediate devices
   MUST neither set nor modify this field.



 


Elkins                  Expires October 2, 2015                [Page 11]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


3.6 5-tuple Aging

   Within the operating system, metrics must be kept on a 5-tuple basis.

   The 5-tuple is:

   SADDR : IP address of the sender SPORT : Port for sender DADDR : IP
   address of the destination DPORT : Port for destination PROTC :
   Protocol for upper layer (ex. TCP, UDP, ICMP)

   The question comes of when to stop keeping data or restarting the
   numbering for a 5-tuple.  For example, in the case of TCP, at some
   point, the connection will terminate.  Keeping data in control blocks
   forever, will have unfortunate consequences for the operating system.

   So, the recommendation is to use a known aging parameter such as Max
   Segment Lifetime (MSL) as defined in Transmission Control Protocol
   [RFC0793] to reuse or drop the control block.  The choice of aging
   parameter is left up to the implementation.

4 Considerations of Timing Representation

4.1 Encoding the Delta-Time Values

   This section makes reference to and expands on the document "Encoding
   of Time Intervals for the TCP Timestamp Option" [TRAM-TCPM].

4.2 Timer registers are different on different hardware

   One of the problems with timestamp recording is the variety of
   hardware that generates the time value to be used. Different CPUs
   track the time in registers of different sizes, and the most-
   frequently-iterated bit could be the first on the left or the first
   on the right. In order to generate some examples here it is necessary
   to indicate the type of timer register being used.

   As described in the "IBM z/Architecture Principles of Operation"
   [IBM-POPS], the Time-Of-Day clock in a zSeries CPU is a 104-bit
   register, where bit 51 is incremented approximately every
   microsecond:

                                                                     1
   0        1         2         3         4         5         6      0
   +--------+---------+---------+---------+---------+---------+--+...+
   |        |         |         |         |         |*        |      |
   +--------+---------+---------+---------+---------+---------+--+...+
   ^                                                 ^               ^
   0                                                51 = 1 usec    103
 


Elkins                  Expires October 2, 2015                [Page 12]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   To represent these values concisely a hexadecimal representation will
   be used, where each digit represents 4 binary bits. Thus:

   0000 0000 0000 0001 = 1 timer unit (2**-12 usec, or about 244 psec)
   0000 0000 0000 1000 = 1 microsecond
   0000 0000 003E 8000 = 1 millisecond
   0000 0000 F424 0000 = 1 second
   0000 0039 3870 0000 = 1 minute
   0000 0D69 3A40 0000 = 1 hour
   0001 41DD 7600 0000 = 1 day

   Note that only the first 64 bits of the register are commonly
   represented, as that represents a count of timer units on this
   hardware.  Commonly the first 52 bits are all that are displayed, as
   that represents a count of microseconds. 


4.3 Timer Units on Other Systems

   This encoding method works the same with other hardware clock
   formats. The method uses a microsecond as the basic value and allows
   for large time differentials.


4.4 Time Base

   This specification allows for the fact that different CPU TOD clocks
   use different binary points. For some clocks, a value of 1 could
   indicate 1 microsecond, whereas other clocks could use the value 1 to
   indicate 1 millisecond. In the former case, the binary digits to the
   right of that binary point measure 2**(-n) microseconds, and in the
   latter case, 2**(-n) milliseconds.

   The Time Base allows us to ensure we have a common reference, at the
   very least, common knowledge of what the binary point is for the
   transmitted values.

   We propose a base unit for the time.  This is a 2-bit integer
   indicating the lowest granularity possible for this device.  That is,
   for a value of 00 in the Time Base field, a value of 1 in the DELTA
   fields indicates 1 picosecond.

   The possible values of Time Base are as follows:

           00 - milliseconds 
           01 - microseconds
           10 - nanoseconds
           11 - picoseconds
 


Elkins                  Expires October 2, 2015                [Page 13]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   Time base is not necessarily equivalent to length of one timer tick. 
   That is, on many, if not all, systems, the timer tick value will not
   be in complete units of nanoseconds, milliseconds, etc.  For example,
   on an IBM zSeries machine, one timer tick (or clock unit) is 2 to the
   -12th microseconds.

   Therefore, some amount of conversion may be needed to approximate
   Time Base units.

4.5 Timer-value scaling

   As discussed in [TRAM-TCPM] we propose storing not an entire time-
   interval value, but just the most significant bits of that value,
   along with a scaling factor to indicate the magnitude of the time-
   interval value.  In our case, we will use the high-order 16 bits. The
   scaling value will be the number of bits in the timer register to the
   right of the 16th significant bit. That is, if the timer register
   contains this binary value:

             1110100011010100101001010001000000000000
             <-16 bits     -><-24 bits             ->

   then, the values stored would be  1110 1000 1101 0100 in binary (E8D4
   hexadecimal) for the time value and 24 for the scaling value. Note
   that the displayed value is the binary equivalent of 1 second
   expressed in picoseconds. 

   The below table represents a device which has a TimeBase of
   picosecond (or 00).  The smallest and simplest value to represent is
   1 picosecond; the time value stored is 1, and the scaling value is 0.
   Using values from the table below, we have:

                          Time value in     Encoded    Scaling   
         Delta time        picoseconds       value     decimal   
      --------------------------------------------------------
         1 picosecond               1           1         0 
         1 nanosecond             3E8         3E8         0      
         1 microsecond          F4240        F424         4
         1 millisecond       3B9ACA00        3B9A        16    
         1 second          E8D4A51000        E8D4        24    
         1 minute        3691D6AFC000        3691        32
         1 hour         cca2e51310000        CCA2        36     
         1 day        132f4579c980000        132F        44
         365 days   1b5a660ea44b80000        1B5A        52




 


Elkins                  Expires October 2, 2015                [Page 14]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   Sample binary values (high order 16 bits taken)   

   1 psec            1                                              0001
   1 nsec          3E8                                    0011 1110 1000
   1 usec        F4240                          1111 0100 0010 0100 0000
   1 msec     3B9ACA00           0011 1011 1001 1010 1100 1010 0000 0000
   1 sec    E8D4A51000 1110 1000 1101 0100 1010 0101 0001 0000 0000 0000

4.6 Limitations with this encoding method

   If we follow the specification in [TRAM-TCPM], the size of one of
   these time-interval fields is limited to this 11-bit value and five-
   bit scale, so that they fit into a 16-bit space. With that
   limitation, the maximum value that could be stored in 16 bits is:

       11-bit value   Scale
       =============  ======
       1111 1111 111  1 1111

   or an encoded value of 3FF and a scale value of 31. This value
   corresponds to any time differential between:


                 |<Count of zeroes is the Scale value>|
   11 1111 1111 1000 0000 0000 0000 0000 0000 0000 0000  (binary)
   3  F    F    8    0    0    0    0    0    0    0     (hexadecimal)

   and

   11 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111  (binary)
   3  F    F    F    F    F    F    F    F    F    F     (hexadecimal)


   This time value, 3FFFFFFFFFF, converts to 50 days, 21 hours, 40
   minutes and 46.511103 seconds. A time differential 1 microsecond
   longer won't fit into 16 bits using this encoding method.

4.7 Lack of precision induced by timer value truncation

   When the bit values following the first 11 significant bits are
   truncated, obviously loss of precision in the value. The range of
   values that will be truncated to the same encoded value is
   2**(Scale)-1 microseconds.





 


Elkins                  Expires October 2, 2015                [Page 15]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   The smallest time differential value that will be truncated is

     1000 0000 0000 = 2.048 msec

   The value

     1000 0000 0001 = 2.049 msec

   will be truncated to the same encoded value, which is 400 in hex,
   with a scale value of 1. With the scale value of 1, the value range
   is calculated as 2**1 - 1, or 1 usec, which you can see is the
   difference between these minimum and maximum values.

   With that in mind, let's look at that table of delta time values
   again, where the Precision is the range from the smallest value
   corresponding to this encoded value to the largest:


                    Time value in     Encoded         
   Delta time       microseconds       value   Scale      Precision
   1 microsecond              1           1      0        0:00.000000
   1 millisecond            38E         38E      0        0:00.000000
   1 second               F4240         7A1      9        0:00.000511
   1 minute             3938700         727     15        0:00.032767
   1 hour              D693A400         6B4     21        0:02.097151
   1 day             141DD76000         507     26        1:07.108863
   Maximum value    3FFFFFFFFFF         7FF     31       35:47.483647

   So, when measuring the delay between transmission of two packets, or
   between the reception of two packets, any delay shorter than 50 days
   21 hours and change can be stored in this encoded fashion within 16
   bits. When you encode, for example, a DTN response time delay of 50
   days, 21 hours and 40 minutes, you can be assured of accuracy within
   35 minutes.

5 PDM Flow - Simple Client Server 

   Following is a sample simple flow for the PDM with one packet sent
   from Host A and one packet received by Host B.  The PDM does not
   require time synchronization between Host A and Host B.  The
   calculations to derive meaningful metrics for network diagnostics are
   shown below each packet sent or received.






 


Elkins                  Expires October 2, 2015                [Page 16]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   Each packet, in addition to the PDM contains information on the
   sender and receiver. As discussed before, a 5-tuple consists of:

      SADDR : IP address of the sender
      SPORT : Port for sender
      DADDR : IP address of the destination
      DPORT : Port for destination
      PROTC : Protocol for upper layer (ex. TCP, UDP, ICMP)

   It should be understood that the packet identification information is
   in each packet. We will not repeat that in each of the following
   steps.

5.1 Step 1

   Packet 1 is sent from Host A to Host B.  The time for Host A is set
   initially to 10:00AM.

   The time and packet sequence number are saved by the sender
   internally.  The packet sequence number and delta times are sent in
   the packet.

   Packet 1

                       +----------+             +----------+
                       |          |             |          |
                       |   Host   | ----------> |   Host   |
                       |    A     |             |    B     |
                       |          |             |          |
                       +----------+             +----------+

   PDM Contents:

   PSNTP    : Packet Sequence Number This Packet:     25 
   PSNLR    : Packet Sequence Number Last Received:   - 
   DELTATLR : Delta Time Last Received:               - 
   SCALEDTLR: Scale of Delta Time Last Received:      0
   DELTATLS : Delta Time Last Sent:                   -
   SCALEDTLS: Scale of Delta Time Last Sent:          0 
   TIMEBASE : Granularity of Time:                   00 (Milliseconds)


   Internally, within the sender, Host A, it must keep:

   Packet Sequence Number of the last packet sent:     25
   Time the last packet was sent:                10:00:00

   Note, the initial PSNTP from Host A starts at a random number.  In
 


Elkins                  Expires October 2, 2015                [Page 17]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   this case, 25.  The time in these examples is shown in seconds for
   the sake of simplicity.

5.2 Step 2

   Packet 1 is received at Host B.  Its time is set to one hour later
   than Host A.  In this case, 11:00AM

   Internally, within the receiver, Host B, it must note:

   Packet Sequence Number of the last packet received:    25
   Time the last packet was received                 :    11:00:03

   Note, this timestamp is in Host B time.  It has nothing whatsoever to
   do with Host A time.  The Packet Sequence Number of the last packet
   received will become PSNLR which will be sent out in the packet sent
   by Host B in the next step.  The time last received will be used to
   calculate the DELTALR value to be sent out in the packet sent by Host
   B in the next step.


5.3 Step 3

   Packet 2 is sent by Host B to Host A.  Note, the initial packet
   sequence number (PSNTP) from Host B starts at a random number.  In
   this case, 12.   Before sending the packet, Host B does a calculation
   of deltas.  Since Host B knows when it is sending the packet, and it
   knows when it received the previous packet, it can do the following
   calculation:

   Sending time (packet 2) - receive time (packet 1)

   We will call the result of this calculation: Delta Time Last
   Received

   That is:

   DELTATLR = Sending time (packet 2) - receive time (packet 1)

   Note, both sending time and receive time are saved internally in Host
   B.  They do not travel in the packet. Only the Delta is in the
   packet.

   Assume that within Host B is the following:

   Packet Sequence Number of the last packet received:     25
   Time the last packet was received:                      11:00:03
   Packet Sequence Number of this packet:                  12
 


Elkins                  Expires October 2, 2015                [Page 18]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   Time this packet is being sent:                         11:00:07

   We can now calculate a delta value to be sent out in the packet. 
   DELTATLR becomes:

   4 seconds = 11:00:07 - 11:00:03

   This is the derived metric: Server Delay.   The time and scaling
   factor must be calculated.  Then, this value, along with the packet
   sequence numbers will be sent to Host A as follows:

   Packet 2

                       +----------+             +----------+
                       |          |             |          |
                       |   Host   | <---------- |   Host   |
                       |    A     |             |    B     |
                       |          |             |          |
                       +----------+             +----------+

   PDM Contents:

   PSNTP    : Packet Sequence Number This Packet:    12
   PSNLR    : Packet Sequence Number Last Received:  25
   DELTATLR : Delta Time Last Received:            3A35 (4 seconds)
   SCALEDTLR: Scale of Delta Time Last Received:     25
   DELTATLS : Delta Time Last Sent:                   -
   SCALEDTLS: Scale of Delta Time Last Sent:          0
   TIMEBASE : Granularity of Time:                   00 (Milliseconds)

   The metric left to be calculated is the Round-Trip Delay. This will
   be calculated by Host A when it receives Packet 2.

5.4 Step 4

   Packet 2 is received at Host A.  Remember, its time is set to one
   hour earlier than Host B. Internally, it must note:

   Packet Sequence Number of the last packet received:    12
   Time the last packet was received                 :    10:00:12

   Note, this timestamp is in Host A time.  It has nothing whatsoever to
   do with Host B time.

   So, now, Host A can calculate total end-to-end time. That is:

   End-to-End Time = Time Last Received - Time Last Sent

 


Elkins                  Expires October 2, 2015                [Page 19]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   For example, packet 25 was sent by Host A at 10:00:00. Packet 12 was
   received by Host A at 10:00:12 so:

   End-to-End time = 10:00:12 - 10:00:00 or 12 (Server and Network RT
   delay combined).    This time may also be called total Overall Round-
   trip time (which includes Network RTT and Host Response Time). 

   This derived metric we will call DELTATLS or Delta Time Last Sent.

   We can now also calculate round trip delay.  The formula is:

   Round trip delay = DELTATLS - DELTATLR

   Or:

   Round trip delay = 12 - 4 or 8

   Now, the only problem is that at this point all metrics are in Host A
   only and not exposed in a packet. To do that, we need a third packet.

   Note:  this simple example assumes one send and one receive.   That
   is done only for purposes of explaining the function of the PDM.  In
   cases where there are multiple packets returned, one would take the
   time in the last packet in the sequence.   The calculations of such
   timings and intelligent processing is the function of post-processing
   of the data. 

5.5 Step 5

   Packet 3 is sent from Host A to Host B.

                       +----------+             +----------+
                       |          |             |          |
                       |   Host   | ----------> |   Host   |
                       |    A     |             |    B     |
                       |          |             |          |
                       +----------+             +----------+

   PDM Contents:

   PSNTP    : Packet Sequence Number This Packet:    26 
   PSNLR    : Packet Sequence Number Last Received:  12 
   DELTATLR : Delta Time Last Received:               0 
   SCALEDTLS: Scale of Delta Time Last Received       0
   DELTATLS : Delta Time Last Sent:                105e (12 seconds)
   SCALEDTLR: Scale of Delta Time Last Received:     26
   TIMEBASE : Granularity of Time:                   00 (Milliseconds)

 


Elkins                  Expires October 2, 2015                [Page 20]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   To calculate Two-Way Delay, any packet capture device may look at
   these packets and do what is necessary.

6 Other Flows

   What we have discussed so far is a simple flow with one packet sent
   and one returned.   Let's look at how PDM may be useful in other
   types of flows.








































 


Elkins                  Expires October 2, 2015                [Page 21]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


6.1 PDM Flow - One Way Traffic

   The flow on a particular session may not be a send-receive paradigm. 
    Let us consider some other situations.   In the case of a one-way
   flow, one might see the following:


   Packet   Sender      PSN            PSN        Delta Time  Delta Time
                     This Packet    Last Recvd    Last Recvd  Last Sent
   =====================================================================
   1        Server       1              0              0            0	
   2        Server       2              0              0            5
   3        Server       3              0              0           12
   4        Server       4              0              0           20


   What does this mean and how is it useful?

   In a one-way flow, only the Delta Time Last Sent will be seen as
   used.    Recall, Delta Time Last Sent is the difference between the
   send of one packet from a device and the next.  This is a measure of
   throughput for the sender - according to the sender's point of view. 
   That is, it is a measure of how fast is the application itself (with
   stack time included) able to send packets.  

   How might this be useful?  If one is having a performance issue at
   the client and sees that packet 2, for example, is sent after 5
   microseconds from the server but takes 3 minutes to arrive at the
   destination,  then one may safely conclude that there are delays in
   the path other than at the server which may be causing the delivery
   issue of that packet.  Such delays may include the network links,
   middle-boxes, etc.

   Now, true one-way traffic is quite rare.   What people often mean by
   "one-way" traffic is an application such as FTP where a group of
   packets (for example, a TCP window size worth) is sent, then the
   sender waits for acknowledgment.  This type of flow would actually
   fall into the "multiple-send" traffic model.










 


Elkins                  Expires October 2, 2015                [Page 22]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


6.2 PDM Flow - Multiple Send Traffic

   Assume that two packets are sent for each ACK from the server.

   Packet   Sender      PSN            PSN       Delta Time  Delta Time 
                     This Packet    Last Recvd    Last Recvd  Last Sent
   =====================================================================
   1        Server       1              0              0           0	
   2        Server       2              0              0           5
   3        Client       1              2             20           0
   4        Server       3              1             10          15 

   How might this be used?

   Notice that in packet 3, the client has a value of Delta Time Last
   received of 20.   Recall that Delta Time Last Received is the Send
   time of packet 3 - receive time of packet 2.   So, what does one know
   now?   In this case, Delta Time Last Received is the processing time
   for the Client to send the next packet.    

   How to interpret this depends on what is actually being sent.  
   Remember, PDM is not being used in isolation, but to supplement the
   fields found in other headers.   Let's take some examples:

   1.  Client is sending a standalone TCP ACK.   One would find this by
   looking at the payload length in the IPv6 header and the TCP
   Acknowledgement field in the TCP header.   So, in this case, the
   client is taking 20 units to send back the ACK.   This may or may not
   be interesting.

   2.  Client is sending data with the packet.  Again, one would find
   this by looking at the payload length in the IPv6 header and the TCP
   Acknowledgement field in the TCP header.   So, in this case, the
   client is taking 20 units to send back data.   This may represent
   "User Think Time".   Again, this may or may not be interesting, in
   isolation.   But, if there is a performance problem receiving data at
   the server, then taken in conjunction with RTT or other packet timing
   information, this information may be quite interesting.

   Of course, one also needs to look at the PSN Last Received field to
   make sure of the interpretation of this data.   That is,  to make
   sure that the Delta Last Received corresponds to the packet of
   interest.

   The benefits of PDM are that we have such information available in a
   uniform manner for all applications and all protocols without
   extensive changes required to applications.

 


Elkins                  Expires October 2, 2015                [Page 23]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


6.3 PDM Flow - Multiple Send with Errors

   One might wonder if all of the functions of PDM might be better
   suited to TCP or a TCP option.   Let us take the case of how PDM may
   help in a case of TCP retransmissions in a way that TCP options or
   TCP ACK / SEQ would not.

   Assume that three packets are sent with each send from the server.

   From the server, this is what is seen.

   Pkt Sender    PSN        PSN     Delta Time Delta Time TCP     Data
               This Pkt  LastRecvd  LastRecvd  LastSent   SEQ    Bytes
   =====================================================================
   1   Server      1        0           0          0      123     100
   2   Server      2        0           0          5      223     100
   3   Server      3        0           0          5      333     100


   The client however, does not get all the packets.  From the client,
   this is what is seen for the packets sent from the server.

   Pkt Sender    PSN        PSN    Delta Time Delta Time TCP    Data
              This Pkt  LastRecvd  LastRecvd  LastSent   SEQ    Bytes
   =====================================================================
   1   Server     1         0          0          0        123   100
   2   Server     3         0          0          5        333   100

   Let's assume that the server now retransmits the packet.  
   (Obviously, a duplicate acknowledgment sequence for fast retransmit
   or a retransmit timeout would occur.  To illustrate the point, these
   packets are being left out.)

   So, then if a TCP retransmission is done, then from the client, this
   is what is seen for the packets sent from the server.

   Pkt Sender    PSN        PSN    Delta Time  Delta Time TCP    Data
              This Pkt  LastRecvd  LastRecvd   LastSent   SEQ    Bytes
   =====================================================================
   1   Server    4          0           0          30    223      100

   The server has resent the old packet 2 with TCP sequence number of
   223.   The retransmitted packet now has a PSN This Packet value of 4.
   The Delta Last Sent is 30 - the time between sending the packet with
   PSN of 3 and this current packet.

   Let's say that packet 4 STILL does not make it.  Then, after some
   amount of time (RTO) then the packet with TCP sequence number of 223
 


Elkins                  Expires October 2, 2015                [Page 24]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   is resent.

   From the client, this is what is seen for the packets sent from the
   server.

   Pkt Sender    PSN        PSN    Delta Time Delta Time TCP    Data
              This Pkt  LastRecvd  LastRecvd  LastSent   SEQ    Bytes
   =====================================================================
   1   Server    5          0          0        60       223     100

   If now, this packet makes it, one has a very good idea that packets
   exist which are being sent from the server as retransmissions and not
   making it to the client.   This is because the PSN of the resent
   packet from the server is 5 rather than 4.  If we had used TCP
   sequence number alone, we would never have seen this situation.  
   Because the TCP sequence number in all situations is 223.

   This situation would be experienced by the user of the application
   (the human being actually sitting somewhere) as a "hangs" or long
   delay between packets.  On large networks, to diagnose problems such
   as these where packets are lost somewhere on the network,  one has to
   take multiple traces to find out exactly where.

   The first thing is to start with doing a trace at the client and the
   server.  So, we can see if the server sent a particular packet and
   the client received it.  If the client did not receive it, then we
   start tracking back to trace points at the router right after the
   server and the router right before the client.  Did they get these
   packets which the server has sent?   This is a time consuming
   activity.

   With PDM,  we can speed up the diagnostic time because we may be able
   to use only the trace taken at the client to see what the server is
   sending.   

7 Potential Overhead Considerations

   Questions have been posed as to the potential overhead of PDM.  
   First, PDM is entirely optional.   That is, a site may choose to
   implement PDM or not as they wish.   If they are happy with the costs
   of PDM vs. the benefits, then the choice should be theirs.

   Below is a table outlining the potential overhead in terms of
   additional time to deliver the response to the end user for various
   assumed RTTs.



 


Elkins                  Expires October 2, 2015                [Page 25]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


   Packet 
   Bytes         RTT           Bytes      Bytes      New   Overhead
   in Packet                Per Milli     in PDM     RTT           
   =====================================================================
   1000       1000 milli         1       16       1016.000  16.000 milli
   1000        100 milli        10       16        101.600   1.600 milli
   1000         10 milli       100       16         10.160    .160 milli
   1000          1 milli      1000       16          1.016    .016 milli

   Below are some examples of actual RTTs for packets traversing large
   enterprise networks.   The first example is for packets going to
   multiple business partners.

      Packet 
      Bytes         RTT           Bytes      Bytes      New   Overhead
      in Packet                Per Milli     in PDM     RTT           
   =====================================================================
   1000       17 milli         58           16       17.360   .360 milli


   The second example is for packets at a large enterprise customer
   within a data center.  Notice that the scale is now in microseconds
   rather than milliseconds.


      Packet 
      Bytes         RTT           Bytes      Bytes      New   Overhead
      in Packet                Per Micro     in PDM     RTT           
   =====================================================================
    1000       20 micro        50         16       20.320   .320 micro



8 Security Considerations

   The PDM MUST NOT be changed dynamically via packet flow as this
   creates a possibility for potential security violations or DoS
   attacks by numerous packets turning the header on and off.

   Attackers may also send many packets from multiple ports, for example
   by doing a port scan.  This will cause the stack to create many
   control blocks.  This is the same problem as seen for SYN flood
   attacks. Similar protections should be implemented by the stack to
   preserve the integrity of memory.




 


Elkins                  Expires October 2, 2015                [Page 26]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


9 IANA Considerations

   Option Type to be assigned by IANA [RFC2780].    

10 References

10.1 Normative References

   [RFC0793]  Postel, J., "Transmission Control Protocol", STD 7, RFC
   793, September 1981.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
   Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC2460]  Deering, S. and R. Hinden, "Internet Protocol, Version 6
   (IPv6) Specification", RFC 2460, December 1998.

   [RFC2681]  Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip
   Delay Metric for IPPM", RFC 2681, September 1999.

   [RFC2780]  Bradner, S. and V. Paxson, "IANA Allocation Guidelines For
   Values In the Internet Protocol and Related Headers", BCP 37, RFC
   2780, March 2000.

10.2 Informative References

   [TRAM-TCPM] Trammel, B., "Encoding of Time Intervals for the TCP
   Timestamp Option-01", Internet Draft,  July 2013. [Work in Progress]

   [IBM-POPS] IBM Corporation, "IBM z/Architecture Principles of
   Operation", SA22-7832, 1990-2012

11 Acknowledgments

   The authors would like to thank Keven Haining, Al Morton, Brian
   Trammel, David Boyes, Bill Jouris, Richard Scheffenegger, and Rick
   Troth for their comments and assistance.

Authors' Addresses

       Nalini Elkins
       Inside Products, Inc.
       36A Upper Circle
       Carmel Valley, CA 93924
       United States
       Phone: +1 831 659 8360
       Email: nalini.elkins@insidethestack.com
       http://www.insidethestack.com
 


Elkins                  Expires October 2, 2015                [Page 27]

INTERNET DRAFT       elkins-ippm-6man-pdm-option-00       March 31, 2015


       Robert Hamilton
       Chemical Abstracts Service
       A Division of the American Chemical Society
       2540 Olentangy River Road
       Columbus, Ohio  43202
       United States
       Phone: +1 614 447 3600 x2517
       Email: rhamilton@cas.org
       http://www.cas.org

       Michael S. Ackermann
       Blue Cross Blue Shield of Michigan
       P.O. Box 2888
       Detroit, Michigan 48231
       United States
       Phone: +1 310 460 4080
       Email: mackermann@bcbsmi.com
       http://www.bcbsmi.com

































Elkins                  Expires October 2, 2015                [Page 28]