Internet DRAFT - draft-feher-benchresres

draft-feher-benchresres



Network Working Group                                  Gabor Feher, BUTE 
INTERNET-DRAFT                                     Istvan Cselenyi, TRAB 
Expiration Date: January 2001                     Krisztian Nemeth, BUTE 

                                                               July 2000 


      Benchmarking Terminology and Methodology for Routers Supporting 
                           Resource Reservation 
                     <draft-feher-benchresres-00.txt> 


1. Status of this Memo 

   This document is an Internet-Draft and is in full conformance with 
   all provisions of Section 10 of RFC2026. 

   Internet-Drafts are working documents of the Internet Engineering 
   Task Force (IETF), its areas, and its working groups. Note that other 
   groups may also distribute working documents as Internet- 
   Drafts. 

   Internet-Drafts are draft documents valid for a maximum of six months 
   and may be updated, replaced, or obsoleted by other documents at any 
   time.  It is inappropriate to use Internet-Drafts as reference 
   material or to cite them other than as "work in progress." 

   The list of current Internet-Drafts can be accessed at                 
   http://www.ietf.org/ietf/1id-abstracts.txt                             

   The list of Internet-Draft shadow directories can be accessed at       
   http://www.ietf.org/shadow.html 

   This memo provides information for the Internet community. This memo 
   does not specify an Internet standard of any kind. Distribution of 
   this memo is unlimited. 

2. Table of content 

   1. Status of this Memo.............................................1 
   2. Table of content................................................1 
   3. Abstract........................................................2 
   4. Introduction....................................................2 
   5. Existing definitions............................................3 
   6. Definitions of Terms............................................3 
      6.1 Resource Reservation Protocol...............................3 
         6.1.1 Resource Reservation Session...........................4 
         6.1.2 Multicast Resource Reservation Session.................4 
         6.1.3 Reservation Capable Router.............................4 
         6.1.4 Signaling End-point....................................5 
         6.1.5 Reservation Initiator..................................5 
         6.1.6 Signaling Path.........................................6 
         6.1.7 Signaling Message Sequence.............................6 
         6.1.8 Multicast aware Resource Reservation Protocol..........7 
         6.1.9 Scalability Limit......................................7 


Feher, Cselenyi, Nemeth   Expires January 2001                  [Page 1]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

      6.2 Traffic Types...............................................8 
         6.2.1 Premium Traffic........................................8 
         6.2.2 Best-Effort Traffic....................................8 
      6.3 Router Load Types...........................................8 
         6.3.1 Signaling Load.........................................8 
         6.3.2 Signaling Burst........................................9 
         6.3.3 Reservation Load.......................................9 
      6.4 Performance Metrics........................................10 
         6.4.1 Signaling Message Handling Time.......................10 
         6.4.2 Premium Traffic Extra Delay...........................11 
         6.4.3 Best-effort Traffic Extra Delay.......................12 
         6.4.4 Signaling Message Loss................................12 
   7. Methodology....................................................13 
      7.1 Evaluating the Results.....................................13 
      7.2 Test Set up................................................13 
         7.2.1 One Tester............................................13 
         7.2.2 Two Testers...........................................14 
         7.2.3 Test Set up for Unicast Resource Reservation..........14 
         7.2.4 Test Set up for Multicast Resource Reservation........14 
         7.2.5 Signaling Message Verification........................15 
      7.3 Scalability Tests..........................................15 
         7.3.1 Maximum Signaling Message Burst Size -- Unicast Case..16 
         7.3.2 Maximum Signaling Message Burst Size -- Multicast Case16 
         7.3.3 Maximum Signaling Intensity -- Unicast Case...........17 
         7.3.4 Maximum Signaling Intensity -- Multicast Case.........18 
         7.3.5 Maximum Number of Reservation Sessions -- Unicast Case18 
         7.3.6 Maximum Number of Reservation Sessions -- Multicast Case
         ............................................................19 
      7.4 Benchmarking Tests.........................................19 
         7.4.1 Signaling Message Handling Time -- Unicast Case.......20 
         7.4.2 Signaling Message Handling Time -- Multicast Case.....22 
         7.4.3 Premium Traffic Extra Delay -- Unicast Case...........22 
         7.4.4 Premium Traffic Extra Delay -- Multicast Case.........23 
         7.4.5 Best-effort Traffic Extra Delay.......................24 
   8. Acknowledgement................................................24 
   9. References.....................................................24 
   10. Authors' Addresses:...........................................25 

3. Abstract 

   The purpose of this document is to define terminology and methodology 
   specific to the performance benchmarking the resource reservation 
   signaling capability of IP routers.  

   In addition to defining the performance metrics and the tests, this 
   document also describes specific formats for reporting the results of 
   the tests. 

4. Introduction 

   The IntServ over DiffServ framework [3] outlines a heterogeneous 
   Quality of Service (QoS) architecture for multi domain Internet 
   services. Signaling based resource reservation (e.g. by RSVP [5]) is 

 
Feher, Cselenyi, Nemeth   Expires January 2001                  [Page 2]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

   an integral part of that model. While this way scalability 
   constraints are released for most of the core routers, the 
   performance of border routers that handle signaling is still crucial. 
   Therefore network operators, who are planning to deploy this model, 
   shall scrutinize the scalability limitations in reservation capable 
   routers and the impact of signaling on the routers’ forwarding 
   performance. 

   An objective way for quantifying the scalability constraints of QoS 
   signaling is to perform measurements on routers that are capable of 
   resource reservation. This document defines a specific set of tests 
   that vendors or network operators can use to measure and report the 
   signaling performance characteristics of router devices that support 
   resource reservation protocols. The results of these tests will 
   provide comparable data for different products supporting the 
   decision process of purchase. Moreover, these measurements provide 
   input characteristics for the dimensioning of a network in which 
   resources are provisioned dynamically by signaling. Finally, these 
   test are applicable for characterizing the impact of control plane 
   signaling on the forwarding performance of routers. 

   The first part of the document, "Terminology", defines the terms that 
   are used later on in the document. The second part is the 
   "Methodology" that defines measurement methods to find the 
   scalability limits and characterize the signaling performance of the 
   tested router. 

   This benchmarking terminology and methodology is based on the 
   knowledge gained by examination of four very different resource 
   reservation protocols: RSVP [5], Boomerang [6], YESSIR [7] and ST2+ 
   [8]. Nevertheless, this document aspires to compose terms and methods 
   that are valid in general and not restricted to these four protocols. 

5. Existing definitions 

   RFC 1242 [1] "Benchmarking Terminology for Network Interconnect 
   Devices" and RFC 2285 [2] "Benchmarking Terminology for LAN Switching 
   Devices" contains discussions of a number of terms relevant to the 
   benchmarking of signaling performance of reservation capable routers 
   and should be consulted before attempting to make use of this 
   document. 

   For the sake of clarity and continuity this document adopts the 
   template for definitions set out in Section 2 of RFC 1242 [1]. 
   Definitions are indexed and grouped together in sections for ease of 
   reference. 

6. Definitions of Terms 

6.1 Resource Reservation Protocol  

   This group of definitions applies to various resource reservation 
   protocols implemented in router devices. 

 
Feher, Cselenyi, Nemeth   Expires January 2001                  [Page 3]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 


6.1.1 Resource Reservation Session 

   Definition: 
      A resource reservation session (or shortly reservation) expresses 
      that certain QoS treatment is assigned to the packets of a traffic 
      flow along the path between two hosts.  

   Discussion: 
      The QoS treatment can be specified explicitly by giving the amount 
      of networking resources (e.g. forwarding capacity or buffer space) 
      or it can be specified implicitly by referring to a forwarding 
      behavior (e.g. EF PHB in case of Differentiated Services [9]). The 
      flow can be specified either by the five-tuple (i.e. as a micro-
      flow in [9]) or as traffic aggregate. 

   See Also: 
      Signaling Path 

6.1.2 Multicast Resource Reservation Session 

   Definition: 
      Multicast resource reservation session (or shortly multicast 
      reservation) denotes that certain QoS treatment is assigned to the 
      packets of a traffic flow related to a multicast address. 

   Discussion: 
      There are resource reservation protocols that allow different 
      amount of resource dedication in reservation capable routers for 
      one multicast reservation session. These protocols distinguish 
      different traffic sources and different traffic destinations and 
      can have more than one reservation models to express the resource 
      needs within the reservation. (e.g. RSVP SE/WF/FF [5]) 

   Issues: 
      - Reservation aggregation 

   See Also: 
      Signaling Path 

6.1.3 Reservation Capable Router 

   Definition: 
      A router is reservation capable if it understands a resource 
      reservation protocol that signals the set up, tear down or 
      modification of a reservation in the router. 

   Discussion: 
      Resource reservation protocols can be divided into two groups, the 
      soft state and the hard state protocols. Both kinds of protocol 
      tear a resource reservation entry upon request but in case of soft 
      state protocols the router also tears an entry if a refresh 
      message does not renew the resource reservation entry within a 

 
Feher, Cselenyi, Nemeth   Expires January 2001                  [Page 4]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

      refresh period. Thus in case of soft state protocols the number of 
      refresh messages may cause considerable signaling load, while on 
      the other hand it is easier to build a robust protocol that is 
      soft state. 

6.1.4 Signaling End-point 

   Definition: 
      A Signaling End-point can be a host or a network node, which is 
      capable to initiate and/or terminate or proxy signaling sessions. 

   Discussion: 
      Usually, signaling end-points have a protocol stack that is 
      capable to generate and understand signaling messages. However in 
      some cases this is not necessary. When the messages of the 
      resource reservation protocol are payloads of other protocols that 
      the host understands and the resource reservation protocol does 
      not require any special action from signaling end-points then the 
      host becomes a signaling end-point without knowing the resource 
      reservation protocol (e.g. Boomerang [6] encapsulates the 
      reservation requests to an ICMP Echo message). 

      Reservation proxies are protocol translators that translate the 
      signaling messages of one resource reservation protocol into 
      messages of other resource reservation protocols bridging 
      different networks with different resource reservation protocols. 
      Thus the reservation proxy is two signaling end-points in one, as 
      it is both a signaling terminator and a signaling initiator. 

6.1.5 Reservation Initiator 
    
   Definition: 
      The reservation initiator is the signaling end-point, which 
      initiates the reservation set up. 
       
   Discussions: 
      Resource reservation protocols can be divided into different 
      groups according to the initiator of the reservation request, what 
      can be the traffic source(s), the traffic destination(s), or both 
      of them. Typical example for the later is RSVP, where the data 
      traffic destination(s) requests the resource reservation. This 
      scheme is called receiver-oriented where "receiver" refers to the 
      data traffic receiver. 
       
      A receiver-oriented protocol must assure that the data flow(s) 
      coming from the traffic source(s) to the traffic destination(s) 
      takes the same path as the reservation request, which travels in 
      the opposite direction. IP routing does not guarantee the same 
      path for the different directions between end-points, so the 
      resource reservation protocol has to take care of it. Therefore 
      RSVP has a signaling message primitive, called the PATH message, 
      which establishes the signaling path(s) among traffic sources and 
      traffic destinations before the reservation can happen and 

 
Feher, Cselenyi, Nemeth   Expires January 2001                  [Page 5]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

      afterwards the signaling messages are forwarded on this path 
      backward. 
       
   See Also: 
      Signaling end-point 
      Signaling path 
    
6.1.6 Signaling Path 
    
   Definition: 
      A signaling path is a sequence of network nodes and links on which 
      signaling messages travel from one signaling end-point to the 
      other. 
       
   Discussion: 
      In case of reservation session where the data traffic is unicast 
      and goes point-to-point, the signaling path is mostly the same as 
      the data path. However in case of multicast reservations there are 
      more than one signaling path belonging to one reservation session 
      according to the definition. There is one signaling path between 
      each traffic source and traffic destination. 
       
      Both in the unicast and multicast case, it might happen that the 
      path of the reservation is not the same as the path of the data 
      stream, as recent reservation capable routers do not take effort 
      to guarantee it. Still, the current practice is that the signaling 
      messages and the data packets are addressed with the same IP 
      address, trusting that in this case they are forwarded on the same 
      path. 
       
   Issues: 
      - Multicast signaling path graph 
      - Route change 
       
    
6.1.7 Signaling Message Sequence 
    
   Definition: 
      Signaling message sequence is defined as a series of signaling 
      messages, which are related to the same reservation session and 
      whose order follows the specification of the reservation protocol. 
       
   Discussion: 
      A typical signaling message sequence is a reservation set up and 
      reservation tear down pair, where the first signaling message sets 
      up a reservation in the router and the second one tears that down. 
       
      Signaling message sequences are used in measurements where the 
      changes in the states of the router affect the measurement. Using 
      signaling message sequences the router states are restored after 
      the last message in the sequence. (e.g.: RSVP's PATH and PATHTEAR 
      message pair) 
       

 
Feher, Cselenyi, Nemeth   Expires January 2001                  [Page 6]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

6.1.8 Multicast aware Resource Reservation Protocol 
    
   Definition: 
      A resource reservation protocol is multicast aware when it 
      supports resource reservation for multicast sessions. 
    
   Discussion: 
      There are two types of multicast support, one is the many-to-many 
      multicast and the other is the one-to-many multicast. The former 
      allows reservations for many traffic sources in the multicast 
      group, while the latter allows for only one traffic source in the 
      group. When there are more than one data traffic sources in a 
      multicast reservation session then the protocol might aggregate 
      the reservation towards the data traffic destinations. 
       
      Certainly a multicast aware protocol is more complex than a non-
      multicast aware. Currently perhaps the most well-known IP resource 
      reservation protocol, RSVP [5], does support many-to-many 
      multicast, while another protocol, Boomerang [6], supports one-to-
      many multicast.  
    
   Issues: 
      - Multicast traffic 
      - IP multicast 
    
6.1.9 Scalability Limit 
    
   Definition: 
      Scalability limit is the border between the steady and the 
      overloaded state of the Device Under Test (DUT). 
       
   Discussion: 
      All existing routers have finite memory buffer range and finite 
      processing power (CPU). In the steady state of the router the 
      memory buffers are not fully utilized and the processing power is 
      enough to cope with the different tasks of the router. However 
      when the load rises in the router there is a certain point where 
      no more memory space is available or the processing power is not 
      enough to satisfy the needs of all the tasks and thus the router 
      becomes overloaded. The critical load condition, when the router 
      is still in the steady state but the smallest amount of load 
      increase would drive it to the overloaded state is the scalability 
      limit of the router. 
       
      Usually the overloaded state of the router can be recognized by 
      some kinds of data loss. A resource reservation capable router may 
      drop signaling messages, data packets or reservation entries, as 
      it does not have enough capacity to process or maintain them. 
        
      Smart task scheduling is able to overcome on this for short 
      periods when the load is above the scalability limit, thus the 
      router remains in the steady state. Therefore the scalability 
      limit should be determined under a long lasting constant load. 

 
Feher, Cselenyi, Nemeth   Expires January 2001                  [Page 7]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

       
   Issues: 
      - Scalability tests 
       
6.2 Traffic Types 
    
   This group of definitions defines traffic types forwarded by resource 
   reservation capable routers. 
    
6.2.1 Premium Traffic 
    
   Definition: 
      Premium traffic is a traffic type that the router distinguishes 
      from the best-effort traffic and forwards its packets according to 
      a QoS agreement. 
       
   Discussion: 
      Each premium traffic flow has a resource reservation entry in the 
      router set up by signaling messages. The router might define more 
      than one premium traffic type (e.g. delay sensitive traffic, loss 
      sensitive traffic) and different premium traffic types have 
      different QoS treatments. 
       
6.2.2 Best-Effort Traffic 
    
   Definition: 
      Best-effort traffic is the traffic that has no reservation entry 
      in the router. 
       
   Discussion: 
      Traffic flows that do not require QoS guaranties along their path 
      are considered as best-effort traffic. Best effort means that the 
      router takes its best effort to forward every data packets, but it 
      cannot give any guarantee. This type of traffic is the standard in 
      today’s Internet. 
       
6.3 Router Load Types 
    
   This group of definitions defines different load components, which 
   are used for testing the resource reservation performance of the 
   reservation capable router.  
    
6.3.1 Signaling Load 
    
   Definition: 
      The signaling load is characterized by the signaling intensity, 
      which expresses, how many signaling messages arrive to the 
      reservation capable router within a time unit. 
       
   Discussion: 
      As the processing of signaling messages consumes router power, the 
      signaling intensity is correlated with the load. This load is 
      called signaling load because it is generated by signaling. The 

 
Feher, Cselenyi, Nemeth   Expires January 2001                  [Page 8]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

      higher the signaling intensity is, the more signaling messages hit 
      the router within a time unit. Therefore the processing capacity 
      spent on signaling handling increases, resulting higher load in 
      the router. 
       
      Most of the resource reservation protocols have several protocol 
      primitives realized by different signaling message types. Each of 
      these message types requires different processing capacity from 
      the router. 
       
   Issues: 
      - Different signaling message types load the router differently 
      - Message generation burstiness 
    
   Measurement unit: 
      Number of signaling messages per second [1/s] 
    
   See Also: 
      Signaling burst 
    
6.3.2 Signaling Burst 
    
   Definition: 
      The signaling burst denotes a certain number of signaling 
      messages, which arrive to the input port(s) of the router 
      continuously causing persistent load for the signaling message 
      handler. The single parameter of the signaling burst is the number 
      of signaling messages in the burst. 
       
   Discussion: 
      Back-to-back signaling messages on one port of the router form a 
      typical signaling burst. But other cases can be imagined also 
      where signaling messages arrive on different ports simultaneously 
      or with an overlap in time (i.e. when the tail of one signaling 
      message is behind the head of another one arriving on another 
      port). 
       
      Certainly, as in case of signaling load, the signaling message 
      type is an issue here, too. 
       
   Issues: 
      - Different types of signaling messages 
    
   Measurement unit: 
      The signaling burst size is the number of messages arrived during 
      the burst 
    
   See Also: 
      Signaling intensity 
       
6.3.3 Reservation Load 
    
   Definition: 

 
Feher, Cselenyi, Nemeth   Expires January 2001                  [Page 9]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

      The reservation load is characterized by the number of resource 
      reservation sessions set up in the router. 
       
   Discussion: 
      In case of micro-flow classification the packet classifier shall 
      distinguish the flows, which have reservations in the router from 
      the others, which do not. Therefore the more reservation session 
      is set up in the router the more complex the traffic 
      classification is. This is one factor that contributes to the 
      reservation load. Moreover, most signaling based resource 
      reservation protocols require that the routers maintain some kind 
      of state for each flow they keep reservation for. The growth of 
      this state space is a scalability issue as on one hand it requires 
      memory capacity and on the other hand searching through a larger 
      state space may increase the signaling handling time.  
       
      Note that some protocols (e.g. RSVP, Boomerang) allow making 
      reservation for the aggregation of several micro-flows. In this 
      case the number of reserved sessions can be smaller than the 
      number of micro-flows. 
       
      In case of soft state protocols the refresh messages required to 
      maintain the reservations cause a considerable signaling load as 
      well beside the reservation load. Although, some soft state 
      protocols might be capable to aggregate refresh messages that 
      decreases significantly the signaling load. 
    
   Issues: 
      - Micro-flow aggregation 
      - Refresh message aggregation 
    
   Measurement unit: 
      Number of reservation sessions in the router 
    
6.4 Performance Metrics 
    
   This group of definitions is the collection of the measurable effects 
   of the impact of the resource reservation on a router device. 
    
6.4.1 Signaling Message Handling Time 
    
   Definition: 
      The signaling message handling time (or shortly signal handling 
      time) is the time that a signaling message spends inside the 
      router before it is forwarded to the next hop. 
    
   Discussion: 
      Usually signaling messages are issued by a signaling end-point and 
      forwarded via the signaling path by the routers. However, beside 
      the message forwarding the router interprets the content of the 
      messages and acts according to it. Thus the message handling time 
      is longer than forwarding time of data packets of the same size. 
      Moreover, there are signaling message primitives that are altered 

 
Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 10]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

      during the processing and there might be messages also that are 
      drained in the router or generated by the router. Thus, the signal 
      message handling time is the time difference of a signaling 
      message entering time and the leaving time of the corresponding 
      processed signaling message. When a message is drained by the 
      router then the signal handling time is immeasurable therefore it 
      is not defined. 
       
      In case of signaling messages that carry instructions for 
      multicast flows, the router might issue multiple signaling 
      messages after processing. In this case, by definition, the signal 
      handling time is the time interval elapsed between the arrival of 
      the incoming signaling message to the router and the departure of 
      the last, related signaling message. 
       
      Signal handling time is an important measure as it directly 
      affects the setup time of a session. On the other hand, it is also 
      an indication of the signal processing capacity of the measured 
      router as it is correlated to the number of signaling messages 
      that can be processed within a time unit. 
       
      This metric is also dependent on the type of the signaling 
      message. For example, it takes generally smaller time to tear down 
      a session within a router than to set it up. 
    
   Issues: 
      - Refresh message aggregation 
      - The type of the signaling message 
       
   Measurement unit: 
      seconds 
       
6.4.2 Premium Traffic Extra Delay 
    
   Definition: 
      Premium traffic extra delay is the delay that a data packet of a 
      reserved flow suffers because of the resource reservation protocol 
      running on the router. 
    
   Discussion: 
      This metric shows the effect of the classification, policing and 
      shaping along with the cross-effect of the control plane on the 
      data plane.  
       
      Classification [9] is the mechanism for filtering out premium 
      traffic. All data packet belonging to premium traffic must be 
      classified in order to find the right resource bounds for the flow 
      where the packet belongs. In systems using only one processor for 
      both the processing of signaling messages and for data forwarding, 
      this cross influence is expected to be important. 
       
   Issues: 
      - Smart classification, policing, shaping routines 

 
Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 11]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

      - Core and border routers 
    
   Measurement unit: 
      seconds 
    
   See Also: 
      Best-effort traffic extra delay 
    
6.4.3 Best-effort Traffic Extra Delay 
    
   Definition: 
      Best-effort traffic extra delay is the delay that a non-
      prioritized data packet transfer because the resource reservation 
      protocol is running on the router. 
    
   Discussion: 
      It is obvious that classification or policing algorithms do not 
      address the best-effort traffic. However the cross-effect between 
      the control and data plane influences the traffic and adds an 
      extra delay to the forward procedure of each packet. 
    
   Measurement unit: 
      seconds 
    
   See Also: 
      Premium traffic extra delay 
    
6.4.4 Signaling Message Loss 
    
   Definition: 
      Signaling message loss is the ratio of the signaling messages that 
      has actually left the router and signaling messages that are 
      expected to leave the router as a response to the corresponding 
      signaling message entering to the router. 
    
   Discussion: 
      As in case of signaling message handling time there is a problem 
      here as well with the signaling messages that drained or generated 
      inside the router. Those signaling messages are immeasurable and 
      therefore the definition does not apply to them. Signaling loss 
      considers signaling messages only that leave the router as a 
      consequence of processing an entering signaling message. Note, 
      that signaling messages in a multicast reservation session might 
      trigger several signaling messages. 
        
      Losing a signaling message is almost always an indication of 
      saturation of the router. This measure is therefore suitable for 
      sounding out the scalability limits of the router.  
       
   Issues: 
      - Multicast signaling 
      - Refresh message aggregation 
       

 
Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 12]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

   Measurement unit: 
      percentage value, but in many cases the existence of signaling 
      loss is enough 
       
7. Methodology 
    
7.1 Evaluating the Results 
    
   RFC2544 [4] describes considerations regarding the implementation and 
   evaluation of tests, which are certainly valid for this test suite 
   too. Namely, we shall emphasize that we intended to create a system 
   from commercially available measurement instruments and devices for 
   the sake of easy implementation of the described tests. Simple test 
   scripts and test software for Linux are available from the Boomerang 
   homepage [10]. 
    
   Secondly, care should be taken for selecting the proper tests for a 
   specific router, since not all of the tests apply to all types of 
   Devices Under Test (DUTs).  
    
   Finally, the selection of the relevant measurement data and their 
   evaluation requires experience and it must be done with an 
   understanding of generally accepted testing practices regarding 
   repeatability, variance and statistical significance of small numbers 
   of trials. 
    
7.2 Test Set up 
    
7.2.1 One Tester 
    
   The ideal way to perform the measurements is to connect a tester 
   device to both the incoming and outgoing network interfaces of the 
   DUT. The tester sends signaling messages and data traffic on its 
   outgoing port to one of the incoming ports of the reservation capable 
   router device, while the outgoing network port of the router, where 
   the processed signaling messages and the forwarded packets appear, is 
   connected to an incoming port of the tester device. Thus the tester 
   device is capable to measure the signaling message handling time, the 
   various traffic forwarding times and the signaling loss. This 
   scenario can be seen in Figure 1 [4]. In this case the tester is both 
   the signaling and data traffic source and the destination device in 
   one. 
    
    
                               +------------+ 
                               |            | 
                  +------------|  tester    |<-------------+ 
                  |            |            |              | 
                  |            +------------+              | 
                  |                                        | 
                  |            +------------+              | 
                  |            |            |              | 
                  +----------->|    DUT     |--------------+ 

 
Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 13]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

                               |            | 
                               +------------+ 
                                 Figure 1 
    
7.2.2 Two Testers 
    
   The measurements can be performed with two tester devices as well, 
   separating the sender and receiver functionality into two pieces of 
   equipment. In this case the sender test device is the driver of the 
   input network interface of the DUT and the second one, the receiver 
   test device, is connected to the output network interface of the DUT 
   and collects the messages and traffic packets leaving from the DUT. 
   This scenario can be seen in Figure 2. Using this test scenario the 
   synchronization of the clocks in the tester devices must be assured 
   for certain tests. Nevertheless, the scalability tests do not depend 
   on time synchronization. 
    
            +--------+         +------------+          +----------+ 
            |        |         |            |          |          | 
            | sender |-------->|    DUT     |--------->| receiver | 
            |        |         |            |          |          | 
            +--------+         +------------+          +----------+ 
                                 Figure 2 
    
   The main benefit of using the single tester device scenario described 
   above is that the time measurement happens within a single device and 
   does not require any time synchronization. Using one tester device is 
   recommended during all of the tests. The description of the 
   benchmarking methodology uses the expressions of sender and receiver 
   devices but they do not have to be two physically separate 
   appliances. 
    
   When the DUT runs a multicast aware resource reservation protocol 
   then the test must be performed in the multicast resource reservation 
   test scenario as well as in the standard unicast scenario. 
    
7.2.3 Test Set up for Unicast Resource Reservation 
    
   The test set up in the unicast test scenario is the same as described 
   generally in the test set up section. During the test the tester 
   device must use unicast addresses for the data traffic and must send 
   resource reservation requests for host-to-host (i.e. unicast) 
   reservation. 
    
7.2.4 Test Set up for Multicast Resource Reservation 
    
   The test set up in the multicast test scenario is similar to the one 
   described in the beginning of this chapter (see sections 7.2.1 and 
   7.2.2), but all the outgoing ports (where the router emits signaling 
   messages or data packets as a response to one signaling message or 
   data packet on the incoming port) must be connected to the tester 
   device or tester devices. Furthermore the data traffic from the 


 
Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 14]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

   tester must be sent to a multicast address and the tester device must 
   ask resource reservation requests for multiple hosts. 
    
   There are resource reservation protocols, like RSVP, which has more 
   than one reservation scheme for multicast flows. In this case all 
   reservation schemes must be tested. 
    
   There are many of ways to possibly define a multicast reservation 
   session. The possibly large number of end-points and the distribution 
   of traffic sources and traffic destinations within the group results 
   in a very large number of combinations. The benchmarking procedure 
   does not require measuring all possible multicast sessions, just one 
   scenario is required, while others are still recommended. The 
   proposed multicast scenario consists of a multicast group with 4 end-
   points where there are one traffic originator and three traffic 
   drainers. The traffic originator should be placed to the sender side 
   of the DUT, while the drainers should be placed to the receiver side. 
    
   The benchmarking reports carried on multicast scenarios always have 
   to include the full description of the scenario itself. 
    
7.2.5 Signaling Message Verification 
    
   However the conformance testing of the resource reservation is beyond 
   the scope of this document, defective signaling message processing 
   can be expected in an overloaded router. Thus, during the tests, when 
   signaling messages are processed in the DUT, the receiver device must 
   verify the messages whether they fully conform to the message format 
   of the protocol specification and they are the expected signaling 
   messages at the given situation. When any of the messages do not 
   conform to the protocol specification the report must indicate 
   implementation errors. However, further analysis of the conformance 
   of a malfunctioning protocol implementation is beyond the scope of 
   this test suit that targets performance analysis. 
    
   Verifying data traffic packets are not required, since the signaling 
   performance benchmarking of reservation capable routers should not 
   deal with data traffic. For this purpose there are other benchmarking 
   methodologies that verify data traffic during the measurements, like 
   the one described in RFC 2544 [4]. 
    
7.3 Scalability Tests 
    
   Unicast scalability tests are defined to explore the scalability 
   limits of a reservation capable router when it deals with unicast 
   resource reservation requests. According to our definition, a router 
   reaches its scalability limit, when it steps out of the steady state 
   operation and steps into a phase where it cannot process all of the 
   signaling messages and begins to drop some of them. 
    
   Multicast scalability tests are applied for multicast-aware resource 
   reservation protocols only. They are similar to unicast scalability 
   tests, but all the resource reservations should ask resources for 

 
Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 15]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

   multicast reservation sessions and the multicast test scenario must 
   be used for the measurements. 
    
7.3.1 Maximum Signaling Message Burst Size -- Unicast Case 
    
   Objective: 
   To determine the maximum number of the signaling messages in a burst 
   that the router is still able to handle without signaling loss. 
    
   Procedure: 
   1. Select one signaling message or a signaling message sequence from 
   the specification of the resource reservation protocol. 
    
   2. A number of signaling messages or signaling message sequences must 
   be sent back-to-back in order to form a burst. All the signaling 
   messages in the burst must be valid messages and the sender must 
   check if the router is able to process them without any error. The 
   signaling messages must be addressed in a way that they go through 
   the DUT and the receiver should be able to catch them all. 
    
   3. Send the burst towards the DUT and count the signaling messages 
   received by the receiver.  
    
   4. When the number of sent messages equals to the number of received 
   messages, the number of messages in a burst must be increased and the 
   test is to be repeated. When the receiver receives less signaling 
   messages than the sender has sent, the DUT is over its scalability 
   limit. The scalability limit for the maximum signaling message burst 
   size is found when the sender is able to send a signaling burst 
   without signaling loss but one more signaling message in the burst 
   would result signaling loss. 
    
   During the test the DUT must not receive other data packets than the 
   signaling messages that the sender sends. The test should be repeated 
   at least 30 times for each signaling message or signaling message 
   sequence. The DUT should be reset to its initial state before each 
   step of the test. 
    
   Reporting format: 
   The report should indicate the type of the signaling message or 
   signaling message sequence and the median of the measured maximum 
   signaling burst size. 
    
   Note: 
   There are resource reservation protocols, like RSVP, where certain 
   kind of signaling messages (e.g. RESV) are valid only when some other 
   signaling messages (e.g. PATH) have already established corresponding 
   states in the router. In this situation the sender must ensure the 
   establishment of such states in the router before the test begins. 
    
7.3.2 Maximum Signaling Message Burst Size -- Multicast Case 
    
   Objective: 

 
Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 16]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

   To determine the maximum number of the signaling messages in a burst 
   that the router is still able to handle without signaling loss. 
    
   Procedure: 
   The procedure is the same as it is defined for the unicast case, 
   except all the signaling messages should refer to multicast sessions. 
   The test stops when there is a signaling loss on any of the outgoing 
   interfaces of the DUT. 
    
   Reporting format: 
   The reporting format is the same as in the unicast case. 
    
7.3.3 Maximum Signaling Intensity -- Unicast Case 
    
   Objective: 
   To determine the maximum number of signaling messages that can be 
   sent to the router within a time unit. 
    
   Procedure: 
   1. Select one signaling message or a signaling message sequence from 
   the specification of resource reservation protocol. 
    
   2. Construct a flow of valid signaling messages or signaling message 
   sequences where the individual signaling messages are equally spaced 
   and there are a specified number of messages in one second according 
   to the signaling intensity. The signaling messages must be addressed 
   in a way that they must go through the DUT and the receiver should be 
   able to catch them all. 
    
   3. Send the flow towards the DUT for at least one minute. Count the 
   signaling messages received by the receiver. 
    
   4. When the number of sent messages equals to the number of received 
   messages then the signaling intensity of the flow must be increased 
   and the test must be repeated. When the receiver receives less 
   signaling messages than the sender has sent, the DUT is over its 
   scalability limit. The scalability limit for the maximum signaling 
   intensity is found, when the signaling flow arrives to the receiver 
   without any signaling loss but the DUT would lose signaling messages, 
   if the signaling intensity would be larger. 
    
   During the test the DUT must not forward other data packets than the 
   signaling messages that the sender sends. The test should be repeated 
   at least 30 times for each signaling message or signaling message 
   sequence. The DUT should be reset to its initial state before each 
   step of the test. 
    
   Reporting format: 
   The report should indicate the type of signaling message or signaling 
   message sequence and the median of the measured maximum signaling 
   intensity. 
    
   Note: 

 
Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 17]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

   There are resource reservation protocols, like RSVP, where certain 
   kind of signaling messages are valid only when some other signaling 
   messages have already established corresponding states in the router. 
   In this situation the sender must ensure the establishment of such 
   states in the router before the test begins. 
    
7.3.4 Maximum Signaling Intensity -- Multicast Case 
    
   Objective: 
   To determine the maximum number of signaling messages that can be 
   sent to the router within a time unit. 
    
   Procedure: 
   The procedure is the same as it is defined for the unicast case, 
   except that each signaling messages should refer to multicast 
   sessions. The test stops when there is a signaling loss on any of the 
   outgoing interfaces of the DUT. 
    
   Reporting format: 
   The reporting format is the same as in the unicast case. 
    
7.3.5 Maximum Number of Reservation Sessions -- Unicast Case 
    
   Objective: 
   To determine the maximum number of reservation sessions that can 
   exist simultaneously in a reservation capable router. 
    
   Procedure: 
   1. Set up a reservation session in the reservation capable router by 
   sending the appropriate signaling message sequence over the DUT. 
    
   2. After a successful reservation setup, wait for a specified amount 
   of time (T) while still maintaining the established reservations. 
    
   Care should be taken of the maintenance of established reservation 
   sessions. If the DUT uses soft state resource reservation protocol, 
   then the waiting time, T must be at least as long as the protocol 
   specification requires for reservation time out. This waiting is 
   necessary to assure that DUT is able to refresh the reservations. In 
   case of hard state resource reservation protocols it is not necessary 
   to wait thus T can be zero. . 
    
   3. Repeat the previous steps until the router is not able to set up 
   any new reservation or it is not able to maintain the existing 
   reservations. That is, if you have set up N reservations and the DUT 
   starts dropping reservations during the waiting time or after it, 
   then the maximum number of reserved sessions is passed over and the 
   actual number of maximum reservation sessions is N-1.  
 
   The test should be repeated at least 5 times and the DUT should be 
   reset between the tests cycles. 
    
   Reporting format: 

 
Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 18]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

   The report must indicate the median of the measured maximum number of 
   reservation sessions. When the number of reserved sessions grow over 
   a number that is surely enough in the given technology conditions, 
   then the test can be canceled and the report can state that the 
   resource reservation protocol implementation performs the maximum 
   number of reservation sessions over that limit. (e.g. "Over 10.000 
   sessions") 
    
   Checking the reservation sessions in a reservation capable router 
   might be difficult if the router does not support any interface to 
   monitor its interior states. Lack of such support the signaling-end 
   point might try to send overrated data traffic across all the 
   reservation sessions and dropping the right amount of data traffic 
   means that the reservation sessions are alive. Of course other smart 
   methods can be used also. 
    
7.3.6 Maximum Number of Reservation Sessions -- Multicast Case 
    
   Objective: 
   To determine the maximum number of reservation sessions that can 
   exist simultaneously in a reservation capable router. 
    
   Procedure: 
   The procedure is the same as it is defined for the unicast case, 
   except all the signaling messages should refer to multicast 
   reservation sessions. 
    
   Reporting format: 
   The reporting format is the same as in the unicast case. 
    
7.4 Benchmarking Tests 
    
   Benchmarking tests are defined to measure the QoS signaling related 
   performance metrics on the router device. 
    
   During the tests the DUT must stay in steady state. This means that 
   the router must not drop any signaling message or data packet, i.e. 
   it should operate below its scalability limits. In case of signaling 
   message or data traffic loss, the test must be stopped, and the 
   parameters of the test must be adjusted to prevent the DUT to leave 
   its steady state operating range. 
    
   During all of the benchmarking tests the sender tester loads the DUT 
   by sending a specified amount of signaling and data traffic to the 
   receiver device across the DUT. Moreover, the signaling end-points 
   must also assure that the DUT maintains a certain number of 
   reservation sessions. All of the performance metrics are measured 
   under different load conditions, where the load is a combination of: 
    
   a. Signaling load 
   b. Premium traffic load 
   c. Best-effort traffic load 
   d. Reservation load 

 
Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 19]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

    
   Signaling load must be generated by sending signaling messages 
   periodically to the DUT. This way the signaling load is measured by 
   the signaling intensity and expressed with the same signaling 
   measurement unit as it is in case of signaling intensity. The 
   signaling messages sent by the sender device should be equally 
   distributed on the time scale. Most of the resource reservation 
   protocols have more than one signaling message type and the 
   benchmarking is complete when the test has been carried over for each 
   signaling message type.  
    
   Premium traffic load must be generated by sending a data flow from 
   the sender to the receiver across the DUT where the DUT must have a 
   valid reservation for that flow. The traffic must consist of equally 
   spaced and equally sized data packets. The premium traffic must be 
   reported by its traffic parameters: data packet size in octets and 
   the calculated bandwidth of the stream in kbps unit. The data packet 
   size should include both the payload and the header of the IP packet. 
   Signaling messages maintaining the flow do not belong to the data 
   traffic flow and must be ignored when calculating the data traffic 
   parameters. 
    
   The best-effort traffic load must be generated by sending a data flow 
   from the sender tester to the receiver tester across the DUT where 
   the DUT must not have any reservation for that flow. The traffic must 
   consist of equally spaced and equally sized data packets. The best-
   effort traffic must be reported by traffic parameters as it is 
   described in case of the premium traffic. 
    
   The reservation load must be generated by reserving resources for 
   traffic flows via signaling in the DUT. During the test the sender 
   device must maintain the reservation sessions with signaling messages 
   if the resource reservation protocol requires it. The reservation 
   sessions should not need to be loaded with data traffic. The 
   reservations must differ at least in one of the IP addresses or port 
   number of the endpoints in the reservation entry. 
    
   The four load types have influence on each other from their nature, 
   but during the tests these cross-effects must be minimized. The 
   signaling load must establish as few temporary resource reservations 
   in the DUT as possible. When a new resource reservation is set up in 
   the DUT the signaling end-points must arrange to restore the number 
   of reservations in the router as soon as possible. Signaling messages 
   are datagrams in the real word, but during the measurements they are 
   not treated as premium or best-effort traffic. 
    
7.4.1 Signaling Message Handling Time -- Unicast Case 
    
   Objective: 
   To determine how fast the resource reservation protocol responds to 
   one type of signaling message or signaling message sequence in 
   various load conditions. 
    

 
Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 20]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

   Procedure: 
   1. Select one signaling message or a signaling message sequence from 
   the resource reservation protocol specification. 
    
   2. Load the DUT with a certain kind of load except signaling load. 
   The DUT should preserve its steady state operation during the load.  
    
   3. Form a signaling message flow from the selected signaling message 
   type or signaling message sequence. The signaling messages should be 
   distributed equally in time, this way the signaling flow has a 
   constant signaling intensity. 
    
   4. Send the signaling flow to the receiver across the DUT and measure 
   the time intervals that signaling messages spend inside the router as 
   the difference of absolute time (e.g. time stamp) when a signaling 
   message enters to the DUT and the time when the corresponding 
   signaling message leaves it. The signaling flow must be up for at 
   least one minute and the measurement should begin 30 seconds after 
   the first signaling message is processed in the DUT and should last 
   at least 30 seconds. The result of the measurement is the average of 
   the individually measured signaling message handling times grouped by 
   signaling message types.  
    
   5. Repeat the test 30 times for each desired signaling intensity 
   value for the signaling flow. Always reset the DUT between two test 
   cycles. 
    
   6. Alter the signaling intensity and the load conditions and repeat 
   the whole test. The signaling message handling time must be measured 
   in function of many different load conditions. Measuring all possible 
   load condition however is almost impossible so it is not required, 
   instead the parameters of the load generation is given in a range and 
   a step. 
    
   The data traffic parameters have to be selected from common traffic 
   parameters. It is recommended to choose a packet size of: 54, 64, 
   128, 256, 1024, 1518, 2048 and 4472. The same values used in RFC 2544 
   that gives methodology for benchmarking network interconnect devices. 
   The packet rate is recommended to be 1, 10, 100 and 1000 kbit/s. At 
   least 2 different data traffic parameter set should be chosen both 
   for the premium traffic and the best-effort traffic. It is 
   recommended to use UDP packets constructing data traffic but any 
   other transfer protocol can be used. 
    
   The number of reserved sessions in the DUT should be in the range of 
   1 and the maximum number of reservation sessions for the tested 
   resource reservation protocol implementation. At least 3 different 
   values must be chosen. 
    
   The signaling intensity of the signaling message flow should be in 
   the range of 1 and the maximum signaling intensity for the tested 
   resource reservation protocol implementation. At least 3 different 
   signaling message intensity values must be chosen. 

 
Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 21]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

    
   Additionally, the signaling message handling time must be measured in 
   unloaded conditions as well. In this case there is no data traffic on 
   the DUT, no reserved resources and the signaling intensity should 
   take the 1 signaling message per second value. 
    
   Reporting format: 
   The results should be reported in a table. The columns of the table 
   are the values of the four load components and the fifth column is 
   the measured signaling message handling time in seconds. The table 
   entries for the traffic parameters must include the packet size, the 
   packet rate and the transfer protocol type. 
    
   There should be one table for each signaling message type or 
   signaling message sequence. 
    
   Note: The number of the tests is the product of the number of 
   different load parameters. The minimum is 37 which includes: 2 
   parameter sets for the premium traffic, 2 parameter sets for best-
   effort traffic, 3 different reservation session number and 3 
   different signaling intensity values for the signaling sequence plus 
   one measurement where the signaling load is the only load on the DUT. 
    
7.4.2 Signaling Message Handling Time -- Multicast Case 
    
   Objective: 
   To determine how fast the resource reservation protocol responds to 
   one type of signaling messages or signaling message sequence in 
   various load conditions. 
    
   Procedure: 
   The procedure is similar to the unicast case but the signaling 
   messages should refer to multicast sessions, the reserved sessions in 
   the DUT must be entries for multicast reservation sessions and the 
   premium traffic must be addressed to a multicast group. 
    
   Reporting format: 
   The reporting format is the same as in the unicast case. 
    
7.4.3 Premium Traffic Extra Delay -- Unicast Case 
    
   Objective:  
   To determine how the resource reservation protocol affects the 
   premium traffic forwarding performance of the router when it handles 
   resource reservations for unicast flows. 
    
   Procedure: 
   1. Load the DUT with a certain kind of load except premium traffic 
   load. The DUT should preserve its steady state operation during the 
   load. 
    
   2. Construct a premium traffic flow from data packets of the same 
   size. The flow should have a constant bitrate. 


Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 22]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 


   3. Send the premium flow to the receiver across the DUT with 
   different traffic parameters and measure the time intervals that data 
   packets spend inside the router as they are being forwarded. The 
   signaling must set up a reservation session for this flow in the DUT 
   before the test starts and it should not be torn down during the 
   test. The traffic flow must be active for at least one minute. The 
   measurement should begin 30 seconds after the first data packet 
   belonging to the measured flow is forwarded in the DUT, and should 
   last at least 30 seconds. It is not necessary to measure all premium 
   packets, but at least 20 samples are required in a second and the 
   samples should be distributed equally in time scale. The result of 
   the measurement is the average of the individually measured packet 
   forwarding time values. 

   4. Repeat the test 30 times for each desired parameter set of the 
   premium traffic. Always reset the DUT between two test cycles. 

   5. Alter the parameters of the premium traffic and the load 
   conditions and repeat the whole test. The premium traffic extra delay 
   must be measured in function of many different load conditions. 

   Choose the load parameters according to the test description of the 
   signaling message handling time, but here at least 3 traffic 
   parameter sets are required to be tested in case of the premium flow. 

   Additionally, the premium traffic extra delay must be measured in 
   unloaded conditions as well. In this case there is neither best-
   effort traffic and nor signaling load on the DUT and there is only 
   one resource reservation entry, which one reserves the resources for 
   the premium traffic flow used in the test. 

   Reporting format: 
   The results should be reported in a table. The columns of the table 
   are the values of the four load components, the fifth column is the 
   premium traffic delay in seconds and the sixth column is the 
   difference of the premium traffic delay with the given load 
   conditions and the premium traffic delay measured in the unloaded 
   router. The table entries for the traffic parameters must include the 
   packet size, the packet rate and the transfer protocol type. The 
   column of the signaling load must contain the signaling message type 
   or the signaling message sequence along with the signaling intensity. 

   Note: The number of the tests is at least as large as in case of the 
   signaling message handling time test. Using the same parameters as in 
   case of the signaling message handling time test is highly 
   recommended. 

7.4.4 Premium Traffic Extra Delay -- Multicast Case 

   Objective: 




Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 23]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

   To determine how the resource reservation protocol affects the 
   premium traffic forwarding performance of the router when it handles 
   resource reservations for multicast flows. 

   Procedure: 
   The procedure is similar to the unicast case but the signaling 
   messages should refer to multicast sessions, the reservation sessions 
   in the DUT must be multicast reservations sessions and the premium 
   traffic must be addressed to a multicast group. 

   Reporting format: 
   The reporting format is the same as in the unicast case. 

7.4.5 Best-effort Traffic Extra Delay 

   Objective:  
   To determine how the resource reservation protocol affects the best-
   effort traffic forwarding performance of the router when it handles 
   resource reservations for unicast flows. 

   Procedure:  
   The procedure is almost the same as in case of the premium traffic 
   extra delay test, but the best-effort traffic must be measured 
   instead of the premium traffic and it should be measured for at least 
   3 different traffic parameter sets. Of course the best-effort traffic 
   does not require any reservation session entry. 
    
   Reporting format:  
   The reporting format is the same as in case of the premium traffic 
   delay test, except that the fifth column is for the best-effort 
   traffic instead of the premium traffic. 

   Note:  
   Measuring the best-effort traffic delay for multicast flows is not 
   necessary as the vast amount of the best-effort traffic is unicast 
   and considered to remain unicast in the future as multicast flows 
   usually requires QoS services. 

8. Acknowledgement 

   We would like to thank the following individuals for their help in 
   designing and implementing the benchmarking framework presented in 
   this document: Joakim Bergkvist and Norbert Vegh from Telia Research 
   AB, Sweden, Gabor Kovacs, Anrdas Korn, Balazs Szabo from High Speed 
   Netwroks Laboratory of BUTE. 

9. References 

   [1]  S. Bradner, "Benchmarking Terminology for Network 
        Interconnection Devices", RFC 1242, July 1991 

   [2]  R. Mandeville, "Benchmarking Terminology for LAN Switching 
        Devices", RFC 2285, February 1998 


Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 24]

INTERNET-DRAFT     <draft-feher-benchresres-00.txt>            July 2000 

   [3]  Y. Bernet, et. al., "A Framework For Integrated Services 
        Operation Over Diffserv Networks", Internet Draft, May 2000, 
        <draft-ietf-issll-diffserv-rsvp-05.txt> 

   [4]  S. Bradner, J. McQuaid, "Benchmarking Methodology for Network 
        Interconnect Devices", RFC 2544, March 1999 

   [5]  B. Braden, Ed., et. al., "Resource Reservation Protocol (RSVP) - 
        Version 1 Functional Specification", RFC 2205, September 1997. 

   [6]  J. Bergkvist, I. Cselenyi, "Boomerang Protocol Specification", 
        Internet Draft, June 1999, <draft-bergkvist-boomerang-spec-
        00.txt> 

   [7]  P. Pan, H. Schulzrinne, "YESSIR: A Simple Reservation Mechanism 
        for the Internet", Computer Communication Review, on-line 
        version, volume 29, number 2, April 1999 

   [8]  L. Delgrossi, L. Berger, "Internet Stream Protocol Version 2 
        (ST2) Protocol Specification - Version ST2+", RFC 1819, August 
        1995 

   [9]  D. Black, S. Blake, M. Carlson, E. Davies, Z. Wang and W. Weiss, 
        "An Architecture for Differentiated Services", RFC 2475, 
        December 1998 

   [10] Boomerang Team, "Boomerang homepage - Benchmarking Tools", 
        http://boomerang.ttt.bme.hu 

10. Authors' Addresses: 

   Gabor Feher 
   Budapest University of Technology and Economics (BUTE) 
   Department of Telecommunications and Telematics 
   Pazmany Peter Setany 1/D, H-1117, Budapest, 
   Phone: +36 1 463-3110 
   Email: feher@ttt-atm.ttt.bme.hu 

   Istvan Cselenyi 
   Telia Research AB 
   Vitsandsgatan 9B 
   SE 12386, Farsta 
   SWEDEN, 
   Phone: +46 8 713-8173 
   Email: istvan.i.cselenyi@telia.se 

   Krisztian Nemeth 
   Budapest University of Technology and Economics (BUTE) 
   Department of Telecommunications and Telematics 
   Pazmany Peter Setany 1/D, H-1117, Budapest, 
   Phone: +36 1 463-2225 
   Email: nemeth_k@ttt-atm.ttt.bme.hu 



Feher, Cselenyi, Nemeth   Expires January 2001                 [Page 25]