Network Working Group R. Mandeville Internet-Draft European Network Laboratories Expiration Date: February 1999 J. Keene Netcom Systems August 1998 Benchmarking Methodology for LAN Switching Devices Status of this Memo This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF),its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." To view the entire list of current Internet-Drafts, please check the "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net (Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. Distribution of this memo is unlimited. Abstract This document is intended to provide methodology for the benchmarking of local area network (LAN) switching devices. It extends the methodology already defined for benchmarking network interconnecting devices in RFC 1944 to switching devices. This RFC primarily deals with devices which switch frames at the Medium Access Control (MAC) layer. It provides a methodology for benchmarking switching devices, forwarding performance, congestion control, latency, address handling and filtering. In addition to defining the tests, this document also describes specific formats for reporting the results of the tests. 1. Introduction Mandeville, Keene [Page 1] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 This document defines a specific set of tests to measure and report the performance characteristics of network switching devices. A previous document, "Benchmarking Terminology for LAN Switching Devices" (RFC 2285), defined many of the terms that are used in this document. The terminology document SHOULD be consulted before attempting to make use of this document. 2. Requirements The following RFCs SHOULD be consulted before attempting to make use of this document: * RFC 1242 "Benchmarking Terminology for Network Interconnect Devices" * RFC 1944 "Benchmarking Methodology for Network Interconnect Devices" * RFC 2285 "Benchmarking Terminology for LAN Switching Devices" For the sake of clarity and continuity, this RFC adopts the template for benchmarking tests set out in Section 26 of RFC 1944. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 3. Test setup This document extends the general test setup described in section 6 of RFC 1944 to the benchmarking of LAN switching devices. RFC 1944 primarily describes non-meshed traffic where input and output interfaces are grouped in mutually exclusive sending and receiving pairs. In fully meshed traffic, each interface of a DUT/SUT is set up to both receive and transmit frames to all the other interfaces under test. Prior to each test run, the DUT/SUT MUST learn the MAC addresses used in the test and the address learning MUST be verified to avoid flooded frames being counted as correctly received frames. The forwarding rate, namely the rate at which address learning frames are offered may have to be adjusted to be as low as 50 frames per second or even less, to guarantee successful learning. The DUT/SUT address aging time SHOULD be configured to be greater than the period between the learning phase of the test and the test run; in non-meshed and partially meshed tests, the aging time SHOULD at a minimum be set to at least the length of the test period. More than one trial may be needed for the association Mandeville, Keene [Page 2] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 of the address to the port to occur. If a DUT/SUT uses a hashing algorithm with address learning, the DUT/SUT may not learn the necessary addresses to perform the tests. The format of the MAC addresses MUST be adjustable so that the address mapping may be re-arranged to make a DUT/SUT learn addresses without confusion. It is recommended that SNMP and Spanning Tree be disabled when bench- marking switching devices unless investigating overhead behavior. If such protocols cannot be turned off, it is recommended that the levels of offered load be reduced (less than 100%) to allow for the additional management frames. 4. Frame formats and sizes For frame formats and sizes, refer to RFC 1944, sections 8 and 9 and Appendix C. There are three frame formats for layer 2 Ethernet switches: standard MAC Ethernet frames, standard MAC Ethernet frames with vendor-specific tags added to them, and IEEE 802.3ab frames tagged to accommodate 802.3p,q. The two types of tagged frames may exceed the standard maximum length frame of 1518 bytes, and may not be accepted by the interface controllers of some DUT/SUTs. It is recommended to check the compatibility of the DUT/SUT with tagged frames before testing. Devices switching tagged frames of over 1518 bytes will have a lower maximum forwarding rate than standard untagged frames. 5. Benchmarking Tests The following tests offer objectives, procedures, and reporting formats for benchmarking LAN switching devices. 5.1 Fully meshed throughput, frame loss and forwarding rates 5.2 Partially meshed overloading 5.3 Head of line blocking 5.4 Partially meshed multiple devices 5.5 Multiple streams of unidirectional traffic 5.6 Filter illegal frames 5.7 Broadcast frame handling and latency 5.8 Maximum forwarding rate and minimum interframe gap 5.9 Address caching capacity 5.10 Address learning rate Mandeville, Keene [Page 3] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 5.1 Fully meshed throughput, frame loss and forwarding rates Objective: To determine the throughput, frame loss and forwarding rates of DUT/SUTs offered fully meshed traffic as defined in RFC 2285. Procedure: RFC 2285 points out that when offering bursty meshed traffic, the variables which MUST be defined are frame size, bust size, interframe gap, interburst gap, and load. Each variable is configured with the following considerations. Interframe Gap (IFG) - The IFG between frames inside a burst MUST be at the minimum specified by the standard (9.6 us for 10Mbps Ethernet and 0.96 us for 100Mbps Ethernet). Interburst Gap (IBG) - This is the interval between bursts of traffic. Refer to Appendix A, Calculating Interburst Gap, for the formula used to compute IBG. Load / Port - The test SHOULD be run multiple times with a different load per port in each case. The 100% load translates to a transmit load of 50% for half duplex. This type of test SHOULD also be run at higher than 100% loads. In half duplex mode, exactly half of the target load SHOULD be sent to each of the ports under test. For example, with a 100% load of 64-byte frames, the target load for each port under test is 7440 frames received per second and 7440 frames transmitted per second (for 10Mbps Ethernet). Frame Size - Recommended frame sizes are 64, 128, 256, 512, 1024, 1280 and 1518 bytes, per RFC 1944 section 9. Burst Size - The burst size defines the number of packets sent back-to-back at the minimum legal IFG (96 bit times) before pausing transmission to receive frames. Burst sizes SHOULD vary between 1 and 930 frames. To avoid truncating bursts, a burst size SHOULD be an even multiple into 7440. 7440 is the maximum frames per second for half duplex mode; 14880 is the maximum frames per second for full duplex mode. Refer to Appendix A for a table of recommended burst values. Mandeville, Keene [Page 4] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 Each port in the test sends frames to all other ports in a round- robin type fashion. The following table shows how each port in a test SHOULD transmit frames to all other ports in the test. In this example, there are six ports with 1 address per port: Source Port, then Destination Ports (in order of transmission) Port #1 2 3 4 5 6 2... Port #2 3 4 5 6 1 3... Port #3 4 5 6 1 2 4... Port #4 5 6 1 2 3 5... Port #5 6 1 2 3 4 6... Port #6 1 2 3 4 5 1... As shown in the table, there is an equal distribution of destination addresses for each transmit opportunity. This keeps the test balanced so that one destination port is not overloaded by the test algorithm and all ports are equally and fully loaded throughout the test. For tests using multiple addresses per port, the actual port destinations are the same as described above and the actual source/destination address pairs are chosen randomly to exercise the DUT/SUT's ability to perform address lookups. For every address, the testing device sends learning packets to allow the DUT/SUT to load its address tables properly. To measure the DUT/SUT's ability to switch traffic while performing many different address lookups, the number of addresses per port SHOULD be increased in a series of tests. Reporting format: In these tests, a port SHOULD transmit and receive the same amount of packets. Each port MUST count the packets received with a valid address. Any packet received which does not have a valid address MUST not be counted as a received packet and can be counted as part of a flood count as described in 3.8.3 in RFC 2285. The results for these tests SHOULD be reported in the form of numerical data or a graph with text to indicate the data type. The data types for the throughput test and for the frame loss rate test are described in RFC 1944. Forwarding rate (FR) of the DUT/SUT SHOULD be reported as the number of frames per second that the device is observed to successfully transmit to the correct destination interface in response to a specified offered load. The offered load MUST also be cited. Mandeville, Keene [Page 5] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 Forwarding rate at maximum offered load (FRMOL ) SHOULD be reported as the number of frames per second that a device can successfully transmit to the correct destination interface in response to the maximum offered load as defined in RFC 2285, section 3.6. The maximum offered load MUST also be cited. Maximum forwarding rate (MFR) SHOULD be reported as the highest forwarding rate of a DUT/SUT taken from an iterative set of forwarding rate measurements. The load applied to the device MUST also be cited. 5. 2 Partially meshed overloading To be done. 5. 3 Head-of-line blocking To be done. 5.4 Partially Meshed Multiple Devices To be done. 5.5 Multiple streams of unidirectional traffic To be done. 5.6 Filter illegal frames To be done. 5.7 Broadcast frame handling and latency test To be done. 5.8 Maximum forwarding rate and minimum interframe gap To be done. 5.9 Address Caching Capacity Objective: To determine the address caching capacity of a LAN switching device as defined in RFC 2285, section 3.8.1. Procedure: Mandeville, Keene [Page 6] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 This test SHOULD at a minimum be performed in a three-port configuration as described below. The first port (port 1) of the testing device is connected to the DUT/SUT and is the port from which a testing device sends frames with varying source addresses and a fixed destination address corresponding to the MAC address of the receiving port. By receiving frames with varying source addresses, the DUT/SUT will learn these new addresses from the sending port of the test device. A second port (port 2) of the testing device is connected to the DUT/SUT and acts as the receiving port for the address learning frames. This port also sends "control" frames back to the addresses learned on the first port. The algorithm for this is explained below. -- A third port (port 3) on the testing device MUST be connected to a port on the DUT/SUT and act as a monitoring port to listen for flooded frames. The algorithm for the test is as follows: BEGIN Set Initial Value of N to user specified number, where N is the number of addresses to be verified in each iteration WHILE NOT Finished DO PAUSE for the aging time specified Address learning: Port 1 sends N frames with varying source addresses to Port 2 to attempt to fill the DUT/SUT address table for Port #1. Controlling: Port 2 sends N frames with varying destination addresses corresponding to Port 1. IF (Port 3 received a frame during Control phase) OR (Port 1 did not receive the correct # of frames) THEN Address Table of DUT/SUT was full Set N to lower number (in binary search method) ELSE Address Table of DUT/SUT was NOT full Set N to higher number (in binary search method) IF High and Low Values of N Meet THEN Test is Finished, Value of N equals number of addresses supported by DUT/SUT ELSE Continue Test END WHILE DONE Mandeville, Keene [Page 7] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 Using a binary search approach, the test targets the exact number of addresses supported per port with minimal test iterations. Due to the aging time of DUT/SUT address tables, each iteration may take some time during the waiting period for the addresses to clear. If possible, configure the DUT/SUT for a low value for the aging time. Once the high and low values of N meet, then the threshold of address handling has been found. For this test, the aging time of the DUT/SUT MUST be known. Use a short aging time if possible to reduce the time needed to run the test. The aging time MUST be longer than the time necessary to produce packets at the specified rate. If a low frame rate is used for the test, then it may be possible that sending a large amount of frames may actually take longer than the aging time. Keep in mind that the test actually sends twice as many frames as the number of addresses being tested due to the learning phase and the controlling phase. Set the initial value of addresses per port to a number slightly higher than the number of addresses that the DUT/SUT can handle. This step primes the binary search and can reduce the number of iterations required to determine the exact number supported. Set the forwarding rate to a number that is reasonable to be handled by the DUT/SUT and one that is high enough so that the test iterations are not too long. Reporting format: After the test is run, results for each iteration SHOULD be displayed in a table to include: -- the number of addresses used for each test iteration. -- the frame rate used for each test iteration. -- number of control frames that were transmitted by test port number 2. Control frames are the frames sent with varying destination addresses to confirm that the DUT/SUT has learned all of the addresses for each test iteration. -- the number of frames received by test port 2 during the control portion of each test. If the number is non-zero, this is an indication of the DUT/SUT flooding a frame in which the destination address is not in the address table. -- the number of frames that test port 1 received during the control Mandeville, Keene [Page 8] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 portion of the test. This number will include those frames which are Spanning tree frames. For a normal test iteration, this number SHOULD be equal to the number of frames transmitted by port 2 during the control phase. -- the number of frames that test port 1 received during the control phase not including Spanning Tree or other non test generated frames. In all cases, this number SHOULD be equal to the number of frames transmitted by port 2 during the control phase of the test. -- the number of frames test port 3 received during the control phase of the test. If the value is not zero, then this indicates that for that test iteration, the DUT/SUT could not determine the proper destination port for that many frames. In other words, the DUT/SUT flooded the frame to all ports since its address table was full. 5.10 Address Learning Rate Objective: Once the maximum number of addresses supported per port by the DUT/SUT is known, the addressing learning rate (the rate at which the DUT/SUT learns these addresses) can be determined. Procedure: An algorithm similar to the one used to determine address caching capacity can be used to determine the address learning rate. This test iterates the rate at which address learning frames are offered by the test device connected to the DUT/SUT. It is recommended to set the number of addresses offered to the DUT/SUT in this test to the maximum caching capacity. However, the address learning rate might be determined for different numbers of addresses but in each test run, the number MUST remain constant. Initializing the forwarding rate primes the binary search algorithm and can help to shorten the overall test duration. A third port on the DUT/SUT MUST listen for flooded frames. In this test, the forwarding rate SHOULD be varied until a rate is found that is the threshold for the rate supported by the DUT/SUT. It may be useful to pick a value slightly higher than the advertised forwarding rate. Reporting format: Mandeville, Keene [Page 9] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 To be done. Section 6. Security Considerations This document does not yet address Security Considerations. Section 7. Authors' Address Robert Mandeville European Network Laboratories (ENL) 2, rue Helene Boucher 87286 Guyancourt Cedex France Phone: + 33 1 39 44 12 05 EMail: bob@enl.net Judy Keene Netcom Systems 20550 Nordhoff St. Chatsworth, CA 91311 USA Phone: +1 818 700 5100 Email: judy_keene@netcomsystems.com Appendix A: Calculating the Interburst Gap IBG is defined in RFC 2285 as a function of maximum media rate (also known as line rate), the length of the frames in the bursts with the preamble and the interframe gap (IFG), the number of frames in the bursts, and the intended load. Using the burst size, frame size and the load per port, the IBG can be calculated: Burst size * ((preamble 64 + (frame size * 8 bits) + 96 IFG bit times)) * 1 (for full duplex) or .5 (for half duplex) (10 bit times equal 1 microsecond) Example: Burst size = 24 Mandeville, Keene [Page 10] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 Frame size = 64 24 * (64 + 64*8 + 96) = 24 * (672) = 16,128 bit/times = 1612.8 us IBG In half duplex mode, exactly half of the target load SHOULD be sent to each of the ports under test. For example, with a 100% load of 64-byte frames, the target load for each port under test is 7440 frames received per second and 7440 frames transmitted per second (for 10Mbps Ethernet). Recommended characteristic values for burst size are: 2 15 60 240 930 3 16 62 248 4 20 80 310 5 24 93 372 6 30 120 465 8 31 124 496 10 40 155 620 12 48 186 744 Mandeville, Keene [Page 11] Network Working Group R. Mandeville Internet-Draft European Network Laboratories Expiration Date: February 1999 J. Keene Netcom Systems August 1998 Benchmarking Methodology for LAN Switching Devices Status of this Memo This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF),its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." To view the entire list of current Internet-Drafts, please check the "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net (Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. Distribution of this memo is unlimited. Abstract This document is intended to provide methodology for the benchmarking of local area network (LAN) switching devices. It extends the methodology already defined for benchmarking network interconnecting devices in RFC 1944 to switching devices. This RFC primarily deals with devices which switch frames at the Medium Access Control (MAC) layer. Itprovides a methodology for benchmarking switching devices, forwarding performance, congestion control, latency, address handling and filtering. In addition to defining the tests, this document also describes specific formats for reporting the results of the tests. 1. Introduction Mandeville, Keene [Page 1] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 This document defines a specific set of tests to measure and report the performance characteristics of network switching devices. A previous document, "Benchmarking Terminology for LAN Switching Devices" (RFC 2285), defined many of the terms that are used in this document. The terminology document SHOULD be consulted before attempting to make use of this document. 2. Requirements The following RFCs SHOULD be consulted before attempting to make use of this document: * RFC 1242 "Benchmarking Terminology for Network Interconnect Devices" * RFC 1944 "Benchmarking Methodology for Network Interconnect Devices" * RFC 2285 "Benchmarking Terminology for LAN Switching Devices" For the sake of clarity and continuity, this RFC adopts the template for benchmarking tests set out in Section 26 of RFC 1944. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 3. Test setup This document extends the general test setup described in section 6 of RFC 1944 to the benchmarking of LAN switching devices. RFC 1944 primarily describes non-meshed traffic where input and output interfaces are grouped in mutually exclusive sending and receiving pairs. In fully meshed traffic, each interface of a DUT/SUT is set up to both receive and transmit frames to all the other interfaces under test. Prior to each test run, the DUT/SUT MUST learn the MAC addresses used in the test and the address learning MUST be verified to avoid flooded frames being counted as correctly received frames. The forwarding rate, namely the rate at which address learning frames are offered may have to be adjusted to be as low as 50 frames per second or even less, to guarantee successful learning. The DUT/SUT address aging time SHOULD be configured to be greater than the period between the learning phase of the test and the test run; in non-meshed and partially meshed tests, the aging time SHOULD at a minimum be set to at least the length of the test period. More than one trial may be needed for the association Mandeville, Keene [Page 2] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 of the address to the port to occur. If a DUT/SUT uses a hashing algorithm with address learning, the DUT/SUT may not learn the necessary addresses to perform the tests. The format of the MAC addresses MUST be adjustable so that the address mapping may be re-arranged to make a DUT/SUT learn addresses without confusion. It is recommended that SNMP and Spanning Tree be disabled when bench- marking switching devices unless investigating overhead behavior. If such protocols cannot be turned off, it is recommended that the levels of offered load be reduced (less than 100%) to allow for the additional management frames. 4. Frame formats and sizes For frame formats and sizes, refer to RFC 1944, sections 8 and 9 and Appendix C. There are three frame formats for layer 2 Ethernet switches: standard MAC Ethernet frames, standard MAC Ethernet frames with vendor-specific tags added to them, and IEEE 802.3ab frames tagged to accommodate 802.3p,q. The two types of tagged frames may exceed the standard maximum length frame of 1518 bytes, and may not be accepted by the interface controllers of some DUT/SUTs. It is recommended to check the compatibility of the DUT/SUT with tagged frames before testing. Devices switching tagged frames of over 1518 bytes will have a lower maximum forwarding rate than standard untagged frames. 5. Benchmarking Tests The following tests offer objectives, procedures, and reporting formats for benchmarking LAN switching devices. 5.1 Fully meshed throughput, frame loss and forwarding rates 5.2 Partially meshed overloading 5.3 Head of line blocking 5.4 Partially meshed multiple devices 5.5 Multiple streams of unidirectional traffic 5.6 Filter illegal frames 5.7 Broadcast frame handling and latency 5.8 Maximum forwarding rate and minimum interframe gap 5.9 Address caching capacity 5.10 Address learning rate Mandeville, Keene [Page 3] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 5.1 Fully meshed throughput, frame loss and forwarding rates Objective: To determine the throughput, frame loss and forwarding rates of DUT/SUTs offered fully meshed traffic as defined in RFC 2285. Procedure: RFC 2285 points out that when offering bursty meshed traffic, the variables which MUST be defined are frame size, bust size, interframe gap, interburst gap, and load. Each variable is configured with the following considerations. Interframe Gap (IFG) - The IFG between frames inside a burst MUST be at the minimum specified by the standard (9.6 us for 10Mbps Ethernet and 0.96 us for 100Mbps Ethernet). Interburst Gap (IBG) - This is the interval between bursts of traffic. Refer to Appendix A, Calculating Interburst Gap, for the formula used to compute IBG. Load / Port - The test SHOULD be run multiple times with a different load per port in each case. The 100% load translates to a transmit load of 50% for half duplex. This type of test SHOULD also be run at higher than 100% loads. In half duplex mode, exactly half of the target load SHOULD be sent to each of the ports under test. For example, with a 100% load of 64-byte frames, the target load for each port under test is 7440 frames received per second and 7440 frames transmitted per second (for 10Mbps Ethernet). Frame Size - Recommended frame sizes are 64, 128, 256, 512, 1024, 1280 and 1518 bytes, per RFC 1944 section 9. Burst Size - The burst size defines the number of packets sent back-to-back at the minimum legal IFG (96 bit times) before pausing transmission to receive frames. Burst sizes SHOULD vary between 1 and 930 frames. To avoid truncating bursts, a burst size SHOULD be an even multiple into 7440. 7440 is the maximum frames per second for half duplex mode; 14880 is the maximum frames per second for full duplex mode. Refer to Appendix A for a table of recommended burst values. Mandeville, Keene [Page 4] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 Each port in the test sends frames to all other ports in a round- robin type fashion. The following table shows how each port in a test SHOULD transmit frames to all other ports in the test. In this example, there are six ports with 1 address per port: Source Port, then Destination Ports (in order of transmission) Port #1 2 3 4 5 6 2... Port #2 3 4 5 6 1 3... Port #3 4 5 6 1 2 4... Port #4 5 6 1 2 3 5... Port #5 6 1 2 3 4 6... Port #6 1 2 3 4 5 1... As shown in the table, there is an equal distribution of destination addresses for each transmit opportunity. This keeps the test balanced so that one destination port is not overloaded by the test algorithm and all ports are equally and fully loaded throughout the test. For tests using multiple addresses per port, the actual port destinations are the same as described above and the actual source/destination address pairs are chosen randomly to exercise the DUT/SUT's ability to perform address lookups. For every address, the testing device sends learning packets to allow the DUT/SUT to load its address tables properly. To measure the DUT/SUT's ability to switch traffic while performing many different address lookups, the number of addresses per port SHOULD be increased in a series of tests. Reporting format: In these tests, a port SHOULD transmit and receive the same amount of packets. Each port MUST count the packets received with a valid address. Any packet received which does not have a valid address MUST not be counted as a received packet and can be counted as part of a flood count as described in 3.8.3 in RFC 2285. The results for these tests SHOULD be reported in the form of numerical data or a graph with text to indicate the data type. The data types for the throughput test and for the frame loss rate test are described in RFC 1944. Forwarding rate (FR) of the DUT/SUT SHOULD be reported as the number of frames per second that the device is observed to successfully transmit to the correct destination interface in response to a specified offered load. The offered load MUST also be cited. Mandeville, Keene [Page 5] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 Forwarding rate at maximum offered load (FRMOL ) SHOULD be reported as the number of frames per second that a device can successfully transmit to the correct destination interface in response to the maximum offered load as defined in RFC 2285, section 3.6. The maximum offered load MUST also be cited. Maximum forwarding rate (MFR) SHOULD be reported as the highest forwarding rate of a DUT/SUT taken from an iterative set of forwarding rate measurements. The load applied to the device MUST also be cited. 5. 2 Partially meshed overloading To be done. 5. 3 Head-of-line blocking To be done. 5.4 Partially Meshed Multiple Devices To be done. 5.5 Multiple streams of unidirectional traffic To be done. 5.6 Filter illegal frames To be done. 5.7 Broadcast frame handling and latency test To be done. 5.8 Maximum forwarding rate and minimum interframe gap To be done. 5.9 Address Caching Capacity Objective: To determine the address caching capacity of a LAN switching device as defined in RFC 2285, section 3.8.1. Procedure: Mandeville, Keene [Page 6] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 This test SHOULD at a minimum be performed in a three-port configuration as described below. The first port (port 1) of the testing device is connected to the DUT/SUT and is the port from which a testing device sends frames with varying source addresses and a fixed destination address corresponding to the MAC address of the receiving port. By receiving frames with varying source addresses, the DUT/SUT will learn these new addresses from the sending port of the test device. A second port (port 2) of the testing device is connected to the DUT/SUT and acts as the receiving port for the address learning frames. This port also sends "control" frames back to the addresses learned on the first port. The algorithm for this is explained below. -- A third port (port 3) on the testing device MUST be connected to a port on the DUT/SUT and act as a monitoring port to listen for flooded frames. The algorithm for the test is as follows: BEGIN Set Initial Value of N to user specified number, where N is the number of addresses to be verified in each iteration WHILE NOT Finished DO PAUSE for the aging time specified Address learning: Port 1 sends N frames with varying source addresses to Port 2 to attempt to fill the DUT/SUT address table for Port #1. Controlling: Port 2 sends N frames with varying destination addresses corresponding to Port 1. IF (Port 3 received a frame during Control phase) OR (Port 1 did not receive the correct # of frames) THEN Address Table of DUT/SUT was full Set N to lower number (in binary search method) ELSE Address Table of DUT/SUT was NOT full Set N to higher number (in binary search method) IF High and Low Values of N Meet THEN Test is Finished, Value of N equals number of addresses supported by DUT/SUT ELSE Continue Test END WHILE DONE Mandeville, Keene [Page 7] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 Using a binary search approach, the test targets the exact number of addresses supported per port with minimal test iterations. Due to the aging time of DUT/SUT address tables, each iteration may take some time during the waiting period for the addresses to clear. If possible, configure the DUT/SUT for a low value for the aging time. Once the high and low values of N meet, then the threshold of address handling has been found. For this test, the aging time of the DUT/SUT MUST be known. Use a short aging time if possible to reduce the time needed to run the test. The aging time MUST be longer than the time necessary to produce packets at the specified rate. If a low frame rate is used for the test, then it may be possible that sending a large amount of frames may actually take longer than the aging time. Keep in mind that the test actually sends twice as many frames as the number of addresses being tested due to the learning phase and the controlling phase. Set the initial value of addresses per port to a number slightly higher than the number of addresses that the DUT/SUT can handle. This step primes the binary search and can reduce the number of iterations required to determine the exact number supported. Set the forwarding rate to a number that is reasonable to be handled by the DUT/SUT and one that is high enough so that the test iterations are not too long. Reporting format: After the test is run, results for each iteration SHOULD be displayed in a table to include: -- the number of addresses used for each test iteration. -- the frame rate used for each test iteration. -- number of control frames that were transmitted by test port number 2. Control frames are the frames sent with varying destination addresses to confirm that the DUT/SUT has learned all of the addresses for each test iteration. -- the number of frames received by test port 2 during the control portion of each test. If the number is non-zero, this is an indication of the DUT/SUT flooding a frame in which the destination address is not in the address table. -- the number of frames that test port 1 received during the control Mandeville, Keene [Page 8] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 portion of the test. This number will include those frames which are Spanning tree frames. For a normal test iteration, this number SHOULD be equal to the number of frames transmitted by port 2 during the control phase. -- the number of frames that test port 1 received during the control phase not including Spanning Tree or other non test generated frames. In all cases, this number SHOULD be equal to the number of frames transmitted by port 2 during the control phase of the test. -- the number of frames test port 3 received during the control phase of the test. If the value is not zero, then this indicates that for that test iteration, the DUT/SUT could not determine the proper destination port for that many frames. In other words, the DUT/SUT flooded the frame to all ports since its address table was full. 5.10 Address Learning Rate Objective: Once the maximum number of addresses supported per port by the DUT/SUT is known, the addressing learning rate (the rate at which the DUT/SUT learns these addresses) can be determined. Procedure: An algorithm similar to the one used to determine address caching capacity can be used to determine the address learning rate. This test iterates the rate at which address learning frames are offered by the test device connected to the DUT/SUT. It is recommended to set the number of addresses offered to the DUT/SUT in this test to the maximum caching capacity. However, the address learning rate might be determined for different numbers of addresses but in each test run, the number MUST remain constant. Initializing the forwarding rate primes the binary search algorithm and can help to shorten the overall test duration. A third port on the DUT/SUT MUST listen for flooded frames. In this test, the forwarding rate SHOULD be varied until a rate is found that is the threshold for the rate supported by the DUT/SUT. It may be useful to pick a value slightly higher than the advertised forwarding rate. Reporting format: Mandeville, Keene [Page 9] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 To be done. Section 6. Security Considerations This document does not yet address Security Considerations. Section 7. Authors' Address Robert Mandeville European Network Laboratories (ENL) 2, rue Helene Boucher 87286 Guyancourt Cedex France Phone: + 33 1 39 44 12 05 EMail: bob@enl.net Judy Keene Netcom Systems 20550 Nordhoff St. Chatsworth, CA 91311 USA Phone: +1 818 700 5100 Email: judy_keene@netcomsystems.com Appendix A: Calculating the Interburst Gap IBG is defined in RFC 2285 as a function of maximum media rate (also known as line rate), the length of the frames in the bursts with the preamble and the interframe gap (IFG), the number of frames in the bursts, and the intended load. Using the burst size, frame size and the load per port, the IBG can be calculated: Burst size * ((preamble 64 + (frame size * 8 bits) + 96 IFG bit times)) * 1 (for full duplex) or .5 (for half duplex) (10 bit times equal 1 microsecond) Example: Burst size = 24 Mandeville, Keene [Page 10] INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998 Frame size = 64 24 * (64 + 64*8 + 96) = 24 * (672) = 16,128 bit/times = 1612.8 us IBG In half duplex mode, exactly half of the target load SHOULD be sent to each of the ports under test. For example, with a 100% load of 64-byte frames, the target load for each port under test is 7440 frames received per second and 7440 frames transmitted per second (for 10Mbps Ethernet). Recommended characteristic values for burst size are: 2 15 60 240 930 3 16 62 248 4 20 80 310 5 24 93 372 6 30 120 465 8 31 124 496 10 40 155 620 12 48 186 744 Mandeville, Keene [Page 11]