TRILL Working Group D. Bond Internet-Draft UNH-IOL Intended status: Standards Track V. Manral Expires: January 4, 2012 HP Networking July 3, 2011 RBridges: Operations, Administration, and Maintenance (OAM) Support draft-ietf-trill-rbridge-oam-00 Abstract The IETF has standardized RBridges, devices that implement the TRILL protocol, a solution for transparent least-cost frame routing in multi-hop networks with arbitrary topologies, using a link-state routing protocol technology and encapsulation with a hop-count. As RBridges are deployed in real-world situations, operators will need tools for debugging problems that arise. This document specifies a set of RBridge features for operations, administration, and maintenance (OAM) purposes in RBridge campuses. The features specified in this document include tools for traceroute, ping, and error reporting. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on January 4, 2012. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of Bond & Manral Expires January 4, 2012 [Page 1] Internet-Draft RBridges: OAM Support July 2011 publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 2. Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. TRILL OAM Message . . . . . . . . . . . . . . . . . . . . . . 5 4. RBridge Tools . . . . . . . . . . . . . . . . . . . . . . . . 8 4.1. Application RBridge Tools . . . . . . . . . . . . . . . . 8 4.1.1. Hop Count Traceroute . . . . . . . . . . . . . . . . . 8 4.1.1.1. Multi-Destination Targets . . . . . . . . . . . . 9 4.1.1.2. Hop Count Traceroute Example . . . . . . . . . . . 10 4.1.2. RBridge Ping . . . . . . . . . . . . . . . . . . . . . 11 4.1.2.1. Ping Example . . . . . . . . . . . . . . . . . . . 13 4.2. Error Reporting . . . . . . . . . . . . . . . . . . . . . 14 4.2.1. Hop Count Zero Error . . . . . . . . . . . . . . . . . 15 4.2.2. MTU Error . . . . . . . . . . . . . . . . . . . . . . 15 5. RBridge Channel Message Format . . . . . . . . . . . . . . . . 16 5.1. RBridge Channel Header and Sequence Number . . . . . . . . 16 6. OAM Protocol Formats . . . . . . . . . . . . . . . . . . . . . 16 6.1. Protocol Application Codes Formats . . . . . . . . . . . . 16 6.1.1. Echo Request . . . . . . . . . . . . . . . . . . . . . 17 6.1.2. Echo Reply . . . . . . . . . . . . . . . . . . . . . . 17 6.2. Error Notification Format . . . . . . . . . . . . . . . . 18 6.2.1. Error Specifiers . . . . . . . . . . . . . . . . . . . 19 7. Type, Length, Value (TLV) Encodings . . . . . . . . . . . . . 22 7.1. Next Hop Nickname . . . . . . . . . . . . . . . . . . . . 24 7.2. Incoming Port ID . . . . . . . . . . . . . . . . . . . . . 24 7.3. Outgoing Port ID . . . . . . . . . . . . . . . . . . . . . 25 7.4. Outgoing Port MTU . . . . . . . . . . . . . . . . . . . . 25 7.5. ISIS System ID . . . . . . . . . . . . . . . . . . . . . . 26 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 26 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26 10. Security Considerations . . . . . . . . . . . . . . . . . . . 26 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 27 11.1. Normative References . . . . . . . . . . . . . . . . . . . 27 11.2. Informative References . . . . . . . . . . . . . . . . . . 28 Bond & Manral Expires January 4, 2012 [Page 2] Internet-Draft RBridges: OAM Support July 2011 1. Introduction The IETF has standardized RBridges, devices that implement the TRILL protocol, a solution for transparent least-cost frame routing in multi-hop networks with arbitrary topologies, using a link-state routing protocol technology and encapsulation with a hop-count [RFCtrill]. As RBridges are deployed, operators will face problems that require tools for troubleshooting of connectivity issues in the network. TRILL uses IS-IS for the control plane [IS-IS] [RFC6165] [RFCtisis]. IS-IS has a link-state database which contains the information of all links in the TRILL domain and IS-IS has a routing table. This information can be used for trouble shooting purposes. In addition, RBridges should support SNMP, as described in [RFCtrill] and [RBridgeMIB]. SNMP, the routing table, and the link-state database are insufficient as the only OAM tools because while the control plane within an RBridge campus may be functioning successfully the data plane may not be. This motivates the need for OAM tools that allow an operator to test the data plane. Protocols such as IP, MPLS, and IEEE 802.1 have features enabling an operator to exercise the data plane [RFC4443] [RFC0792] [IEEE.802-1ag]. There is a need for a similar set of tools in TRILL. Likewise, there is a need for error reporting capabilities inside an RBridge campus. For instance, if a TRILL Inner.VLAN tag has an illegal value there should be a way for devices to report this error. This would asist administrators of an RBridge campus in quickly locating a problem device in the network. This document specifies a set of RBridge features for operations, administration, and maintenance purposes in RBridge campuses along with a frame format. The features specified in this document include tools for traceroute, ping, and error reporting. Section 3 of this document specifies the general usage of a defined message format. Section 4 specifies some additional applications of the message format. Section 5 specifies the format of the messages on the wire. 1.1. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 2. Acronyms o BPDU - Bridge PDU Bond & Manral Expires January 4, 2012 [Page 3] Internet-Draft RBridges: OAM Support July 2011 o CHbH - Critical Hop-by-Hop o CItE - Critical Ingress-to-Egress o DA - Destination Address o DR - Designated Router o DRB - Designated RBridge o ES - End Station o ESa - End Station A o ESb - End Station B o ECMP - Equal-Cost Multi-Path o ESADI - End Station Address Distribution Instance o FCS - Frame Check Sequence o ID - Identification o IEEE - Institute of Electrical and Electronics Engineers o IETF - Internet Engineering Task Force o IP - Internet Protocol o IS-IS - Intermediate System to Intermediate System o MAC - Media Access Control o MPLS - Multiprotocol Label Switching o MTU - Maximum Transmission Unit o OAM - Operations, Administration, and Maintenance o P2P - Point-to-point o PDU - Protocol Data Unit o RBridge - Routing Bridge o SA - Source Address Bond & Manral Expires January 4, 2012 [Page 4] Internet-Draft RBridges: OAM Support July 2011 o SNMP - Simple Network Management Protocol o TLV - Type, Length, Value o TRILL - TRansparent Interconnection of Lots of Links o VLAN - Virtual Local Area Network 3. TRILL OAM Message To facilitate message passing as needed by the OAM requirements, the TRILL RBridge Channel facility [RBridgeChannel] is utilized. There are two types of TRILL OAM messages defined in this document carried within an RBridge Channel: application and error notification. Frames with an error notification MUST NOT be generated in response to frames that are an error notification. Implementations SHOULD rate limit the origination of error notifications. Whereas unknown unicast frames are sent as multi- destination messages, sending unknown unicast frames with an error can lead to an amplification attack. As such special care and rate limiting are necessary for error notifications. The specification of rate limiting is beyond the scope of this document. An RBridge SHOULD maintain counters for each type of error generated. Error notification messages contain the error-causing frame or the initial part thereof after its OAM message. The following are two figures showing application and error notification message structure. Section 5 goes into the details of these formats. +----------------------------+ | Outer Link Header | +----------------------------+ | TRILL Header | +----------------------------+ | Inner Ethernet Header | +----------------------------+ | RBridge Channel Header | +----------------------------+ | OAM Protocol Spec. Payload | +----------------------------+ Application Frame Bond & Manral Expires January 4, 2012 [Page 5] Internet-Draft RBridges: OAM Support July 2011 Figure 1 +---------------------------------------+ | Outer Link Header | +---------------------------------------+ | TRILL Header | +---------------------------------------+ | Inner Ethernet Header | +---------------------------------------+ | RBridge Channel Header | +---------------------------------------+ | OAM Protocol Specific Payload | +---------------------------------------+ | Offending Frame TRILL Header | +---------------------------------------+ | Offending Frame Inner Link Header | +---------------------------------------+ | Offending Frame Payload | +---------------------------------------+ Error Notification Frame Figure 2 Frames with a TRILL OAM message generated in response to another TRILL data frame have fields set as follows unless otherwise specified: +-------------+----------------+------------------------------------+ | Frame Type | Field | Value | +-------------+----------------+------------------------------------+ +-------------+----------------+------------------------------------+ | Application | Inner.MacSA | If the Inner.MacDA of the received | | or Error | | frame is one of the MAC addresses | | | | of the RBridge generating the | | | | frame, the value MUST be that MAC | | | | address. Otherwise, it MUST be | | | | one of the RBridge's MAC | | | | addresses. | +-------------+----------------+------------------------------------+ | Application | Inner.MacDA | The value SHOULD be | | or Error | | All-Egress-RBridges. The | | | | Inner.MacDA MAY be other values as | | | | specified in subsequent sections. | +-------------+----------------+------------------------------------+ Bond & Manral Expires January 4, 2012 [Page 6] Internet-Draft RBridges: OAM Support July 2011 +-------------+----------------+------------------------------------+ | Application | Inner.VLAN ID | If the frame is generated in | | or Error | | response to another frame with a | | | | legal Inner.VLAN ID, it MUST be | | | | the Inner.VLAN ID of the received | | | | frame. In other cases, it SHOULD | | | | be the default VLAN ID 1. | +-------------+----------------+------------------------------------+ | Application | Ingress | If the egress RBridge nickname of | | or Error | RBridge | the received frame is a nickname | | | nickname | of the RBridge generating the | | | | frame, then the value MUST be that | | | | nickname. Otherwise, it MUST be | | | | one of the RBridge's nicknames. | +-------------+----------------+------------------------------------+ | Application | Egress RBridge | The value MUST be the ingress | | or Error | nickname | RBridge nickname of the received | | | | frame. Except that, if the | | | | ingress RBridge nickname received | | | | is unknown or reserved the frame | | | | MUST be generated on the port the | | | | frame was received on with an | | | | Outer.MacDA and egress RBridge | | | | nickname of the previous-hop | | | | RBridge if this is known. | +-------------+----------------+------------------------------------+ | Error | Offending | The value MUST be N bytes of the | | | Encapsulated | frame that had the error where N | | | Frame | is the minimum of the frame size | | | | and the number of bytes that would | | | | bring the resulting error frame up | | | | to 1470 bytes. This MUST include | | | | the TRILL header and MUST NOT | | | | include the link-layer header. | +-------------+----------------+------------------------------------+ | Error | M Bit | The value MUST be zero. | +-------------+----------------+------------------------------------+ | Application | Inner.Priority | The value SHOULD be one less than | | or Error | | the priority of the received | | | | frame, but not less than the | | | | lowest priority. Defaults to zero | | | | for sent frames. | +-------------+----------------+------------------------------------+ Table 1: Frame Field Values RBridge campuses do not, in general, guarantee lossless transport of frames so a frame containing a TRILL OAM Message, possibly generated Bond & Manral Expires January 4, 2012 [Page 7] Internet-Draft RBridges: OAM Support July 2011 in response to some other frame, might be lost. 4. RBridge Tools This section specifies a number of RBridge OAM tools. For classification purposes they are divided into two sections, applications and error tools. 4.1. Application RBridge Tools 4.1.1. Hop Count Traceroute The ability to trace the path the data takes through the network is an invaluable debugging tool. RBridge traceroute provides this functionality through use of the TRILL OAM message (See Section 3). In a hop-count traceroute, the originating RBridge starts by transmitting one TRILL data frame with a TRILL OAM message. This message contains a protocol code of an echo request. (See Section 6.1.1) The ingress RBridge MUST be the RBridge originating the frame. When a traceroute is initiated, it is either targeting a known unicast target or a multi-destination target as specified by the operator. If the hop-count traceroute is for a known unicast target, the egress RBridge is the destination RBridge to which connectivity will be checked and the M bit MUST be zero. Otherwise, if the hop- count traceroute is for a multi-destination target, the egress RBridge is the distribution tree nickname for the traceroute. Multi- destination targets are handled the same as known unicast targets but require a small amount of additional logic as specified in Section 4.1.1.1. The first echo request frame transmitted MUST have a hop-count of one. The RBridge will continue transmitting these echo requests, incrementing the hop-count by one each time until a hop-count error notification or echo reply is received from the destination. Each of these requests in turn will generate a hop-count error notification until the egress RBridge is reached. If a transit RBridge decrements the hop-count by more than one it may transmit multiple hop-count error notifications. The purpose of the traceroute is to confirm connectivity of the data plane, and therefore options defined in future drafts MAY be included. The purpose of allowing the addition of options is so that the frame mimics a data frame that follows the same path through the data plane that a 'real' data frame would. The echo request MAY have an arbitrary 28-bit unsigned integer Bond & Manral Expires January 4, 2012 [Page 8] Internet-Draft RBridges: OAM Support July 2011 sequence number to assist in matching reply messages to the request. This is important for the hop-count traceroute since replies may return to the ingress RBridge in a different order then their matching requests were sent. The Inner.VLAN, Inner.MacSA, Inner.MacDA, Inner.Priority, and Ingress Nickname SHOULD default to the values specified in Table 1. RBridges SHOULD be configurable to change these values to assign the TRILL data frame to a flow. The replying RBridge MUST include its 16-bit port ID from the port on which the hop-count error generating frame was received in the incoming port field of the reply. It MUST also include its 16-bit port ID from the port on which the frame would be forwarded if the frame did not have a hop-count error. A port ID of 0xFFFF indicates the frame would not have been forwarded and would be consumed by the RBridge itself. Finally the reply MUST include a 16-bit nickname of the next hop RBridge the frame would have been sent to if there were no error. If the request is a multi-destination frame, this field MUST be set to the nickname of the RBridge the frame was received from. This is the previous hop RBridge. This is to facilitate knowledge of a more precise path through the campus as seen in RFC 5837 [RFC5837]. The advantage of this traceroute method is that the transit RBridges do not have to do any special processing of the frames until a hop- count error is detected, a condition they are required to detect by the TRILL base protocol. The disadvantage is the request-orginating RBridge needs to transmit as many frames as there are hops between itself and the destination RBridge. The end stations are not involved in this process. RBridge traceroutes are from RBridge to RBridge. While the frames sent may emulate data sent from ESa to ESb, the end stations are not, in fact, involved. 4.1.1.1. Multi-Destination Targets For multi-destination targets at each branch in the tree the tagged frame will be replicated causing each RBridge in the tree, possibly pruned by VLAN and/or IP multicast group, to send a response to the echo request. If all RBridges in the possibly pruned distribution tree support the echo request message, then the ingressing RBridge will receive an error notification from each of them. The ingressing RBridge can compile all of these notifications, using the parent pointers located in the nexthop nickname field, into an output of the tree the traffic traversed. A traceroute application SHOULD report any errors received, such as an invalid distribution tree nickname, Bond & Manral Expires January 4, 2012 [Page 9] Internet-Draft RBridges: OAM Support July 2011 caused by the hop-count traceroute frames. RBridges receiving a multicast destination echo request MUST NOT transmit an echo reply if the multi-destination bit is set. Echo requests that are not used with the hop-count traceroute come from the ping tool, and ping messages are not valid as multi-destination traffic. In a hop count traceroute, devices will already be transmitting a hop-count error notification and so there is no reason to transmit a double set of replies. A multi-destination hop-count traceroute SHOULD not stop when an echo reply is received. It stops when the transmitted hop count reaches the maximum, 0x3F. 4.1.1.2. Hop Count Traceroute Example Figure 3 contains a campus with three RBridges. Consider a hop-count traceroute from RB0 to RB2. +-----+ +-------+ +-------+ +-------+ +-----+ | ESa +--+ RB0 +---+ RB1 +---+ RB2 +--+ ESb | +-----+ |ingress| |transit| |egress | +-----+ +-------+ +-------+ +-------+ Time RB0 RB1 RB2 . (1)-------> | | . | <------- (2) | . (3)-------> (3) -------> | . | <------- (4) <-------(4) Hop Count Traceroute Example Topology Figure 3 In this diagram RB0 transmits frame (1) destined to RB2. This frame contains the echo request message and a hop-count of 1. When RB1 receives this frame it drops it and transmits a hop-count-exceeded message, (2), to RB0. RB0 then transmits a frame, (3), with a hop- count of 2. RB1 decrements this hop-count by 1 to 1 and forwards it to RB2. RB2 drops frame (3) and transmits a hop-count-exceeded message, (4), to RB0. The traceroute is now complete. Below are some select fields for the frames: Bond & Manral Expires January 4, 2012 [Page 10] Internet-Draft RBridges: OAM Support July 2011 +--------+-----------+-----------+------------+------------+--------+ | Frame | Ingress | Egress | TRILL OAM | Sequence | Hop | | # | RBridge | RBridge | Protocol | Number | Count | +--------+-----------+-----------+------------+------------+--------+ +--------+-----------+-----------+------------+------------+--------+ | (1) | RB0 | RB2 | Echo | 1 | 1 | | | | | Request | | | +--------+-----------+-----------+------------+------------+--------+ | (2) | RB1 | RB0 | Hop Count | 1 | N/A | | | | | Error | | | +--------+-----------+-----------+------------+------------+--------+ | (3) @ | RB0 | RB2 | Echo | 2 | 2 | | RB1 | | | Request | | | +--------+-----------+-----------+------------+------------+--------+ | (3) @ | RB0 | RB2 | Echo | 2 | 1 | | RB2 | | | Request | | | +--------+-----------+-----------+------------+------------+--------+ | (4) @ | RB2 | RB0 | Hop Count | 2 | N/A | | RB1 | | | Error | | | +--------+-----------+-----------+------------+------------+--------+ | (4) @ | RB2 | RB0 | Hop Count | 2 | N/A | | RB0 | | | Error | | | +--------+-----------+-----------+------------+------------+--------+ Table 2: Hop Count Traceroute Example Frames For example, if the nicknames for RB0, RB1, and RB2 are 0x1111, 0x2222, and 0x3333 respectively, the console output from such a trace might be: Hop Count Tracing RBridge Incoming Port Id Outgoing Port Id RBridge Nexthop Nickname ------- ---------------- ---------------- ------------------------ 0x1111 0xFFFF 0x0001 0x2222 0x2222 0x0000 0x0001 0x3333 0x3333 0x0000 0xFFFF 0x0000 Table 3: Hop Count Traceroute Example Output In this example, the first line of output is generated from local information, no hop-count frames are sent to generate it. 4.1.2. RBridge Ping Ping is a tool for verifying RBridge connectivity. As with an RBridge traceroute, the ping-originating RBridge transmits one or more TRILL data frames with a TRILL OAM message. This message Bond & Manral Expires January 4, 2012 [Page 11] Internet-Draft RBridges: OAM Support July 2011 contains the code of an echo request (See Section 6.1.1). The ingress RBridge MUST be the frame-originating RBridge. The egress RBridge is the destination RBridge to which connectivity will be checked. The M bit MUST be zero. The purpose of the ping is to confirm connectivity of the data plane, and therefore options defined in future drafts MAY be included. The purpose of allowing the addition of options is so that the frame mimics a data frame that follows the same path through the data plane that a 'real' data frame would. The echo request MAY have an arbitrary 28-bit unsigned integer sequence number to assist in matching reply messages to the request. In most circumstances, a single echo request is needed to complete the ping but it might be desirable for a single RBridge to ping multiple egress RBridges, or trace differing flows simultaneously. Assigning differing sequence numbers to each frame aids in matching which trace the reply belongs to. The Inner.VLAN, Inner.MacSA, Inner.MacDA, Inner.Priority, and Ingress Nickname SHOULD default to the values specified in Table 1. RBridges SHOULD provide the ability to change these values so as to assign the TRILL data frame to a flow. The payload of the frame is arbitrary and MAY contain any value. This value can have an influence on which flow the frame is assigned to. RBridges implementing ping SHOULD issue a reply in response to this request. See Section 10 for reasons that RBridges are allowed to choose not to respond to a request. If an RBridge chooses to respond to the request, the reply MUST consist of one TRILL data frame per request with an OAM message containing the protocol code of an echo reply. The echo reply MUST have the same sequence number as the request being matched. For the echo reply the ingress RBridge field MUST be the reply- originating RBridge's nickname. The egress RBridge MUST be the request-originating RBridge's nickname. The Inner.VLAN, Inner.MacSA, and Inner.MacDA SHOULD default to the values specified in Table 1. The M bit MUST be zero. The reply-originating RBridge MUST include its 16-bit port ID from the port on which the request was received in the incoming port field of the reply. It MUST also include its 16-bit port ID from the port on which the frame is forwarded. A port ID of 0xFFFF indicates the frame would not have been forwarder and was consumed by the RBridge itself. The nickname field in the generated frame MUST be set to all zeros on transmission and ignored on reception. Bond & Manral Expires January 4, 2012 [Page 12] Internet-Draft RBridges: OAM Support July 2011 The reply frame need not follow the same path though the campus as the request. The reply messages are not meant to test the data plane. End stations are not involved in this the ping process. RBridge pings are from RBridge to RBridge. While the frames sent may emulate data sent from ESa to ESb, the end stations are not, in fact, involved. The transmitting RBridge MUST wait for a reply frame until a time-out occurs. At that time, the RBridge SHOULD assume the frame was lost, and this SHOULD be indicated to the operator. The length of this time-out beyond the scope of this document. 4.1.2.1. Ping Example Figure 4 contains a campus with three RBridges. Consider a ping from RB0 to RB2. +-----+ +-------+ +-------+ +-------+ +-----+ | ESa +--+ RB0 +---+ RB1 +---+ RB2 +--+ ESb | +-----+ |ingress| |transit| |egress | +-----+ +-------+ +-------+ +-------+ Time RB0 RB1 RB2 . (1)-------> (1) -------> | . | <------- (2) <-------(2) Ping Example Topology Figure 4 In this diagram RB0 transmits frame (1) destined to RB2. This frame contains the echo request message. When RB1 receives this frame it forwards it to RB2. When RB2 receives this frame it transmits and echo reply frame (2) destined to RB0. RB1 receives this frame and forwards it to RB0. Below are some select fields for the frames: Bond & Manral Expires January 4, 2012 [Page 13] Internet-Draft RBridges: OAM Support July 2011 +--------+-------------+-------------+---------------+--------------+ | Frame | Ingress | Egress | TRILL OAM | Sequence | | # | RBridge | RBridge | Protocol | Number | +--------+-------------+-------------+---------------+--------------+ +--------+-------------+-------------+---------------+--------------+ | (1) | RB0 | RB2 | Echo Request | 1 | +--------+-------------+-------------+---------------+--------------+ | (2) | RB2 | RB0 | Echo Reply | 1 | +--------+-------------+-------------+---------------+--------------+ Table 4: Ping Example Frames For example, if the nicknames for RB0, RB1, and RB2 are 0x1111, 0x2222, and 0x3333 respectively, the console output from such a ping might be: Pinging -------------------------------------------- ... from 0x1111 to 0x3333... 0x3333 is alive ... from 0x1111 to 0x3333... 0x3333 is alive ... from 0x1111 to 0x3333... 0x3333 is alive Table 5: Ping Example Output In this example, the ping was repeated three times with the sequence number (not shown) being changed each time. 4.2. Error Reporting Errors can occur in received TRILL data frames. For this purpose, the error notification format is specified. These are generated due to various events as specified subsequently. When a TRILL data frame is received with an error, an error notification frame SHOULD be generated. See Section 10 for reasons some RBridges are allowed to choose not to respond to a request. The generated reply MUST contain the error notification. The sub-code MUST contain a code specifying the error encountered. The valid sub-code values are specified in Section 6.2.1. Two of these sub-codes provide for TLVs with additional information. The error notification also contains a 3 bit error type field which describes the error. This frame has a TRILL header and it contains, as its payload, the frame received with the error. If the size of the received frame would cause the generated frame to exceed 1470 bytes, the frame MUST be truncated to the 1470 bytes. The payload MUST include the TRILL header of the received frame and MUST NOT include the link-layer header. The generated reply MUST contain the error notification message specific to the error. Bond & Manral Expires January 4, 2012 [Page 14] Internet-Draft RBridges: OAM Support July 2011 When the original ingress RBridge receives the error frame, at a minimum, the RBridge SHOULD update a counter specifying the number of error frames received for the causing error. The encapsulated frame MUST NOT be egressed. Each RBridge SHOULD also keep a set of counters for errors reported by other RBridges. The two sub-codes that provide for TLVs with additional information are described below. All other sub-codes specified in this document do not contain TLVs. 4.2.1. Hop Count Zero Error When a TRILL data frame is received with a hop-count of zero, an error notification frame MUST be generated unless rate limiting or some particular difficulty, as described below stops the sending of such an error notification. The generated reply MUST contain the hop-count zero error sub-code. If the received frame has the echo request message, the hop-count zero error notification MUST have a sequence number matching the echo request. Otherwise, the sequence number MUST be set to zero. The incoming port ID MUST be the port ID the received frame arrived on. The outgoing port ID MUST be the port ID of the port the received frame would have been forwarded onto if the hop-count was not zero. Finally, the error notification MUST include a 16-bit nickname of the next hop RBridge the frame would have been sent to. If the request is a multi-destination frame, this field MUST be set to all zeros on transmission and ignored on reception. If the RBridge transmitting the request is the egress RBridge, this field MUST be set to 0x0000. 4.2.2. MTU Error When a TRILL data frame is received with a payload that would exceed the MTU of the port the frame would otherwise be forwarded to, an error notification frame MAY be generated. The generated reply MUST contain the MTU error sub-code. The outgoing port MTU field MUST have the MTU of the port the frame would have otherwise been transmitted on. The incoming port ID MUST be the port ID the received frame arrived on. The outgoing port ID MUST be the port ID of the port the received frame would have been forwarded onto if the frame size was not too large. Finally, the error notification message MUST include a 16-bit nickname of the next hop RBridge the frame would have been sent to. If this is a multi-destination frame this field MUST be set to all zeros on transmission and ignored on reception. If the RBridge transmitting the request is the egress RBridge, this field MUST be set to 0x0000. Bond & Manral Expires January 4, 2012 [Page 15] Internet-Draft RBridges: OAM Support July 2011 5. RBridge Channel Message Format This section specifies the format of the TRILL OAM payload after the RBridge Channel header and values of the fields in the RBridge Channel Header [RBridgeChannel]. 5.1. RBridge Channel Header and Sequence Number The RBridge Channel Header [RBridgeChannel] fields and flags and following sequence number are as follows: o CHV (Channel Header Version) is zero. o Protocol code values are: * 0x004 (Suggested): Echo * 0x005 (Suggested): Error Notification o Flags: The SL and NA bits SHOULD be zero, the MH bit SHOULD be one o ERR: The ERR field MUST be zero. o Sequence Number: For the Echo and Error Notification protocols, the RBridge Channel Header is always followed by a nibble sub- protocol identifier (SPID) and a 28-bit Sequence Number. This 28- bit field is used to sequence or match frames for certain uses. The SPID is used to provide additional op-code room for a protocol to further multiplex its messages. Not all TRILL OAM messages utilize the sequence number field or the SPID. 6. OAM Protocol Formats The formats of Echo Request, Echo Reply, and Error Notification OAM Messages are given below. 6.1. Protocol Application Codes Formats Bond & Manral Expires January 4, 2012 [Page 16] Internet-Draft RBridges: OAM Support July 2011 6.1.1. Echo Request | 0| 1| 2| 3| 4| 5| 6| 7| 8| 9|10|11|12|13|14|15| +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | RBridge Channel | | Header | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | SPID | Sequence | | Number | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ Echo Request Figure 5 This message is used by ingress RBridges to request an echo reply from the egress RBridge. Further uses are specified in Section 4.1.1 and Section 4.1.2 o SPID: The SPID MUST be zero to indicate an echo request o Sequence Number: An arbitrary 28-bit unsigned integer used to aid in matching reply messages to echo requests. It MAY be zero. 6.1.2. Echo Reply | 0| 1| 2| 3| 4| 5| 6| 7| 8| 9|10|11|12|13|14|15| +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | RBridge Channel | | Header | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | SPID | Sequence | | Number | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ . . . TLVs . . . +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ Echo Reply Format Figure 6 This message is used by egress RBridges to reply to an echo request Bond & Manral Expires January 4, 2012 [Page 17] Internet-Draft RBridges: OAM Support July 2011 from the ingress RBridge. Further uses are specified in Section 4.1.1 and Section 4.1.2. o SPID: The SPID MUST be one to indicate an echo reply o Sequence Number: A 28-bit unsigned integer used to aid in matching reply messages to echo requests. Set to the sequence number field of the Echo Request that cause this Echo Reply. o TLVs: A set of type, length, value encoded fields as specified in Section 7. The next hop nickname, outgoing port ID, and incoming port ID TLVs are required. 6.2. Error Notification Format | 0| 1| 2| 3| 4| 5| 6| 7| 8| 9|10|11|12|13|14|15| +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | RBridge Channel | | Header | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | SPID | Sequence | | Number | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Err. T.| Subcode | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ . . . TLVs . . . +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ Error Format Figure 7 This message is used by RBridges to signal that an error has been detected. o SPID: The SPID MUST be set to all zeros on transmission and is ignored on reception. It is unused by the error notification protocol. o Sequence Number: For all sub-codes except for the hop count error this field is unused. It is set to zero on transmission and ignored on reception. For the hop count error this is a 28-bit unsigned integer used to aid in matching reply messages to echo requests requests. If the frame whose hop-count dropped to zero Bond & Manral Expires January 4, 2012 [Page 18] Internet-Draft RBridges: OAM Support July 2011 contains the echo request message (See Section 6.1.1), this MUST match the sequence number Echo Request found in that message. If this is not in reply to an Echo Request, then the sequence number MUST be set to zero. o Error Type: MUST be a specifier of the error type describing the error. The values are: 0 (Permanent Error), 1 (Transient Error), 2 (Warning), 3 (Comment). Values 4 through 7 are available for allocation by IETF Review. o Subcode: MUST be a specifier of the error discovered in the frame. The valid values are specified in Section 6.2.1 o TLVs: A set of type, length, value encoded fields as specified in Section 7. For next hop errors the next hop nickname, outgoing port ID, and incoming port ID TLVs MUST be present. For MTU errors the outgoing port MTU, next hop nickname, outgoing port ID, and incoming port ID TLVs MUST be present. For all other errors the TLVs are not used by default and the length of this section is set to zero. An RBridge MAY include additional TLVs on any error however. 6.2.1. Error Specifiers The sub-code values fall into three categories: errors, warnings, and comments. All sub-codes represent something out of the ordinary that has gone wrong, but certain ones are more important than others. Sub-codes that are classified as errors are the most severe with warning sub-codes being less severe. These are enabled by default. Sub-codes classified as comments are minor and are disabled by default. They may be useful for operators debugging a network. All error generations are optional and therefore MAY be generated or not generated depending on security and implementation constraints. The error specifiers sub-code values are: Error Sub-codes o 0: Unknown Error: Indicates an error has occurred. o 1: Corrupt Frame: Frame received with invalid FCS or that was not an 8-bit multiple in length. It could be impossible for a device to signal this if the low-level port hardware hides this from the software. o 2: Invalid Outer.MacDA: Indicates the MAC Address is a multicast address and the M bit is zero, the MAC Address is not a multicast address and the M bit is one, or the M bit is zero and the frame Bond & Manral Expires January 4, 2012 [Page 19] Internet-Draft RBridges: OAM Support July 2011 carried is an ESADI frame. o 3: Illegal Outer.VLAN: Indicates the Outer.VLAN ID is 0xFFF. o 4: Invalid Outer.VLAN: Indicates the Outer.VLAN ID was not the designated VLAN ID. o 5: Unknown TRILL Version: Indicates the TRILL Header Version is unknown. o 6: Op-Length Exceeds Frame Length: Indicates the Op-Length says the options field extends beyond the end of the received frame length. o 7: Unknown Egress RBridge: Indicates the Egress RBridge in a received frame is unknown. o 8: Unknown Ingress RBridge: Indicates the Ingress RBridge in a received frame is unknown. (RBridges are not required to test for this error.) o 9: Unsupported Critical Hop-by-hop Option: Indicates an unsupported critical hop-by-hop option was received. o 10: Unsupported Critical Ingress-to-Egress Option: Indicates an unsupported critical ingress-to-egress option was received. o 11: Hop Count Zero: Indicates a frame exceeded its hop count in transit. (Used for pings and traceroute.) o 12-84: Available for allocation by IETF Review o 85: Reserved for Private Experimentation Warning Sub-codes o 86: Illegal Inner.VLAN: Indicates the Inner.VLAN ID is 0xFFF. o 87: Inner/Outer VLAN Priority Mismatch: Indicates the priority values in the inner and outer VLANs do not match. o 88: P2P Hello on TRILL Hello Link: Indicates a P2P Hello was received on a TRILL Hello Link. o 89: TRILL Hello on P2P Hello Link: Indicates a TRILL Hello was received on a P2P Hello Link. Bond & Manral Expires January 4, 2012 [Page 20] Internet-Draft RBridges: OAM Support July 2011 o 90: No Adjacency: Indicates a TRILL data frame was sent from an RBridge the receiving RBridge is not adjacent to. (RBridges MAY be configured to accept such frames in which case this is not an error.) o 91: Encapsulated Layer 2 Control Frame: A TRILL Data Frame containing a Layer 2 Control frame with a destination MAC in the range 01-80-C2-00-00-00 to 01-80-C2-00-00-0F or a 01-80-C2-00-00-21 (VRP) frame was received. o 92: Invalid Mutability Flag: Indicates the mutability flag was set on a received CHbH Option. o 93: Invalid TLV Option Length: Indicates the option length field of a TLV option was between 121 and 127. o 94: Options Ordering Error: Indicates the TLV options are ordered incorrectly. o 95: Configured Nickname Collision: Indicates an RBridge was detected in the campus with the same nickname (Configured or not). o 96: Multiple appointed forwarders detected. o 97-169: Available for allocation by IETF Review o 170: Reserved for Private Experimentation Comment Sub-codes o 171: Inner.VLAN C-Bit Set: Indicates the C-Bit in the Inner.VLAN is set. o 172: Unknown Inner.MacDA: Indicates the Inner.MacDA is unknown. This may occur if devices are configured to explicitly register end stations and an unknown Inner.MacDA occurs in a unicast TRILL data frame. This also only applies at egress and could indicate that the Inner.MacDA was a learned address that has timed out. o 173: Unknown Inner.MacSA: Indicates the Inner.MacSA is unknown. This may occur if devices are configured to explicitly register end stations and an unknown Inner.MacSA occurs in a TRILL data frame. o 174: Outer.VLAN C-Bit Set: Indicates the C-Bit in the Outer.VLAN is set for an Ethernet frame. Bond & Manral Expires January 4, 2012 [Page 21] Internet-Draft RBridges: OAM Support July 2011 o 175: Invalid Reserved Bits: Indicates the reserved bits are non- zero in a received frame. o 176: Invalid Nickname: Indicates a nickname in the reserved space of 0xFFC1 to 0xFFFF was received that is not implemented at the receiving RBridge. o 177: Unsupported Non-Critical Hop-by-hop Option: Indicates an unsupported non-critical hop-by-hop option was received. While sending a non-critical option to an unsupported device is not an error, this could be used to support identification of devices needing an upgrade. o 178: Unsupported Non-Critical Ingress-to-Egress Option: Indicates an unsupported non-critical ingress-to-egress option was received. While sending a non-critical option to an unsupported device is not an error, this could be used to support identification of devices needing an upgrade. o 179: Performance Exceeded: Indicates a frame was discarded due to performance problems such as a buffer overflow. o 180-254: Available for allocation by IETF Review o 255: Reserved for Private Experimentation 7. Type, Length, Value (TLV) Encodings To facilitate future interoperable expansion of the data carried in OAM sub-messages some sub-messages use a TLV encoding. These TLV sections consist of a list of type, length, value encoded data where the type signals to the RBridge how to interpret the value, and the length tells the RBridge the length of the value in bytes. The type and length are both 8 bit fields. A length of zero indicates the value is a UTF-8 string with a NULL ('\0') terminating byte. Preceding the list of TLVs is a 16 bit total length field which specifies the total length of all the length fields in octet units. TLVs with an unknown Type may be ignored and skipped over. Bond & Manral Expires January 4, 2012 [Page 22] Internet-Draft RBridges: OAM Support July 2011 | 0| 1| 2| 3| 4| 5| 6| 7| 8| 9|10|11|12|13|14|15| +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Total Length | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ . . . TLV List . . . +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ TLVs Format Figure 8 Each TLV in the TLV List appears on the wire encoded as follows: | 0| 1| 2| 3| 4| 5| 6| 7| 8| 9|10|11|12|13|14|15| +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Type | Length | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ . . . Value . . . +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ TLV Format Figure 9 The type values are: o 0: Next Hop Nickname, See Section 7.1 o 1: Outgoing Port ID, See Section 7.3 o 2: Incoming Port ID, See Section 7.2 o 3: Outgoing Port MTU, See Section 7.4 o 4: IS-IS System ID, See Section 7.5 o 5-253: Available for allocation by IETF Review o 254: Reserved for Private Use Bond & Manral Expires January 4, 2012 [Page 23] Internet-Draft RBridges: OAM Support July 2011 o 255: Reserved 7.1. Next Hop Nickname | 0| 1| 2| 3| 4| 5| 6| 7| 8| 9|10|11|12|13|14|15| +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Type = 0x00 | Length = 0x02 | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Next Hop Nickname | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ Next Hop Nickname Format Figure 10 For traceroutes targeting known unicast destinations, hop-count errors, and MTU errors, this TLV MUST be a 16-bit nickname of the next hop RBridge the frame is being or would have been sent to. If the RBridge transmitting the TLV is the egress RBridge this field MUST be set to 0x0000. For traceroutes targeting multi-destination destinations, e.g. with the TRILL M bit high, this field contains a nickname of the RBridge the frame being responded to is from. For pings, this field MUST be set to all zeros on transmission and ignored on reception. For multi-destination hop-count errors this field contains a nickname of the RBridge the frame with the exceeded hop-count was sent from. For multi-destination MTU error traffic, this field MUST be set to all zeros on transmission and ignored on reception. 7.2. Incoming Port ID | 0| 1| 2| 3| 4| 5| 6| 7| 8| 9|10|11|12|13|14|15| +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Type = 0x01 | Length = 0x02 | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Incoming Port ID | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ Incoming Port ID Format Figure 11 This TLV MUST be set to the Port ID found in 'The Special VLANs and Flags sub-TLV' for the port the request being replied to was received Bond & Manral Expires January 4, 2012 [Page 24] Internet-Draft RBridges: OAM Support July 2011 on [RFCtisis]. 7.3. Outgoing Port ID | 0| 1| 2| 3| 4| 5| 6| 7| 8| 9|10|11|12|13|14|15| +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Type = 0x02 | Length = 0x02 | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Outgoing Port ID | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ Outgoing Port ID Format Figure 12 This TLV MUST be set to the Port ID found in 'The Special VLANs and Flags sub-TLV' for the port the frame is being forwarded on to (or would have been for an echo request/hop-count error). [RFCtisis] If the request was consumed by the replying RBridge, the port ID MUST be 0xFFFF. 7.4. Outgoing Port MTU | 0| 1| 2| 3| 4| 5| 6| 7| 8| 9|10|11|12|13|14|15| +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Type = 0x03 | Length = 0x02 | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Outgoing Port MTU | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ Outgoing Port MTU Format Figure 13 This TLV MUST be the MTU of the outgoing port specified in the outgoing port ID TLV. Bond & Manral Expires January 4, 2012 [Page 25] Internet-Draft RBridges: OAM Support July 2011 7.5. ISIS System ID | 0| 1| 2| 3| 4| 5| 6| 7| 8| 9|10|11|12|13|14|15| +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Type = 0x04 | Length = 0x06 | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | | | ISIS System ID | | | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ ISIS System ID Format Figure 14 This TLV MUST be the ISIS System ID of the system generating the message. This TLV MAY be included in all/any error messages. 8. Acknowledgments Many people have contributed to this work, including the following, in alphabetic order: Sam Aldrin, Dinesh Dutt, Donald E. Eastlake 3rd, Anoop Ghanwani, Jeff Laird, and Marc Sklar 9. IANA Considerations IANA is request to create a new subregistry within the TRILL Parameter registry for "TRILL OAM Message Error Sub-Message Error Specifiers". This subregistry that is initially populated as specified in Section 6.2.1. Additional values are allocated by IETF Review [RFC5226]. IANA is requested to create a new subregistry within the TRILL Parameter registry for "TRILL Error Reporting Protocol TLV Types" with initial values as listed in Section 5.3. Additional values are allocated by IETF Review [RFC5226]. This draft also requires action to reserve the TRILL RBridge Channel protocol codes. IANA is requested to allocate the TRILL RBridge Channel protocol codes for as listed in Section 5.1. 10. Security Considerations The nature of the OAM Messages can lead to security concerns. By providing information about the topology and status of a network, attackers can gain greater knowledge of a network in order to exploit Bond & Manral Expires January 4, 2012 [Page 26] Internet-Draft RBridges: OAM Support July 2011 the network. Passive attacks such as reading frames with an OAM message could be used to gain such knowledge or active attacks where an attacker mimics an RBridge can be used to probe the network. Authentication, data integrity, protection against replay attacks, and confidentiality for TRILL OAM frames may be provided using a to- be-specified TRILL Security Option. Using such a security option would mitigate both the passive and active attacks. For instance, data origin authentication could be provided in the future using a security options in the TRILL Header by verifying a hash using shared keys or a mechanism like SEND with CGA. To prevent replay attacks rate limiting, sequence numbers as well as some nonce based mechanism could be provided. Confidentiality for TRILL OAM frames could be provided based on some future security option extension which encypts TRILL frames. In a network where one does not wish to configure a security option, the threat of attackers is still present. For this reason, generation of any TRILL OAM Message frames is optional and SHOULD be configurable by an operator on a per RBridge basis. An RBridge MAY have this configurable on a per port basis. For instance, an operator SHOULD be able to disable hop-count traceroute reply messages or error notification message generation per port. Another security threat is denial of service through use of OAM messages. For this reason, RBridges MUST rate limit the generation of OAM message frames. For multi-destination frames, the frames MAY be discarded silently to prevent any denial of service attacks in case of an errored packet such as an 'options not recognized' error notification. 11. References 11.1. Normative References [RBridgeChannel] Eastlake, D., Manral, V., Yizhou, L., Aldrin, S., and D. Ward, "RBridges: TRILL RBridge Channel Support", draft-ietf-trill-rbridge-channel-01 (work in progress), June 2011. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu, D., and S. Mansfield, "Guidelines for the Use of the "OAM" Acronym in the IETF", BCP 161, RFC 6291, June 2011. Bond & Manral Expires January 4, 2012 [Page 27] Internet-Draft RBridges: OAM Support July 2011 [RFCtrill] Eastlake, D., Dutt, D., Gai, S., Ghanwani, A., and R. Perlman, "Rbridges: Base Protocol Specification", draft-ietf-trill-rbridge-protocol-16 (work in progress), March 2010. 11.2. Informative References [IEEE.802-1ag] Institute of Electrical and Electronics Engineers, "IEEE Stadard for Local and metropolitian area networks / Virtual Bridged Local Area Networks / Connectivity Fault Management", IEEE Standard 802.1Q, December 2007. [IS-IS] International Organization for Standardization, "Intermediate system to intermediate system intra- domain-routing routine information exchange protocol for use in conjunction with the protocol for providing the connectionless-mode Network Service (ISO 8473)", ISO/IEC 10589:2002, Second Edition, Nov 2002. [RBridgeMIB] Rijhsinghani, A. and K. Zebrose, "Definitions of Managed Objects for RBridges", draft-ietf-trill-rbridge-mib-02 (work in progress), March 2011. [RFC0792] Postel, J., "Internet Control Message Protocol", STD 5, RFC 792, September 1981. [RFC4443] Conta, A., Deering, S., and M. Gupta, "Internet Control Message Protocol (ICMPv6) for the Internet Protocol Version 6 (IPv6) Specification", RFC 4443, March 2006. [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008. [RFC5837] Atlas, A., Bonica, R., Pignataro, C., Shen, N., and JR. Rivers, "Extending ICMP for Interface and Next- Hop Identification", RFC 5837, April 2010. [RFC6165] Banerjee, A. and D. Ward, "Extensions to IS-IS for Layer-2 Systems", RFC 6165, April 2011. [RFCtisis] Eastlake, D., Banerjee, A., Dutt, D., Perlman, R., and A. Ghanwani, "TRILL Use of IS-IS", Bond & Manral Expires January 4, 2012 [Page 28] Internet-Draft RBridges: OAM Support July 2011 draft-ietf-isis-trill-05 (work in progress), February 2011. Authors' Addresses David Michael Bond University of New Hampshire InterOperability Laboratory 121 Technology Drive Suite #2 Durham, New Hampshire 03824 US Phone: +1-603-339-7575 EMail: mokon@mokon.net URI: http://mokon.net Vishwas Manral Hewlett-Packard Co. 19111 Pruneridge Ave. Cupertino, CA 95014 US Phone: 408-447-0000 EMail: vishwas.manral@hp.com Bond & Manral Expires January 4, 2012 [Page 29]