TRILL Working Group Donald Eastlake 3rd INTERNET-DRAFT Stellar Switches Intended status: Proposed Standard Manoj Wadekar Updates: RFCtrill QLogic Anoop Ghanwani Brocade Expires: February 16, 2011 August 17, 2010 RBridges: Support of IEEE 802.1Qbb, 802.1Qaz, and 802.1Qau Abstract IEEE 802.1 is developing standards as part of its Data Center Bridging (DCB) activity that amend the IEEE 802.1Q standard. These include 802.1Qau, 802.1Qaz, and 802.1Qbb. The intent of these three standards is (1) to efficiently minimize data loss due to queue overflow for selected classes of traffic within Local Area Networks (LANs) meeting certain conditions and (2) to provide means to allocate the available bandwidth to different classes of traffic. IEEE 802.1 is specifying theses standards and the behavior needed to support them in bridges and end stations. This document briefly explains the standards and specifies the implementation of these standards for RBridges. Status of This Document This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Distribution of this document is unlimited. Comments should be sent to the TRILL working group mailing list. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html D. Eastlake, M. Wadekar, A. Ghanwani [Page 1] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support Table of Contents 1. Introduction............................................3 1.1 Overview of These Standards............................3 1.2 Terminology............................................5 1.3 Additional Acronyms....................................5 2. Priority-Based Flow Control.............................6 3. Enhanced Transmission Selection.........................7 4. The DCB Exchange Protocol...............................7 5. Congestion Notification.................................8 5.1 Congestion Notification Domains........................9 5.2 Congestion Notification Tag Details...................11 5.3 Congestion Notification Message Details...............11 5.4 Additions to TRILL for Congestion Notification........12 5.4.1 RBridge Ingress Details.............................13 5.4.2 Transit RBridge Details.............................16 5.4.2.1 Transit RBridge Input Port........................16 5.4.2.2 Transit RBridge Output Port.......................16 5.4.3 RBridge Egress Details..............................17 6. Management Considerations..............................18 7. IANA Considerations....................................18 8. Security Considerations................................18 9. References.............................................19 9.1 Normative References..................................19 9.2 Informative References................................19 D. Eastlake, M. Wadekar, A. Ghanwani [Page 2] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support 1. Introduction IEEE 802.1 is developing standards that amend IEEE [802.1Q] as part of its Data Center Bridging (DCB) activity. These include 802.1Qau, 802.1Qaz, and 802.1Qbb. The intent of these three standards is (1) to efficiently eliminate data loss due to queue overflow for selected classes of traffic within Local Area Networks (LANs) having a limited delay bandwidth product and meeting other conditions and (2) to provide limited means to allocate the available bandwidth to different classes of traffic. Intended uses include the support of loss sensitive services, such as Fiber Channel over Ethernet [FCoE], in data centers. The existing optional PAUSE feature of IEEE 802.3 (Annex 31B of [802.3]) can, with appropriate engineering, provide Ethernet service without loss of frames due to queue overflow. However, PAUSE has problems as follows: 1. Traffic for some protocols, for example TCP, require frame losses to signal congestion for flow control. Elimination of frame drops due to congestion would prevent TCP flow control, unless some other mechanism were added. 2. Some traffic comprises time critical network control frames, for example BPDUs. PAUSE is relatively indiscriminant and pauses such frames, except for some MAC Control frames such as PAUSE frames themselves, along with pausing the loss sensitive multi-hop traffic that we are primarily worried about. This might compromise continued network connectivity. 3. PAUSE can result in intermittent waves of spreading traffic paralysis, crippling network throughput, as follows: When a switch S1 receives a PAUSE on a port P1 and can no longer transmit frames out that port it is likely that output queues to P1 will fill up quickly. When that happens, to avoid frame loss, S1 must send PAUSE frames out on each of its ports that might receive a frame for output to P1. For example, it might have to PAUSE input on both P2 and P3, unnecessarily blocking traffic between those two ports, to be sure it will not receive input on either of them for P1. This can repeat in switches connected to S1, switches connected to those switches, etc., 1.1 Overview of These Standards Overviews of the three DCB standards documents are given below. IEEE 802.1 is specifying theses standards and the behavior needed to support them in bridges and end stations. This document specifies the implementation of these standards for RBridges [RFCtrill]. D. Eastlake, M. Wadekar, A. Ghanwani [Page 3] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support IEEE [802.1Qbb], Priority-based Flow Control (PFC), refines the Ethernet PAUSE feature in a priority based fashion as described in Section 2. If a switch implements separate queues for different priorities at each port, this can eliminate the first and second problems above. Traffic requiring frame drops due to congestion can be assigned a priority for which PFC is not enabled. And pause is not normally enabled for the two highest priorities, 6 and 7, which are typically used for time sensitive control frames. PFC also reduces the third problem as any congestion spreading would only affect the priority or priorities with PFC enabled. IEEE [802.1Qaz] is a standards document that actually covers two things. One, Enhanced Transmission Selection (ETS), allocates bandwidth between traffic class groups indicated by priority. It is described in Section 3. The second, 802.1Qaz, contains the specification of the Data Center Bridging Exchange Protocol (DCBX) for discovering and configuring the three standards that this document covers, as described in Section 4. IEEE [802.1Qau], Congestion Notification, enables switches to manage congestion by signaling congestion on a per flow basis to end stations. As a part of the standard, participating end stations are required to implement per flow rate limiting that typically doesn't exist in currently deployed end stations. 802.1Qau is enabled on a per priority basis and, with appropriate engineering, minimizes frame drops due to queue overflow in a LAN Congestion Notification Domain within which all switches and end stations implement it. Thus 802.1Qau provides a complement to the 802.1bb Priority-Based Flow Control, for helping eliminate such frame drops. 802.1Qau tries to reduce congestion by proactively reducing frame ingress rates at the source end station(s). For some congestion cases this may be insufficient to stop buffer overflow at the congestions point. PFC provides an emergency brake for such cases and avoids frame loss. 802.1Qau eliminates the first problem listed above for PAUSE in that frames that require congestion drops can be assigned a priority for which 802.1Qau is not enabled. It avoids the second problem because it is not normally used to limit priorities 6 and 7, which are typically used for time sensitive control frames. And it avoids the third problem listed above for PAUSE because it acts by restraining end station flow sources rather than blocking transmission on intermediate switch ports. Section 5 below provides additional information on 802.1Qau and specifies additions to the TRILL protocol to support it. These three DCB standards may be implemented independently or in any combination except that implementation of any of them implies implementation of DCBX, specified in IEEE 802.1Qaz. D. Eastlake, M. Wadekar, A. Ghanwani [Page 4] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support 1.2 Terminology The terminology and acronyms of [RFCtrill] are used in this document. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 1.3 Additional Acronyms The following acronyms are used in this document in addition to those defined in [RFCtrill]. AVB - Audio-Visual Bridging CNM - Congestion Notification Message CNtag - Congestion Notification tag DCB - Data Center Bridging DCBX - DCB Exchange protocol ETS - Enhanced Transmission Selection (IEEE 802.1Qaz) FCoE - Fiber Channel over Ethernet LLDP - Link Layer Discovery Protocol (IEEE 802.1AB) PFC - Priority-based Flow Control (IEEE 802.1Qbb, 802.3bd) PHY - Physical layer D. Eastlake, M. Wadekar, A. Ghanwani [Page 5] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support 2. Priority-Based Flow Control IEEE [802.1Qbb], Priority-Based Flow Control (PFC), refines the IEEE 802.3 PAUSE feature to permit separately requesting, on a physical link, pausing and unpausing the traffic of each of the eight available frame priority levels. The actual priority-based pause Ethernet control frame is being specified in [802.3bd]. Such queue pausing occurs within the transmission logic associated with a port and requires no changes to the TRILL protocol, which is implemented above such port logic, as described in [RFCtrill]. LLDP/DCBX is used in PFC discovery and agreement with peers as described in Section 4. A station implementing the PFC standard MUST implement DCBX, signaling PFC support and configuration. Guarantee of lossless handling of frames with a particular priority in an RBridge campus requires implementation and enablement of PFC for that priority at all end stations that originate frames and all RBridges and bridges in that campus as well as meeting the PFC engineering requirements specified in [802.1Qbb]. The PFC control frames specified in 802.3bd are MAC control frames that are not VLAN tagged. Their transmission normally bypasses the output queue at a port so they are transmitted immediately, or as soon as the frame currently being transmitted is sent, so as to meet the timing requirements of PFC. D. Eastlake, M. Wadekar, A. Ghanwani [Page 6] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support 3. Enhanced Transmission Selection Enhanced Transmission Selection (ETS), specified in IEEE [802.1Qaz], allocates bandwidth, between traffic classes, through each of the ports of a switch or end station. (To be more precise, it modifies the algorithm used to select, from multiple priority-based output queues at a port, the next frame to transmit. Provision is made for proprietary algorithms and 802.1 is specifying an algorithm in connection with precise frame timing (AVB), but we are only concerned with the default DCB algorithm.) Transmission selection occurs within the logic associated with a port and requires no changes to the TRILL protocol, which is implemented above such port logic, as described in [RFCtrill]. An RBridge implementing the ETS standard MUST implement DCBX (see Section 4) signaling of ETS support and configuration. For ETS to be effective, traffic in different ETS groups cannot share an output queue. 4. The DCB Exchange Protocol The DCB Exchange Protocol (DCBX) is specified in IEEE [802.1Qaz], which also specifies ETS as described in Section 3. DCBX is built on the Link Layer Discovery Protocol (LLDP), which is specified in IEEE [802.1AB]. DCBX is used for the discovery of DCB capabilities of peer switches, for the detection of inconsistent configuration of DCB features between peer switches, and for the propagation of DCB features to switches configured to accept configuration via DCBX. For purposes of TRILL protocol peering, RBridges ignore intervening bridges, but for the purposes of LLDP and DCBX all stations, including RBridges, 802.1 bridges, and end stations are considered peers. RBridges implementing any of the three DCB protocols MUST also implement LLDP and DCBX. D. Eastlake, M. Wadekar, A. Ghanwani [Page 7] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support 5. Congestion Notification Congestion Notification, IEEE [802.1Qau], can limit flows to minimize frame loss by having congestion points that detect congestion and send congestion notification messages back to reaction points in end stations that can limit flows. See [802.1Qau] for the specification of the algorithms to perform at congestion and reaction points. Congestion Notification is designed to operate best in minimizing frame loss of unicast flows in a LAN composed of point-to-point physical links where all switches have implemented Congestion Notification. An RBridge may act as an end point, for example when sourcing or sinking SNMP management frames, and thus may contain one or more reaction points, as well as congestion points at its output queues, if it implements Congestion Notification. Reaction points are in end stations where flows originate and are the mechanism to limit flows. The granularity of reaction points is beyond the scope of IEEE 802.1Qau and this document but cannot be larger than a priority within an end station. If the granularity is smaller and there are multiple reaction points in an end station for a given priority, then the end station must label outgoing frames with a Congestion Notification tag (CNtag) that includes an end station flow ID. (This flow ID is an opaque field to the rest or the network.) Reaction points are typically implemented within the native frame origination logic of an end station. +-----------------------------------------------+ | Ethernet Header (possibly including VLAN Tag) | +-----------------------------------------------+ | Optional CNtag | +-----------------------------------------------+ | Ethernet Payload | +-----------------------------------------------+ | Ethernet FCS | +-----------------------------------------------+ Figure 1: Native Ethernet Frame in a Congestion Notification Domain Congestion points are at queues in forwarding devices, normally port output queues. The functions of a congestion point are (1) to conditionally send Congestion Notification Messages (CNMs) to the source of a frame and (2) in the normal case where they are at an output port, to conditionally strip Congestion Notification tags (CNtags) out of a frame being forwarded. When a frame is to be inserted into an output queue with a congestion point, the procedures specified in IEEE 802.1Qau are used to D. Eastlake, M. Wadekar, A. Ghanwani [Page 8] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support determine if a CNM should be sent to the frame's source and if so to determine various fields in that CNM. When a frame is to be inserted into an output queue with a congestion point, the congestion point may remove any CNtag in the frame as discussed in Section 5.1. Congestion points are implemented within the logic associated with a port and require no changes to the TRILL protocol for the output of native frames, as TRILL is implemented above such port logic as described in [RFCtrill]; however, when outputting a TRILL data frame, any CNM generated needs to be for the TRILL encapsulated frame rather than for the entire TRILL data frame and the CNM needs to be TRILL encapsulated. +-----------------------------------------------+ | Ethernet Header (possibly including VLAN Tag) | +-----------------------------------------------+ | CNtag | +-----------------------------------------------+ | Congestion Notification Message Fixed Fields | + - - - - - - - - - - - - - - - - - - - - - - -+ | Initial bytes of frame causing CNM | +-----------------------------------------------+ | Ethernet FCS | +-----------------------------------------------+ Figure 2: Native CNM Within a contiguous part of the campus where Congestion Notification is enabled (see Section 5.1), you would see the same frames with the same tags as in a similar bridged LAN except that those frames will be TRILL encapsulated as shown in Figures 3 and 4. The exception is when a TRILL-ignorant bridge within the campus produces a CNM in response to a TRILL data frame as shown in Figure 6. The resulting CNM is adjusted by the first RBridge it encounters, which will be the previous-hop RBridge. 5.1 Congestion Notification Domains IEEE 802.1Qau reduces frame drops due to output queue overflow in a Congestion Notification Domain. There could be many such domains, each limited to a specific priority value and contiguous set of network stations (end stations, RBridges, or 802.1 bridges), within an RBridge campus. For example, two Congestion Notification Domains, one at priority X and one at priority Y, could cover the same set of contiguous stations, overlapping but different sets of such stations, or completely disjoint sets of such stations, in a campus. D. Eastlake, M. Wadekar, A. Ghanwani [Page 9] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support IEEE 802.1Qau includes mechanisms to "defend" Congestion Notification Domains, that is, make sure only congestion managed flows of frames enter congestion point queues. The edge of a domain, i.e. the set of station ports in the domain connected by a physical link to a station not in the domain, is determined by a combination of auto-detection using LLDP (see Section 4) and management configuration. Bridges that implement Congestion Notification defend a domain by (1) prohibiting priority mapping inside the domain, (2) mapping the priority of any frame entering the domain from a station outside the domain to a priority that is not a congestion managed priority, and (3) prohibiting the mapping of the priority of any frame entering the domain from a station outside the domain to the domain's priority. Note that, because of item 2 in the previous paragraph, a station can be a member of no more than 7 different Congestion Notification Domains because there must be at least one priority that is not congestion managed for use as the mapped priority of entering frames from outside the domain and which are therefore not part of a congestion managed flow. As a practical matter, it is unlikely that a station would be a member of more than 4 or 5 different Congestion Notification Domains as priorities 6 and 7 are normally used for high priority control frames and are not congestion controlled and at least one low priority is kept not congestion managed for mapping as above. The per port per priority state of a switch or end station will be one of the following four values, which have the effects indicated: o Disabled: - On native frame input, frame priority can be mapped to or from this priority. - If this is an end-station output port, CNtags are not added. - If this is a switch output port, CNtags are not stripped. o Edge: - On frame input, a frame with this priority is mapped to a non- congestion control priority and no frame can be mapped to this priority, regardless of the priority-mapping table at the port. - If this is an end-station output port, CNtags are not added. - If this is a switch output port, CNtags are stripped. o Interior: - On frame input, a frame in this priority is not mapped to another priority and no frame can be mapped to this priority, regardless of the priority-mapping table at this port. - If this is an end-station output port, CNtags are not added. - If this is a switch output port, CNtags are stripped. D. Eastlake, M. Wadekar, A. Ghanwani [Page 10] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support o InteriorReady: - On frame input, a frame in this priority is not mapped to another priority and no frame can be mapped to this priority, regardless of the priority-mapping table at this port. - If this is an end-station output port, CNtags may be added. - If this is a switch output port, CNtags are not stripped. Note that when the priority of a TRILL encapsulated frame is mapped, this can only be done by changing the priority field in the Inner.VLAN tag. The Outer.VLAN tag priority is effectively discarded on frame receipt. 5.2 Congestion Notification Tag Details An end station originating a native frame may add a Congestion Notification tag (CNtag), if the end station and the next hop device are part of a Congestion Domain, to identify the reaction point in that end station that controls the end station flow to which that native frame belongs. A CNtag is 4 bytes long, consisting of a 2 bytes Ethertype (0x22E9) followed by a 2 bytes flow ID, and appears after any VLAN tag but before the frame body. The inclusion of a CNtag is optional as the originating end station may be able to identify the corresponding reaction point from other information returned in a Congestion Notification Message such as the priority. As described in Section 5.3, CNtags are always added to Congestion Notification Messages when they are created. 5.3 Congestion Notification Message Details A Congestion Notification Message (CNM) is, under certain circumstances, created by a congestion point, as described in IEEE 802.1Qau, when a frame is entered into the queue associated with that congestion point. The complete CNM frame always includes a Congestion Notification tag (CNtag, see Section 5.2). The CNtag includes a zero flow id if the frame provoking the CNM did not have a CNtag. The body of the CNM itself, after the CNtag, starts with the CNM Ethertype (0x22E7) followed by the information below: - CNM version information, currently zero - Quantized congestion feedback information as specified in IEEE 802.1Qau - An 8 byte opaque ID of the congestion point generating the CNM - The priority of the frame causing the CNM - The destination MAC address of the frame causing the CNM - The number of bytes included from the beginning of the body of D. Eastlake, M. Wadekar, A. Ghanwani [Page 11] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support the frame causing the CNM - The first up to 64 bytes of the body of the frame causing the CNM Except that input bytes/frame counters are not incremented, a CNM generated at an output queue for a port is treated as if it had been received on that port above the EISS. CNMs are considered to be in the same VLAN as the frame that provoked them and have configurable priority that defaults to priority 6. It is undesirable, but not an error, for a CNM to be sent in response to a CNM frame which encounters congestion. This is normally avoided by sending CNM frames with a priority which does not have congestion notification enabled. As described in Section 5.4.1.3 below, when a CNM is generated by an RBridge when queuing a TRILL data frame, it is generated for the enclosed frame, not for the entire TRILL data frame. This will cause the CNM to be addressed to the source end station of the data, not just a previous RBridge hop. 5.4 Additions to TRILL for Congestion Notification The figure below is used in the discussion in this section. The assumption is that a frame is generated at End Station "a" (ESa) destined for End Station "b" (ESb) and this frame is forwarded through the sequence of 802.1 bridges (Bn) and RBridges (RBn) shown. For native frames from ESa, RB1 acts as the ingress RBridge, encapsulating and directing them to egress RBridge RB3 for decapsulation and delivery to ESb. The arrows indicate the flow of a data frame. Any resulting CNM will flow in the opposite direction. +-----+ +-----+ +-----+ +-----+ | ESa +-->--+ B1 | + RB3 |-->--+ B3 + +-----+ +--+--+ +--+--+ +--+--+ | | | V ^ V | | | +--+--+ +-----+ +--+--+ +--+--+ | RB1 +-->--+ RB2 +-->--+ B2 + | ESb | +-----+ +-----+ +-----+ +-----+ Figure 5: Example Frame Path TRILL can make no difference to the actions at any reaction points in ESa or any congestion points at the output queues of B1, B2, or B3, since they are not RBridges, although any Congestion Notification Message (CNM) generated at B2 will be in response to a TRILL D. Eastlake, M. Wadekar, A. Ghanwani [Page 12] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support encapsulated native frame. The situation at the output queue of RB3 is actually the same as B3 since, as egress, RB3 will have decapsulated any traffic for ESb before it tries to insert it in an output queue. Thus the frame RB3 is enqueuing will be a native frame, a congestion point at the RB3 output can act, for such a frame, exactly as an 802.1 bridge congestion point, and any CNM generated in the RB3 output from that native frame will be treated as if it was received by the RB3 port. A CNM created at the RB1 or RB2 output queue is straightforward. Assume the CNM is created in response to TRILL frame 1 (TF1) and the TF1 encapsulates native frame 1 (NF1). The CNM would be created as a TRILL encapsulated CNM with the ingress RBridge of TF1 as its egress. The Inner.MacDA would be ESa. The Inner.MacSA would be the MAC address of the port on which the TRILL encapsulated CNM was initially sent, that is, the same as the Outer.MacSA. The encapsulated CNM itself would be filled in as if in response to NF1, not TF1. Similarly, a CNM created at B3 would have ESa as its destination address and would be encapsulated when it arrived at RB3 as RB3 would be its ingress RBridge. 5.4.1 RBridge Ingress Details This section specifies special actions for Congestion Notification at an RBridge input port receiving a native frame, that is, the RBridge ingress function. The usual 802.1Q processing on the priority of the input TRILL data frame, modified as described in Section 5.1, is done. Special actions are required only when the native frame received is a CNM. The ingress process at an RBridge, say RB2, supporting Congestion Notification MUST detect the case of a native CNM created by a bridge in response to a TRILL data frame, say by B2 in Figure 5, and transform it as described below. If such a CNM was generated in response to a TRILL control (IS-IS) frame, it is discarded. No other changes are needed in the RBridge ingress process. Such a native CNM requiring special actions is easily recognized as it's MAC destination address will be the RBridge and it will have the CNM Ethertype. (A CNM not addressed to the RBridge must have been generated in response to an unencapsulated native frame, for example at B3 in the diagram above, and can be encapsulated and generally forwarded by transit Rbridges in the same way as other native frame.) Such a native CNM resulting from a TRILL data frame at B2 has the contents generally shown in Figure 6 and listed further below. D. Eastlake, M. Wadekar, A. Ghanwani [Page 13] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support +-----------------------------------------------+ | Ethernet Header (possibly including VLAN Tag} | +-----------------------------------------------+ | CNtag | +-----------------------------------------------+ | Congestion Notification Message Fixed Fields | + - - - - - - - - - - - - - - - - - - - - - - -+ | Up to 64 initial bytes of the following: | | +-----------------------------------------+ | | | TRILL Ethertype and Header | | | +-----------------------------------------+ | | | Inner Ethernet Header incl. VLAN Tag | | | +-----------------------------------------+ | | | Optional CNtag | | | +-----------------------------------------+ | | | Ethernet Payload | | | +-----------------------------------------+ | | | +-----------------------------------------------+ | Ethernet FCS | +-----------------------------------------------+ Figure 6: Native CNM Caused by TRILL Data Frame 1 + Outer.MacDA, MAC address of RB2 2 + Outer.MacSA, MAC address of B2, the bridge generating this CNM 3 + Outer.VLAN tag for the designated VLAN on the RB2 to RB3 link with the priority configured at B2 for CNMs (default priority 6) 4 + CNtag (CNtag Ethertype 0x22E9 followed by Flow ID of zero) + CNM 5 o CNM Ethertype 0x22E7 6 o CNM version information, quantized congestion feedback information, and an 8 byte opaque ID of the congestion point generating the CNM 7 o The priority of the TRILL encapsulation frame causing the CNM 8 o The destination MAC address of the TRILL encapsulation frame causing the CNM, RB3 in this case 9 o The number of bytes included below from the beginning of the body of the TRILL encapsulation frame causing the CNM + Initial bytes of body of TRILL encapsulation frame causing the CNM o TRILL Header of the frame causing the CNM 10 - TRILL Ethertype 0x22F3 11 - Flags, hop count, options length 12 - Egress nickname, RB3 in this case 13 - Ingress nickname, RB1 in this case 14 - Options, if any 15 o Inner.MacDA, MAC address of ESb D. Eastlake, M. Wadekar, A. Ghanwani [Page 14] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support 16 o Inner.MacSA, MAC address of ESa 17 o Inner.VLAN tag of the TRILL encapsulated frame causing the CNM 18 o Optional CNtag 19 o Encapsulated native frame body The ingressing RBridge RB2 transforms this CNM above into the following TRILL encapsulated CNM. + Outer.MacDA, MAC address of next hop RBridge (RB1) toward originating end station + Outer.MacSA, MAC address of RB2 port on which this TRILL encapsulated CNM frame is to be sent + Outer.VLAN tag for the designated VLAN on the RB2 to RB1 link with priority copied from incoming Outer.VLAN, field #3 above + TRILL Header to get the CNM to the right end station o TRILL Ethertype 0x22F3 o Flags, hop count, options length o Egress nickname, RB1 in this case, from ingress nickname in the TRILL header in the received CNM, field #13 above o Ingress nickname, RB2 in this case, the nickname of the RBridge doing this transformation o Options, if any + Inner.MacDA, MAC address of ESa, field #16 above + Inner.MacSA, MAC address of B2, field #2 above + Inner.VLAN Tag with VLAN ID from field #17 above and priority from field #3 above + CNtag, with flow ID from field #18 above, if #18 is present, otherwise flow ID of zero + CNM o CNM Ethertype 0x22E7 o CNM version information, quantized congestion feedback information, and an 8 byte opaque ID of the congestion point generating the CNM, field #6 above o The priority of the native frame who's encapsulated form caused the CNM, from Inner.VLAN, field #17 above o The destination MAC address of the frame whose encapsulated form caused the CNM, the Inner.MacDA, field #15 above o The number of bytes included below from the beginning of the body of the frame whose encapsulated form caused the CNM. This will be 24 smaller (but not less than zero) than the same field (#9) in the CNM tag received due to dropping the TRILL Header (8 bytes), MAC addresses (12 bytes), and Inner.VLAN (4 bytes). + Initial bytes of the body of the frame whose encapsulated form caused the CNM, field #19 above Because of the reduction in the number of bytes of the body of the frame that would have caused the CNM if it weren't encapsulated, it is RECOMMENDED that bridges and RBridges implementing Congestion D. Eastlake, M. Wadekar, A. Ghanwani [Page 15] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support Notification in a campus be configured to include the maximum (64) number of bytes when generating a CNM. 5.4.2 Transit RBridge Details The subsections below describe transit RBridge support of Congestion Notification at input and output ports. As this is considering an RBridge in its transit role, only the handling of TRILL data frames is discussed. If the RBridge is receiving a native frame, it will be an ingress as described in Section 5.4.2 and if it is sending a native frame, it will be an egress as described in Section 5.4.3. However, this section does apply to the output of an encapsulated frame that was ingressed at an RBridge and to the input, in TRILL encapsulated form, of a frame to be egressed at the RBridge. 5.4.2.1 Transit RBridge Input Port The usual 802.1Q processing on the priority of the input TRILL data frame, modified as described in Section 5.1, is done. 5.4.2.2 Transit RBridge Output Port As discussed in Section 5.1, a CNtag is stripped under some circumstances; however, such a CNtag will appear as part of the encapsulated frame, not on the outside of the TRILL data frame, so the CNtag is stripped from deeper in the frame. When there is a Congestion Point enabled at an RBridge output queue a CNM is not generated as the result of trying to queue a TRILL control (IS-IS) frame for output at an RBridge. A TRILL encapsulated CNM is generated in response to a TRILL data frame, when to do so is specified by 802.1Qau, composed as below. The TRILL data frame causing the CNM is referred to as TF1 and its encapsulated native frame as NF1. + Outer.MacDA - MAC address of the next hop RBridge towards the egress nickname used in the TRILL Header (see below) + Outer.MacSA - MAC address of the output port on which the TRILL encapsulated CNM is to be sent + Outer.VLAN - Designated VLAN of the link on which the TRILL encapsulated CNM is to be sent + TRILL Header o TRILL Ethertype 0x22F3 o Flags, hop count, options length o Egress nickname, from ingress nickname in TF1 o Ingress nickname, a nickname of the RBridge generating the D. Eastlake, M. Wadekar, A. Ghanwani [Page 16] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support CNM o Options, if any + Inner.MacDA - set to the Inner.MacSA of TF1, that is, the source MAC address of NF1 + Inner.MacSA - same as Outer.MacSA of TF1 + Inner.VLAN - same as the Inner.VLAN of TF1, that is, the VLAN tag of NF1 + CNtag - with flow ID from the CNtag of NF1 or zero if NF1 did not have a CNtag + CNM - message generated for NF1 5.4.3 RBridge Egress Details After decapsulation, processing of the decapsulated native frame is the same as at an 802.1 bridge output port. As discussed in Section 5.1, any CNtag present is stripped under some circumstances. If the output queue is congested, then a native CNM will be generated in response to the decapsulated native frame. This native CNM will then be treated as if it had been received on the port. D. Eastlake, M. Wadekar, A. Ghanwani [Page 17] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support 6. Management Considerations ---TBD--- 7. IANA Considerations This document requires no IANA actions. This section should be deleted by the RFC Editor before publication. 8. Security Considerations See [RFCtrill] for general RBridge Security Considerations. ---more TBD--- D. Eastlake, M. Wadekar, A. Ghanwani [Page 18] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support 9. References Normative and informational references for this document are given below. 9.1 Normative References [802.1AB] - IEEE, "IEEE Standard for Local and metropolitan area networks / Station and Media Access Control Connectivity Discovery", IEEE 802.1AB-2009, 17 September 2009. [802.1Q] - IEEE, "IEEE Standard for Local and metropolitan area networks / Virtual Bridged Local Area Networks", IEEE 802.1Q-2005, 19 May 2006. [802.1Qau] - "IEEE Standard for Local and metropolitan area networks / Virtual Bridged Local Area Networks / Amendment 13: Congestion Notification", IEEE 802.1Qau-2010, 23 April 2010. [RFC2119] - Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 [RFCtrill] - R. Perlman, D. Eastlake, D. Dutt, S. Gai, and A. Ghanwani, "RBridges: Base Protocol Specification", draft-ietf- trill-rbridge-protocol-16.txt, in RFC Editor queue. 9.2 Informative References [802.1Qaz] - IEEE, "Draft Standard for Local and Metropolitan Area Networks / Virtual Bridged Local Area Networks / Amendment XX: Enhanced Transmission Selection for Bandwidth Sharing Between Traffic Classes", Work in Progress, 4 August 2010. [802.1Qbb] - IEEE, "Draft Standard for Local and Metropolitan Area Networks / Virtual Bridged Local Area Networks / Amendment: Priority-based Flow Control", Work in Progress, 25 May 2010. [802.3] IEEE, "IEEE Standard for Information technology / Telecommunications and information exchange between systems / Local and metropolitan area networks / Specific requirements Part 3: Carrier sense multiple access with collision detection (CSMA/CD) access method and physical layer specifications", IEEE 802.3-2008, 26 December 2008. D. Eastlake, M. Wadekar, A. Ghanwani [Page 19] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support [802.3bd] - IEEE, "Draft Standard for Information technology / Telecommunications and information exchange between systems / Local and Metropolitan Area Networks / Specific requirements Part 3: Carrier Sense Multiple Access with Collision Detection (CSMA/CD) Access Method and Physical Layer Specifications / Amendment: MAC Control Frame for Priority-based Flow Control", Work in Progress, 14 July 2010. [FCoE] - http://fcoe.com/ D. Eastlake, M. Wadekar, A. Ghanwani [Page 20] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support Authors' Addresses Donald Eastlake 3rd Stellar Switches 155 Beaver Street Milford, MA 01757 USA Tel: +1-508-333-2270 Email: d3e3e3@gmail.com Manoj Wadekar QLogic Corporation 26650 Aliso Viejo Pkwy Aliso Viejo, CA 92656 USA Tel: +1-949-389-6000 Email: manoj.wadekar@qlogic.com Anoop Ghanwani Brocade Communications Systems 1745 Technology Drive San Jose, CA 95110 USA Phone: +1-408-333-7149 Email: anoop@brocade.com D. Eastlake, M. Wadekar, A. Ghanwani [Page 21] INTERNET-DRAFT RBridges: Qbb, Qaz, & Qau Support Copyright, Disclaimer, and Additional IPR Provisions Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the BSD License. The definitive version of an IETF Document is that published by, or under the auspices of, the IETF. Versions of IETF Documents that are published by third parties, including those that are translated into other languages, should not be considered to be definitive versions of IETF Documents. The definitive version of these Legal Provisions is that published by, or under the auspices of, the IETF. Versions of these Legal Provisions that are published by third parties, including those that are translated into other languages, should not be considered to be definitive versions of these Legal Provisions. For the avoidance of doubt, each Contributor to the IETF Standards Process licenses each Contribution that he or she makes as part of the IETF Standards Process to the IETF Trust pursuant to the provisions of RFC 5378. No language to the contrary, or terms, conditions or rights that differ from or are inconsistent with the rights and licenses granted under RFC 5378, shall have any effect and shall be null and void, whether published or posted by such Contributor, or included with or in such Contribution. D. Eastlake, M. Wadekar, A. Ghanwani [Page 22]