Douglas Otis INTERNET-DRAFT SANlight (Expires March 21, 2001) FC over SCTP/IP (FC/SCTP/IP) Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC 2026 [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as Reference material or to cite them other than as "work in progress". The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/lid-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Copyright (C) The Internet Society (2000). All Rights Reserved. Abstract This specification incorporates an encapsulation scheme for Fibre- Channel (FC) using SCTP. The essential advantages for using SCTP compared to TCP as it relates to FC are as follows: - Headers contained within one frame. - Objects aligned at 32-bit boundaries. - Out of sequence frame processing. - Standard authentication. - Independent streams under common control. - Session restart. - Improved error detection. - Prevention of blind spoofing and denial of service attacks. - Standard Heartbeat and multi-homing. (optional) The principle focus of this document is to provide an implementation of SCTP as an encapsulation with network related management required for FC traffic. As FC traffic is predominately related to storage, functions in this area effect storage as well as a possible means of direct IP access. Stream filtering to block non-storage related traffic is an example of a storage specific function covered. Normative References: RFC 826, Ethernet Address Resolution Protocol(ARP) RFC 792, Internet Control Message Protocol (ICMP) RFC 2132, DHCP Options and BOOTP Vendor Extensions (DHCP-BOOTP) RFC 2589, Lightweight Directory Access Protocol (LDAPv3) Internet Draft, 21-Sep-00 FC/SCTP/IP [Page 2] RFC 1157, Simple Network Management Protocol (SNMP) RFC 1907, Management Information Base for SNMP (SNMPv2-MIB) RFC "wip", Stream Control Transmission Protocol (SCTP) RFC "wip", LDAP structures for FC/SCTP/IP (LDAP/FC) Definitions Fibre-Channel (FC) Frames are encapsulated as Data Payload chunks within the SCTP protocol. See FC Frame Format for a definition of the FC delimiter encoding. The Payload Protocol Identifier is determined by IANA. All Payload Data types will be sent Unordered. Stream Management Stream 0 is reserved as a control stream that exchanges configuration structures to assign FC port and attributes to streams within Data Payload Chunks. These structures are as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Request Tag |Revision Major |Revision Minor | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Management Request | Management Response | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Fig. 1 Management Request/Response Header The Request Tag is assigned by the client to track a Management Response. The Revision Major indicates compatibility level, whereas, the Revision Minor indicates additional features. The Management Request header and arguments are sent by the client and echoed in Response. The Management Response field is first set to zero by the client. Management Request: - 0 = Report supported Revisions - 1 = Assign Stream - 2 = Release Stream Management Response: - 0 = Good - 1 = Invalid Request - 2 = Unavailable Resource - 3 = Busy - 4 = Refused - 5 = Revision Conflict Report Supported Revisions: Nothing other than the header is sent on the request. The response is a list of highest to lowest with the last revision being 0.0, which is mandatory. Internet Draft, 21-Sep-00 FC/SCTP/IP [Page 3] 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Revision Major |Revision Minor |(Revision Major|Revision Minor)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | (additional revisions) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Fig. 2 Revision Response Assign Stream: This request is to obtain a stream connected to a physical port. This request also provides an advisory priority desired. It can also assert a time limit for dropping stale frames. The response will return the request with the Management Response indicating acceptance and if Good is returned, the Physical Port field will be modified to indicate the Stream assigned otherwise it will be set to zero. Priority and Time Limit may be modified by the server to reflect any adjustments. Should the Time Limit be zero, the Timestamp field is not valid and no stale frames are dropped. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Physical Port/Stream | Priority | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time Limit (milli-seconds) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Fig. 3 Assign Stream 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Stream | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Fig. 4 Release Stream Authentication Management Authentication management is facilitated with LDAP/FC. Agents implementing FC/SCTP/IP are intended to use LDAP to obtain the required information for establishing communications. These structures extend into stream management attributes in some cases. Advisory Devices implementing FC/SCTP/IP will also implement ARP, ICMP and optionally DHCP-BOOTP, and SNMP. Only FC Class 3 protocol is supported. As such, SOFi3, SOFn3, EOFn EOFt delimiters are expected. The Exchange Error Policy is "Abort, discard multiple sequences." To send SCSI commands, the R_CTL field is set with 0x6 to indicate an unsolicited command, 0x1 to indicate solicited data. R_CTL 0x5 indicates Data Descriptor, and 0x7 indicates Response. The TYPE field with 0x8 indicates SCSI-FCP. The S_ID identify the initiator and D_ID the target. Every exchange Internet Draft, 21-Sep-00 FC/SCTP/IP [Page 4] between the initiator and target is uniquely identified by the initiator with the Originator Exchange ID, OX_ID, with a value other than 0xFFFF. OX_ID is analogous to the Queue Tag but the number of open sequences is limited by the SEQ_ID which is assigned by the sequence initiator to a unique number to identify the sequence. SEQ_CNT uniquely identifies FC frames within a sequence and the first frame always starts at 0. A target may assign a RX_ID to an exchange as an alias for S_ID and OX_ID but not used by the initiator. The Frame Control field, F_CTL, contains state information: - bit 8 = From Responder - bit 9 = Recipient/Initiator Sequence Context - bit 10 = First Sequence - bit 11 = Last Sequence - bit 12 = Last Frame of Sequence - bit 13-14 = don't care - bit 15 = Transfer/Hold Sequence Initiative - bit 16-17 = don't care - bit 18-21 = reserved - bit 22-27 = don't care - bit 28 = Parameter equals Relative Offset (set by target) - bit 29 = reserved - bit 30-31 = Number of fill bytes. The DF_CTL specifies the presents of optional header and should be set to 0. The 4 byte CRC field does not include the SOF and EOF fields and uses the polynomial: x32+x26+x23+x22+x16+x12+x11+x10+x8+x7+x5+x4+x2+x+1 FC Frame Format Encapsulation: The exact size of each frame varies depending on the size of the variable fields. The size of the variable field ranges from 0 to 2112-bytes as shown in the FC Frame Format in Fig. 5 resulting in the minimum size FC Frame of 36 bytes and the maximum size FC frame of 2148 bytes. The FC Frame will be prefixed with 8 bytes of additional information to assist the SCTP transport as a Data Payload chunk as follows: Internet Draft, 21-Sep-00 FC/SCTP/IP [Page 5] 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Timestamp (milli-seconds) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | OSS |N|C| Tcredit | SOF | EOF | >+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+< | R_CTL | D_ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved | S_ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TYPE | F_CTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SEQ_ID | DF_CTL | SEQ_CNT | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | OX_ID | RX_ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Parameter | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | (Optional Header) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | (Payload) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CRC | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Fig. 5 Encapsulated FC Frame Ordered Set Sequences, OSS, are detected between FC frames and are indicated within this field. Should SOF and EOF have the value of zero, the remainder of the frame is not valid and may be truncated. Such a truncated field will not affect credit/debit accounting. The N bit indicates the CRC field is not valid and must be constructed before passing to FC. The C bit indicates login credits are asserted by the server. This bit is returned by the client as acknowledgement. Tcredit indicates either return of a used credit or possibly additional credits for active streams where additional buffers are available. Should the C bit be set by the server, then this field indicates the current debit while de-allocating any additional credits that have been granted. The client does not set Tcredit. The credit is established as Buffer-to-Buffer credits during login. The SCTP server agent may add additional credits via TCredit. The remaining portion of this chunk contains the FC Frame. The Frame Header is 24-bytes long and has several fields associated with the identification and control of the payload. Internet Draft, 21-Sep-00 FC/SCTP/IP [Page 6] +------+--------+-----------+----//-------+------+------+ | SOF |Frame |Optional | Frame | CRC | EOF | | (4B) |Header |Header | Payload | (4B) | (4B) | | |(24B) |<----------------------->| | | | | | Data Field = (0-2112B) | | | +------+--------+-----------+----//-------+------+------+ Fig. 6 FC Frame Format SOF and EOF Delimiters: On a FC link, SOF and EOF are called Ordered Sets and are sent as special out-of-band words constructed from the 10-bit comma character (K28.5) followed by 3 additional 10-bit data characters. On a non-Fibre Channel link the Start of Frame (SOF) and End of Frame (EOF) delimiters are both byte-encoded and 4-bytes long. On a FC link the SOF delimiter serves to identify the beginning of a frame and prepares the receiver for frame reception. The SOF contains information about the frame's Class of Service, position within a sequence, and in some cases, connection status. The EOF delimiter identifies the end of the frame and the final frame of a sequence. In addition, it serves to force the running disparity to negative. The EOF is used to end the connection in connection-oriented classes of service. It is therefore important to preserve the information conveyed by the delimiters across the IP-based network, so that the receiving FC to IP device can correctly construct the FC frame in its original SOF and EOF format before forwarding it to its ultimate FC destination on the FC link if acting as a bridge. Start of Frame (SOF) and End of Frame (EOF) byte-encoding are defined in Annex A. The SOF and EOF codes are encoded as a single byte to represent each FC Ordered Set. Frame Header: The Frame Header is 24-bytes long and has several fields that are associated with the identification and control of the payload. Current FC Standards allow up to 3 Optional Header fields: - Network Header (16-bytes) - Association Header (32-bytes) - Device Header (up to 64-bytes) Frame Payload: The FC Frame Payload is transparent to the FC to IP device. An FC application level payload is called an Information Unit at the FC-4 Level. This is mapped into the Frame Payload of the FC Frame. A large Information Unit is segmented using a structure consisting of FC Sequences. Typically, a Sequence consists of more than one FC frame. FC to IP does not maintain any state information regarding the relationship of frames within a FC Sequence. Internet Draft, 21-Sep-00 FC/SCTP/IP [Page 7] CRC: The CRC is 4-bytes long and uses the same 32-bit polynomial used in FDDI and is specified in ANSI X3.139 Fiber Distributed Data Interface. When FC frames are encapsulated into IP packets, the CRC is untouched. APPENDIX A: Fibre Channel EOF and SOF Encoding A.1 Ordered Sets On a FC link, Ordered Sets (OS) are sent as special out-of-band words constructed of the 10-bit comma character (K28.5) followed by 3 additional 10-bit data characters. The Ordered Sets defined by FC include the Frame Delimiter, Start of Frame (SOF) and End of Frame (EOF), and other Primitive Signals. When FC frames are encapsulated in an IP packet, the Byte-encoded frame format is used. The Byte-encoded frame format uses 32-bit OS Code Words to represent valid FC frame delimiter. This format uses a single-byte OS Code to represent each FC Ordered Set. For frame delimiters, the encoding is mapped using the least significant hexadecimal digit of the second character added to the forth character which is insensitive to disparity. FC frame delimiters shall be encoded in the format shown in Table below. Internet Draft, 21-Sep-00 FC/SCTP/IP [Page 8] A.2 Encoded FC Frame Delimiters K28.5, Daa.b, Dcc.d, Dcc.d +-----------------+----------------------------------------------+ | OS-Code (hex) | Delimiter Name | +-----------------+----------------------------------------------+ | 0x1C | SOFc1 Connect Class 1 | +-----------------+----------------------------------------------+ | 0x5C | SOFi1 Initiate Class 1 | +-----------------+----------------------------------------------+ | 0x3C | SOFn1 Normal Class 1 | +-----------------+----------------------------------------------+ | 0x5A | SOFi2 Initiate Class 2 | +-----------------+----------------------------------------------+ | 0x3A | SOFn2 Normal Class 2 | +-----------------+----------------------------------------------+ | 0x5B * | SOFi3 Initiate Class 3 (Initialize Loop)| +-----------------+----------------------------------------------+ | 0x3B * | SOFn3 Normal Class 3 | +-----------------+----------------------------------------------+ | 0x1E | SOFc4 Activate Class 4 | +-----------------+----------------------------------------------+ | 0x5E | SOFi4 Initiate Class 4 | +-----------------+----------------------------------------------+ | 0x3E | SOFn4 Normal Class 4 | +-----------------+----------------------------------------------+ | 0x5D | SOFf Fabric | +-----------------+----------------------------------------------+ | 0x7A * | EOFt Terminate | +-----------------+----------------------------------------------+ | 0x9A | EOFdt Disconnect-Terminate | +-----------------+----------------------------------------------+ | 0xFA | EOFa Abort | +-----------------+----------------------------------------------+ | 0xDA * | EOFn Normal | +-----------------+----------------------------------------------+ | 0x9F | EOFdti Disconnect Terminate-Invalid | +-----------------+----------------------------------------------+ | 0xDF | EOFni Normal Invalid | +-----------------+----------------------------------------------+ | 0xBA | Idle | +-----------------+----------------------------------------------+ | 0x4A | R_RDY Receiver Ready | +-----------------+----------------------------------------------+ * Indicates used for Class 3 SCSI-FCP Internet Draft, 21-Sep-00 FC/SCTP/IP [Page 9] A.3 Encoded Primitive Sequences K28.5, Daa.b, Dcc.d, Dee.f +-----------------+---------------------------------+ | OSS (hex) | Delimiter Name | +-----------------+---------------------------------+ | 0x0 | No Sequence | +-----------------+---------------------------------+ | 0x1 | NOS Not Operational | +-----------------+---------------------------------+ | 0x2 | LRR Link Reset Response | +-----------------+---------------------------------+ | 0x3 | LR Link Reset | +-----------------+---------------------------------+ | 0x4 | OLS Offline | +-----------------+---------------------------------+ Author's Address: Douglas Otis SANlight Inc. 160 Saratoga Ave, #46 Santa Clara, CA 95051 Tel: (408) 260-1400 x2001 dotis@sanlight.net Full Copyright Statement Copyright (C) The Internet Society (2000). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.