Network Working Group R. Stewart Internet-Draft Cisco Systems, Inc. Expires: April 1, 2003 October 1, 2002 Stream Control Transmission Protocol (SCTP) Remote Direct Memory Access (RDMA) Direct Data Placement (DDP) Adaption draft-stewart-rddp-sctp-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http:// www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on April 1, 2003. Copyright Notice Copyright (C) The Internet Society (2002). All Rights Reserved. Abstract This document describes a method to adapt Direct Data Placment (DDP) and Remote Direct Memory Access (RDMA) to Stream Control Transmission Protocol (SCTP) RFC2960 [2] using a generic description found in [RDMA-Draft] [4] and [DDP-Draft] [3] This adaption provides a method for two peers to know that each side is performing DDP or RDMA thus enabling hardware acceleration if available. Stewart Expires April 1, 2003 [Page 1] Internet-Draft SCTP RDMA/DDP Adaptation October 2002 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Adaptation Layer Indicator . . . . . . . . . . . . . . . . . . 4 3. Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.1 Association Initialization . . . . . . . . . . . . . . . . . . 5 3.2 RDMA and DDP Placement behavior with SCTP . . . . . . . . . . 6 4. IANA considerations . . . . . . . . . . . . . . . . . . . . . 7 5. Security Considerations . . . . . . . . . . . . . . . . . . . 8 6. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 9 References . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Author's Address . . . . . . . . . . . . . . . . . . . . . . . 10 Full Copyright Statement . . . . . . . . . . . . . . . . . . . 11 Stewart Expires April 1, 2003 [Page 2] Internet-Draft SCTP RDMA/DDP Adaptation October 2002 1. Introduction This document describes a method to adapt Direct Data Placment (DDP) and Remote Direct Memory Access (RDMA) to Stream Control Transmission Protocol (SCTP) RFC2960 [2] using a generic description found in [RDMA-Draft] [4] and [DDP-Draft] [3] This adaption provides a method for two peers to know that each side is performing DDP or RDMA thus enabling hardware acceleration if available. Some implementations may include this adaptation layer within their SCTP implementations to obtain maximum performance but the behavior of SCTP will be unaffected. In order to accomplish this we specify the use of the new adaptation layer indication as defined in [ADDIP- Draft] [6] 1.1 Conventions The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL, when they appear in this document, are to be interpreted as described in RFC2119 [1]. Stewart Expires April 1, 2003 [Page 3] Internet-Draft SCTP RDMA/DDP Adaptation October 2002 2. Adaptation Layer Indicator We define a adaption indication which MUST appear in the INIT or INIT-ACK with the following format as defined in [ADDIP-Draft] [6] 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type =0xC006 | Length = Variable | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Adaptation Indication | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Adaptation Indication: The following value is defined for DDP in this document: RDMA - 0x00000001 RDMA+DDP - 0x00000002 Stewart Expires April 1, 2003 [Page 4] Internet-Draft SCTP RDMA/DDP Adaptation October 2002 3. Procedures 3.1 Association Initialization At the startup of an association, an endpoint wishing to perform DDP, RDMA, or DDP+RDMA placement MUST include an adaptation layer indication in its INIT or INIT-ACK (as defined in 2.1). After the exchange of the first two messages (INIT and INIT-ACK), an endpoint MUST verify and inspect the adaptation indication and compare it to the following table to determine proper action. Indication | Action type | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | This indicates that the peer DOES NOT NONE | support ANY DDP or RDMA adaption and thus | RDMA and DDP procedures MUST NOT be | performed upon this association. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | This indicates that the peer DOES support RDMA | RDMA (but not DDP). Procedures outlined in | [RDMA-Draft] MUST be followed. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | This indicates that the peer supports BOTH DDP+RDMA | RDMA and DDP. If the receiving endpoint | indicated the same, then the procedures in | both [RDMA-Draft] and [DDP-Draft] | MUST be followed. If the local endpoint only indicated | RDMA, then ONLY the procedures in | [RDMA-Draft] MUST be followed. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | This indicates that the peer DOES NOT ANY-OTHER | support ANY DDP or RDMA adaption and thus Indication | RDMA and DDP procedures MUST NOT be | performed upon this association. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Editors Notes/Questions: 1. Do we need to have a control channel, stream 0 for instance? 2. Do we need to dictate control messages that can be sent back to negotiate different aspects? 3. If we do this do we even need DDP, just have RDMA and then negotiate things from there? 4. Do we need to better describe how to use unordered data with the Stewart Expires April 1, 2003 [Page 5] Internet-Draft SCTP RDMA/DDP Adaptation October 2002 TSN and the CUM-ACK out of the sockets-api? If the endpoints agree to use either DDP or RDMA each endpoint MUST enable the SCTP_DISABLE_FRAGMENTS option as defined in [Sctpsockets- Draft] [5]. 3.2 RDMA and DDP Placement behavior with SCTP When RDMA or DDP is used in combination with SCTP no further procedures are needed other than both endpoints knowing that specialized placement is occurring. SCTP, once the SCTP_DISABLE_FRAGMENTS option is set, will always assure that EACH send message fits in one complete SCTP chunk. The receiver will NOT need to be concerned with message boundaries and loss events since each SCTP DATA chunk will always have the BE bits set. The placement algorithm will only need to look into the bytes described in their respective drafts and follow those procedures to perform RDMA or DDP. If the sender specifies Unordered delivery (by setting the U bit) the receiver, after placement, MAY notify the receiving application that the data has arrived. The cumulative TSN and TSN values defined in [Sctpsockets-Draft] [5] can be used to determine if data is ready to be delivered to upper layer protocols. Stewart Expires April 1, 2003 [Page 6] Internet-Draft SCTP RDMA/DDP Adaptation October 2002 4. IANA considerations This document defines two new Adaptation Layer Indication codepoints: RDMA - 0x00000001 RDMA+DDP - 0x00000002 Stewart Expires April 1, 2003 [Page 7] Internet-Draft SCTP RDMA/DDP Adaptation October 2002 5. Security Considerations Any direct placement of memory poses a significant security risk, these threats should be addressed in the appropriate DDP [DDP-Draft] [3] or RDMA [RDMA-Draft] [4] drafts. This document does not add any additional security risks over those found in RFC2960 [2]. Stewart Expires April 1, 2003 [Page 8] Internet-Draft SCTP RDMA/DDP Adaptation October 2002 6. Acknowledgments The author would like to thank the following people that have provided comments and input Stephen Bailey, David Black, Caitlin Bestler,Douglas Otis, and Allyn Romanow. Stewart Expires April 1, 2003 [Page 9] Internet-Draft SCTP RDMA/DDP Adaptation October 2002 References [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [2] Stewart, R., Xie, Q., Morneault, K., Sharp, C., Schwarzbauer, H., Taylor, T., Rytina, I., Kalla, M., Zhang, L. and V. Paxson, "Stream Control Transmission Protocol", RFC 2960, October 2000. [3] Culley, P., "Direct Data Placement over Reliable Transports", draft-shah-iwarp-ddp-00 (work in progress), September 2002. [4] Recio, R., Garcia, D. and P. Culley, "An RDMA Protocol Specification", draft-recio-iwarp-rdma-00 (work in progress), September 2002. [5] Stewart, R., "Sockets API Extensions for Stream Control Transmission Protocol", draft-ietf-tsvwg-sctpsocket-05 (work in progress), September 2002. [6] Stewart, R., "Stream Control Transmission Protocol (SCTP) Dynamic Address Reconfiguration", draft-ietf-tsvwg-addip-sctp- 06 (work in progress), September 2002. Author's Address Randall R. Stewart Cisco Systems, Inc. 8725 West Higgins Road Suite 300 Chicago, IL 60631 USA Phone: +1-815-477-2127 EMail: rrs@cisco.com Stewart Expires April 1, 2003 [Page 10] Internet-Draft SCTP RDMA/DDP Adaptation October 2002 Full Copyright Statement Copyright (C) The Internet Society (2002). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. Stewart Expires April 1, 2003 [Page 11]