Network Working Group J. Pelissier INTERNET-DRAFT Sanera Systems Expires October, 2001 P. Grun Intel Corporation A. Murching Microsoft Corporation R. Recio IBM April 2001 IP and ARP over InfiniBand(TM) Architecture Status of this Memo This document is an Internet-Draft and is subject to all provisions of Section 10 of RFC2026 [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.html. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) The Internet Society (2001). All Rights Reserved. Abstract This memo defines an initial application of IP and ARP in InfiniBand(TM) Architecture (IBA) network. IBA itself describes a link, network, and transport layer. In this application, these IBA layers act as the link layer for IP with the specifics of IBA being largely transparent to IP. Pelissier, et. al. [Page 1] IP and ARP over InfiniBand(TM) Architecture April, 2001 This memo specifies the method for encapsulating an IP datagram within an IBA packet and the method for resolving IP addresses to the node identifying components included in IBA headers. This memo introduces general IBA technology and nomenclature. Readers are encouraged to review the specifications and other supporting material available from the InfiniBand(SM) Trade Association at their website www.infinibandta.org. Comments Comments should be sent to the IP over IB working group mailing list (ipoverib@mailbag.intel.com) or to the authors. Acknowledgments The authors gratefully acknowledge the technical contribution and review of many members of the InfiniBand(SM) Trade Association. 1. Conventions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [2]. 2. Introduction The goal of this specification is to allow compatible and interoperable implementations for transmitting IP datagrams and IBA Address Resolution Protocol (IBARP) requests and replies encapsulated in IBA unreliable datagrams. IBA provides an unreliable datagram service that is similar to the delivery service provided by many other LAN technologies such as Ethernet. IBA also provides other reliable and unreliable services; however, their applicability to IP is outside the scope of this memo. An IBA unreliable datagram packet contains the following headers: o Local Route Header (LRH) - provides routing information for relay of an IBA packet within an IBA subnet. o Global Route Header (GRH) - provides routing information for relay of an IBA packet between IBA subnets. o Base Transport Header (BTH) - provides various information for the IBA transport services. This header is what identifies the packet and an IBA unreliable datagram. Pelissier, et. al. [Page 2] IP and ARP over InfiniBand(TM) Architecture April, 2001 o Datagram Extended Header (DETH) - provides additional IBA specific information unique to IBA datagrams. The IBA link level address space provides for approximately 48k unicast addresses and 16k multicast addresses. The IBA network layer (i.e. the relay enabled by the IBA GRH) provides for 128-bit addresses similar to those in IPv6 [3]. 3. Encapsulation of IP datagrams in IBA Unreliable Datagrams The figure below illustrates the format of an IBA Unreliable Datagram. +-----+----+-----+------+---------+-----+------+ | LRH |GRH | BTH | DETH | Payload |ICRC | VCRC | | 8B |40B | 12B | 8B | 0-4096B | 4B | 2B | +-----+----+-----+------+---------+-----+------+ IP datagrams SHALL be encapsulated in IBA unreliable datagrams in the payload portion as illustrated below: +----------+----------+------------+ | Reserved |Ethertype |IP Datagram | | 2B | 2B | 0-4092B | +----------+----------+------------+ The Reserved field shall be set to 0x0000. Ethertype SHALL be set to 0x0800. 3.1 Byte Order As described in Appendix B of the Internet Protocol Specification [4], the IP datagram SHALL be transmitted over the IBA network as a series of 8 bit bytes, most significant byte first (also known as big endian byte ordering). 3.2 Maximum Transmission Unit IBA supports a number of permissible MTU sizes. The methods in this memo assume and are limited to IBA paths that support 2048 byte or 4096 byte MTU. The default MTU size for IP packets on IBA is 2048 octets. This size may be reduced by a Router Advertisement [5] containing an MTU option that specifies a smaller MTU, or by manual configuration of each node. If a Router Advertisement is received on an IBA interface has an MTU option specifying an MTU larger than 2048, or larger than a manually configured value, that MTU option MAY be logged to system management but MUST otherwise be ignored. MTU size information Pelissier, et. al. [Page 3] IP and ARP over InfiniBand(TM) Architecture April, 2001 received from DHCP is considered "manually configured" for the purpose of the preceding requirement. 4. Stateless Autoconfiguration (IPv6) The Interface Identifier [3] is formed from the EUI-64 portion of the IBA interface's GID at GID index 0. The "Universal/Local" (U/L) bit, which is the next-to-lowest order bit of the first octet of the EUI- 64 is complemented. (The U/L bit in an EUI-64 has the opposite meaning of that in an IPv6 address. The U/L bit set to 0 in an EUI-64 indicates universally unique; the U/L bit set to 1 has the same meaning in an IPv6 address. Hence, the bit must be complemented. 4.1 Link-Local Address (IPv6) The IPv6 link-local address [3] for an IBA interface is formed by appending the Interface Identifier previously defined to FE80::/64. 5. Address Mapping - Unicast (IPv6) The procedure for mapping an IPv6 unicast address into an IBA link- layer address is described in [6]. The link-layer address has the following format when the link layer is IBA: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Length | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | EUI-64[63-32] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | EUI-64[31-0] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved8 | QPN | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | QKey | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | GID[127-96] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | GID[127-96] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | GID[95-64] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | GID[63-32] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | GID[31-0] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Pelissier, et. al. [Page 4] IP and ARP over InfiniBand(TM) Architecture April, 2001 Option Fields: Type 1 for Source Link-layer address. 2 for Target Link-layer address. Length 5 (in units of 8 octets) EUI-64 EUI-64 portion of the GID at GID index 0 for the IBA interface. QPN Queue Pair Number that the IBA interface uses for IP traffic QKey QKey to be used with the above Queue Pair. GID A GID assigned to the IBA interface. 6. Address Resolution (IPv4) IBA address resolution (IBARP) is modeled closely after that for Ethernet [6]. The same general process is used with the hardware address field being modified appropriately for IBA. Ethernet requires only a MAC address to properly target a node with IP traffic. Unlike Ethernet, IBA has several parameters that are required to properly target a node for IP traffic. These are: EUI-64: A globally unique, time invariant identifier that identifies the individual IBA port. Specifically, this is the EUI-64 portion of the port's IBA Global Identifier at index location zero. LID: The locally (i.e. IBA subnet local) unique, time variant identifier of the IBA port. This is the identifier that IBA switches use for relaying IBA packets. GID: The unique, time variant identifier of the IBA port. This is the identifier that IBA routers used for relaying IBA packets. Like IPv6 addresses, the extent to which a GID is unique is limited by IBA scope rules. Q_Key: The key that is provided by recipients of IBA unreliable datagrams to initiators of IBA unreliable datagrams that authorizes access to the queue pair that is to receive the IBA unreliable datagrams. QPN: The (or one of many) queue pair number that a receiving port is using for IP unicast traffic. Pelissier, et. al. [Page 5] IP and ARP over InfiniBand(TM) Architecture April, 2001 6.1 IBARP General Operation A device wishing to resolve an IP address to an IBA hardware address constructs an IBARP request packet and multicasts it to all of the devices on the IP subnet. For the purposes of IBARP, the device sending the IBARP request is referred to as the sender. The IBARP packet contains various parameters (defined in detail in the following sections of this memo) including the IP address for which hardware address resolution is desired and identification of the sender. All IP devices on the IP subnet examine the IBARP request packet to determine if the target IP address within the packet matches one of their IP addresses. If so, the device that matches generates an IBARP response. For the purposes of IBARP, the device with the matching IP address is referred to as the target. The IBARP response is sent unicast to the sender and contains the hardware address of the target (again, the details are defined in the following sections of this memo). Any device that receives an IBARP request (regardless of whether or not the device was the target device) MAY use the hardware address included in the IBARP request associated with the sender for future IP communication with the sender. 6.2 IBARP Multicast Paramaters Several parameters are required in order to construct the IBARP multicast packet required by the sender of an IBARP message. These parameters are: LRH: SL, DLID GRH: TClass, FlowLabel, HopLmt, DGID BTH: PKey DETH: QKey These parameters are obtained using the IBA Service ID Resolution Protocol (SIDR) [7]. SIDR provides specific fields for the establishment of an IBA unreliable datagram flow and provides a private data area for additional information that may be required. The fields covered by the specific data are QPN and QKey. The remaining fields must be provided in the private data area. Pelissier, et. al. [Page 6] IP and ARP over InfiniBand(TM) Architecture April, 2001 The private data area in SIDR responses for the IBARP service SHALL be formatted as follows: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Ver | SL | TClass | DLID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | HopLmt | Resv. | FlowLabel | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Resv. | PKey | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | QKey | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | DGID[127-96] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | DGID[95-64] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | DGID[63-32] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | DGID[31-0] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | / Reserved (180 Bytes) / | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The Ver field SHALL be set to 1. 6.3 IBARP Packet Format IBARP packets (both request and reply) SHALL be formatted as follows: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | / IBA Headers / | (72 bytes if IBA global - 32 bytes if IBA local) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | / IBA Payload / IBARP contents / | (92 bytes) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | / IBA CRCs / | (6 bytes) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Pelissier, et. al. [Page 7] IP and ARP over InfiniBand(TM) Architecture April, 2001 The format of the IBA headers is as follows (for an IBA globally routed packet - for local routing remove the fields between IPVer and DGID, inclusive): +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | VL | LVer | SL | R2|LNH| DLID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Resv5 | PktLen | SLID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPVer | TClass | FlowLabel | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PayLen | NxtHdr | HopLmt | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SGID[127-96] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SGID[95-64] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SGID[63-32] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SGID[31-0] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | DGID[127-96] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | DGID[95-64] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | DGID[63-32] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | DGID[31-0] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | OpCode |S|M|Pad| TVer | PKey | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved8 | DestinationQP | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |A| Reserved7 | PSN | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | QKey | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved8 | SourceQP | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Pelissier, et. al. [Page 8] IP and ARP over InfiniBand(TM) Architecture April, 2001 The format of the IBA payload section of an IBARP packet SHALL be: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved16 | Ethertype | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Hardware Type | Protocol Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | HLEN | PLEN | Operation | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sender EUI-64[63-32] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sender EUI-64[31-0] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved8 | Sender QPN | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sender QKey | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SenderGID[127-96] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SenderGID[127-96] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SenderGID[95-64] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SenderGID[63-32] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SenderGID[31-0] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sender IP Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Target EUI-64[63-32] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Target EUI-64[31-0] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved8 | Target QPN | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Target QKey | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TargetGID[127-96] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TargetGID[127-96] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TargetGID[95-64] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TargetGID[63-32] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TargetGID[31-0] | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Target IP Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Pelissier, et. al. [Page 9] IP and ARP over InfiniBand(TM) Architecture April, 2001 The format of IBA CRCs is as follows: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Invariant CRC | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Variant CRC | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 6.4 IBARP Packet Content The contents of the following fields are defined by IBA specifications and do not have any IP specific characteristics: LRH: VL, LVer, LNH, PktLen, SLID GRH: IPVer, PayLen, NxtHdr, SGID BTH: S, M, Pad, TVer, A, PSN DETH: SourceQP CRCs: Invariant and Variant The content of the following fields in IBARP packets SHALL be set as indicated (the fields marked SIDR indicate that the value is to be obtained using the IBA SIDR protocol): +----------------+----------------------+-------------------------+ | FIELD | SENDER | TARGET | +----------------+----------------------+-------------------------+ | LRH: | +----------------+----------------------+-------------------------+ | SL | SIDR | SL obtained from SA | | | | path record query | | | | using IBARP.SenderGID | | | | as DGID and the | | | | target's GID as SGID in | | | | the query template | +----------------+----------------------+-------------------------+ | DLID | SIDR | LID obtained from SA | | | | path record query | | | | using IBARP.SenderGID | | | | as DGID and the | | | | target's GID as SGID in | | | | the query template | +----------------+----------------------+-------------------------+ | GRH: | +----------------+----------------------+-------------------------+ Pelissier, et. al. [Page 10] IP and ARP over InfiniBand(TM) Architecture April, 2001 | TClass | SIDR | TClass obtained from SA | | | | path record query | | | | using IBARP.SenderGID | | | | as DGID and the | | | | target's GID as SGID in | | | | the query template | +----------------+----------------------+-------------------------+ | FlowLabel | SIDR | FlowLabel obtained from | | | | SA path record query | | | | using IBARP.SenderGID | | | | as DGID and the | | | | target's GID as SGID in | | | | the query template | +----------------+----------------------+-------------------------+ | HopLmt | SIDR | HopLmt obtained from SA | | | | path record query | | | | using IBARP.SenderGID | | | | as DGID and the | | | | target's GID as SGID in | | | | the query template | +----------------+----------------------+-------------------------+ | DGID | SIDR | IBARP:SenderGID | +----------------+----------------------+-------------------------+ | BTH: | +----------------+----------------------+-------------------------+ | OpCode | OpCode specified by | OpCode specified by | | | IBA for unreliable | IBA for unreliable | | | datagram service | datagram service | +----------------+----------------------+-------------------------+ | PKey | SIDR | TBD | +----------------+----------------------+-------------------------+ | DestQP | As specified by IBA | IBARP:SenderQPN | | | for multicast | | | | packets | | +----------------+----------------------+-------------------------+ | DETH: | +----------------+----------------------+-------------------------+ | QKey | SIDR | IBARP:SenderQKey | +----------------+----------------------+-------------------------+ | ETYP: | +----------------+----------------------+-------------------------+ | Ethertype | 0x0800 for IPv4 | 0x0800 for IPv4 | | | 0x86DD for IPv6 | 0x86DD for IPv6 | +----------------+----------------------+-------------------------+ | IBARP: | +----------------+----------------------+-------------------------+ | Hardware Type | TBD for IBA | TBD for IBA | +----------------+----------------------+-------------------------+ | Protocol Type | 0x0800 | 0x0800 | +----------------+----------------------+-------------------------+ Pelissier, et. al. [Page 11] IP and ARP over InfiniBand(TM) Architecture April, 2001 | HLEN | 32 | 32 | +----------------+----------------------+-------------------------+ | PLEN | 4 | 4 | +----------------+----------------------+-------------------------+ | Operation | 1 | 2 | +----------------+----------------------+-------------------------+ | SenderEUI-64 | EUI-64 Portion of | Copied from request | | | the sender's GID | | | | at GID index 0 | | +----------------+----------------------+-------------------------+ | SenderQPN | QPN sender | Copied from request | | | expects in received | | | | IP datagrams | | | | addressed to | | | | IBARP.SenderIPAddress| | +----------------+----------------------+-------------------------+ | SenderQKey | QKey sender | Copied from request | | | expects in received | | | | IP datagrams | | | | addressed to | | | | IBARP.SenderIPAddress| | +----------------+----------------------+-------------------------+ | SenderGID | DGID sender | Copied from request | | | expects in received | | | | IP datagrams | | | | addressed to | | | | IBARP.SenderIPAddress| | +----------------+----------------------+-------------------------+ | SenderIPAddress| A valid IP address | Copied from request | | | of the sender | | +----------------+----------------------+-------------------------+ | TargetEUI-64 | Anything | EUI-64 Portion of the | | | | target's GID at GID | | | | index 0 | +----------------+----------------------+-------------------------+ | TargetQPN | Anything | QPN target | | | | expects in received IP | | | | datagrams addressed to | | | | IBARP.TargetIPAddress | +----------------+----------------------+-------------------------+ | TarketQKey | Anything | QKey target | | | | expects in received IP | | | | datagrams addressed to | | | | IBARP.TargetIPAddress | +----------------+----------------------+-------------------------+ | TargetGID | Anything | DGID target | | | | expects in received IP | | | | datagrams addressed to | | | | IBARP.TargetIPAddress | +----------------+----------------------+-------------------------+ Pelissier, et. al. [Page 12] IP and ARP over InfiniBand(TM) Architecture April, 2001 | TargetIPAddress| IP address of the | Copied from request | | | device for which the | | | | sender is attempting | | | | to determine the | | | | hardware address | | +----------------+----------------------+-------------------------+ 7. References [1] Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October, 1996. [2] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [3] Hinden, R. and S. Deering, "IP Version 6 Addressing Architecture", RFC 2372, July 1998. [4] Postel, J., "Internet Protocol", RFC 791, USC/Information Sciences Institute, September 1991. [5] Narten, T., Nordmark, E. and W. Simpson, "Neighbor Discovery for IP Version 6 (IPv6)", RFC 2461, December 1998. [6] Plummer, David C., "An Ethernet Address Resolution Protocol", RFC 826, November 1982. [7] "InfiniBand(TM) Architecture Specification Volume 1, Release 1.0", InfiniBand(SM) Trade Association, October 24, 2000. 8. Authors' Addresses Joseph Pelissier Sanera Systems 1925 NW AmberGlen Parkway Suite 155 Beaverton, OR 97006 Phone: +1 503 601 0259 Email: joep@sanera.net Paul Grun Intel Corporation 5200 NE Elam Young Parkway Hillsboro, OR 97124 Phone: +1 503 712 4332 Email: paul.grun@intel.com Pelissier, et. al. [Page 13] IP and ARP over InfiniBand(TM) Architecture April, 2001 Arvind Murching Microsoft Corporation One Microsoft Way Redmond WA 98052 Phone: +1 425 703 2364 Email: arvindm@microsoft.com Renato J Recio IBM Internal Zip 9043 11400 Burnett Road Austin, Texas 78759 Phone: +1 512 838 3685 Email: recio@us.ibm.com 9. Full Copyright Statement Copyright (C) The Internet Society (2001). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Pelissier, et. al. [Page 14]