Internet DRAFT - draft-white-protocol-stack


HTTP/1.1 200 OK
Date: Tue, 09 Apr 2002 12:10:19 GMT
Server: Apache/1.3.20 (Unix)
Last-Modified: Tue, 17 Mar 1998 16:34:00 GMT
ETag: "3ddc17-f1ae-350ea5f8"
Accept-Ranges: bytes
Content-Length: 61870
Connection: close
Content-Type: text/plain

Network Working Group                                   John C. C. White
Internet-Draft                                     The MITRE Corporation
Obsoletes: RFC998                                             April 1997
Category: Proposed Standard
Expires in six months

                NETBLT (Network Block Transfer Protocol)


Status of this Memo

   This document is an Internet-Draft.  Internet-Drafts are working
   documents of the Internet Engineering Task Force (IETF), its areas,
   and its working groups.  Note that other groups may also distribute
   working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet- Drafts as reference
   material or to cite them other than as ``work in progress.''

   To learn the current status of any Internet-Draft, please check the
   ``1id-abstracts.txt'' listing contained in the Internet- Drafts
   Shadow Directories on (Africa), (Europe), (Pacific Rim), (US East Coast), or (US West Coast).


   The NETBLT protocol [RFC998] was designed as an experimental
   transport layer protocol, intended for moving large quantities of
   data across a wide variety of networks. It provides reliable bulk
   transfer with an end-to-end flow-control mechanism meant to deal with
   network congestion by throttling the rate at which data is inserted
   into the network. However, experiments with NETBLT across shared
   links revealed problems with fairness; traffic from one connection
   could hog most of a link's bandwidth, and there seems to be no way to
   prevent this under the current rate-control scheme, so further
   application of NETBLT was not pursued by its original developers.

   However, NETBLT has a number of characteristics which make it very
   attractive for use across noisy, long-delay, slow-turnaround, or
   asymmetric communications links. Such links are common in military
   usage, and may become more widespread with the development of mobile
   computing. NETBLT's attractive characteristics include selective
   retransmission of lost packets, potentially large transmission
   windows, and control of transmission from the receiving, rather than
   the sending side; the latter makes NETBLT relatively insensitive to

White                                                           [Page 1]
INTERNET DRAFT                   NETBLT                       April 1997

   network delays. NETBLT, with minor modifications, was adopted as the
   transport layer of the military standard TACO2 (Tactical
   Communications Protocol 2) [MIL- STD], which is intended for the
   transmission of images and other bulk data across links that cannot
   support the usual TCP/IP operation. This document describes NETBLT as
   it is currently used, and is intended partly to encourage
   consideration of NETBLT in other difficult communications

1. Document History

   The military standard definition of NETBLT was developed from RFC998
   by expunging most of the tutorial information and translating the
   remainder into the language required by military standards. This
   document was then prepared from the military standard; as a result,
   there may be some unnecessary rearrangements and rewordings. In any
   case, most of the protocol remains as designed by the original
   developers. Modifications have been made to simplify protocol
   operation and to extend its capability.

2. NETBLT Overview

   The bulk data transfer protocol NETBLT works by opening a connection
   between a sender and a receiver, transferring a message in one or
   more buffers, and closing the connection.  Each buffer is transferred
   as a sequence of packets; the interaction between sender and receiver
   is primarily on a per-buffer basis.  This section provides an
   overview of NETBLT; further explanation and detailed requirements are
   found in the following sections.  The material here assumes the
   existence of a full- duplex connection between sender and receiver,
   such that information can be transferred in both directions more or
   less concurrently.  Changes for half-duplex and simplex operation are
   provided later.

   Specific packet types are identified in the following sections by
   upper-case names (e.g., DATA packets), in contrast with packet
   functions (e.g., keepalive packets) which are accomplished by more
   than one packet type.

2.1 Single-buffer operation.

   In its simplest form, a NETBLT transfer works as follows: first, a
   connection is opened between a sending and a receiving NETBLT. That
   step includes negotiation of various transmission parameters. The
   sending client loads a buffer of data and passes it to the NETBLT
   layer to be transferred.  NETBLT breaks the buffer up into packets
   and sends these packets across the network via datagrams.  The
   receiving NETBLT loads these packets into a matching buffer.  When

White                                                           [Page 2]
INTERNET DRAFT                   NETBLT                       April 1997

   the last packet in the buffer should have arrived at the receiver,
   the receiving NETBLT checks whether all packets in that buffer have
   been received correctly.  If some packets have not been received
   correctly, the receiving NETBLT requests that they be resent.  When
   the buffer has been completely received, the receiving NETBLT passes
   it to the receiving client, and confirms its receipt to the sender.
   When a new buffer is ready to receive more data, the receiving NETBLT
   notifies the sender that the new buffer is ready, and the sender
   prepares and sends the next buffer in the same manner.  This
   continues until all the data has been sent; at that time the sender
   notifies the receiver that the transmission has been completed.  The
   two sides then close the connection.

2.2 Multiple-buffer operation.

   As described above the NETBLT protocol is "lock-step".  Action halts
   after a buffer is transmitted, and begins again after confirmation is
   received from the receiver of data.  NETBLT provides for multiple
   buffering, so that the sending NETBLT can transmit new buffers while
   waiting for confirmation of earlier buffers from the receiving

2.3 Buffers and packets.

   The data to be transmitted is broken up into buffers by the sending
   client. All buffers are the same size, except for the last buffer.
   During connection setup, the sending and receiving NETBLTs negotiate
   the buffer size.  Buffer sizes are in bytes; data is placed in
   buffers on byte boundaries.

   Each buffer is broken down by NETBLT into a sequence of DATA packets
   terminated by an LDATA packet.  DATA packet size is negotiated
   between the sending and receiving NETBLTs during connection setup.
   All DATA packets are the same size.  DATA and LDATA packets are
   identical in format except for the packet type.

2.4 Flow control.

   NETBLT uses two strategies for flow control, one at the client level
   and one internal.

2.4.1 Client level flow control.

   The sending and receiving NETBLTs transmit data in buffers; client
   flow control is therefore by buffer.  Before a buffer can be
   transmitted, NETBLT confirms that both clients have set up matching
   buffers, that one is ready to send data, and that the other is ready
   to receive data. Either client can therefore control the flow of data

White                                                           [Page 3]
INTERNET DRAFT                   NETBLT                       April 1997

   by not providing a new buffer.  Clients cannot stop a buffer transfer
   once it is in progress, except by aborting the entire transfer.

2.4.2 Internal flow control.

   The internal flow control mechanism for NETBLT is rate control.  The
   transmission rate is negotiated by the sending and receiving NETBLTs
   during connection setup and after each buffer transmission.  The
   sender uses timers to maintain the negotiated rate, by controlling
   the time to transmit groups of packets.  The sender transmits a burst
   of packets over the negotiated time interval, and sends another burst
   in the next interval. NETBLT's rate control therefore has two parts,
   a burst size and a burst interval, with (burst interval)/(burst size)
   equal to the average transmission time per packet.

   A burst interval value of zero means that internal flow control is
   turned off, so that only client level flow control is in effect.  In
   this case, the sending NETBLT will transmit packets without regard
   for the rate control mechanism.

   All NETBLT flow control parameters (packet size, buffer size, number
   of buffers outstanding, burst size, and burst interval) are
   negotiated during connection setup.  The negotiation process is the
   same for all parameters.  The client initiating the connection (the
   active side) sends a value for each parameter in its OPEN packet.
   The other client (the passive side) will compare these values with
   the highest- performance values it can support.  The passive side can
   modify any of the parameters, but only by making them more
   restrictive; i. e., smaller packet size, smaller buffer size, fewer
   buffers, smaller burst size, and larger burst interval.  The
   (possibly modified) parameters are sent back to the active side in
   the RESPONSE packet.

   The burst size and burst interval may also be re-negotiated after
   each buffer transmission to adjust the transfer rate according to the
   performance observed from transferring the previous buffer.  The
   receiving end sends burst size and burst interval values in its OK
   messages (which acknowledge successful receipt of a buffer) and in
   its RESEND messages (which request retransmission of specific
   packets).  The sender will compare these values with the values it
   can support.  Again, it may modify either of these parameters, but
   only by making them more restrictive.  The modified parameters will
   then be communicated to the receiver in DATA, LDATA, or NULL-ACK

2.5 Checksumming.

   NETBLT automatically checksums each packet header and, optionally,

White                                                           [Page 4]
INTERNET DRAFT                   NETBLT                       April 1997

   the data portion of each DATA and LDATA packet. The checksum value is
   the bitwise negation of the ones-complement sum of the 16-bit words
   being checksummed. If a packet to be transferred has an odd number of
   bytes, it is padded with a final null byte (binary 0's) to make the
   number of bytes even for the purpose of checksum calculation.  The
   extra byte is not transmitted as part of the packet, but its
   existence is assumed at the receiving end for checksum verification.

3.  NETBLT detailed operation

   Each NETBLT transfer has three stages: connection setup, data
   transfer, and connection close.  The stages are described in detail
   below, along with methods for insuring that each stage completes
   reliably.  State diagrams are provided at the end of the description
   for each stage of the transfer.  Each transition in the diagrams is
   labelled with the event that causes the transition, and optionally,
   in parentheses, actions that occur at the time of the transition.

3.1 Connection setup.

   A NETBLT connection is set up by an exchange of two packets between
   the active NETBLT and the passive NETBLT. The active end sends an
   OPEN packet; the passive end acknowledges the OPEN packet in one of
   two ways: it either sends a REFUSED packet, indicating that the
   connection cannot be completed for some reason, or it completes the
   connection setup by sending a RESPONSE packet.  After a successful
   connection setup, the transfer can begin.  Figure 1 illustrates the
   opening of a connection by the active end, and figure 2 shows the
   same process for the passive end.

   Each side of the connection transmits its death-timeout value in
   seconds in the OPEN or the RESPONSE packet.  The death-timeout value
   is used to determine the frequency with which to send keepalive
   packets during idle periods of an opened connection (death timers and
   keepalive packets are discussed later).

   The sending NETBLT specifies a passive client through a client-
   specific "well-known" 16 bit logical port number on which the
   receiving end listens. The sending client identifies itself through a
   32 bit Internet address and a unique 16 bit port number.

   An unstructured, variable-length client message field is provided in
   OPEN and RESPONSE packets for any client-specific information that
   may be required. In addition, a "reason for refusal" field is
   provided in REFUSED packets.

   Recovery for lost OPEN and RESPONSE packets is provided by the use of
   timers.  The sending end sets a timer when it sends an OPEN packet.

White                                                           [Page 5]
INTERNET DRAFT                   NETBLT                       April 1997

   When the timer expires, another OPEN packet is sent, until some
   predetermined maximum number (at least five) of OPEN packets have
   been sent.  The timer is cleared upon receipt of a RESPONSE or
   REFUSED packet.

   To prevent duplication of OPEN and RESPONSE packets, the OPEN packet
   contains a 32 bit connection unique ID (UID) that must be returned in
   the RESPONSE packet.  This unique ID prevents the initiator from
   confusing the response to the current request with the response to an
   earlier connection request (there can only be one connection open
   between any pair of logical ports).  Any OPEN or RESPONSE packet with
   a port pair matching that of an open connection will have its unique
   ID checked.  If the unique ID of the packet matches the unique ID of
   the connection, then the packet type is checked.  If it is a RESPONSE
   packet, it is treated as a duplicate and ignored.  If it is an OPEN
   packet, the passive NETBLT will send another RESPONSE  (on the
   assumption that a previous RESPONSE packet was sent and lost, causing
   the initiating NETBLT to retransmit its OPEN packet).  A non-matching
   unique ID is treated as an attempt to open a second connection
   between the port pair and is rejected by sending a REFUSED message.

     |            |--------<-----------------------------+
     |  Inactive  |-->-+                                 |
     |            |    |                                 |
     +------------+    |                                 |
       ^        Connect request from client              |
       |        (Send OPEN, start Open Timer)            |
    REFUSED received   |                         <=max # OPENs sent
       |               |                                 |
     +------------+    |                                 |
     |  Opening   |-<--+---<---------+                   ^
     |            |                  |                   |
     |            |->-+      <max # OPENs sent           |
     +------------+   |   (send OPEN, start Open Timer)  |
           |          |              |                   |
           |  Open Timer timeout     |                   |
           |          +------>-------+-------------------+
     RESPONSE received
     (clear Open Timer)
           |       +------------+
           |       |            |
           +-->----| Connected  |
                   |            |

   Figure 1. Active side open state diagram

White                                                           [Page 6]
INTERNET DRAFT                   NETBLT                       April 1997

             |            |--->----------+
        +-<--|  Inactive  |     Unacceptable OPEN received
        |    |            |        (send REFUSED)
        |    |            |---<----------+
        |    +------------+
     Acceptable OPEN received
     (send RESPONSE)             +-------+--------->-----------+
        |    +------------+      |       |                     |
        |    |            |--->--+ Acceptable OPEN    Unacceptable OPEN
        +->--| Connected  |         received            received
             |            |        (send RESPONSE)     (send REFUSED)
             |            |--<---+       |                     |
             +------------+      +-------+---------<-----------+

   Figure 2. Passive side open state diagram

3.2 Data transfer

   The simplest full-duplex mode of data transfer proceeds as follows.
   The sending client sets up a buffer full of data.  The receiving
   NETBLT sends a GO message inside a CONTROL packet to the sender,
   signifying that it too has set up a buffer and is ready to receive
   data.  Once the GO message is received, the sender transmits the
   buffer as a series of DATA packets followed by an LDATA packet. When
   the last packet in the buffer should have been received (as
   determined by a timer), if any packets in the buffer have not been
   received the receiver sends a RESEND message inside a CONTROL packet
   containing a list of packets that were not received.  The sender will
   resend these packets.  This process continues until there are no
   missing packets.  At that time the receiver sends an OK message
   inside a CONTROL packet, sets up another buffer to receive data, and
   sends another GO message.  The sender, having received the OK
   message, will set up another buffer, wait for the GO message, and
   repeat the process.

   A more efficient full-duplex transfer mode uses multiple buffering,
   in which the sender and receiver allocate and transfer buffers in a
   manner that allows error recovery or successful transmission
   confirmation of previous buffers to be concurrent with transmission
   of the current buffer. During the connection setup phase, one of the
   negotiated parameters is the number of concurrent buffers permitted
   during the transfer. If there is more than one buffer available,
   transfer of the next buffer will start right after the current buffer
   finishes, and the receiver is ready to receive the buffer.  The
   receiver signals that it is ready for the next buffer by sending a GO
   message.  This is illustrated in the following example: Assume the
   sender has available two buffers A and B in a multiple-buffer

White                                                           [Page 7]
INTERNET DRAFT                   NETBLT                       April 1997

   transfer, with A preceding B. When A has been transferred and the
   sending NETBLT is waiting for either an OK or a RESEND message for
   it, the sending NETBLT can start sending B immediately.  If the
   receiver of data sends an OK for A, all is well; if it sends a
   RESEND, the missing packets specified in the RESEND message are
   retransmitted. In the multiple-buffer transfer mode, all packets to
   be sent are ordered by buffer number (lowest number first).  Since
   buffer numbers increase monotonically, packets from an earlier buffer
   will precede packets from a later buffer.

3.2.1 Control Messages.

   NETBLT uses a single long-lived control packet; the packet is treated
   like a FIFO queue, with new control messages added on at the end and
   acknowledged control messages removed from the front.  The
   implementation places control messages in the control packet and
   transmits the entire control packet, consisting of any unacknowledged
   control messages plus new messages just added. Since control packet
   transmissions are fairly frequent, unacknowledged messages may be
   transmitted several times before they are finally acknowledged.  The
   receiver may send zero or more control messages (OK, GO, or RESEND)
   within a single CONTROL packet. In order to limit the size of the
   control packet, it is permissible to send fewer than the full set of
   unacknowledged control messages in a control packet; it is however
   required that the control messages in a control packet be
   consecutive, starting with the lowest-numbered unacknowledged control

   Each control message includes a sequence number, which starts at one
   and increases by one for each control message generated.  The sending
   NETBLT checks the sequence number of every incoming control and
   stores the highest sequence number below which all other sequence
   numbers have been received (in following paragraphs this is called
   the high-acknowledged- sequence-number). It returns this number in
   every packet flowing back to the receiver.  The receiver removes
   control messages with sequence numbers less than or equal to the
   high-acknowledged-sequence-number from the control packet.

   Whenever the receiver sends a control packet, it starts a control
   timer. When the control timer expires, the receiving NETBLT will
   resend the control packet and reset the timer.  The receiving NETBLT
   will continue to resend control packets in response to control timer
   expiration until either the control timer is cleared or the receiving
   NETBLT's death timer (described later) expires (at which time it will
   shut down the connection). The control timer may have as its initial
   value an arbitrary number.  Subsequent control timer values are based
   on the network round-trip transit time (the time between sending the
   control packet and receiving the acknowledgment of all messages in

White                                                           [Page 8]
INTERNET DRAFT                   NETBLT                       April 1997

   the control packet) plus a variance factor.  The timer value is
   regularly updated, based on a smoothed average of collected round-
   trip transit times.  The control timer is set to the keepalive value
   when a packet is received from the sender with high-acknowledged-
   sequence-number equal to the highest sequence number in the control
   packet most recently sent.

   The exact algorithm for control timer calculation is not mandated.
   The suggested algorithm, similar to that [Jacobsen] now used in TCP,
   is as follows:

   Initially, the round trip time is set to one-half the keepalive value
   and the deviation is set to zero. When a control packet is sent, if
   the round-trip-delay (RTD) timer is not running, its highest sequence
   number is stored and the RTD timer is started. When acknowledgement
   of that sequence number is received, the RTD timer value is used to
   calculate a new RTD estimate. However, if a control packet is
   retransmitted, the RTD timer is zeroed, its value is not used (to
   avoid ambiguous RTD measurements), and the estimated RTD is increased
   by 1/2 of its value (to allow recovery from too low an RTD estimate).
   To calculate the new RTD estimate,

   New smoothed round trip time = (1-a) * old smoothed round trip time +
   a * latest round trip measurement

   New deviation = (1-b) * old deviation + b * |latest round trip
   measurement - old smoothed round trip time|

   where a = 1/8 and b = 1/4, allowing computations to be done with add
   and shift operations.  The control timer is set equal to the new
   smoothed round trip time plus twice the new deviation, or to the
   keepalive value, whichever is less, if the control packet is not
   empty.  If the control packet is empty, the control timer is set to
   the keepalive value.

   The sending NETBLT, upon receiving a previously unseen control
   message, will either set up a new buffer (upon receipt of an OK
   message for a previous buffer), mark data for resending (upon receipt
   of a RESEND message), or prepare a buffer for sending (upon receipt
   of a GO message).  If the sending NETBLT is not in a position to send
   data, it sends a NULL-ACK packet, which contains its high-
   acknowledged-sequence- number (this permits the receiving NETBLT to
   resend any outstanding control messages or to clear its control
   timer), and waits until it can send more data.

3.2.2 Send buffer state sequence.

   The state sequence for a sending buffer is as follows: when a GO

White                                                           [Page 9]
INTERNET DRAFT                   NETBLT                       April 1997

   message for the buffer is received, the buffer is created, filled
   with data, and placed in a SENDING state.  When an OK for that buffer
   has been received, it goes into a SENT state and may be released.
   Figure 3 illustrates this sequence.

                                     |           |
     GO for buffer n received --->---|   Ready   |-->-------+
     (create and fill buffer n)      |           |          |
                                     +-----------+          |
                                                Start sending buffer n
                                          (set last-buffer-touched to n)
                                     +-----------+          |
                                     |           |          |
                       +------<------|  Sending  |--<-------+
                       |             |           |
      OK for buffer n received       +-----------+
                       |             +-----------+
                       +------>------|           |
                                     |   Sent    |-->- (remove buffer n)
                                     |           |

   Figure 3. Sending buffer state diagram

3.2.3 Receive buffer state sequence.

   The state sequence for a receiving buffer is more complicated.
   Assume existence of a Buffer A. When a control message for Buffer A
   is sent, the buffer will move into state ACK-WAIT (it is waiting for
   acknowledgement of the control message).  As soon as the control
   message has been acknowledged, Buffer A will move from the ACK-WAIT
   state into the ACKED state (it is now waiting for DATA packets to
   arrive).  At this point, the control message is removed from the
   control packet.  Buffer A will stay in the ACKED state until a DATA,
   LDATA, or NULL-ACK packet arrives with its "Last Buffer Touched"
   number greater than or equal to Buffer A's number.  At this time,
   Buffer A's data timer is set to the time expected for the remaining
   packets in the buffer to be received plus a variance, and Buffer A
   will move to the RECEIVING state.  (Note: This mechanism is different
   from, and simpler than, the "loose/tight" timer mechanism described
   in RFC 998).  When all DATA packets for A have been received, it will
   move from the RECEIVING state to the RECEIVED state and may be passed
   to the receiving client.  Had any packets been missing, Buffer A's
   data timer would have expired; in that case, Buffer A will move into
   the ACK-WAIT state after sending a RESEND message. The sending of a
   RESEND message will cause the data timers of all buffers currently in

White                                                          [Page 10]
INTERNET DRAFT                   NETBLT                       April 1997

   the RECEIVING state to be recalculated, since the presence of re-sent
   packets will change the expected completion time for later buffers.
   The state progression would then move as in the above example. Figure
   4 illustrates this sequence.

     < maximum # buffers exist &
      last buffer not detected --->---------------------+
     (create buffer n; send GO n)                       |
                                                |              |
              +--<--ACK for buffer n GO or --<--|  ACK-wait    |
              |     RESEND message received     |              |
              |                                 +--------------+
        +------------+                                  |
        |            |                                  ^
        |   ACKed    |                                  |
        |            |-->---+                   RESEND sent
        +------------+      |                   (set all receiving
                            |                   buffer data timers)
          DATA/LDATA/NULL-ACK with                      |
         last-buffer-touched >= n received              |
          (set buffer n data timer)             +-------------+
                            |                   |             |
                            |                   | Resend-wait |
                            |                   |             |
                            |                   +-------------+
       +-------------+      |                           |
       |             |---<--+                           |
       |  Receiving  |                                  |
       |             |-->-- Buffer n data timeout & ->--+
       +-------------+      buffer n not complete
              |            (add RESEND to control packet)
       Buffer n complete    +------------+
              |             |            |
              +---->--------|  Received  |--->--- Buffer n flushed
                            |            |        (remove buffer n)

   Figure 4. Receiving buffer state diagram

3.2.4 Data Timers.

   NETBLT solves the problem of DATA and LDATA packet loss by using a
   data timer for each buffer at the receiving end.  The simplest data
   timer model has a data timer set when a buffer is ready to be
   received; if the data timer expires, the receiving NETBLT will send a

White                                                          [Page 11]
INTERNET DRAFT                   NETBLT                       April 1997

   RESEND message requesting all missing DATA/LDATA packets in the
   buffer.  When all packets have been received, the timer is cleared.
   Data timer values are based on the amount of time taken to transfer a
   buffer plus a variance factor.

   The exact algorithm for data timer estimation is not mandated.  The
   suggested algorithm is to compute the number of packets expected
   before the buffer is complete, multiply that by the time required to
   transmit a packet, and add a variance. The receiver uses both the
   minimum time per packet established by the burst size/burst interval,
   and the measured time per packet with mean deviation, to establish
   two estimates of the expected time per packet. These two estimates
   are then used to calculate data timer settings for each buffer, and
   the maximum value is used for that buffer's data timer. This
   combination allows the demand placed on the net capacity by a given
   transfer to be limited, while still avoiding unnecessary
   retransmissions if the available net capacity is less than that
   requested. The algorithm is as follows:

   The average time-per-packet A and the mean deviation D are
   initialized. The first-packet time and packet number are stored when
   the first packet from a given buffer arrives, and the latest-packet
   time and packet number are stored or updated as each packet from that
   same buffer arrives.

   When a packet from a different buffer or a NULL-ACK arrives, or a
   RESEND is sent  for the given buffer, the difference between the
   first-packet time and the latest-packet time is divided by the
   difference between the first-packet number and the latest-packet
   number, to provide a sample time-per-packet. Then,

   New smoothed time-per-packet = (1-a) * old smoothed time-per-packet +
   a * latest time-per-packet measurement

   New time-per-packet deviation = (1-b) * old time-per-packet deviation
   + b * |latest time-per-packet measurement - old smoothed time-per-
   packet |

   where a = 1/8 and b = 1/4, allowing computations to be done with add
   and shift operations, as with the control timer.  When the data timer
   for a buffer must be set, two times are calculated using N, the
   number of packets which must be received before the buffer is
   completely filled:

    T1 = 1.25 * N * smoothed time-per-packet + 2 * time-per-packet

    T2 = 1.5 * N * burst interval / burst size.

White                                                          [Page 12]
INTERNET DRAFT                   NETBLT                       April 1997

   The data timer is then set to the maximum of (T1, T2).

3.2.5 Death timers.

   At connection startup, each NETBLT sends its death value to the other
   end in either the OPEN or the RESPONSE packet. As soon as the
   connection is opened, each end sets its death timer to its chosen
   value; this timer is reset every time a packet is received.  When a
   NETBLT's death timer expires, it will close the connection without
   sending any more packets.

3.2.6 Keepalive packets.

   NETBLT includes a keepalive function, which sends packets repeatedly
   at fixed intervals when a NETBLT has no other reason to send packets.
   The sender uses NULL-ACKs as keepalive packets; the receiver uses
   empty CONTROL packets. If the sending NETBLT is not ready to send
   upon receipt of a control packet, it sends a single NULL-ACK packet
   to clear any outstanding control timers at the receiving end.  Each
   end uses the other end's death-timeout value to compute a frequency
   with which to send keepalive packets. The keepalive frequency should
   be high enough that several keepalive packets can be lost before the
   other end's death timer expires; suitable values are the sender's
   death timer value divided by seven for the receiver, and the
   receiver's death timer value divided by eight for the sender
   (keepalive intervals should be different to avoid repeated collisions
   in half-duplex operations).

3.3 Connection termination.

   There are four conditions under which a connection is terminated:  a
   successful transfer, a client quit, a NETBLT abort, and a death timer

3.3.1 Successful transfer.

   After a successful data transfer, NETBLT closes the connection.

   When the sender is transmitting the last buffer of data, it sets a
   "last-buffer" flag on every DATA packet in the buffer. The receiver
   will recognize that the transfer has completed successfully when all
   of the following are true: (1) it has received DATA packets with a
   "last- buffer" flag set, (2) all its control messages have been
   acknowledged, and (3) it has no outstanding buffers with missing
   packets.  The DONE packet is transmitted when the receiver recognizes
   that the transfer has been successfully completed.  At that point,
   the receiver closes its half of the connection.  Figure 5 illustrates
   this sequence.

White                                                          [Page 13]
INTERNET DRAFT                   NETBLT                       April 1997

    +-------------+                                    +------------+
    |             |                                    |            |
    |  Connected  |-->-- Last buffer received & --->---|  Inactive  |
    |             |      all buffers disposed of &     |            |
    +-------------+      all messages acked            +------------+
                             (send DONE)

   Figure 5. Receiver successful close state diagram

   The sender will recognize that the transfer has completed when the
   following are true: (1) it has transmitted DATA packets with a "last-
   buffer" flag set and (2) it has received OK messages for all its
   buffers.  At that point, it will "dally" for a predetermined period
   of time before closing its half of the connection.  If the NULL-ACK
   packet acknowledging the receiver's last OK message was lost, the
   receiver has time to retransmit the OK message, receive a new NULL-
   ACK, and recognize a successful transfer.  The dally timer value is
   based on the receiver's control timer value; it should be long enough
   to allow the receiver's control timer to expire so that the OK
   message can be re-sent. A value of twice the receiver's control timer
   value is suitable for the dally timer.  When the sender receives a
   DONE packet, it clears its dally timer and close its half of the
   connection.  Figure 6 illustrates this sequence.

          |           |
          | Connected |--->---+
          |           |       |
          +-----------+  All buffers flushed
                         (send NULL-ACK;
                          set dally timer)
          +-----------+       |
          |           |---<---+-------<-----------------+
          | Dallying  |                                 |
          |           |----->--- OK message received ->-+
          +-----------+          (send NULL-ACK;
                |                 set dally timer)        +----------+
                |                                         |          |
                +-->-- DONE received or dally timeout ->--| Inactive |
                                                          |          |

   Figure 6. Sender successful close state diagram

3.3.2 Client QUIT.

White                                                          [Page 14]
INTERNET DRAFT                   NETBLT                       April 1997

   During a NETBLT transfer, one client may send a QUIT packet to the
   other, to terminate the transfer prematurely. The NETBLT receiving
   the QUIT packet will take no action other than immediately notifying
   its client and transmitting a QUITACK packet.  The QUIT sender will
   time out and retransmit until a QUITACK has been received or its
   death timer expires.  The sender of the QUITACK will dally before
   quitting, so that it can respond to a retransmitted QUIT.  Figure 7
   illustrates this sequence.

       |           |
   +---| Connected |--->--- Quit request from client ----->---+
   |   |           |        (send QUIT; set quit timer)       |
   |   +-----------+                                   +-----------+
   |                          +- Quit timer timeout ->-|           |
   +->- QUIT received -->--+  |    (send QUIT)         | Quit-sent |
        (send QUIT-ACK;    |  +--------<---------------|           |
         set dally timer)  |                           +-----------+
       +------------+      |                                  |
       |            |--<---+--------<-----------+             +->--+
   +---| Ouit-rcvd  |                           |                  |
   |   |            |---->----- QUIT received --+                  |
   |   +------------+           (send QUIT-ACK;                    |
   |                            set dally timer)                   |
   |                          +------------+                       |
   |                          |            |                       |
   +-->-- Dally timeout -->---|  Inactive  |-<--QUIT-ACK received -+
                              |            |     or death timeout

   Figure 7. Quit state diagram


   An ABORT will take place when an unrecoverable malfunction occurs.
   Since the ABORT originates in the NETBLT layer, it may be sent at any
   time.  The ABORT implies a malfunction, so no transmit reliability is
   expected, and the sender will immediately close its connection.
   Figure 8 illustrates this sequence.

White                                                          [Page 15]
INTERNET DRAFT                   NETBLT                       April 1997

                |            |
           +-<--| Connected  |-->---+
           |    |            |      |
           |    +------------+      |
           |                        |
      ABORT received            Internal malfunction
           |                     (send ABORT)
           |    +------------+      |
           |    |            |      |
           +->--|  Inactive  |--<---+
                |            |

   Figure 8. Abort state diagram

3.3.4 Death timer timeout.

   When a NETBLT's death timer expires, it closes the connection without
   sending further packets.

4. Protocol layering structure.

   NETBLT may be implemented directly on top of the Internet Protocol
   (IP), in which case it has been assigned an official protocol number
   of 30 (decimal), which is 0x1e (hexadecimal).  In other instances, it
   has been implemented over UDP, for which an official protocol number
   will be requested.

5. Packet formats.

   NETBLT packets are divided into three categories, all of which share
   a common 12-byte packet header.

     a.  There are three packet types that travel only from data sender
   to receiver; these include the high-acknowledged-sequence-numbers
   which the receiver uses for control of message transmission
   reliability.  They are the NULL-ACK, DATA, and LDATA packets.

     b.  There are two packet types that travels only from receiver to
   sender.  One is the CONTROL packet.  Each CONTROL packet can contain
   an arbitrary number of control messages (GO, OK, or RESEND), each
   with its own sequence number. The other is the unreliably-transmitted
   DONE packet.

     c.  There are six packet types which can travel in either

White                                                          [Page 16]
INTERNET DRAFT                   NETBLT                       April 1997

   direction. These packet types either have special ways of insuring
   reliability, or are not transmitted reliably.  They are the OPEN,
   travels from active side to passive side; the RESPONSE and REFUSED
   packets travel from passive side to active side; and the QUIT,
   QUITACK, and ABORT packets can be sent by either side.

   All packet headers are "longword-aligned," such that all packet
   headers are a multiple of four bytes in length and all four-byte
   fields start on a longword boundary. The content of the longword
   alignment fields is zeros.  The Client String field is terminated
   with at least one null byte, with extra null bytes added at the end
   to create a field that is a multiple of four bytes long.  All numeric
   values are coded as binary integers.

   OPEN (type 0) and RESPONSE (type 1)

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    |           Checksum            |    Version    |     Type      |
    |           Length              |           Local Port          |
    |        Foreign Port           | Longword Alignment Padding    |
    |                       Connection Unique ID                    |
    |                         Buffer Size                           |
    |        DATA packet size       |          Burst Size           |
    |           Burst Interval      |       Death Timer Value       |
    |  Reserved (must be zero)  |C|M| Maximum # Outstanding Buffers |
    | Client String ...
                                      Longword Alignment Padding    |

   a. Checksum: to generate the checksum, the checksum field itself is
    cleared, the 16-bit ones-complement sum is computed over the packet,
    and the ones complement of this sum is placed in the checksum field.

   b. Version: the NETBLT protocol version number. This document describes
   version 4 of NETBLT.

White                                                          [Page 17]
INTERNET DRAFT                   NETBLT                       April 1997

   c. Type: the NETBLT packet type number (OPEN = 0, RESPONSE = 1, etc.)

   d. Length: the total length (NETBLT header plus data, if present) of the
   NETBLT  packet in bytes

   e. Local Port: the local NETBLT's 16-bit port number

   f. Foreign Port: the foreign NETBLT's 16-bit port number

   g. Connection UID: the 32 bit connection unique identifier. Connection
   UID may be any randomly-selected value, which is unique in that if more
   than one NETBLT connection is supported by a single host interface, it
   will not be duplicated.

   h. Buffer size: the size in bytes of each NETBLT buffer (except the

   i. Data packet size: length of each DATA packet in bytes

   j. Burst Size: Number of DATA packets in a burst

   k. Burst Interval: Transmit time in milliseconds of a single burst

   l. Death timer: Packet sender's death timer value in seconds

   m. "C": the DATA/LDATA packet data checksum flag (0 = do not checksum
   DATA and LDATA packet data, 1 = do).

   n. "M": the transfer mode (0 = READ, 1 = WRITE).

   o. Maximum Outstanding Buffers: maximum number of buffers that can be
   transferred  before waiting for an OK message from the receiving NETBLT.

   p. Client string: an arbitrary, null-terminated, longword-aligned string
   for use by  NETBLT clients.

   QUITACK (type 3), and DONE (type 10)

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    |           Checksum            |    Version    |     Type      |
    |           Length              |           Local Port          |
    |        Foreign Port           | Longword Alignment Padding    |

White                                                          [Page 18]
INTERNET DRAFT                   NETBLT                       April 1997

   QUIT (type 2), ABORT (type 4), and REFUSED (type 9)

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    |           Checksum            |    Version    |     Type      |
    |           Length              |           Local Port          |
    |        Foreign Port           | Longword Alignment Padding    |
    | Reason for QUIT/ABORT/REFUSE...
                                      Longword Alignment Padding    |

   DATA (type 5) and LDATA (type 6)

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    |           Checksum            |    Version    |     Type      |
    |           Length              |           Local Port          |
    |        Foreign Port           | Longword Alignment Padding    |
    |                       Buffer Number                           |
    |                     Last Buffer Touched                       |
    | High Consecutive Seq Num Rcvd |         Packet Number         |
    |    Data Area Checksum Value   |      Reserved (MBZ)         |L|
    |      New Burst Size           |        New Burst Interval     |

   a. Checksum: checksum of the packet header only, including the Data Area
   Checksum  Value.

   b. Buffer number: a 32 bit unique number assigned to every buffer.
   Buffers are sequentially numbered, starting with 1.

   c. Last Buffer Touched: the number of the highest buffer transmitted so

White                                                          [Page 19]
INTERNET DRAFT                   NETBLT                       April 1997

   d. High Consecutive Sequence Number Received: Highest control message
   sequence number below which all control messages have been received.

   e. Packet number: sequential, monotonically increasing DATA packet
   identifier, starting with 0 in each buffer.

   f. Data Area Checksum Value: Checksum of the DATA packet's data.
   Algorithm used  is the same as that used to compute checksums of other
   NETBLT packets.

   g. "L" is a bit that is set to 1 when the buffer that this DATA packet
   belongs to is the last  buffer in the transfer.

   h. New Burst Size:  Burst size as negotiated from value given by
   receiving NETBLT  in OK message.

   i. New Burst Interval: Burst interval as negotiated from value given by
   receiving  NETBLT in OK message.  Value is in milliseconds.

   NULL-ACK (type 7)

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    |           Checksum            |    Version    |     Type      |
    |           Length              |           Local Port          |
    |        Foreign Port           | Longword Alignment Padding    |
    |                    Last Buffer Touched                        |
    | High Consecutive Seq Num Rcvd |        New Burst Size         |
    |       New Burst Interval      |  Longword Alignment Padding |L|

   a. Last Buffer Touched: the number of the highest buffer transmitted so

   b. High Consecutive Sequence Number Received: same as in DATA/LDATA

   c. New Burst Size:  Burst size as negotiated (half- and full-duplex
   only) from value given by receiving NETBLT in OK message.

White                                                          [Page 20]
INTERNET DRAFT                   NETBLT                       April 1997

   d. New Burst Interval: Burst interval as negotiated (half- and full-
   duplex only) from value given by receiving  NETBLT in OK message.  Value
   is in milliseconds.

   e. "L" is a bit that is set to 1 when the buffer identified in the Last
   Buffer Touched field is the last buffer in the transfer.

   CONTROL (type 8)

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    |           Checksum            |    Version    |     Type      |
    |           Length              |           Local Port          |
    |        Foreign Port           | Longword Alignment Padding    |

   Followed by any number of messages, each of which is longword
   aligned, with the following formats:

   GO message (subtype 0)

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    |   Subtype     | Word Padding  |       Sequence Number         |
    |                        Buffer Number                          |

   a. Subtype: message type (GO = 0, OK = 1, RESEND = 2)

   b. Sequence number: A 16 bit unique message number.  Sequence numbers
   must be  monotonically increasing, starting with 1.

   c. Buffer number: as in DATA/LDATA packet

White                                                          [Page 21]
INTERNET DRAFT                   NETBLT                       April 1997

   OK message (subtype 1).

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    |   Subtype     | Word Padding  |       Sequence Number         |
    |                        Buffer Number                          |
    |    New Offered Burst Size     |   New Offered Burst Interval  |
    | Current control timer value   | Longword Alignment Padding    |

   a. New offered burst size: burst size for subsequent buffer transfers,
   possibly based  on performance information for previous buffer

   b. New offered burst interval: burst rate for subsequent buffer
   transfers, possibly  based on performance information for previous
   buffer transfers.  Rate is in  milliseconds.

   c. Current control timer value: Receiving NETBLT's control timer value
   in  milliseconds.

   RESEND message (subtype 2)

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    |   Subtype     | Word Padding  |       Sequence Number         |
    |                        Buffer Number                          |
    |  Number of Missing Packets    |  New Offered Burst Size       |
    |    New Offered Burst Interval |   Longword Alignment Padding  |
    | Packet Number (2 bytes/packet)| ...
                                    |    Padding (if necessary)     |

   a. Packet number:  the 16 bit data packet identifier of a  DATA packet,
   from the buffer identified by Buffer Number, whose retransmission  is
   requested. Multiple packet numbers may occur in one RESEND message.

6. NETBLT modes of operation.

White                                                          [Page 22]
INTERNET DRAFT                   NETBLT                       April 1997

   NETBLT supports three modes of operation; simplex, half-duplex, and
   full-duplex.  This section identifies the required components of
   NETBLT for simplex and half-duplex modes of operation. Across full-
   duplex connections the normal NETBLT as described above is used.

6.1 Simplex.

   The only NETBLT packet types used in the simplex case are the

   a. OPEN
   b. QUIT
   c. ABORT
   d. DATA
   e. LDATA
   f. NULL-ACK

6.1.1 Sender simplex operation.

   Operation of NETBLT in simplex send mode is as follows: the OPEN
   message is sent; DATA and LDATA packets are sent; and the connection
   is closed. Any packet may be sent more than once, for redundancy, but
   for all n, packets from buffer(n - 1) will not be sent after packets
   from buffer(n).  QUIT and ABORT packets may be sent at any time, and
   will have the same effect.  The Maximum Number of Outstanding Buffers
   (in the OPEN packet) is set to 2.

6.1.2 Receiver simplex operation.

   Operation of NETBLT in simplex receive mode is as follows: when an
   OPEN packet is received, a connection is considered to be
   established. Packets received are stored into NETBLT buffers. The
   receiving NETBLT will pass a buffer to the client when the buffer is
   filled with correct packets or when good packets for a higher-
   numbered buffer are received. A list of packets which are possibly
   bad, or missing, is passed to the client. When the last buffer (L
   flag set in packet headers) has been passed to the client, or when
   the death timeout has expired, the receiving connection is

   The receiving NETBLT will discard redundant packets.  In the case of
   errors, the following rules apply at the receiving NETBLT:

   a. A NETBLT packet with a bad header checksum is discarded.

   b. A NETBLT DATA or LDATA packet with a good header checksum and a
   bad data area checksum may optionally be saved but flagged as
   possibly bad. Reasonableness checks may be used to insure that good

White                                                          [Page 23]
INTERNET DRAFT                   NETBLT                       April 1997

   data is not affected by the possibly bad packet data.  If a good
   NETBLT packet (redundantly transmitted) is received with the same
   buffer and packet number as a possibly bad  one, the possibly bad
   packet is replaced with the good one.

6.2 Half-duplex.

   The normal, full-duplex version of NETBLT operates across half-duplex
   connections with the following modification: keepalive packets will
   not be sent by the receiver while it is in the process of receiving a
   packet.  The burst timer and burst size counter are reset at the
   start of each transmission period.  If the Maximum Number of
   Outstanding Buffers (in the OPEN packet) is set to 1, the sending and
   receiving NETBLTs will operate in lockstep.  If the Maximum Number of
   Outstanding Buffers is set to a value N greater than 1, the receiving
   NETBLT will wait until N buffers have been completely received or
   have had their data timers expire before sending a CONTROL packet.
   An exception occurs when the last buffer is sent; when all buffers up
   to and including the last buffer have been completely received or
   have had their data timers expire, the receiving NETBLT is permitted
   to send its CONTROL packet. The last buffer is identified by the
   receiver as the buffer for which the "L" bit is set in a DATA/LDATA
   packet, or as the Last Buffer Touched in a NULL-ACK packet with its
   "L" bit set to 1.

7. Security Considerations

   Security considerations for NETBLT operation have not been addressed
   in this document.

8. Possible Extensions

   Two forms of extension to NETBLT are being examined. The first would
   make it possible for NETBLT to operate as a unidirectional stream
   protocol, by allowing all buffers, not just the last one, to be of
   any size less than or equal to the negotiated size. The second would
   provide a "Transaction NETBLT", using an approach similar to
   Transaction TCP [RFC1379].

9. References

   [Jacobsen]  Jacobsen, V. "Congestion Avoidance and Control", ACM
   SIGCOMM 88 Symposium Proceedings, August 1988

   [MIL-STD]  MIL-STD-2045-44500 "Tactical Communications Protocol 2
   (TACO2) for the National Imagery Transmission Format Standard", June

White                                                          [Page 24]
INTERNET DRAFT                   NETBLT                       April 1997

   [RFC998]  Clark, D., Lambert, M., and Zhang, L. "NETBLT: A Bulk Data
   Transfer Protocol", RFC 998, March 1987

   [RFC1379]  Braden, R., "Extending TCP for Transactions -- Concepts",
   RFC 1379, November 1992

10. Author's Address

   John C. C. White
   The MITRE Corporation
   202 Burlington Road
   Bedford, MA 01730-1420

   Phone: 617-271-3284
   Fax:   617-271-2721

White                                                          [Page 25]