Internet DRAFT - draft-xie-stewart-sigtran-ddp

draft-xie-stewart-sigtran-ddp



Network Working Group                                      R. R. Stewart
INTERNET-DRAFT                                                    Q. Xie
                                                                Motorola

expires in six months                                      April 12,2000


                     Data Distribution Protocol (DDP)
                  <draft-xie-stewart-sigtran-ddp-01.txt>

Status of This Memo

This document is an Internet-Draft and is in full conformance with all
provisions of Section 10 of RFC 2026. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts.

The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt

The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.


Xie, Stewart                                                [Page  1]

Internet Draft   Data Distribution Protocol             April 2000



Abstract

Data Distribution Protocol (DDP) provides a fault tolerant data
transfer mechanism over IP networks. DDP uses a name-based addressing
model which isolates a logical communication endpoint from its IP
address(es), thus effectively eliminating the binding between the
communication endpoint and its physical IP address(es) which normally
constitutes a single point of failure.

In addition, DDP defines each logical communication destination as a
named group, providing full transparent support for server-pooling and
load sharing. It also allows dynamic system scalability - members of a
server pool can be added or removed at any time without interrupting
the service.

DDP is designed to take full advantage of the network level redundancy
provided by the Simple Transmission Control Protocol (SCTP) [1]. But
it can also use other transport protocol like TCP.

Internally, DDP is made up of two parts, namely the Data Distribution
Control Part (DDCP) and the Endpoint Name Resolution Part (ENRP).
DDCP provides the user interface for name to address translation, load
sharing management, and fault management.  ENRP defines the fault
tolerant name translation service.




Table Of Contents

1. Introduction
  1.1 Motivation
    1.1.1 CORBA and Its Limitations
    1.1.2 DNS and Its Limitations
  1.2 Protocol overview
  1.3 Organization of this document
  1.4 Definitions
2. The DDP Interfaces
  2.1 DDP User <-> DDCP Layer: Primitives and Notifications
    2.1.1 Registration.Request Primitive
    2.1.2 Deregistration.Request Primitive
    2.1.3 Connection.Request Primitive
    2.1.4 Disconnection.Request Primitive
    2.1.5 Data.Request Primitive
      2.1.5.1 Send by Name
        2.1.5.1.1 Receiver Endpoint Selection
        2.1.5.1.2 Round Robin
        2.1.5.1.3 Weighted Round Robin
        2.1.5.1.4 Least Used
        2.1.5.1.5 Least Used with Degradation
      2.1.5.2 Send by Handle
      2.1.5.3 Send by Transport Address
      2.1.5.4 Options
    2.1.6 Data.Received Notification
    2.1.7 Error.Report Notification
  2.2 DDP Layer <-> SCTP: Primitives and Notifications
    2.2.1 SCTP SEND Primitive
    2.2.2 SCTP RECEIVE Primitive
    2.2.3 SCTP SET.PRIMARY Primitive
    2.2.4 SCTP DATA.ARRIVE Notification
    2.2.5 SCTP SEND.FAILURE Notification
    2.2.6 SCTP COMMUNICATION.LOST Notification
    2.2.7 SCTP NETWORK.STATUS.CHANGE Notification
  2.4 Examples
    2.4.1 Send to an Unknown Name
    2.4.2 Send to a Cached Name
  2.5 Handle DDP to ENRP Communication Failures
    2.5.1 SCTP Send Failure
    2.5.2 T1-ENRPrequest Timer Expiration
    2.5.3 Handle ENDPOINT_KEEP_ALIVE Messages
3. The ENRP interface
  3.1 Functional Summary
    3.1.1 Basic ENRP Operations
      3.1.1.1 Endpoint Registration
      3.1.1.2 Endpoint De-registration
      3.1.1.3 Name Translation
      3.1.1.4 Server Name Space Update
        3.1.1.4.1 Addition of a New DDP Endpoint
        3.1.1.4.2 Removal of a DDP Endpoint
        3.1.1.4.3 Removal of a DDP Endpoint with no Ownership
        3.1.1.4.4 Update Endpoint Attributes
        3.1.1.4.5 Claim Endpoint Ownership
        3.1.1.4.6 Report Endpoint Failure
      3.1.1.5 Endpoint Change Routing Value
      3.1.1.6 Server Down Load Name Space from a Peer
      3.1.1.7 Server Monitor Peer Status
      3.1.1.8 Server Down Load Peer List
      3.1.1.9 Endpoint Initialization
      3.1.1.10 Server Initialization
    3.1.2 Fault Management Operations
      3.1.2.1 Detect and Report Unreachable Endpoint
      3.1.2.2 ENRP Server Heartbeat
      3.1.2.3 ENRP Server Hunt
      3.1.2.4 ENRP Server Detect and Take-over Inactive Peer 
      3.1.2.5 Register Homeless Endpoints
    3.1.3 Maintenance Operations
      3.1.3.1 Forceful Removal of Endpoint
      3.1.3.2 Dump Home Endpoint List
      3.1.3.3 Dump Remote Endpoint List
      3.1.3.4 Dump Peer Server List
  3.2 Message Summary
    3.2.1 Endpoint Entry
    3.2.2 REGISTRATION message
    3.2.3 DEREGISTRATION message
    3.2.4 REGISTRATION_RESPONSE message
    3.2.5 NAME_REQUEST message
    3.2.6 NAME_INFORMATION message
    3.2.7 NAME_UNKNOWN message
    3.2.8 UPDATE_ROUTING_VALUE message
    3.2.9 PEER_NAME_TABLE_REQUEST message
    3.2.10 PEER_NAME_TABLE_RESPONSE message
    3.2.11 PEER_LIST_REQUEST message
    3.2.12 PEER_LIST_RESPONSE message
    3.2.13 PEER_NAME_UPDATE message
    3.2.14 ENDPOINT_KEEP_ALIVE message
    3.2.15 PEER_PRESENCE message
    3.2.16 ENDPOINT_UNREACHABLE message
    3.2.17 TAKEOVER_INITIATE message
    3.2.18 TAKEOVER_INITIATE_RESPONSE message
    3.2.19 TAKEOVER_PEER_SERVER message
    3.2.20 SERVER_HUNT message
    3.2.21 SERVER_HUNT_RESPONSE message
    3.2.22 SERVER_DUMP message
    3.2.23 SERVER_DUMP_RESPONSE message
4 Variables, Time Values, and Thresholds
  4.1 Variables
  4.2 Time values
  4.3 Thresholds
5. References






1. Introduction

Data Distribution Protocol (DDP) has been designed to provide
fault-tolerant location transparent message distribution amongst
networked communication endpoints. DDP uses a name-based addressing
model which isolates a communication destination endpoint from its IP
address(es), thus effectively eliminating the binding between that
communication endpoint and its physical IP address(es) which normally
constitutes a single point of failure.

When multiple receiver instances exist under the same name, a.k.a,
server pool redundancy, DDP will select one receiver instance, based
on the current load sharing policy indicated by the name, and deliver
the message to the selected receiver instance.

While delivering the message, DDP monitors the reachability of
the selected destination instance. If it is found unreachable, before
notifying the sender the failure, DDP can automatically select another
instance (if one exists) under that name and attempt to deliver the
message to that instance. In other words, DDP is capable of
transparent fail-over amongst instances of a server pool if the
selected instance is unreachable.

DDP is composed of two parts, namely the Data Distribution Control
Part (DDCP) and the Endpoint Name Resolution Part (ENRP).  DDCP is
responsible for the abstraction of the underlying transport
technologies, load distribution management, fault management, as well
as the presentation to the upper layer (i.e., the DDP user) a unified
primitive interface.

ENRP is designed to provide a fully distributed fault-tolerant
real-time translation service that maps a name to a set of transport
addresses pointing to a specific group of networked communication
endpoints registered under that name.  ENRP employs a client-server
model with which an ENRP server will respond to the name translation
service requests from endpoint clients on both the local host and
remote hosts.

When SCTP [1] is used as the transport layer protocol, DDP can
seamlessly incorporate the link-layer redundancy provided by the SCTP.

This document defines both DDCP, ENRP client-server interface and the
ENRP server functionalities, including the establishment and
management of a fully distributed fault-tolerant endpoint name space.

In below, when describing the communication interface, the term DDP
layer and DDCP sub-layer will be used interchangeably unless otherwise
stated.


1.1 Motivation

In this section, we will discuss the motivation for developing
DDP. Our discussion will be focused on the analysis of the
inadequateness of two existing technologies, namely CORBA and DNS,
in providing solutions to fault-tolerance design of IP distributed
applications.

  [Editor's Note: we evaluated a number of other options before and
   during the development of DDP. Over 100 different DPE
   products/designs were considered. After extensive research we could
   not find any that addressed the fault-tolerant concerns and
   real-time in concert.] 


1.1.1 CORBA and Its Limitations

Often referred to as a Distributed Processing Environment (DPE),
CORBA was mainly desinged to provide location transparency for
distributed applications. However, the following limitations may exist
when applying CORBA to the design of real time fault-tolerant system:

 1) CORBA is traditionally weak in fault tolerance. The recent
    development of a fault tolerant version of CORBA by OMG is perhaps
    a step in the right direction towards fixing this
    weakness. Nevertheless, the maturity, implementablity, and
    real-time performance of the design is yet to be proven.

    [Editor's Note: the fault tolerance mechanism being developed by
     OMG for CORBA bears quite some similarities to DDP.]

 2) CORBA's distribution model encourages an object-based view,
    i.e., each communication endpoint is normally an object. We
    consider this kind of granularity too fine to be efficient and
    effective for designing real-time fault-tolerant applications. 

 3) CORBA in general has a large signature that makes the use of it a
    challenge in real-time environments. Small devices with limited
    memory and CPU resource (e.g., H323 or SIP terminals) will find
    CORBA hard to fit in. 

 4) CORBA uses TCP as its transport (Note, some effort is currently
    underway to separate CORBA from its transport). This makes CORBA
    suffer from the same limitations of TCP in terms of real-time and
    fault-tolerance performance.

 5) CORBA has long lacked easily usable support for the asynchronous
    communication model, and this may be an issue in many 
    applications. An apparently improved API for asynchronous
    communication has been added to the CORBA standards only recently,
    and many, if not most, CORBA implementations still do not support
    it. There is as yet insufficient user experience with it to make
    conclusions regarding this new feature's usability. 


1.1.2 DNS and Its Limitations

Undoubtedly DNS is the best-known and provn IP namespace mechanism.
However, namespace function alone does NOT provide real time
fault-tolerance solution. Other mechanisms and procedures, such as
server process failure detection, back-up server control and
selection, fast fail-over/switch-over, load balancing, etc., also play
crucial roles. These functions are supported by DDP but not by DNS
(unless substantial enhancement can be made to DNS).

As will be further elaborated later in this document, DDP design
consists of two parts, namely DDCP and ENRP, where ENRP defines a
light-weight yet highly efficient namespace mechanism optimized for
building real time fault-tolerant applications. Nevertheless, DDP does
not restrict itself to ENRP for namespace services. In fact, it is not
only feasible but also desirable in the future to generalize DDP
design so that the ENRP can provide a generic interface that is
capable of inter-working with different namespace, including DNS, at
the DDP implementor's choice.

In the following, we list some limitations of the current DNS
namespace capability when compared to that of ENRP:

 1) DNS name registration and translation services have been primarily
    optimized for host names. But we consider namespace services
    optimized for process names or endpoint names more appropriate and
    efficient for supporting real time server-level fault-tolerance
    applications.

 2) DNS is primarily passive. It provides a query/response database
    that allows one to find information, but does NOT provide 
    monitoring of hosts or processes to assure consistency within
    its database.

 3) DNS has dynamic extensions but is not designed around the
    dynamic fast changing process address space that is typical to
    real time distributed applications.

It has also been suggested that DDP be extended to work with DNS to
bridge multiple DDP planes and provide an "inter-DDP-Domain" bridging
function. 


1.2 Protocol overview

At startup, each endpoint in a DDP operational domain registers its
name to the name space. Here a name is defined as a NULL terminated
ASCII string of fixed length.

When sending a message, the sender addresses the receiver endpoint by
its name and passes the message to its DDP layer. The DDP layer, with
the help from the ENRP name space daemon server(s), translates the
name to a valid transport address (or a list of transport addresses if
the receiver is multi-homed) and route the message to the receiving
endpoint.

The following diagram illustrates the components of DDP and their
relationships.

                           Figure 1.


                          Data Sender          Data Receiver
              ENRP        (DDP-user)            (DDP-user)
             Server       +---------+           +---------+
                          |   DDCP  |<---//---->|   DDCP  |
            +------+      |------+  |           |------+  |
     <----->| ENRP |<---->| ENRP |  |           | ENRP |  |
 To peer    +------+      +---------+           +---------+
 ENRP server| SCTP |      |   SCTP  |           |   SCTP  |
            +------+      +---------+           +---------+
            |  IP  |      |    IP   |           |    IP   |
            +------+      +---------+           +---------+
               |_______________|_____________________|


Multiple endpoints can register themselves under the same name. In
that case they will be treated as a receiver pool, and DDP, when
routing a message destined to that name, will use a predefined
load-sharing policy to determine which endpoint(s) in the pool will
receive the message.


Stewart & Xie                                                  [Page  4]

                      Data Distribution Protocol                Feb 1999


DDP design has a high emphasis on seamless support of "server
pooling", system fault tolerance, dynamic scalability, and
close-to-real-time name translation. 

In particular, DDP can be characterized by: 

A) Seamless support of "server pooling" --- DDP allows multiple
   servers to register under the same name. It also allows servers to
   be dynamically add to or remove from a server pool without any
   reconfiguration. 

B) Support automatic receiver "fail-over" --- when the chosen message
   receiver fails, DDP, with pre-stated permission from its upper
   layer, can automatically re-direct the message to an alternative
   server under the same name if one exists.  

C) Transaction management by nickname or "association handle" --- this
   is to allow a continuous transaction or session consisting of
   multiple interactions be held between a client endpoint and and one
   particular server in a server pool. 

D) Fully distributed name space --- For achieving a high degree of
   fault tolerance and operation efficiency, the ENRP daemons which
   provide name translation service and the name space data are
   distributed across the operational scope of the network. 

   ENRP daemon servers can be added to or removed from the operation
   scope dynamically, without interrupting the current name translation
   service. 

   For example, a node may be originally configured to operate without a
   local ENRP server. When the load condition changes, one can start a
   new ENRP server on that node to increase the operation capacity. The
   new ENRP server will automatically integrate itself with the existing
   ENRP server(s) in the scope.

   Similarly, when an ENRP server becomes unavailable for service (e.g.,
   being intentionally shutdown, or suffered failure), its DDP clients
   will be automatically taken-over by a remote ENRP server and
   continuously be served. 

E) Network failure detection and automatic recovery  --- In the case 
   when a major network failure breaks the operation scope into 
   isolated communication islands, the name translation service will
   survive and continue inside each island so long as there is one or
   more ENRP servers present in that island. Endpoints inside each
   island will still continue to be able to communicate with each other.
   Moreover, when the network recovers, the isolated ENRP servers will
   re-discover each other and re-integrate the name space back into
   its original form. 

Figure 2. shows an example of distributed applications operating in a
scope that is connected by a pare of redundant networks. 


                        Figure 2.

        Node 1                      Node 2
       +--------------+   ||       +--------------+
       |              |   ||   ||  |              |
       |  Apps1       |   |+===||==|              |
       |              |   ||   ||  |   Apps2      |
       |              |===+|   ||  |              |
       |        Apps2 |   ||   |+==|        Apps3 |
       |              |   ||   ||  |              |
       |              |========+|  |              |
       | (ENRP Svr)   |   ||   ||  | (ENRP Svr)   |
       +--------------+   ||   ||  +--------------+
                          ||   ||
        Node 3            ||   ||   Node 4         
       +--------------+   ||   ||  +--------------+
       |              |   ||   ||  |              |
       |  Apps2       |   ||   ||  |              |
       |              |========+|  |    Apps3     |
       |              |   ||   ||  |              |
       |        Apps4 |   ||   ||  |              |
       |              |===+|   |+==| Apps1        |
       |              |   ||   ||  |              |
       | (ENRP Svr)   |   |+===||==|              |
       +--------------+   ||   ||  +--------------+
                          ||   || 
                    network1  network2


In this example, there are four nodes in the scope, as shown in Figure
2. On Node 1, Node 2, and Node 3, there is an ENRP server running. On
each of the nodes, there are also some applications running. Each
application has a registered name in the name space collectively
managed by the three ENRP servers. In the example, the registered
names are "Apps1", "Apps2", "Apps3", and "Apps4". Some of the
applications (Apps1, Apps2, and Apps3) are distributed as server
pools.

When sending messages to each other, the sender application simply
addresses the recipient by its registered name. The DDP layer inside
the sender application will query its home ENRP server to obtain the
transport address(es) and load control information of the recipient
before sending the message out.

Also note in the example, there is no ENRP server on Node 4. But the
applications on Node 4 will be served by one of the ENRP servers on
other nodes.




1.3 Organization of this document

In Chapter 2 we entails DDCP, focusing on the communication primitives
between the applications above DDP and DDCP, and the communications
primitives between DDCP and SCTP (or other transport layer). Also
included in this discussion is relevant timers and configurable
parameters as appropriate.

Chapter 3 entails the ENRP description. ENRP defines the messaging
structure and relevant rules for communications between an DDP
endpoint and an ENRP server. This chapter discusses how an endpoint
discovers its ENRP server, the messaging that goes on between the
endpoint and the ENRP server, and the messaging that goes on between
all ENRP servers in the system.


1.4 Definitions

This document uses the following terms:

Operation scope --- the part of the network visible by ENRP.

DDP Endpoint --- a logical entity in the operation scope which
implements the DDP stack and is capable of sending and receiving
messages. 

DDP Node --- a host machine in the network which contains one or
more DDP endpoints. 

Endpoint name --- the registered tag of a DDP endpoint, consisting of
a NULL terminated ASCII string of fixed length.

Named group --- a group of DDP endpoints sharing the same endpoint
name in the name space.

Endpoint handle --- a logical pointer, consisting of a name and the 
primary destination transport address to a particular endpoint in a
named group.

ENRP client --- a DDP endpoint using ENRP to obtain name translation
and other related services. In this document, the term "DDP endpoint" is
exchangeable with "ENRP client", unless otherwise stated.

ENRP maintenance client --- a DDP endpoint that has the additional
capability of exchanging ENRP maintenance messages with an ENRP
server in order to perform certain maintenance functions.

ENRP server --- a server program running on a node that manages the
name space collectively with its peer ENRP servers and replies to the
service requests from any ENRP client.

Home ENRP server --- the ENRP server to which an endpoint currently
belongs. Endpoints normally choose the ENRP server on their local
host as the home ENRP server, if one exists.  An endpoint shall only
have one home ENRP server at any given time, and both the endpoint and
the server shall keep track of this master/slave relationship between
them.

ENRP server takeover --- the event that a remote ENRP server takes the
ownership of all the ENRP endpoints on a node and becomes their home
server.

Caretaker ENRP server --- The ENRP server on a remote node which takes
ownership of all endpoints on the local node because of the absence of
an active local server.

ENRP client channel --- the communication channel through which a DDP
endpoint requests for ENRP service. The ENRP client channel is usually
defined by the transport address of the home server and a well known
port number.

ENRP server channel --- defined by a well known multicast IP address
and a well known port number. All ENRP servers in an operation scope
can communicate with one another through this channel. Endpoints are
also allowed to communicate on this channel occasionally.

ENRP name domain --- defined by the combination of the ENRP client
channel and the ENRP server channel in the operation scope.

Network Byte Order: Most significant byte first, a.k.a Big Endian.

2. The DDP Interfaces

This chapter will focus primarily on the primitives and notifications
that form the interface between the DDP-user and the DDCP and that
between DDCP and its lower layer transport protocol (e.g., SCTP).

Appropriate timers and recovery actions for failure detection and
management are also discussed. 


2.1 DDP User <-> DDCP Layer: Primitives and Notifications

A DDP user passes primitives to the DDCP sub-layer to request certain
actions. Upon the completion of those actions or upon the detection of
certain events, the DDCP will notify the DDP-user with notifications.


2.1.1 Registration.Request Primitive

Format: registration.request(endpointName)

where the endpointName parameter contains a NULL terminated ASCII
string of fixed length.

The DDP user must register its name with the ENRP server by using this
primitive before other DDP endpoints in the name space can send message
to this DDP user by name or by handle (see Sections 2.1.5.1? and
2.1.5.2??).

In response to the registration primitive, the DDP layer shall send
out to its home ENRP server a REGISTRATION message (see Section
3.1.1.1?), and start a T2-registration timer. 

If the T2-registration timer expires before receiving a
REGISTRATION_RESPONSE message, or a SEND.FAILURE notification is
received from the SCTP layer, the DDP layer shall start the Server
Hunt procedure (see Section 2.5.4??) in an attempt to get service from
a remote ENRP server. 


2.1.2 Deregistration.Request Primitive

Format: deregistration.request()

The DDP user invokes this primitive to remove itself from the name
space. This should be used as a part of the graceful shutdown
process by the application. 

A DEREGISTRATION message will be sent by DDP layer to the home ENRP
server (see Section 3.1.1.2?). 


2.1.3 Connection.Request Primitive

Format: connect.request(destinationAddress, typeOfAddress)

If the address type is a name and local name translation cache exists,
the DDP layer should initiate a mapping information query on the name
and update it local cache when the response comes back from the ENRP
server.


2.1.4 Disconnection.Request Primitive

Format: disconnect.request(destinationAddress, typeOfAddress)

If the address type is a name and local name translation cache exists,
the DDP layer should remove the mapping information on the name from
its local cache.


2.1.5 Data.Request Primitive

Format: data.request(destinationAddress, typeOfAddress, message,
                     sizeOfMessage, Options); 

This primitive requests DDCP to send a message to some specified
receiver within the name space.

Depending on the address type used for the send request, the
sender's DDCP layer may perform address translation and receiver
selection before sending the message out.

The data.request primitive can take the following forms of address
type: 


2.1.5.1 Send by Name

In this case the destinationAddress and typeOfAddress together
indicates a name.

This is the simplest form of data.request primitive. By default, this
directs DDP to send the message to one of the endpoints in the named
group.

Before sending the message out to the named group, the sender's DDP
layer must first perform a name to address translation. It may also need
to perform receiver selection if multiple endpoints exist in the group.

If the sender's DDP implementation does not support local cache of the
translation information or if it does not have the translation
information on that named group in its local cache, it will transmit a
request for the name mapping information to the ENRP server, and hold
the outbound message in queue while waiting for the response from the
ENRP server (any further send request to this named group before the
ENRP server responds should also be queued).

Once the necessary mapping information arrives from the ENRP server,
the sender's DDP will:

  A) map the name into a list of transport addresses of the
     destination endpoint,

  B) if no association exists towards the destination, establish a
     new SCTP association, 

     NOTE: if the underlying SCTP implementation supports implicit
     association setup, this step is not needed (see [??]).

  C) send out the queued message to the SCTP association using the
     SEND primitive (see [???]), and, 

  D) if the local cache is implemented, append/update the local cache
     with the mapping information received in the ENRP server's
     response. Also, record the local SCTP association id if a new
     association was created.

For more on the ENRP server request procedures see Section 3.1.1.5??.

If multiple endpoints exist in that named group, DDP will choose one
of them and transmit the message to it.  In that case, the choice of
the receiver is made by DDP layer of the sender based on the routing
policy as discussed in section 2.1.1.1.1??.

Optionally, the DDP layer of the sender may return the SCTP
association handle of the selected receiver endpoint to the
application after sending the message. This handle can then be used
for future transmissions to that receiver endpoint (see Section
2.1.1.2??).

Section ??? defines the fail-over procedures for cases where the
selected endpoint is found unreachable.


2.1.5.1.1 Receiver Endpoint Selection

Each time a DDP user sends a message to a named group that contains
more than one endpoint, the sender's DDP layer must select one of the
endpoints in the named group as the receiver of the current
message. The selection is done according to the current routing policy
of the named group to which the message is sent.

Note, no selection is needed if the DDP_SEND_TOALL option is set (see
Section 2.1.1.4?).

When joining a named group, along with its registration each endpoint
specifies its preferred routing policy for receiving messages sent to
this named group. But only the routing policy specified by the first
endpoint joining the named group will become the current routing
policy of the group.

Moreover, together with the routing policy, each endpoint can also
specify a Routing Value for itself at the registration time. The
meaning of the routing value depends on the current routing policy of
the group. An endpoint can also change its routing value whenever it
desires, see Section 3.1.1.6?? for details.

Note, if this first endpoint removes itself from the named group
(e.g., by de-registration from the name space) and the remaining
endpoints have specified conflicting routing policies at their
corresponding registrations, it is implementation specific to
determine the new current routing policy.


Four basic routing policies are defined in DDP, namely the Round
Robin, Least Used, and Least Used Degrading. The following
sections describes each of these policies.


2.1.5.1.2 Round Robin Policy

When a DDP endpoint sends messages by name and Round-Robin is the
current policy of that named group, the DDP layer of the sender will
select the receiver for each outbound message by round-Robining
through all the registered endpoints in that named group, in an
attempt to achieve an even distribution of outbound messages.


2.1.5.1.4 Least Used Policy

When the destination named group is under the Least Used routing
policy, the DDP layer of the message sender will select the endpoint
that has the lowest routing value in the group as the receiver of
the current message. If more than one endpoint from the group share
the same lowest routing value, the selection will be done round Robin
amongst those endpoints.

It is important to note that this policy means that the same endpoint 
will be always selected as the message receiver by the sender until
the load control information of the name is updated and changed in the
local cache of the sender (see section 2.1.2.4??). 


2.1.5.1.5 Least Used with Degradation Policy

This policy is the same as the Least Used policy with the exception
that, each time the endpoint with the lowest routing value is selected
from the group as the receiver of the current message, its routing
value is incremented, and thus it may no longer be the lowest value in
the group.

This provides a degradation of the policy towards round Robin policy
over time. As with the Least Used policy, every local cache update at
the sender will bring the policy back to Least Used with Degradation.


2.1.5.2 Send by Handle

In this case the destinationAddress and typeOfAddress together
indicates a DDP endpoint handle.

This requests the DDP layer to deliver the message to the endpoint
identified by the handle.

The handle should contains the name and the primary destination
transport address of the destination endpoint.

The DDP layer shall use the address to identify the SCTP association
(or to setup a new one if necessary) and then invoke the SEND
primitive to send the message out.

If local cache is supported and the mapping information for the name
found in the handle is not available in the local cache, the sender's
DDP layer should, after sending the message, also transmit a request
for the name mapping information to the ENRP server. Once the
necessary mapping information arrives from the ENRP server, the
sender's DDP will update its local cache with the newly received
mapping information for that name.

Section ??? defines the fail-over procedures for cases where the
endpoint pointed to by the handle is found unreachable.

Optionally, the DDP layer may return the actual handle to which the
message was sent (this may be different from the handle specified when
the primitive is invoked, due to the possibility of automatic
fail-over). 


2.1.5.3 Send by Transport Address

In this case the destinationAddress and typeOfAddress together
indicates an SCTP transport address.

This directs the sender's DDP layer to send the message out to the
specified transport address.

No endpoint fail-over is support when this form of send request is
used.


2.1.5.4 Options

The Options parameter passed in the various forms of the above
data.request primitive gives directions to the sender's DDP layer on
special handling of the message delivery. 

Options can be grouped as follows: 

  - endpoint fail-over (allowed, or prohibited),
  - whether to send to one endpoint or to the whole named group,
  - whether to send to the same receiver endpoint last sent to within
    the named group, and
  - options passed to the SCTP transport protocol.

The complete list of Options is as follows:

DDP_USE_DEFAULT: 0x0000

   Use default setting.

DDP_SEND_FAILOVER: 0x0001

   Enables endpoint fail-over on this message. In case where the first
   selected receiver endpoint or the endpoint pointed to by the handle
   is found unreachable, this option allows the sender's DDP layer to
   re-select an alternate receiver endpoint from the same named group
   if one exists, and silently re-send the message to this newly
   selected endpoint. 

   Endpoint unreachable is normally indicated by the Communication
   Lost or Send Failure notification from SCTP.


DDP_SEND_NO_FAILOVER: 0x0002

   This option prohibits the sender's DDP layer from re-sending the
   message to any alternate receiver endpoint in case that the first
   selected receiver endpoint or the endpoint pointed to by the handle
   is found unreachable. Instead, the sender's DDP layer shall notify
   its upper layer about the unreachability with an Error.Indication
   and any unsent data.

DDP_SEND_TO_LAST: 0x0004

   This option requests the sender's DDP layer to send the message to
   the same receiver endpoint in the named group that the previous
   message was sent to.  

DDP_SEND_TOALL: 0x0008

   When sending by name, this option directs the sender's DDP layer to
   send a copy of the message to all the endpoints, except for the
   sender itself, in that named group.

DDP_SEND_TOSELF: 0x0010. 

   This option only applies in combination with DDP_SEND_TOALL option.
   It permits the sender's DDP layer also deliver a copy of the
   message to itself (i.e., loopback).

DDP_SCTP_BUNDLE: 0x0100

   This option allows the local SCTP transport layer to bundle the
   outbound messages whenever possible into bigger datagrams before
   transmitting them onto the network.

DDP_SCTP_NO_BUNDLE: 0x0200

   This option disallows the local SCTP transport layer to bundle
   outbound messages. 

DDP_SCTP_HB_ON: 0x0400

   This option instructs the local SCTP transport layer to turn on
   heartbeat on the SCTP association indicated by the
   destinationAddress parameter.

DDP_SCTP_HB_OFF: 0x0800

   This option instructs the local SCTP transport layer to turn off
   heartbeat on the SCTP association indicated by the
   destinationAddress parameter.

DDP_SCTP_UNORDER: 0x1000
   This option instructs the SCTP transport layer to send the current
   message using un-ordered delivery.


2.1.6 Data.Received Notification

Format: data.received(messageReceived, sizeOfMessage, senderAddress,
                        typeOfAddress)

When a new user message is received, the DDP layer of the receiver
uses this notification to pass the message to its upper layer.

Along with the message being passed, the DDP layer of the receiver
should also indicate to its upper layer the message sender's
address. The sender's address can be in the form of either an SCTP
association id, or a DDP endpoint handle. 

  A) If the name translation local cache is implemented at the receiver's
     DDP layer, a reverse mapping from the sender's IP address to the
     endpoint name should be performed and if the mapping is 
     successful, the sender's DDP endpoint handle should be
     constructed and passed in the senderAddress field. 

  B) If there is no local cache or the reverse mapping is not
     successful, the SCTP association handle should be passed in 
     the senderAddress field.


2.1.7 Error.Report Notification

Format: error.report(destinationAddress, typeOfAddress,
                         failedMessage, sizeOfMessage)

An error.report should be generated to notify the DDP user about
failed message delivery as well as other abnormalities (see Section
2.2.3 for details).

The destinationAddress and typeOfAddress together indicates to whom
the message was originally sent. The address type can be either a DDP
endpoint handle, association id, or a transport address.

The original message (or the first portion of it if the message is
too big) and its size should be passed in the failedMessage and
sizeOfMessage fields, respectively.


2.2 DDP Layer <-> SCTP: Primitives and Notifications

This section gives a brief description on some SCTP notifications and
primitives that the DDP layer uses. See Section 9 in [??] for more
information on SCTP primitives and notifications.


2.2.1 SCTP SEND Primitive

Basic Format: 

  SEND(association id, buffer address, byte count, options)
  -> result

The outbound message will be held in the buffer when this primitive is
invoked. The DDP layer shall identify the SCTP association which
connects to the intended destination and fill in the 'association id'.

The options field will hold the options destined to the SCTP transport
layer (see 2.1.1.4??).

The returned 'result' can indicate whether there is any local error
executing the primitive.


2.2.2 SCTP RECEIVE Primitive

Basic Format: 

  RECEIVE(association id, buffer address, buffer size)
  -> byte count

This primitive reads the first user message out from SCTP in-queue if
there is one available, and put the it into the specified buffer. The
size of the message read, in octets, will also be returned.


2.2.3 SCTP SET.PRIMARY Primitive

Basic Format: 

  SET.PRIMARY(association id, destination transport address)
  -> result

This can be used to instructs the SCTP to use the specified
destination transport address as the new primary destination address 
for sending messages.


2.2.4 SCTP DATA.ARRIVE Notification

SCTP layer invokes this notification when a user message is
successfully received and ready for retrieval. This shall prompt the
DDP layer to invoke the RECEIVE primitive to get the data (see
2.2.2??). 


2.2.5 SCTP SEND.FAILURE Notification

If a message can not be delivered to the specified association id
for any reason, SCTP will invoke this notification to notify DDP.

In response, the DDP shall take the following steps:

A) If the message was originally sent by name or handle and with
   option DDP_SEND_FAILOVER set, retransmit the message to an
   alternate endpoint of the same name if one exists in the named
   group. The proper routing policy shall be followed if more than one
   alternates exist in the group.

B) If no alternate exists or option DDP_SEND_FAILOVER is not set when
   the message was originally sent, generate an Error.report to
   report the failure to the DDP user. 


2.2.6 SCTP COMMUNICATION.LOST Notification

When SCTP loses communication to an endpoint completely or detects
that the remote endpoint has performed an abort or graceful shutdown
operation, it invokes this notification to notify DDP layer.

When handling this notification DDP shall report this event to its
ENRP server via an ENDPOINT_UNREACHABLE message with the severity
level set to NORMAL_REPORT (see 3.1.2.2??). 

If local mapping cache is implemented, the DDP layer should also mark
the endpoint as unreachable in its local cache. And if all the
endpoints in that named group are marked as unreachable, the DDP layer
should remove the named group from its local cache.


2.2.7 SCTP NETWORK.STATUS.CHANGE Notification

The SCTP sends this notification to the DDP layer when the
reachability status of a transport address of a specific SCTP
association has changed.

If the local mapping cache is supported, the DDP layer, upon
reception of this notification, should look up the information of
this endpoint in its local cache and record the reachability change.

If the address in question becomes unreachable and is the primary
address of the association, the DDP layer MAY also elect a new primary
for this association by invoking the SET.PRIMARY primitive (Section
2.2.3??).

If the local cache is not support or the reverse look up does not
succeed, DDP takes no action.


2.4 Examples


2.4.1 Send to an Unknown Name

This example shows the event sequence when the user of DDP endpoint 1
(EP1) sends message "hello world" to a name which is not in the 
local mapping cache (assuming local caching is supported). 


   ENRP Server                   EP1                    new-name:EP2
        |                         |                          |
        |                       +---+                        |
        |                       | 1 |                        |
        |   2. NAME_REQUEST     +---+                        |
        |<------------------------|                          |
        |                       +---+                        |
        |                       | 3 |                        |
        |   4. NAME_REPONSE     +---+                        |
        |------------------------>|                          |
        |                       +---+                        |
        |                       | 5 |                        |
        |                       +---+  6. "hello world"      |
        |                         |------------------------->|
        |                         |                          |


  1) The user at EP1 invokes:

     data.request("new-name", name-type, "hello world", 11, 0);

     The DDP layer, in response, looks up the name "new-name" in its
     local cache but fails to find it. 
  2) The DDP layer of EP1 queues the message, and sends a NAME_REQUEST
     request to the ENRP server asking for all information about
     "new-name". 
  3) A T1-ENRPrequest timer is started while the DDP layer is waiting
     for the response from the ENRP server.
  4) The ENRP Server responds to the query with a NAME_REPONSE message
     that contains all the information about "new-name".
  5) DDP at EP1 cancels the T1-ENRPrequest timer and populate its
     local cache with information on "new-name".
  6) Based on the routing policy of "new-name", DDP at EP1 selects the
     receiver (EP2), sets up, if necessary, an SCTP association
     towards EP2 (explicitly or implicitly), and send out "hello
     world" message. 


2.4.2 Send to a Cached Name

This shows the event sequence when the DDP user at EP1 sends another
message to the "new-name".


   ENRP Server                   EP1                    new-name:EP2
        |                         |                          |
        |                       +---+                        |
        |                       | 1 |                        |
        |                       +---+  2. "hello world 2"    |
        |                         |------------------------->|
        |                         |                          |


  1) The user at EP1 invokes:

     data.request("new-name", name-type, "hello world 2", 13, 0);

     The DDP layer, in response, looks up the name "new-name" in its
     local cache and find the mapping information.
  2) Based on the routing policy of "new-name", DDP at EP1 selects the
     receiver (assume EP2 is selected again), and send out "hello
     world 2" message (assume the SCTP association is set up). 


2.5 Handle DDP to ENRP Communication Failures

Three types of failure may occur when the DDP layer at an endpoint
tries to communicate with the ENRP server:

 A) SCTP send failure
 B) T1-ENRPrequest timer expiration
 C) Registration failure

Registration failure is discussed in section 2.1.1??.


2.5.1 SCTP Send Failure

This indicates that the SCTP layer failed to deliver a message sent to
the ENRP server. In other words, the ENRP server is currently
unreachable.

In such a case, the DDP layer should not re-send the failed
message. Instead, it should discard the failed message and start the
ENRP server hunt procedure as described in Section 3.1.2.3??.


2.5.2 T1-ENRPrequest Timer Expiration

When a T1-ENRPrequest timer expires, the DDP should re-send the
original request to the ENRP server and re-start the T1-ENRPrequest
timer. In parallel, a SERVER_HUNT message should be issued per
Section 3.1.2.3??.

This should be repeated up to 'max-request-retransmit' times. After
that, an Error.Report notification should be generated to inform the
DDP user and the ENRP request message associated with the timer should
be discarded.


2.5.3 Handle ENDPOINT_KEEP_ALIVE Messages

At times, a DDP endpoint may receive ENDPOINT_KEEP_ALIVE messages
(see section 3.1.2.1?) from the ENRP server. This message requires no
response and should be silently discarded by the DDP layer. However,
each time when an ENDPOINT_KEEP_ALIVE is received, the endpoint should
update its home ENRP server to the sender of the latest Keep Alive
message.


3. The ENRP interface

This section discusses the messages and procedures for communicating
between the DDP layer of a DDP endpoint and an ENRP name space server,
as well as that between peer ENRP servers.


3.1 Functional Summary

In this section, we discuss the functions defined by ENRP. The
functions are divided into three groups, namely the basic ENRP
operations, fault management, and control and maintenance functions.

Most of the ENRP operations involve message exchanges between an ENRP
server and a DDP endpoint, as well as message exchanges between the
ENRP server and its peers.


3.1.1 Basic ENRP Operations

3.1.1.1 Endpoint Registration

ENRP server <-> endpoint:

A DDP endpoint shall send a REGISTRATION message, over the ENRP client
channel, to its home ENRP server, in order to register itself with the
name space.

In the REGISTRATION message, the endpoint shall indicate its name in
the form of a character string, network access information (e.g., a
list of valid transport addresses with which the endpoint
can be reached), and load control information.

The ENRP server shall handle the REGISTRATION message following the
rules listed below:

o If the name does not exist in the name space, the ENRP server shall
  create the name and add the new endpoint under that name.

o If the name already exists in the name space, the requesting
  endpoint shall be added under the same name and be made a member of
  the named group.

o If both the name and the requesting endpoint already exist in the
  name space, i.e., a case of duplicated registration, the ENRP 
  server shall grant the request without taking any further actions.

o The ENRP server may reject the registration due to reasons such as
  invalid values, lack of resource, etc.

In all the above cases, if the REGISTRATION request is granted, the
ENRP server shall assume the ownership of the requesting endpoint.

In response, the home ENRP server shall reply to the requesting
endpoint with a REGISTRATION_RESPONSE message, and shall indicate in
the message body whether the registration is granted or rejected.

ENRP server <-> peers:

If the registration request is not a duplicate and is granted, the
home ENRP server shall take the name space modification action
described in section 3.1.1.8??. Otherwise, no message shall be
exchanged with its peers.


3.1.1.2 Endpoint De-registration

ENRP server <-> endpoint:

A DDP endpoint shall send a DEREGISTRATION message, over the ENRP
client channel, to its home ENRP server in order to remove itself from
the name space.

If the endpoint is the last one under that name in the name space the
home ENRP server shall remove the name from its space as well.

The ENRP server may reject the de-registration request due to reasons
such as invalid parameters, etc.

In response, the home ENRP server shall send a REGISTRATION_RESPONSE
message to the endpoint, and shall indicate in the message body
whether the request is granted or rejected.

It should be noted that de-registration does not stop the DDP endpoint
from sending or receiving messages. It only means that other DDP
endpoints will no longer be able to send message to that endpoint by
name.

ENRP server <-> peers:

Once the de-registration request is granted and the endpoint removed
from its local copy of the name space, the home ENRP server shall take
the name space modification action described in section 3.1.1.9.


3.1.1.3 Name Translation

ENRP server <-> endpoint:

An endpoint shall send a NAME_REQUEST messages to its home ENRP server
to get a name translation service. In the NAME_REQUEST message, the
endpoint shall include the name it wants to be translated.

If the name exits in the name space, the ENRP server shall send back a
NAME_INFORMATION message that shall carry all information of the DDP
endpoint(s) currently registered under that name, including current
load control policy of the group, routing value of each endpoint in
the group, and a list of transport addresses for each endpoint in the
group with which the endpoint can be reached, etc.

If the name does not exist in the name space, the ENRP server shall
respond with a NAME_UNKNOWN message.

ENRP server <-> peers:

None.


3.1.1.4 Server Name Space Update

This includes a set of update operations used by an ENRP server to
inform its peer(s) the addition a new DDP endpoint, removal of an
existing DDP endpoint, change property of a named group, etc.


3.1.1.4.1 Addition of a New DDP Endpoint

When a new DDP endpoint is granted registration to the name space, the
home ENRP server uses this procedure to inform all its peers.

ENRP server <-> endpoint:

None:

ENRP server <-> peers:

An ENRP server shall multicast over the ENRP server channel a
PEER_NAME_UPDATE message with the appropriate flag set to indicate to its
peers about the addition of the new endpoint to the name space.

Upon the reception of this PEER_NAME_UPDATE message, each of the peer ENRP
servers shall take the following actions:

o If the name does not exist in its local copy of the name space, the
  peer ENRP server shall create the name and add the new endpoint
  under that name in its local name space copy, along with other
  attributes about the endpoint carried in the message.

o If the name already exists in the peer server's local copy of the
  name space, the new endpoint endpoint shall be added as a new member
  of the named group.

o If both the same DDP endpoint already exists in the named group in
  the local copy of the name space of the peer, the peer ENRP server
  shall take no actions.

After adding the endpoint into its local copy of name space, the peer
ENRP server shall check if this endpoint is located on the same host
as the peer ENRP server itself does. If so, the peer ENRP server shall
assume the ownership of the endpoint, and take the ?? actions
described in section 3.1.1.12??.


3.1.1.4.2 Removal of a DDP Endpoint

This procedure is used by an ENRP server to inform its peer(s) to
remove an existing DDP endpoint, regardless of the ownership of the
endpoint. 

ENRP server <-> endpoint:

None:

ENRP server <-> peers:

The ENRP server shall multicast over the ENRP server channel a
PEER_NAME_UPDATE message with the appropriate flag set to instruct its
peers to remove of the endpoint from their local copy of the name
space.

On the reception of this PEER_NAME_UPDATE message, each peer ENRP
server shall find and remove the DDP endpoint from its local copy of
the name space regardless whether or not it has ownership on the
endpoint.


3.1.1.4.3 Removal of a DDP Endpoint with no Ownership

This operation is used by an ENRP server to instruct its peers to
remove an existing DDP endpoint which the peer does not have an 
ownership on.

ENRP server <-> endpoint:

None:

ENRP server <-> peers:

An ENRP server shall multicast over the ENRP server channel a
PEER_NAME_UPDATE message with the appropriate flag set to instruct its
peers to remove the specified endpoint from its local copy of the name
space IF the peer does not have ownership on the endpoint.

On the reception of this PEER_NAME_UPDATE message, a peer ENRP server
shall find and remove the endpoint from its local copy of the name
space only if the peer server does not own this endpoint.


3.1.1.4.4 Update Endpoint Attributes

This operation is used by an ENRP server to inform its peers to update
the attributes of an existing DDP endpoint. 

ENRP server <-> endpoint:

None:

ENRP server <-> peers:

An ENRP server shall multicast over the ENRP server channel a
PEER_NAME_UPDATE message with the appropriate flag set to instruct 
its peers to replace the attributes of an existing DDP endpoint in its
local copy of the name space.

On the reception of this PEER_NAME_UPDATE message, a peer ENRP server
shall replace the attributes of the existing endpoint with the new
information carried in the message if the endpoint exists in its local
copy of the name space. If the specified endpoint is not found in its
local name space copy, the peer server shall add the endpoint
following the steps in Section 3.1.1.4.1??.


3.1.1.4.5 Claim Endpoint Ownership

This operation is used by an ENRP server to claim the ownership on a
specific endpoint and to inform its peers about its claim.

ENRP server <-> endpoint:

An ENRP server shall send an ENDPOINT_KEEP_ALIVE message to the
endpoint. This message will cause the endpoint to adopt this ENRP
server as its new home ENRP server (see Section 2.5.3).

ENRP server <-> peers:

An ENRP server shall multicast over the ENRP server channel a
PEER_NAME_UPDATE message with the appropriate flag set to inform its
peers that it has taken the ownership of the specified endpoint.

Upon the reception of this PEER_NAME_UPDATE message, a peer server
shall check whether it is the current owner of the endpoint. If so,
this peer server shall relinquish its ownership on that
endpoint. Otherwise, no action is needed.


3.1.1.4.6 Report Endpoint Failure

This operation is used by an ENRP server to warn its peers that it has
noticed a potentially unreachable endpoint that the server does not
have ownership on.

ENRP server <-> endpoint:

None:

ENRP server <-> peers:

An ENRP server shall multicast over the ENRP server channel a
PEER_NAME_UPDATE message with the appropriate flag set to indicate
that the specified DDP endpoint is potentially unreachable.

On the reception of this message, each peer ENRP server shall check
whether it owns the specified endpoint. If it does, the peer server
shall increase the <endpoint-report-failures> counter of the specified
endpoint by 1. If the value of the <endpoint-report-failures> counter
has exceeded the protocol parameter Max-Endpoint-Report-Failures, the
peer server shall remove the endpoint from its local name space and
take actions described in Section 3.1.1.4.3. If the peer server does
not own the specified endpoint, it shall take no action.


3.1.1.5 Endpoint Change Routing Value

A DDP endpoint can modify its routing value at any time. Depending on
the current number of members in the named group and the routing
policy, this operation allows the DDP endpoint to control its share of
inbound messages received within the named group dynamically (also see
Section 2.1.5.1 for more on load control).

ENRP server <-> endpoint:

A DDP endpoint shall send an UPDATE_ROUTING_VALUE message over the
ENRP client channel to its home ENRP server in order to modify its
routing value. The new routing value shall be indicated in the
message.

Upon the reception of this UPDATE_ROUTING_VALUE message, the home ENRP
server shall replace the routing value of that endpoint in its local
copy of the name space with the new value indicated in the message.

ENRP server <-> peers:

If the update on its local copy of the name space is successful, the
home ENRP server shall take the Server Name Space Update actions as
described in Section 3.1.1.4.4.


3.1.1.6 Server Down Load Name Space from a Peer

This operation allows an ENRP server to request and receive a copy of
a specific portion of the name space from one of its peer ENRP servers.
This is useful for a newly started ENRP server to initiate its local
copy of the name space, or for correcting name space inconsistency.

ENRP server <-> endpoint:

None.

ENRP server <-> peers:

An ENRP server shall first send a PEER_NAME_TABLE_REQUEST message
directly to one of its peers. In the message, it shall indicate which
portion of the name space is requested.

Upon the reception of this message, the peer server shall initiate a
download session in which the requested portion of the name space
shall be sent to the requesting ENRP server in one or more
PEER_NAME_TABLE_RESPONSE messages.

If the sending ENRP server determines that multiple
PEER_NAME_TABLE_RESPONSE messages are needed for the session, it shall
set the appropriate flag in each PEER_NAME_TABLE_RESPONSE message to
inform the receiving ENRP server whether or not the data in this
message is the last piece of the transfer.

Every time the requesting ENRP server receives a
PEER_NAME_TABLE_RESPONSE message, it shall transfer the data entries
carried in the message into its local name space database, and then
check whether or not the data in this message is the last piece to be
transfered. If more data transfer is indicated, the requesting ENRP
server shall send another PEER_NAME_TABLE_REQUEST message to the same
peer to prompt for the next piece.

When transferring the data entries from the PEER_NAME_TABLE_RESPONSE
message into its local name space database, the requesting ENRP server
shall follow the same procedures as described in section 3.1.1.4.1
when parsing through the endpoints carrying in the message one by one.


3.1.1.7 Server Monitor Peer Status

ENRP server <-> endpoint:

None:

ENRP server <-> peers:

An ENRP server shall keep a record on the status of each of its peers.

If a message of any type is received from a peer, the server shall
update that peers status as <active>.  If a message of any type is
received from a peer previously unknown to this server, i.e., a new
peer, the server shall create a record for the new peer and mark the
new peer as <active>.


3.1.1.8 Server Down Load Peer List

This operation allows an ENRP server to request from a peer server a
copy of its internal peer list. This is useful for a new ENRP server
to initiate its own peer list at startup.

ENRP server <-> endpoint:

None.

ENRP server <-> peers:

An ENRP server shall send a PEER_LIST_REQUEST message to a peer to
request a copy of its peer list.

Upon the reception of this message, the peer server shall reply with a
PEER_LIST_RESPONSE message and include in the message body a copy of its
internal peer list, if the peer itself is in operational state. 

If the peer itself is in the process of startup, it shall response
with a PEER_LIST_RESPONSE message but set the appropriate flag to
indicate that it can not grant the PEER_LIST_REQUEST. In such a case,
the requesting ENRP server shall select another peer and repeat the
peer list request with the new peer at a later time. 


3.1.1.9 Endpoint Initialization

At startup, a DDP endpoint shall always assume the existence of a local
ENRP server on the local host and mark it as its home ENRP server, and
initiate the registration procedure described in 3.1.1.1??. 


3.1.1.10 Server Initialization

At startup, before getting into service, an ENRP server (initiating
server) shall multicast a PEER_PRESENCE message with reply required
flag set over the ENRP server channel, in order to inform any other
active peers in the operation scope about its presence.

Upon the reception of this message, a peer shall send a PEER_PRESENCE
without reply required flag back to the initiating server, in order to
help the initiating server to build its peer list.

If no response to its PEER_PRESENCE message are received, the
initiating server shall assume that it is alone in the operation scope
and shall mark the initialization process as completed.

If there are responses to its PEER_PRESENCE message, the initiating
server shall then take the actions described in 3.1.1.8 to request a
peer list from one of the peers that have responded.

Upon the reception of the PEER_LIST_RESPONSE message from that peer,
the initiating server shall use the information carried in the message
to build a complete peer list, including both active and inactive
peers in the operation scope.

Then, the initiating server shall perform a name database download, as
described in 3.1.1.6, with each of the active peers on the peer list,
indicating that the portion of the name database to download shall
only include the endpoints owned by that peer.

Moreover, the initiating server shall also pick one of the active peer
and request to that peer for a download of the table of remote
endpoints.


3.1.2 Fault Management Operations

The following operations are used to detect and recover from various 
system faults.


3.1.2.1 Detect and Report Unreachable Endpoint

Two mechanisms exist to detect and report an unreachable DDP endpoint: 

1) Home ENRP server periodic sanity check

An ENRP server shall send, in every <Endpoint-sanity-cycle> seconds,
an ENDPOINT_KEEP_ALIVE message to each of the endpoints it owns, and
shall keep the number of consecutive failed send attempts in the
counter <Endpoint-sanity-failures> of that endpoint.

If the value of <Endpoint-sanity-failures> of an endpoint exceeds the
pre-set threshold Max-endpoint-sanity-failures, the home ENRP server
shall remove the endpoint from its copy of the name database and take
the actions described in section 3.1.1.4.3 to inform its peers.

The handling of the ENDPOINT_KEEP_ALIVE message by the endpoint is
described in Section 2.5.3??.


2) Detection by peer endpoints

Whenever a DDP endpoint finds a peer unreachable (e.g., via an SCTP
SEND.FAILURE Notification, see Section 2.2.5??), the endpoint shall
send an ENDPOINT_UNREACHABLE message over the ENRP client channel to
its home ENRP server.  The message shall contain one of the transport
addresses of the unreachable peer and have the severity flag set to
NORMAL_REPORT.

Upon the reception of this message, the home ENRP server shall first
check whether it owns the unreachable endpoint. If not, the server
shall take the actions described in section 3.1.1.4.6??.

Otherwise, the server shall increase the <Endpoint-report-failures>
counter of the unreachable endpoint by 1. If the value of the
<Endpoint-report-failures> counter has exceeded
Max-endpoint-report-failures, the server shall remove the endpoint
from its name database and take actions described in 3.1.1.4.3??.


3.1.2.2 ENRP Server Heartbeat

An ENRP server shall multicast, in every <Peer-heartbeat-cycle>
seconds, a PEER_PRESENCE message over the ENRP server channel to
inform its peers that it is still operational. In the PEER_PRESENCE
message, the sending ENRP server shall set the appropriate flag to
indicate that no reply is required.

>From time to time, an ENRP server may also send a point-to-point
PEER_PRESENCE message to a specific peer server, with the flag setting
in the message indicates that a reply is required. In such a case, the
peer server shall immediately respond to the sender with its own
point-to-point PEER_PRESENCE message, and shall indicate in the 
message that no reply is required.


3.1.2.3 ENRP Server Hunt

An endpoint shall initiate the following home server hunt procedure if 
it fails to send to, or times out on a service request with its
current home server.

In the home server hunt procedure, the endpoint shall multicast a
SERVER_HUNT message over the ENRP client channel, and shall repeat
sending this message every <Timeout-server-hunt> seconds until a
SERVER_HUNT_RESPONSE message is received from an ENRP server. Each
time the 'Timeout-server-hunt' timer expires the criticality should
be raised (initially criticality should be set to LOW_CRITICALITY).

Then the endpoint shall pick one of the servers that have responded as
its new home server, and continue the service request with that
server.

Upon the reception of the SERVER_HUNT message, a server shall reply to
the endpoint with a SERVER_HUNT_RESPONSE message, unless:

1) its peer status information indicates that there is a caretaker
   server other than itself for the node where the endpoint is from, 
   AND 

2) the criticality flag in the SERVER_HUNT message is not
   HIGH_CRITICALITY.


3.1.2.4 ENRP Server Detect and Take-over Inactive Peer 

An ENRP server shall keep track the time when the last message
(multicast or point-to-point) was received from each known peer.

If a peer has not been heard for more than Max-time-last-heard, the
ENRP server shall send a point-to-point PEER_PRESENCE with reply
request to that peer.

If the send fails or the peer does not reply after
Max-time-no-response seconds, the ENRP server shall initiate the
following server take-over procedures:

1) Initiate Server Take-over Arbitration

The ENRP server (the initiating server) shall initiate a take-over
arbitration on the inactive peer (the target server) by multicasting a 
TAKEOVER_INITIATE message over the ENRP server channel. In the
message, the initiating server shall specify the identification of the
target server.

After multicasting the TAKEOVER_INITIATE message, the initiating
server shall wait for a TAKEOVER_INITIATE_RESPONSE message from each
of its active peers.

Upon the reception of this message, other peer servers shall take the 
following actions accordingly:

o If the peer server finds that itself is the target server indicated 
  in the TAKEOVER_INITIATE message, it shall immediately multicast a 
  PEER_PRESENCE message over the ENRP server channel in an attempt of 
  stopping the take-over process. 

o If the peer server finds that itself has also initiated a take-over 
  process on the same target server and its IP address is smaller in 
  value than that of the sender of the TAKEOVER_INITIATE message, it
  shall abort its own take-over process.

o Peers other than the target peer and the peer that is taking-over
  shall mark the target server as <inactive> and mark the initiating
  server as the caretaker of the target server and reply to the
  initiating server with a TAKEOVER_INITIATE_RESPONSE message.

Once it has received TAKEOVER_INITIATE_RESPONSE message from all of
its active peers, the initiating server shall consider it won
the arbitration and shall then take the actions in 2) in order to
complete the take-over.

However, if it receives a PEER_PRESENCE from the target server at any
point of the take-over, the initiating server shall immediately abort
the take-over process and re-mark the target server as <active>.

2) Take-over the target peer server

An ENRP server shall multicast a TAKEOVER_PEER_SERVER message over the
ENRP server channel in order to inform all its peers about the
take-over. In the message, identification of the inactive peer server
targeted for the take-over shall be included.

The server shall mark the target server as <inactive> and mark itself
as the caretaker of the target server.

Then it shall assume ownership on each of the endpoints originally
owned by the target server.

The server shall also check whether there are any other inactive peers
which has designated the target server as their caretaker. The server
shall perform the above take-over procedure on each one of those
inactive peers as well.


3.1.2.5 Register Homeless Endpoints

When an ENRP server receives a REGISTRATION message from an endpoint
located on a remote node, it shall always accept and grant the
registration, unless its peer status information indicates that the
peer on that node is inactive and a caretaker other than itself exists
for that node. In that case, the server shall reject the registration
and take no further actions.

If the server has no record about the peer on that node, the
server shall grant the registration and then create a record about
that peer, mark it as inactive, and initiate a take-over procedure on
it, as described in 3.1.2.4??.


3.1.3 Maintenance Operations

The following operations are used by an ENRP maintenance client to
monitor the name space data and perform maintenances on ENRP servers
in an operation scope. 


3.1.3.1 Forceful Removal of Endpoint

A maintenance endpoint shall send a ENDPOINT_UNREACHABLE message to an
ENRP server, in order to force the removal of another endpoint from
the name space.  The message shall contain one of the transport
addresses of the target endpoint and have the severity flag set to
FINAL_REPORT.

Upon the reception of this message, the ENRP server shall immediately
remove the target endpoint from its copy of the name database and take
actions described in Section 3.1.1.4.2.


3.1.3.2 Dump Home Endpoint List

A maintenance endpoint shall send a SERVER_DUMP message with type flag
set to HOME_LIST to a server, in order to require a copy of the
information on all the endpoints owned by that server.

Upon receiving this message, the server shall response with a
SERVER_DUMP_RESPONSE message with the type flag set to HOME_LIST to
the maintenance endpoint. In the message body, the server shall
include information on all the endpoints the server currently owns.


3.1.3.3 Dump Remote Endpoint List

A maintenance endpoint shall send a SERVER_DUMP message with type flag
set to REMOTE_LIST to a server, in order to require a copy of the
information on all the endpoints NOT owned by that server.

Upon receiving this message, the server shall response with a
SERVER_DUMP_RESPONSE message with the type flag set to REMOTE_LIST to
the maintenance endpoint. In the message body, the server shall
include information on all the endpoints the server currently does NOT
owns.


3.1.3.4 Dump Peer Server List

A maintenance endpoint shall send a SERVER_DUMP message with the type
flag set to PEER_SERVER_LIST to a server, in order to require a copy
of the peer list known to that server.

Upon receiving this message, the server shall response with a
SERVER_DUMP_RESPONSE message with the type flag set to
PEER_SERVER_LIST to the maintenance endpoint. In the message body, the
server shall include information on all the peers that server
currently knows.


3.2 Message Summary

All messages as well as their fields described below shall be in
Network Byte Order during transmission. For fields with a length
bigger than 4 octets, a number in a pair of parentheses may follow the
filed name to indicate the length of the field in number of octets.


3.2.1 Endpoint Entry

This field is used to represent a DDP endpoint and the associated
information, such as its transport address(es), load control, and
other operational status information.

The field is defined to support endpoint with up to 8 different
transport addresses.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        IP address #0                          |
   +-------------------------------+-------------------------------+
   |                        IP address #1                          |
   +-------------------------------+-------------------------------+
   \                     \                      \                  \
   /                     /                      /                  /
   \                     \                      \                  \
   +-------------------------------+-------------------------------+
   |                        IP address #7                          |
   +-------------------------------+-------------------------------+
   |          SCTP Port            |         Padding               |
   +-------------------------------+-------------------------------+
   |         Routing Policy        |       Routing Value           |
   +---------------+---------------+---------------+---------------+

The size of the endpoint entry is 40 octets.


3.2.2 REGISTRATION message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP endpoint message identifier #1 = 0x18038688       |
   +-------------------------------+-------------------------------+
   |        ENRP endpoint message identifier #2 = 0x77734683       |
   +-------------------------------+-------------------------------+
   |                       Type = 0x3                              |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 Endpoint name (32)                            |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 endpoint entry (40)                           |
   |                                                               |
   +-------------------------------+-------------------------------+

The endpoint name field specifies the name to be registered, that
shall be composed of up to 32 characters. The endpoint entry field
shall be filled in by the registrant endpoint to declare its transport
addresses, routing policy and value, and other operation preferences.


3.2.3 DEREGISTRATION message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP endpoint message identifier #1 = 0x18038688       |
   +-------------------------------+-------------------------------+
   |        ENRP endpoint message identifier #2 = 0x77734683       |
   +-------------------------------+-------------------------------+
   |                       Type = 0x4                              |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 Endpoint name (32)                            |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 endpoint entry (40)                           |
   |                                                               |
   +-------------------------------+-------------------------------+

The endpoint sending the DEREGISTRATION shall fill in the name and the
endpoint entry in order to allow the ENRP server to verify the identity
of the endpoint.


3.2.4 REGISTRATION_RESPONSE message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP endpoint message identifier #1 = 0x18038688       |
   +-------------------------------+-------------------------------+
   |        ENRP endpoint message identifier #2 = 0x77734683       |
   +-------------------------------+-------------------------------+
   |                       Type = 0x5                              |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 Endpoint name (32)                            |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                     Result = (see below)                      |
   +-------------------------------+-------------------------------+
   |                   Requested action = (see below)              |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 endpoint entry (40)                           |
   |                                                               |
   +-------------------------------+-------------------------------+

In response to a REGISTRATION, the 'Requested action' field shall be
set to 0x0, and the 'Result' field shall take the following values:
  0x0 -- registration granted
  0x1 -- registration rejected

In response to a DEREGISTRATION, the 'Requested action' field shall be
set to 0x1, and the 'Result' field shall take the following values:
  0x2 -- de-registration granted
  0x3 -- de-registration rejected: endpoint not found
  0x4 -- de-registration rejected: other failures.

endpoint entry shall be filled in for verification purposes.


3.2.5 NAME_REQUEST message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP endpoint message identifier #1 = 0x18038688       |
   +-------------------------------+-------------------------------+
   |        ENRP endpoint message identifier #2 = 0x77734683       |
   +-------------------------------+-------------------------------+
   |                       Type = 0x1                              |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 requested name (32)                           |
   |                                                               |
   +-------------------------------+-------------------------------+


3.2.6 NAME_INFORMATION message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP endpoint message identifier #1 = 0x18038688       |
   +-------------------------------+-------------------------------+
   |        ENRP endpoint message identifier #2 = 0x77734683       |
   +-------------------------------+-------------------------------+
   |                       Type = 0x2                              |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 Endpoint name (32)                            |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                  number of entries = n (see below)            |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 endpoint entry 1 (40)                         |
   |                                                               |
   +-------------------------------+-------------------------------+
   /                                                               /
   \                                                               \
   /                                                               /
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 endpoint entry n (40)                         |
   |                                                               |
   +-------------------------------+-------------------------------+


3.2.7 NAME_UNKNOWN message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP endpoint message identifier #1 = 0x18038688       |
   +-------------------------------+-------------------------------+
   |        ENRP endpoint message identifier #2 = 0x77734683       |
   +-------------------------------+-------------------------------+
   |                       Type = 0x0                              |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 requested name (32)                           |
   |                                                               |
   +-------------------------------+-------------------------------+


3.2.8 UPDATE_ROUTING_VALUE message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP endpoint message identifier #1 = 0x18038688       |
   +-------------------------------+-------------------------------+
   |        ENRP endpoint message identifier #2 = 0x77734683       |
   +-------------------------------+-------------------------------+
   |                       Type = 0x11                             |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                      Endpoint name (32)                       |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                      endpoint entry (40)                      |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                        New routing value                      |
   +-------------------------------+-------------------------------+


3.2.9 PEER_NAME_TABLE_REQUEST message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP server message identifier #1 = 0x27047729         |
   +-------------------------------+-------------------------------+
   |        ENRP server message identifier #2 = 0x53829149         |
   +-------------------------------+-------------------------------+
   |                       Type = 0x102                            |
   +-------------------------------+-------------------------------+
   |                 sending server's IP address                   |
   +-------------------------------+-------------------------------+
   |       sender's SCTP port      |         padding               |
   +-------------------------------+-------------------------------+
   |                receiving server's IP address                  |
   +-------------------------------+-------------------------------+
   |      receiver's SCTP port     |         padding               |
   +-------------------------------+-------------------------------+
   |                      Table type = (see below)                 |
   +-------------------------------+-------------------------------+
 
Note, the receiver's IP address and port do not need to be filled in
if the message is being multicasted.

The requested table type shall take one of the following values:
 0x1 --- HOME_LIST: endpoints owned by the server.
 0x2 --- REMOTE_LIST: endpoint NOT owned by the server.


3.2.10 PEER_NAME_TABLE_RESPONSE message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP server message identifier #1 = 0x27047729         |
   +-------------------------------+-------------------------------+
   |        ENRP server message identifier #2 = 0x53829149         |
   +-------------------------------+-------------------------------+
   |                       Type = 0x103                            |
   +-------------------------------+-------------------------------+
   |                     sender's IP address                       |
   +-------------------------------+-------------------------------+
   |      sender's SCTP port       |         padding               |
   +-------------------------------+-------------------------------+
   |                     receiver's IP address                     |
   +-------------------------------+-------------------------------+
   |      receiver's SCTP port     |         padding               |
   +-------------------------------+-------------------------------+
   |                    Table type = (see below)                   |
   +-------------------------------+-------------------------------+
   |                  More to send = (see below)                   |
   +-------------------------------+-------------------------------+
   |                     number of names = n                       |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                   Name entry 1 (see below)                    |
   |                                                               |
   +-------------------------------+-------------------------------+
   /                                                               /
   \                                                               \
   /                                                               /
   +-------------------------------+-------------------------------+
   |                                                               |
   |                   Name entry n (see below)                    |
   |                                                               |
   +-------------------------------+-------------------------------+


'Table type' shall take one of the following values:
 0x1 --- HOME_LIST: endpoints owned by the server.
 0x2 --- REMOTE_LIST: endpoint NOT owned by the server.

'More to send' flag shall be set to 0x1 if there are more name entries 
to be sent for the requested table type. Otherwise, it shall be set to
0x0.

Each 'Name entry' represents an endpoint and shall consist of the
following: 

   +-------------------------------+-------------------------------+
   |                                                               |
   |                    Endpoint name (32)                         |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                   number of endpoints  = m                    |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 endpoint entry 1 (40)                         |
   |                                                               |
   +-------------------------------+-------------------------------+
   /                                                               /
   \                                                               \
   /                                                               /
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 endpoint entry m (40)                         |
   |                                                               |
   +-------------------------------+-------------------------------+


3.2.11 PEER_LIST_REQUEST message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP server message identifier #1 = 0x27047729         |
   +-------------------------------+-------------------------------+
   |        ENRP server message identifier #2 = 0x53829149         |
   +-------------------------------+-------------------------------+
   |                       Type = 0x10b                            |
   +-------------------------------+-------------------------------+
   |                      sender's IP address                      |
   +-------------------------------+-------------------------------+
   |      sender's SCTP port       |         padding               |
   +-------------------------------+-------------------------------+
   |                     receiver's IP address                     |
   +-------------------------------+-------------------------------+
   |      receiver's SCTP port     |         padding               |
   +-------------------------------+-------------------------------+
 
The receiver's IP address and port do not need to be filled in if
the message is being multicasted.


3.2.12 PEER_LIST_RESPONSE message

This message shall contain all the peer information of the sending server.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP server message identifier #1 = 0x27047729         |
   +-------------------------------+-------------------------------+
   |        ENRP server message identifier #2 = 0x53829149         |
   +-------------------------------+-------------------------------+
   |                       Type = 0x10c                            |
   +-------------------------------+-------------------------------+
   |                     sender's IP address                       |
   +-------------------------------+-------------------------------+
   |      senders SCTP port        |         padding               |
   +-------------------------------+-------------------------------+
   |                    receiver's IP address                      |
   +-------------------------------+-------------------------------+
   |      receiver's SCTP port     |         padding               |
   +-------------------------------+-------------------------------+
   |                  responseIndication (see below)               |
   +-------------------------------+-------------------------------+
   |                     number of peers = n                       |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                   Peer entry 1 (see below)                    |
   |                                                               |
   +-------------------------------+-------------------------------+
   /                                                               /
   \                                                               \
   /                                                               /
   +-------------------------------+-------------------------------+
   |                                                               |
   |                   Peer entry n (see below)                    |
   |                                                               |
   +-------------------------------+-------------------------------+

The 'responseIndication' flag shall be set to 0x2 to indicate a
rejection to the request, and no 'Peer entry' shall be attached if the
request is rejected.

Otherwise, the 'responseIndication' flag shall be set to 0x1 and n
'Peer entries' attached.

Each 'Peer entry' shall consist of the following fields:

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Peer's IP address                       |
   +-------------------------------+-------------------------------+
   |       Peer's SCTP port        |         padding               |
   +-------------------------------+-------------------------------+
   |                     Caretaker's IP address                    |
   +-------------------------------+-------------------------------+
   |     caretaker's SCTP port     |         padding               |
   +-------------------------------+-------------------------------+

The peer's IP address and port number serve as the identification of
that peer. If the peer is inactive, its caretaker's IP address and port
number shall be filled in. Otherwise, the caretaker IP and port fields
shall be set to zeros.


3.2.13 PEER_NAME_UPDATE message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP server message identifier #1 = 0x27047729         |
   +-------------------------------+-------------------------------+
   |        ENRP server message identifier #2 = 0x53829149         |
   +-------------------------------+-------------------------------+
   |                       Type = 0x104                            |
   +-------------------------------+-------------------------------+
   |                     sender's IP address                       |
   +-------------------------------+-------------------------------+
   |      sender's SCTP port       |         padding               |
   +-------------------------------+-------------------------------+
   |                    receiver's IP address                      |
   +-------------------------------+-------------------------------+
   |      receiver's SCTP port     |         padding               |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                     Endpoint name (32)                        |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                     endpoint entry (40)                        |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                 Update action (see below)                     |
   +-------------------------------+-------------------------------+

The receiver's IP address and port do not need to be filled in if
the message is being multicasted.

'Update action' shall take one of the following values:
  0x0 --- ADD_ENDPOINT: add a new endpoint, as specified by 'Endpoint
          name' and 'endpoint entry' fields, to the name space.
  0x1 --- DELETE_ENDPOINT: delete the named endpoint from the name
          database, if the receiving server owns the endpoint.
  0x2 --- REMOVE_ENDPOINT: remove the named endpoint from the name
          database, regardless who owns the endpoint.
  0x3 --- UPDATE_ENDPOINT: replace the endpoint's attributes with the
          new information carried in this message.
  0x4 --- ENDPOINT_FAILURE: warn the receiver that the named endpoint
          is potentially unreachable.
  0x5 --- CLAIM_ENDPOINT: inform the receiver that the sender has
          taken the ownership of the specified endpoint.


3.2.14 ENDPOINT_KEEP_ALIVE message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP endpoint message identifier #1 = 0x18038688       |
   +-------------------------------+-------------------------------+
   |        ENRP endpoint message identifier #2 = 0x77734683       |
   +-------------------------------+-------------------------------+
   |                       Type = 0x6                              |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                      Endpoint name (32)                       |
   |                                                               |
   +-------------------------------+-------------------------------+


3.2.15 PEER_PRESENCE message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP server message identifier #1 = 0x27047729         |
   +-------------------------------+-------------------------------+
   |        ENRP server message identifier #2 = 0x53829149         |
   +-------------------------------+-------------------------------+
   |                       Type = 0x100                            |
   +-------------------------------+-------------------------------+
   |                      sender's IP address                      |
   +-------------------------------+-------------------------------+
   |      sender's SCTP port       |         padding               |
   +-------------------------------+-------------------------------+
   |                    receiver's IP address                      |
   +-------------------------------+-------------------------------+
   |      receiver's SCTP port     |         padding               |
   +-------------------------------+-------------------------------+
   |                       Reply required                          |
   +-------------------------------+-------------------------------+

The receiving server's IP address and port do not need to be filled in
if the message is being multicasted.

'Reply required' shall be set to 0x1 if response to this message is
required, otherwise set to 0x0.


3.2.16 ENDPOINT_UNREACHABLE message 

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP endpoint message identifier #1 = 0x18038688       |
   +-------------------------------+-------------------------------+
   |        ENRP endpoint message identifier #2 = 0x77734683       |
   +-------------------------------+-------------------------------+
   |                       Type = 0xa                              |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                      Endpoint name (32)                       |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                     Endpoint IP address                       |
   +-------------------------------+-------------------------------+
   |      Endpoint's SCTP port     |         padding               |
   +-------------------------------+-------------------------------+
   |                Type of severity (see below)                   |
   +-------------------------------+-------------------------------+

The 'Endpoint name' is not required to be filled in for this message,
however the IP address and SCTP port number of the endpoint must be
supplied in the message.

'Type of severity' shall take one of the following values:

  0x0 --- NORMAL_REPORT: warning to the server.
  0x1 --- FINAL_REPORT: the specified endpoint must be removed by the 
          server immediately. 

3.2.17 TAKEOVER_INITIATE message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP server message identifier #1 = 0x27047729         |
   +-------------------------------+-------------------------------+
   |        ENRP server message identifier #2 = 0x53829149         |
   +-------------------------------+-------------------------------+
   |                       Type = 0x106                            |
   +-------------------------------+-------------------------------+
   |                 sending server's IP address                   |
   +-------------------------------+-------------------------------+
   |      sender's SCTP port       |         padding               |
   +-------------------------------+-------------------------------+
   |                receiving server's IP address                  |
   +-------------------------------+-------------------------------+
   |      receiver's SCTP port     |         padding               |
   +-------------------------------+-------------------------------+
   |                   Target server's IP address                  |
   +-------------------------------+-------------------------------+
   |    Target server's SCTP port  |         padding               |
   +-------------------------------+-------------------------------+

The receiving server's address and port do not need to be filled in if
the message is being multicasted.

'Target server's IP address and port number must be supplied.


3.2.18 TAKEOVER_INITIATE_RESPONSE message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP server message identifier #1 = 0x27047729         |
   +-------------------------------+-------------------------------+
   |        ENRP server message identifier #2 = 0x53829149         |
   +-------------------------------+-------------------------------+
   |                       Type = 0x107                            |
   +-------------------------------+-------------------------------+
   |                 sending server's IP address                   |
   +-------------------------------+-------------------------------+
   |      sender's SCTP port       |         padding               |
   +-------------------------------+-------------------------------+
   |                receiving server's IP address                  |
   +-------------------------------+-------------------------------+
   |      receiver's SCTP port     |         padding               |
   +-------------------------------+-------------------------------+
   |                   Target server's IP address                  |
   +-------------------------------+-------------------------------+
   |    Target server's SCTP port  |         padding               |
   +-------------------------------+-------------------------------+

The receiving server's address and port do not need to be filled in if
the message is being multicasted.

'Target server's IP address and port number must be supplied.


3.2.19 TAKEOVER_PEER_SERVER message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP server message identifier #1 = 0x27047729         |
   +-------------------------------+-------------------------------+
   |        ENRP server message identifier #2 = 0x53829149         |
   +-------------------------------+-------------------------------+
   |                       Type = 0x108                            |
   +-------------------------------+-------------------------------+
   |                 sending server's IP address                   |
   +-------------------------------+-------------------------------+
   |      sender's SCTP port       |         padding               |
   +-------------------------------+-------------------------------+
   |                receiving server's IP address                  |
   +-------------------------------+-------------------------------+
   |      receiver's SCTP port     |         padding               |
   +-------------------------------+-------------------------------+
   |                   Target server's IP address                  |
   +-------------------------------+-------------------------------+
   |    Target server's SCTP port  |         padding               |
   +-------------------------------+-------------------------------+

The receiving server's address and port do not need to be filled in if
the message is being multicasted.

'Target server's IP address and port number must be supplied.


3.2.20 SERVER_HUNT message 

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP endpoint message identifier #1 = 0x18038688       |
   +-------------------------------+-------------------------------+
   |        ENRP endpoint message identifier #2 = 0x77734683       |
   +-------------------------------+-------------------------------+
   |                       Type = 0xb                              |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                      Endpoint name (32)                       |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                     criticality (see below)                   |
   +-------------------------------+-------------------------------+

The 'criticality' field shall take one of the following values:
  0x1 --- LOW_CRITICALITY
  0x2 --- MED_CRITICALITY
  0x3 --- HIGH_CRITICALITY


3.2.21 SERVER_HUNT_RESPONSE message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP endpoint message identifier #1 = 0x18038688       |
   +-------------------------------+-------------------------------+
   |        ENRP endpoint message identifier #2 = 0x77734683       |
   +-------------------------------+-------------------------------+
   |                       Type = 0xc                              |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                      Endpoint name (32)                       |
   |                                                               |
   +-------------------------------+-------------------------------+


3.2.22 SERVER_DUMP message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP endpoint message identifier #1 = 0x18038688       |
   +-------------------------------+-------------------------------+
   |        ENRP endpoint message identifier #2 = 0x77734683       |
   +-------------------------------+-------------------------------+
   |                       Type = 0x7                              |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                      Endpoint name (32)                       |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                      Dump Type (see below)                    |
   +-------------------------------+-------------------------------+

The 'Dump Type' field shall take one of the following values:
  0x0 --- HOME_LIST: dump a copy of the home endpoint portion of the
          name database of the server (i.e., endpoints owned by the
          server).
  0x1 --- REMOTE_LIST: dump a copy of the remote endpoint portion of the
          name database of the server (i.e., endpoints NOT owned by the
          server).
  0x2 --- PEER_LIST: dump a copy of a list containing all the peers
          known to the server.


3.2.23 SERVER_DUMP_RESPONSE message

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        ENRP endpoint message identifier #1 = 0x18038688       |
   +-------------------------------+-------------------------------+
   |        ENRP endpoint message identifier #2 = 0x77734683       |
   +-------------------------------+-------------------------------+
   |                       Type = 0x8                              |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                      Endpoint name (32)                       |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                      Dump Type (see below)                    |
   +-------------------------------+-------------------------------+
   |                  Number of Entries = n (see below)            |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                   Dump entry 1 (see below)                    |
   |                                                               |
   +-------------------------------+-------------------------------+
   /                                                               /
   \                                                               \
   /                                                               /
   +-------------------------------+-------------------------------+
   |                                                               |
   |                   Dump entry n (see below)                    |
   |                                                               |
   +-------------------------------+-------------------------------+

The 'Dump Type' fields shall take the same values as defined in
Section 3.2.29??.

If 'Dump Type' is HOME_LIST, or REMOTE_LIST, the 'Number of Entries'
field shall be the number of endpoint entries carried in the message,
and each 'Dump entry' field shall contain an endpoint entry and shall
be defined as:

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                       Endpoint name (32)                      |
   |                                                               |
   +-------------------------------+-------------------------------+
   |                        number of endpoints = m                |
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 endpoint entry 1 (40)                          |
   |                                                               |
   +-------------------------------+-------------------------------+
   /                                                               /
   \                                                               \
   /                                                               /
   +-------------------------------+-------------------------------+
   |                                                               |
   |                 endpoint entry m (40)                          |
   |                                                               |
   +-------------------------------+-------------------------------+

If 'Dump Type' is PEER_LIST, the 'Number of Entries' field shall be
the number of peer entries carried in the message, and each 'Dump
entry' field shall contain a peer entry and shall be defined as:

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Peer's IP address                       |
   +-------------------------------+-------------------------------+
   |       Peer's SCTP port        |         padding               |
   +-------------------------------+-------------------------------+
   |                     Caretaker's IP address                    |
   +-------------------------------+-------------------------------+
   |     caretaker's SCTP port      |         padding               |
   +-------------------------------+-------------------------------+

In a peer entry, the peer's IP address and port number serve as the
identification of that peer. If the peer is inactive, its caretaker's
IP address and port number shall be filled in. Otherwise, the
caretaker IP and port fields shall be zeroed out.


4.  Variables, Time Values, and Thresholds

The following is a summary of the variables, time values, and pre-set
thresholds used in DDP and ENRP protocol.


4.1 Variables

Endpoint-report-failures --- per endpoint; keeps the number of
endpoint-unreachable reports concerning this endpoint.

Endpoint-sanity-failures --- per endpoint; keeps the number of failed
sanity message send attempts concerning this endpoint.

Peer-server-last-heard --- per peer server; a time stamp on when the
last message was received from this peer server.


4.2 Time values

Endpoint-sanity-cycle --- the period for a home ENRP server to start a
new round of endpoint sanity check.

Peer-heartbeat-cycle ---the period for an ENRP server to send out a
heart heat message.

T1-ENRPrequest - A timer started when a request is sent by DDP to the
ENRP server (providing application information is queued). Normally set
to 15 seconds.

T2-registration - A timer started when sending a registration request 
to the local ENRP server, normally set to 30 seconds.

T3-registration-reattempt - If the registration cycle does not complete
this timer is begun to restart the registration process. Normal value
for this timer is 10 minutes.

T4-reregistration  - This timer is started after successful registration
into the DDP name space and is used to cause a re-registration at a
periodic interval. This timer is normally set to 10 minutes.


4.3 Thresholds

Max-endpoint-sanity-failures --- pre-set threshold for
Endpoint-sanity-failures.

Max-endpoint-report-failures --- pre-set threshold for
Endpoint-report-failures.

Max-time-last-heard --- pre-set threshold for Peer-last-heard.

Max-time-no-response --- pre-set threshold for a peer server to answer
a PEER_PRESENCE message with reply required.

Timeout-registration --- pre-set threshold; how long an endpoint will
wait for the REGISTRATION_RESPONSE from its home ENRP server.

Timeout-server-hunt --- pre-set threshold; how long an endpoint will
wait for the REGISTRATION_RESPONSE from its home ENRP server.

num-of-serverhunts - The current count of server hunt messages that
have been transmitted.

registration-count - The current count of attempted registrations.

max-reg-attempt - The maximum number of registration attempts to be
made before a server hunt is issued.

max-request-retransmit - The maximum number of attempts to be made
when requesting information from the local ENRP server before
a server hunt is issued.


5. References

[1] R. R. Stewart, Q. Xie, K. Morneault, C. Sharp, H. J. Schwarzbauer,
T. Taylor, I. Rytina, M. Kalla, L. Zhang, and, V. Paxson, "Simple
Control Transmission Protocol," <draft-ietf-sigtran-sctp-07.txt>, Feb,
2000. 








 
     This Internet Draft expires in 6 months from February, 2000