Routing Over Large Clouds Working Group James V. Luciani INTERNET-DRAFT (Ascom Nexion) Expires September 1996 Server Cache Synchronization Protocol (SCSP) - NBMA Status of this Memo This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as ``work in progress.'' To learn the current status of any Internet-Draft, please check the ``1id-abstracts.txt'' listing contained in the Internet-Drafts Shadow Directories on ds.internic.net (US East Coast), nic.nordu.net (Europe), ftp.isi.edu (US West Coast), or munnari.oz.au (Pacific Rim). Abstract This document describes the Server Cache Synchronization Protocol (SCSP) for Non Broadcast Multiple Access (NBMA) networks. SCSP attempts to solve the generalized server synchronization/cache- replication problem wherein a set of server entities which are bound to a Server Group (SG) through some means (e.g., all servers belonging to the same Logical IP Subnet (LIS)[1]) wish to synchronize the contents (or a portion thereof) of their caches. These caches contain information on the state of the clients within the scope of interest of the SG. An example of types of information that must be synchronized can be seen in NHRP using IP where the information includes the REGISTERED clients' IP to NBMA mappings in the SG LIS. 1. Introduction It is perhaps an obvious goal for any protocol to not limit itself to a single point of failure such as having a single server in a Luciani [Page 1] INTERNET-DRAFT SCSP-NBMA Expires September 1996 client/server paradigm. Even when there are redundant servers, there still remains the problem of cache synchronization; i.e., when one server becomes aware of a change in state of cache information then that server must propagate the knowledge of the change in state to all servers which are actively mirroring that state information. Further, this must be done in a timely fashion without putting undo resource strains on the servers. Assuming that the state information kept in the server cache is the state of clients of the server, then in order to minimize the burden placed upon the client it is also highly desirable that clients need not have complete knowledge of all servers which they may use. However, any mechanism for synchronization should not preclude a client from having access to several (or all) servers. Further, it is important that the semantics of any server synchronization protocol lend itself easily to supplying sufficient information when realized in the actual syntax of a given protocol. These semantics must also lend themselves to potentially a wide range of authentication methodologies. Of course, any solution must be reasonably scalable and capable of using the slew of autoconfiguration technologies in existence and in progress. This document describes the Server Cache Synchronization Protocol (SCSP). SCSP solves the generalized server synchronization/cache- replication problem while addressing the issues described above. The SCSP synchronizes caches (or a portion of the caches) of a set of server entities which are bound to a Server Group (SG) through some means (e.g., all NHRP servers belonging to a Logical IP Subnet (LIS)[1]). These caches contain information on the state of the clients within the scope of interest of the SG. An example of types of information that must be synchronized can be seen in NHRP[2] using IP where the information includes the REGISTERED clients' IP to NBMA mappings in the SG LIS. While SCSP is meant to solve the general server synchronization problem, the following sections, except where otherwise explicitly stated to the contrary, will draw upon the problem as it exists in NHRP[2]. Luciani [Page 2] INTERNET-DRAFT SCSP-NBMA Expires September 1996 2. Terminology This section introduces the terminology associated with SCSP. 2.1 Abbreviations CA - Cache Alignment Message CAFSM - Cache Alignment Finite State Machine CID - Client ID CRL - CSA Request List CSA - Client State Advertisement CSAS - Client State Advertisement Summary CSU - Client State Update CSUS - Client State Update Solicit DCS - Directly Connected Server DS - Designated Server DSID - Designated Server ID DSP - Designated Server Priority ES - Eligible Server HFSM - Hello Finite State Machine I - Initialize bit LS - Local Server LSID - Local Server ID M - More bit MS - Master/Slave bit RS - Remote Server SG - Server Group SID - Server ID Luciani [Page 3] INTERNET-DRAFT SCSP-NBMA Expires September 1996 2.2 Definitions Cache Alignment message (CA message) These messages allow an LS to synchronize its entire cache with that of the cache of one of its DCSs. Cache Alignment Finite State Machine (CAFSM) The CAFSM monitors the state of the cache alignment between an LS and a particular DCS. There exists one CAFSM per DCS as seen from an LS. Client ID (CID) The CID is an unique token which identifies a client whose state is being kept in a server's cache. This value might be taken from the protocol address of the client. CSA Request List (CRL) When CA messages are exchanged between an LS and one of its DCSs, the LS makes a list of those cache entries which are more recent in the DCS (based on a CSAS sequence number) than the LS's own entry and adds to that list any entry in the DCS which is not already in its cache. This list is the CRL. Client State Advertisement record (CSA record) A CSA is a record within a CSU message which identifies an update to the status of a "particular" client. Client State Advertisement Summary record (CSAS record) A CSAS contains a summary of the information in a CSA. A server will send CSAS records describing its cache entries to another server during the cache alignment process. CSAS records are also included in a CSUS messages when an LS wants to request the entire CSA from the DCS. The LS is requesting the CSA from the DCS because the LS believes that the DCS has a more recent view of the state of the cache entry in question. Client State Update message (CSU message) This is a message sent from an LS to its DCSs when the LS becomes aware of a change in state of a client. Client State Update Solicit message (CSUS message) This message is sent by an LS to its DCS after the LS and DCS have exchanged CA messages. The CSUS message contains one or more CSAS records which represent solicitations for entire CSA records (as opposed to just the summary information held in the CSAS). Directly Connected Server (DCS) The DCS is a server which is directly connected to the LS; e.g., there exists a VC between the LS and DCS. Luciani [Page 4] INTERNET-DRAFT SCSP-NBMA Expires September 1996 This term, along with the terms LS and RS, is used to give a frame of reference when talking about servers and their synchronization. Unless explicitly stated to the contrary, there is no implied difference in functionality between a DCS, LS, and RS. Designated Server (DS) The DS is the contact point within the SG for off-SG stations wishing to query the state of the SG. Designated Server ID (DSID) The DSID is a unique token that identifies the DS in an SG. This value might be taken from the protocol address of the DS. Designated Server Priority (DSP) The DSP identifies the priority of a given server to become the DS. If the DSP is 0 then the server is ineligible to become the DS. Eligible Server (ES) An ES is a server that is eligible to become the DS as a result of having a DSP greater than zero. Hello Finite State Machine (HFSM) An LS has a HFSM associated with each of its DCSs. The HFSM monitors the state of the connectivity between the LS and a particular DCS. Initialize bit (I bit) This bit is included in a CA message. When set, this bit indicates that the sender of the CA wishes to negotiate for Master/Slave server status in the cache alignment process. Local Server (LS) The LS is the server under scrutiny; i.e., all statements are made from the perspective of the LS. This term, along with the terms DCS and RS, is used to give a frame of reference when talking about servers and their synchronization. Unless explicitly stated to the contrary, there is no implied difference in functionality between a DCS, LS, and RS. Local Server ID (LSID) The LSID is a unique token that identifies an LS. This value might be taken from the protocol address of the LS. More bit (M bit) This bit is included in a CA message. When set, this bit indicates that the sender of the CA has more CA messages to send above and beyond the message it is currently sending. Luciani [Page 5] INTERNET-DRAFT SCSP-NBMA Expires September 1996 Master/Slave bit (MS bit) This bit is included in a CA message. When set, this bit indicates that the sender of the CA wishes to be Master of the cache alignment process. Remote Server (RS) An RS is a server that is neither an LS nor a DCS and unless otherwise stated an RS refers to a server in the SG. This term, along with the terms LS and DCS, is used to give a frame of reference when talking about servers and their synchronization. Unless explicitly stated to the contrary, there is no implied difference in functionality between a DCS, LS, and RS. Server Group (SG) The SCSP synchronizes caches (or a portion of the caches) of a set of server entities which are bound to a SG through some means (e.g., all servers belonging to a Logical IP Subnet (LIS)[1]). Thus an SG is just a grouping of servers around some commonality. Server ID (SID) The SID is a unique token that identifies a given server. This value might be taken from the protocol address of the server. 3. Overview SCSP borrows heavily from the link state protocols [3,4]. SCSP uses three message classes: "Hello", "Cache Alignment", and "Client State Update". Following is a brief discussion of the use of each of these message classes. Sections 3.1, 3.2, and 3.3 contain a more in depth explanation of the Hello, Cache Alignment, and Client State Update messages. In order to give a frame of reference for the following discussion, the terms Local Server (LS), Directly Connected Server (DCS), and Remote Server (RS) are introduced. The LS is the server under scrutiny; i.e., all statements are made from the perspective of the LS when discussing the SCSP protocol. The DCS is a server which is directly connected to the LS; e.g., there exists a VC between the LS and DCS. An RS is a server that is neither an LS nor a DCS. Unless explicitly specified to the contrary, an RS is assumed to be in the same SG as the LS and DCS. That is, a DCS is always "one hop away" from an LS whereas an RS always two or more hops away from an LS. Again, the reader is asked to keep in mind that the terms LS, DCS, and RS are terms used to form a frame of reference when talking about how one server interacts with another server and that these terms do not imply any difference in functionality. "Hello" messages ascertain whether a DCS is operational and whether the connections between the LS and DCS are bidirectional, unidirectional, or non-functional. Every LS MUST periodically send Hello messages to each DCS. Every LS MUST also send a Hello Reply in Luciani [Page 6] INTERNET-DRAFT SCSP-NBMA Expires September 1996 response to Hello messages sent from a DCS. "Client State Update" (CSU) messages are used to update the state of cache entries in servers for a given SG. CSU messages are also used to elect a "Designated" Server (DS) from a set of "Eligible" Servers (ESs). The DS is the contact point within the SG for off-SG stations wishing to query the state of the SG. A CSU message is sent from an LS to each of its DCSs when the LS observes changes in the state of one or more clients in the SG. For the purpose of the following, the state held in an LS's cache entry corresponds to the state of one or more clients. The change in state of a particular client is noted in a CSU message via a "Client State Advertisement" (CSA) record within the CSU. Examples of such changes in state are as follows: 1) an LS receives a request to add an entry to its cache (e.g., NHRP Registration Request or an administrative intervention), 2) an LS receives a request to remove an entry from its cache (e.g., NHRP Purge Request or administrative intervention), 3) a cache entry has timed out in the LS's cache, has been refreshed in the LS's cache, or has been administratively modified (e.g., in NHRP, an Internetworking address to NBMA address binding has timed out or has been refreshed). After receiving a CSU, an LS acknowledges it by sending a CSU Reply. Each CSA which the LS has not already seen is propagated to each of its the DCS's except the DCS from which it originally received the CSU. "Cache Alignment" (CA) messages allow an LS to synchronize its entire cache with that of the cache of its DCSs. That is, CA messages allow a booting LS to synchronize with its DCSs. If an LS believes that there may be reason to think that its cache has been corrupted or that it is getting bad or incomplete information from one of its DCSs then the LS may restart the cache alignment process with one or more DCSs. Detection of such corrupted databases is beyond the scope of this work. A CA message contains a CA header followed by zero or more CSA Summary (CSAS) records. CSAS records contain a summary of an entry in a server's cache. When CA messages are exchanged between an LS and one of its DCSs, the LS makes a list of those cache entries which are more recent in the DCS (based on a CSA sequence number) then the LS's own entry and adds to that list any entry in the DCS which is not already in its cache. During this process, the DCS makes a Luciani [Page 7] INTERNET-DRAFT SCSP-NBMA Expires September 1996 similar list of CSAS records. The LS forms CSU Solicit (CSUS) messages from the previously mentioned list. The CSUS messages contain a CSUS header and the CSAS records from the list. The LS then sends the CSUS messages to the DCS. The DCS responds to the LS sending the CSUS messages by sending back CSU messages containing the appropriate CSA records. Note that during this time the DCS is also forming CSUS messages from its own list and sending them to the LS to which to which the LS responds with CSUs containing the appropriate CSA records. In this way, both LS and DCS databases are synchronized. The concepts of DS and ES have been introduced in order to reduce the total amount of resources (e.g., VCs) required for the synchronization of server caches. When an LS is an ES, the LS SHOULD connect to every other ES within the SG so that the ESs are in a fully connected mesh and when an LS is not an ES, the LS SHOULD connect to one or more ESs within the SG. If the previous two conditions are met then it is ensured that changes in cache state information will take no more than 3 hops within the SG (i.e., non-ES to ES, ES to ES, and ES to non-ES). While it is not recommended, an arbitrary topology will work with SCSP assuming that the resultant graph is a spanning tree; however, a larger propagation delay for updates might be incurred and this delay will be proportional to the "diameter" of the SG. Luciani [Page 8] INTERNET-DRAFT SCSP-NBMA Expires September 1996 +---------------+ | | +-------@| DOWN |@-------+ | | | | | +---------------+ | | | | | | | | | | | | | | @ | | +---------------+ | | | | | | | WAITING | | | +--| |--+ | | | +---------------+ | | | | @ @ | | | | | | | | | @ | | @ | +---------------+ +---------------+ | BIDIRECTION |----@| UNIDIRECTION | | | | | | CONNECTION |@----| CONNECTION | +---------------+ +---------------+ Figure 1: Hello Finite State Machine (HFSM) 3.1 Hello Messages "Hello" messages ascertain whether a DCS is operational and whether the connections between the LS and DCS are bidirectional, unidirectional, or non-functional. Every LS MUST periodically send Hello messages to each of its DCSs. Every LS MUST also send a Hello Reply in response to Hello messages received from one of its DCSs. An LS must be configured with a list of DCS NBMA addresses. When an LS is an ES, it is RECOMMENDED that the LS be configured with the NBMA address of every other ES within the SG so that the ESs may be connected in a full mesh (see previous section). When an LS is not an ES, it is RECOMMENDED that the LS be configured with the NBMA address of one or more ESs within the SG. While it is not recommended, an arbitrary topology will also work with SCSP assuming that the resultant graph from is a spanning tree; however, a larger propagation delay for updates might be incurred and this delay will be proportional to the "diameter" of the SG. The mechanism for this configuration is beyond the scope of this document although some possible example mechanisms would be the use of an autoconfiguration server or manual configuration. A Server must also be configured with its Designated Server Priority (DSP) which relates is priority in the election of a DS. Luciani [Page 9] INTERNET-DRAFT SCSP-NBMA Expires September 1996 An LS has a Hello Finite State Machine (HFSM) associated with each of its DCSs (see Figure 1). The HFSM monitors the state of the connectivity between the LS and a particular DCS. The HFSM starts in the "Down" State and transitions to the "Waiting" State after NBMA level connectivity has been established. Once in the Waiting State, the LS starts sending Hello messages to the DCS which include: a hello sequence number, the LS's ID (LSID) in the sender ID field, and a zero in the receiver ID field (see Section 4.4). The DCS, upon receiving the LS's Hello, returns a Hello Reply with the same sequence number and sender ID from the Hello packet it received and it puts its own ID in the receiver ID field of the packet. When an LS receives the Hello Reply the LS transitions to the "Bidirectional Connection" State. If an LS receives a Hello message from the DCS before it receives a Hello Reply (in response to its own Hello) then it transitions to the "Unidirectional Connection" State. If the LS receives a Hello Reply for a Hello that it sent while in the Unidirectional State then it transitions into the Bidirectional Connection State. The Down and Waiting States are considered to be non-functional states as previously described. Any abnormal event, such as receiving a Hello Reply for a Hello that was not sent or a malformed Hello/Hello-Reply being received, causes the HFSM to transition to the Waiting State. A loss of NBMA connectivity causes the HFSM to transition to the Down State. Hello packets also contain a HelloInterval and a DeadFactor. The Hello interval advertises the time between sending of consecutive Hello Packets by the LS. That is, if the time between Hello packets exceeds the HelloInterval then the Hello is to be considered late by the DCS. If the DCS does not receive a Hello packet within the interval HelloInterval*DeadFactor seconds then the DCS MUST consider the LS to be stalled at which point the DCS should transition to the Waiting State. If the LS does not receive a Hello Reply within its HelloInterval then the LS resends the same Hello message it sent previously every HelloInterval until the total time elapsed reaches DeadFactor*HelloInterval at which point one of two things happens: 1) if the LS has received Hello messages from the DCS during this time then the LS transitions to the Unidirectional State; otherwise, 2) the LS transitions to the Waiting State. Luciani [Page 10] INTERNET-DRAFT SCSP-NBMA Expires September 1996 +------------+ | | +---@| DOWN | | | | | +------------+ | | | | | @ | +------------+ | |Master/Slave| | | |@---+ | |Negotiation | | | +------------+ | | | | | | | | @ | | +------------+ | | | | | | | Summarize | | | | | | | +------------+ | | | | | | | | @ | | +------------+ | | | Update | | | | | | | | Cache | | | +------------+ | | | | | | | | @ | | +------------+ | | | | | +----| Aligned |----+ | | +------------+ Figure 2: Cache Alignment Finite State Machine 3.2 Cache Alignment Messages "Cache Alignment" (CA) messages allow an LS to synchronize its entire cache with that of the cache of its DCSs. That is, CA messages allow a booting LS to synchronize with its DCSs. If an LS believes that there may be reason to think that its cache has been corrupted or that it is getting bad or incomplete information from one of its DCSs then the LS may restart the cache alignment process with one or more Luciani [Page 11] INTERNET-DRAFT SCSP-NBMA Expires September 1996 DCSs. Detection of such corrupted databases is beyond the scope of this work. A CA message contains a CA header followed by zero or more CSA Summary (CSAS) records (see Section 4.1). CSAS records contain a summary of an entry in a server's cache (see Section 4.1.1). An LS has a Cache Alignment Finite State Machine (CAFSM) associated (see Figure 2) with each of its DCSs. The CAFSM starts in the Down State. When the HFSM reaches the Bidirectional State, the CAFSM transitions to the Master/Slave Negotiation State. The Master/Slave Negotiation State causes either the LS or DCS to take on the role of master over the cache alignment process. Of course, the server which is not chosen as master then takes on the slave role. When the LS's CAFSM reaches the Master/Slave Negotiation State, it will send a CA message to the DCS to which the CAFSM applies. The first CA message which the LS sends includes no CSAS records and a CA header which contains the LSID in the Sender ID field, a zero for the Receiver ID field, a sequence number, and three bits. These three bits are the MS (Master/Slave) bit, the I (Initialization of master) bit, and the M (More) bit. In the first CA message sent by the LS, all three bits are set to one. If the LS does not receive a CA message from the DCS in CAReXmtInterval seconds then it resends the CA message it just sent. The LS continues to do this until it transitions to the Cache Summarize State or until the HFSM transitions out of the Bidirectional State. Any time the HFSM transitions out of the Bidirectional State, the CAFSM transitions to the Down State. When the LS receives a CA message from the DCS the role the LS plays in the exchange depends on packet processing as follows: 1) If the CA from the DCS has the M, I, and MS bits set and there are no CSAS records in the CA message and the SenderID as specified in the DCS's CA is larger than the LSID then a) The timer counting down the CAReXmtInterval is stopped. b) The LS's CAFSM transitions to the Cache Summarize State as the slave of the master/slave communication about to occur. c) The LS adopts the sequence number it received in the CA message as its own sequence number. d) The LS sends a CA message to the DCS which is formated as follows: the MS and I bits are set to zero, the SenderID field is set to the LSID, the ReceiverID field is set to the DCSID, the sequence number is set to the sequence number that appeared in the DCS's CA message, if there are CSAS records to be sent (i.e., if the LS's cache is not empty) then the M bit is set to one and the initial set of CSAS records are included in the CA message. Luciani [Page 12] INTERNET-DRAFT SCSP-NBMA Expires September 1996 2) If the CA message from the DCS has the MS and I bits off and the SenderID as specified in the DCS's CA message is smaller than the LSID then a) The timer counting down the CAReXmtInterval is stopped. b) The LS's CAFSM transitions to the Cache Summarize State as the master. c) The LS must process any CSAS records in the received CA. An explanation of message processing is given below. d) The LS sends a CA message to the DCS which is formated as follows: the MS bit is set, I bit is set to zero, the SenderID field is set to the LSID, the ReceiverID field is set to the DCSID, the LS's current sequence number is incremented by one and added to the CA message, if there are any CSAS records to be sent from the LS to the DCS (i.e., if the LS's cache is not empty) then the M bit is set to one and the initial set of CSAS records are included in the CA message that the LS is sending to the DCS. 3) Otherwise, the packet must be ignored. At any given time, the master or slave have at most one outstanding CA message. Once the LS's CAFSM has transitioned to the Cache Summarize State the sequence of exchanges of CA messages occurs as follows. 1) If the LS receives a CA message with the MS bit set incorrectly (e.g., the MS bit is set in the CA of the DCS and the LS is master) or if the I bit is set then the CAFSM transitions to the Master/Slave Negotiation State. 2) If the LS is master and the LS receives a CA message with a sequence number which is one less than the LS's current sequence number then the message is a duplicate and the message MUST be discarded. 3) If the LS is master and the LS receives a CA message with a sequence number which is equal to the LS's current sequence number then the CA message MUST be processed. An explanation of message processing is given below. As a result of having received the CA message from the DCS the following will occur: a) The timer counting down the CAReXmtInterval is stopped. b) The LS must process any CSAS records in the received CA message. c) Increment the LS's sequence number by one. d) The summarization continues as follows: 1) If the LS has no more CSAS records to send and the received CA message has the M bit off then the CAFSM transitions to the Update Cache State. 2) If the LS has no more CSAS records to send and the received CA message has the M bit on then the LS sends back a CA message (with new sequence number) which contains no CSAS records and with the M bit off. Reset the timer counting down the Luciani [Page 13] INTERNET-DRAFT SCSP-NBMA Expires September 1996 CAReXmtInterval. 3) If the LS has more CSAS records to send then the LS sends the next CA message with the LS's next set of CSAS records. If LS is sending its last set of CSAS records then the M bit is set off otherwise the M bit is set on. Reset the timer counting down the CAReXmtInterval. 4) If the LS is slave and the LS receives a CA message with a sequence number which is equal to the LS's current sequence number then the CA message is a duplicate and the LS MUST resend the CA message which it had just sent to the DCS. 5) If the LS is slave and the LS receives a CA message with a sequence number which is one more than the LS's current sequence number then the message is valid and MUST be processed. An explanation of message processing is given below. As a result of having received the CA message from the DCS the following will occur: a) The LS must process any CSAS records in the received CA message. b) Set the LS's sequence number to the sequence number in the CA message. c) The summarization continues as follows: 1) If the LS had just sent a CA message with the M bit off and the received CA message has the M bit off then the CAFSM transitions to the Update Cache State and the LS sends a CA message with no CSAS records and with the M bit off. 2) If the LS still has CSAS records to send then the LS MUST send a CA message with CSAS records in it. If the message being sent from the LS to the DCS contains the last CSAS records that the LS needs to send then the CA is sent with the M bit off. 6) If the LS is slave and the LS receives a CA message with a sequence that is neither equal to or one more than the current LS's sequence number then an error has occurred and the CAFSM transitions to the Master/Slave Negotiation State. CA message processing occurs as follows. When CA messages are exchanged between an LS and one of its DCSs, the LS makes a list of those cache entries which are more recent in the DCS (based on a CSAS sequence number) than the LS's own entry and adds to that list any entry in the DCS which is not already in its cache. During this process, the DCS makes a similar list. The previously mentioned list is called the CSA Request List (CRL). If the CRL of the LS is empty upon transition into the Update Cache State then the CAFSM immediately transitions into the Aligned State. If the CRL is not empty then the LS solicits the relevant CSA records from the DCS associated with the CAFSM and when the LS has all the updated CSA record information it transitions into the Aligned State. The LS Luciani [Page 14] INTERNET-DRAFT SCSP-NBMA Expires September 1996 solicits the relevant CSA records by forming CSU Solicit (CSUS) messages from the CRL. The CSUS messages contain a CSUS header and CSAS records from the CRL. CSUS messages contain a CSUS header and the CSAS records from the CRL. The LS then sends the CSUS messages to the DCS. The DCS responds to the CSUS messages by sending CSUs containing the appropriate CSA records to the LS. The DCS acts in a similar manner as does the LS with respect to acquiring updated CSA records for the CSAS records in the CRL. In this way, both LS and DCS databases are synchronized. At most one CSUS message will be outstanding at any given time. Just before the first CSUS message is sent from an LS to the DCS associated with the CAFSM, a timer is set to CSUSReXmtInterval seconds. If all the CSA records corresponding to the CSAS records in the CSUS message have not been received by the time that the timer expires then a new CSUS message will be created which includes all the outstanding CSA records plus additional CSAS records not covered in the previous CSUS message. The new CSUS message is then sent to the DCS. If, at some point before the timer expires, all CSA record updates have been received for all the CSAS records included in the previously sent CSUS message then the timer is stopped and if there are additional CSAS records that were not covered in the previous CSUS message but were in the CRL then the timer is reset and a new CSUS message is created which contains CSAS records from the CRL which have not yet been sent to the DCS. This process continues until all the CSAS records that were in the CRL have been updated in the LS. When the LS has a completely updated cache then the LS's CAFSM transitions to the Aligned State as previously mentioned. 3.3. Client State Update Messages "Client State Update" (CSU) messages are used to update the state of cache entries in servers for a given SG. CSU messages are also used to elect a "Designated" Server (DS) from a set of "Eligible" Servers (ESs). The DS is the contact point within the SG for off-SG stations wishing to query the state of the SG. An ES is a server that is eligible to become the DS by virtue of the fact that it has a Designated Server Priority (DSP) which is greater than zero. An LS may send or receive a CSU only when the CAFSM is in either the Aligned State or the Update Cache State. An LS may send or receive a CSUS message only when the CAFSM is in the Update Cache State. A CSU message is sent from an LS to each of its DCSs when the LS observes changes in the state of one or more clients in the SG. The change in state of a particular client is noted in a CSU message via a "Client State Advertisement" (CSA) record within the CSU. Luciani [Page 15] INTERNET-DRAFT SCSP-NBMA Expires September 1996 Examples of such changes in state are as follows: 1) an LS receives a request to add an entry to its cache (e.g., NHRP Registration Request or an administrative intervention), 2) an LS receives a request to remove an entry from its cache (e.g., NHRP Purge Request or administrative intervention), 3) a cache entry has timed out in the LS's cache, has been refreshed in the LS's cache, or has been administratively modified (e.g., in NHRP, an Internetworking address to NBMA address binding has timed out or has been refreshed). After receiving a CSU, an LS acknowledges it by sending a CSU Reply. Each CSA which the LS has not already seen is propagated to each of its the DCS's except the DCS from which LS originally received the CSU. A CSUS message is sent from an LS to a DCS when the LS is in the Update Cache State (see Cache Alignment Section). This occurs after the LS has exchanged CA messages with the DCS and finds that the DCS has cache entries which the LS does not have or when the DCS has cache entries that are more up to date than the same entries in the LS (based on a CSAS sequence number). The DCS responds to the CSUS messages by sending CSU messages containing the appropriate CSA records to the LS. The LS acknowledges the CSU messages as described above. During this process, the DCS sends its own CSUS messages to the LS so that both LS and DCS databases are synchronized. When an LS has one or more CAFSMs in the Aligned State, the LS is participating in the DS election process for the given SG. Once the CAFSM corresponding to a DCS has reached the Aligned State, the LS starts the DSTimer which is set to DSInitTime. Before this DSTimer expires, the LS MUST not include a Preferred DSID or Preferred DSP in the CSU messages it originates. While the DSTimer is running, the LS keeps track of its preferred DS from knowledge contained in its cache and from knowledge of its own DSP and LSID. The preferred DS is the server with the highest DSP and in the case of a tie, the largest server ID wins. CSU messages (see Section 4.2) contain CSA records (see Section 4.2.1). Each CSA contains the following fields: Client ID (CID), a Client State field, a CSA Sequence Number, a DS bit (which proclaims that the originator believes that it is DS), a C/S bit (which proclaims that the cache entry refers to a Client (bit is zero) or a Server (bit is set to one)), and CSA Originator ID (this field contains the ID of the originator of the CSA). Further, if the C/S bit is set then the CSA also contains a Preferred DSID field and a Preferred DSP field. Note that clients are assumed to have a DSP of Luciani [Page 16] INTERNET-DRAFT SCSP-NBMA Expires September 1996 zero. Servers are assumed to be clients of themselves in the sense of keeping their own state in their own cache; thus a server always advertises itself. When the DSTimer expires the LS chooses its preferred DS and starts advertising it as well as the preferred DSP. The LS then does the following: 1) If the LS thinks that it is the preferred DS then a) If all known servers have chosen this LS as leader then the LS becomes the DS (see below) b) If one or more servers are advertising a different DS from the LS then 1) Start the DSOverrideTimer with DSOverrideInterval in it 2) When the DSOverrideTimer expires a) If 2/3 of the servers believe the LS to be leader then the LS becomes the DS (see below) 2) If the LS becomes DS it does the following: a) It increases its DSP by DSPIncrement or to DSPMax whichever is least b) It sends out a CSU message with its new DSP in Preferred DSP field, its LSID in the preferred DSID field, the DS bit set, the Originator ID field set to its LSID, and the Originator DSP field set to its new DSP. 3) At all times an LS is listening for a new DS with higher DSP then the current preferred DSP (and preferred DSID). If at any time the LS sees a DSP higher then the preferred DSP or a DSP which is equal to the current preferred DSP but with an associated DSID which is larger than the preferred DSID then the LS acts as follows: 1) If the LS was the DS then a) The LS announces that the other server is the DS by sending out a CSU message with the new DS's DSID in the preferred DSID, with the new DS's DSP in the preferred DSP field, the DS bit set off, and the Originator DSP field set to its original DSP (not its incremented DSP). b) The LS sets its DSP to its original value. 2) If the LS was not the DS then a) If the new preferred DS is not the LS then the LS simply advertises the new information pertaining to the new DS b) If the new preferred DS is the LS then restart the election process as if the DSTimer had just expired. If the LS loses "connectivity" with the DS (e.g., the cache entry in the LS for the DS is removed) then the LS acts as follows: 1) The LS starts a Re-electionTimer a) If connectivity is reestablished before the timer expires then stop the timer and continue as normal Luciani [Page 17] INTERNET-DRAFT SCSP-NBMA Expires September 1996 b) else restart the election process as if the DSTimer had just expired If at any time the last CAFSM of the LS for the given SG leaves the Aligned State then all memory of the DS for that SG is erased from the LS and re-election will not take place until at least one CAFSM of the LS for the given SG reaches the Aligned State at which point the election process will start from the beginning. 4. Message/Record Contents The following subsections contain a brief description of the message and record contents for the SCSP. 4.1 CA Message-Cache Alignment Message: The Cache Alignment (CA) message allows an LS to synchronize its entire cache with that of the cache of its DCSs. Reply Indicator: States whether the the message is a reply (when indicator set) or not (when indicator is not set). For example, in NHRP this indication would be acquired through the message type number. Sender ID: This is the ID of the sender of the CA message. Receiver ID: This is the ID of the server which the LS believes is the receiver of the CA message which the LS is sending. CA Sequence Number: This field contains a sequence number that identifies the CA message instance for the given client. A larger sequence number means a more recent advertisement. Larger is defined in the well known lollipop fashion. MS bit-Master/Slave bit: This bit is part of the negotiation process for the cache alignment. When this bit is set then the sender of the CA message is indicating that it wishes to lead the alignment process. I bit-Initialize bit: When set, this bit indicates that the sender of the CA message believes that it is in a state where it is negotiating for the status of master or slave. M bit-More bit: This bit indicates that the sender of the CA message has more CSAS records to send. This implies that the cache alignment process must continue. CSAS records: Luciani [Page 18] INTERNET-DRAFT SCSP-NBMA Expires September 1996 See Section 4.1.1 4.1.1 CSAS Record-Client State Advertisement Summary Record: The client state advertisement summary is a summarization of the CSA. A CSAS contains the following: CID-Client ID: This field identifies the client whose state is being kept. CSA Originator ID: This field contains the server ID of the originator of the CSA. CSA Sequence Number: This field contains a sequence number that identifies the CSA instance for the given client. A larger sequence number means a more recent advertisement. Larger is defined in the well known lollipop fashion. 4.2 CSU (Reply) Message -Client State Update (Reply) Message: The client state update message contains a client state update header and zero or more CSAs. CSU Originator ID: This field contains the server ID of the originator of the CSU. CSU Sequence Number: This field contains a sequence number that identifies the CSU instance for the given server. A larger sequence number means a more recent advertisement. Larger is defined in the well known lollipop fashion. Number of CSAs in CSU: This field contains the number of CSAs in the current CSU. CSA records: See Section 4.2.1 4.2.1 CSA Record-Client State Advertisement Record: The Client State Advertisement (CSA) record contains the information necessary to relate the current state of a client to the servers being synchronized in the SG. A CSA record contains the following: CID-Client ID: This field identifies the client whose state is being kept in the servers cache. Client State Field: This field contains an octet string that identifies the of the client. Luciani [Page 19] INTERNET-DRAFT SCSP-NBMA Expires September 1996 CSA Originator ID: This field contains the server ID of the originator of the CSA records. CSA Sequence Number: This field contains a sequence number that identifies the CSA record instance for the given client. A larger sequence number means a more recent advertisement. Larger is defined in the well known lollipop fashion. Preferred DSID: This field contains the ID of the preferred designated as seen from the perspective of the server creating the CSA record. This field does not exist in a record when the C/S bit is zero. Preferred DSP: This field contains the priority of the preferred designated server as seen from the perspective of the server creating the CSA record. This field does not exist in a record when the C/S bit is zero. DS bit: This bit proclaims that the originator of the CSA believes that it is DS. C/S bit: This bit proclaims that the cache entry refers to a Client (bit is zero) or a Server (bit is set to one). 4.3 CSUS Message-Client State Update Solicit Message: The client state update solicit message contains a client state update header and zero or more CSAS records. CSUS Originator ID: This field contains the server ID of the originator of the CSUS. CSUS Sequence Number: This field contains a sequence number that identifies the CSUS instance for the given server. A larger sequence number means a more recent advertisement. Larger is defined in the well known lollipop fashion. Number of CSASs in CSUS: This field contains the number of CSAs in the current CSU. CSAS records (See Section 4.1.1) Luciani [Page 20] INTERNET-DRAFT SCSP-NBMA Expires September 1996 4.4 Hello (Reply) Message: The Hello/Hello-Reply message is used to check connectivity between an LS and one of its DCSs. Reply Indicator: States whether the the message is a reply (when indicator set) or not (when indicator is not set). For example, in NHRP this indication would be acquired through the message type number. Sender ID: This is the ID of the sender of the Hello message. Receiver ID: This is the ID of the server which the LS believes is the receiver of the Hello message. Hello Sequence Number: This field contains a sequence number that identifies the Hello message instance for the given client. A larger sequence number means a more recent advertisement. Larger is defined in the well known lollipop fashion. HelloInterval Hello interval advertises the time between sending of consecutive Hello Packets by an LS. If the time between Hello packets exceeds the HelloInterval then the Hello is to be considered late by the DCS. On the other hand, if the LS does not receive a Hello Reply within its HelloInterval then the LS resends the same Hello message it sent previously DeadFactor This is a multiplier to the HelloInterval. If a DCS does not receive a Hello packet within the interval HelloInterval*DeadFactor from an LS that advertised the HelloInterval then the DCS MUST consider the LS to be stalled at which point the DCS should transition to the Waiting State. On the other hand, if the LS does not receive a Hello Reply within DeadFactor*HelloInterval then one of two things happens: 1) if the LS has received Hello messages from the DCS during this time then the LS transitions to the Unidirectional State; otherwise, 2) the LS transitions to the Waiting State. Conclusions While the above text is couched in terms of synchronizing the knowledge of the state of a client within the cache of servers contained in a server grouping, this solution generalizes easily to any number of database synchronization problems (e.g., LECS synchronization). In such a case, the Client ID (CID) and Client state would be replaced by a Luciani [Page 21] INTERNET-DRAFT SCSP-NBMA Expires September 1996 unique token and an octet string describing the database entry being synchronized. The appendices below show how SCSP can be implemented in terms of packet syntax specific to a set of other protocols. Appendix A: Packet Formats For NHRP The packet formats shown below show packet types to be added to the NHRP protocol in order to support SCSP. The terms are not exactly the same since I am attempting to map SCSP into a "legacy" protocol and thus the terms used should be those of the legacy protocol and not those of SCSP. However, I have attempted to point out equivalences between terms wherever possible. Caveat Emptor! This Appendix is still under construction. A.1 NHRP Cache Alignment (NHRP CA) The Cache Alignment (CA) message allows an LS to synchronize its entire cache with that of the cache of its DCSs. The NHRP CA message has a type code of 13. The Mandatory Part has the following format: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Src Proto Len | Dst Proto Len |M|I|O| unused | No. of CSASs | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Request ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CSAS Record | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ....... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CSAS Record | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Src Proto Len This field holds the length in octets of the Source Protocol Address. Dst Proto Len This field holds the length in octets of the Destination Protocol Address. Luciani [Page 22] INTERNET-DRAFT SCSP-NBMA Expires September 1996 M This bit is part of the negotiation process for the cache alignment. When this bit is set then the sender of the CA message is indicating that it wishes to lead the alignment process. This bit is equivalent to the Master/Slave (MS) bit in SCSP. I When set, this bit indicates that the sender of the CA message believes that it is in a state where it is negotiating for the status of master or slave. This bit is equivalent to the Initialization bit in SCSP. O bit This bit indicates that the sender of the CA message has more CSAS records to send. This implies that the cache alignment process must continue. This bit is equivalent to the More bit in SCSP. No. of CSASs This field contains the number of Client State Advertisements Summaries (CSASs) contained in the NHRP CA message. CA Sequence Number A value which provides a unique identifier to aid in the sequencing of the cache alignment process. The slave NHS always copies the sequence number from the master NHS's previous NHRP CA message into its current NHRP CA message thus acknowledging the master's CA message. When the slave receives a "higher" sequence number then the number that the slave previously sent then the slave's previous NHRP CA message is acknowledged. A "larger" sequence number means a more recent CA message. Larger is defined in the well known lollipop fashion. Source Protocol Address This is the protocol address of the NHS which is sending the NHRP CA message. This value is equivalent to the Sender ID in SCSP. Destination Protocol Address This is the protocol address of the NHS which is to receive the NHRP CA message. This value is equivalent to the Receiver ID in SCSP. CSAS record See Section A.1.1. Luciani [Page 23] INTERNET-DRAFT SCSP-NBMA Expires September 1996 A.1.1 Client State Advertisement Summary Record (CSAS record): The client state advertisement summary is a summarization of the CSA. A CSAS contains the following: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Cli Proto Len | CSA Proto Len | unused | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CSA Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Client Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CSA Originator Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Cli Proto Len This field holds the length in octets of the Client Protocol Address. CSA Proto Len This field holds the length in octets of the CSA Originator Protocol Address. CSA Sequence Number This field contains a sequence number that identifies the CSA record instance for the given client. A "larger" sequence number means a more recent advertisement. Larger is defined in the well known lollipop fashion. Client Protocol Address This field identifies the protocol address of the client whose state is being kept in the NHSs' cache. CSA Originator Protocol Address This field contains the protocol address of the NHS which originated the CSA record. A.2 NHRP Client State Update Request (NHRP CSU Request) The NHRP Client State Update Request (CSU Request) message is used to update the state of cache entries in NHSs which are attached to the NHS sending the message. CSU messages are also used to elect a "Designated" Server NHS (DS). A NHRP CSU Request message is sent from one NHS (the LS) to another directly connected NHS (the DCS) when the LS observes changes in the state of one or more clients. Luciani [Page 24] INTERNET-DRAFT SCSP-NBMA Expires September 1996 This observation may be a result of receiving a CSU from another DCS or as a result of some event occurring for a client that has registered with it. The change in state of a "particular" client is noted in a CSU message via a "Client State Advertisement" (CSA) record within the CSU. The NHRP CSU Request message has a type code of 11. The Mandatory Part has the following format: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Src Proto Len | unused | No. of CSAs | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Request ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CSA Record | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ....... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CSA Record | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Src Proto Len This field holds the length in octets of the Source Protocol Address. No. of CSAs This field contains the number of Client State Advertisements (CSAs) contained in the NHRP CSU message. Request ID A value which, when coupled with the address of the source, provides a unique identifier for the NHRP CSU Request This value is equivalent to the CSU Sequence Number in SCSP. A "larger" sequence number means a more recent advertisement. Larger is defined in the well known lollipop fashion. Source Protocol Address This is the protocol address of the NHS which is sending the NHRP CSU Request. This value is equivalent to the CSU Originator ID in SCSP. CSA Record See Section A.2.1. Luciani [Page 25] INTERNET-DRAFT SCSP-NBMA Expires September 1996 A.2.1 CSA Record The Client State Advertisement (CSA) record contains the information necessary to relate the current state of a client to the NHSs being synchronized. There are zero or more NHRP CSA records in an NHRP CSU Request message. The contents of a record is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Cli Proto Len | CSA Proto Len |D|S| unused | Client State | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Client State (Cntd) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CSA Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Client NBMA Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Client NBMA Subaddress (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Client Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CSA Originator Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Designated NHS State (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Cli Proto Len This field holds the length in octets of the Client Protocol Address. CSA Proto Len This field holds the length in octets of the CSA Originator Protocol Address. D bit: This bit proclaims that the originator of the CSA believes that it is the designated server. S bit: This bit proclaims whether the cache entry refers to a Client (bit is zero) or a Server (bit is set to one). Client State See Section A.2.1.1 CSA Sequence Number This field contains a sequence number that identifies the CSA Luciani [Page 26] INTERNET-DRAFT SCSP-NBMA Expires September 1996 record instance for the given client. A "larger" sequence number means a more recent advertisement. Larger is defined in the well known lollipop fashion. Client NBMA Address The Source NBMA address field is the address of the source station which is sending the Next Hop Resolution Request. If the field's length as specified in ar$shtl is 0 then no storage is allocated for this address at all. Client NBMA SubAddress The Source NBMA subaddress field is the address of the source station which is sending the Next Hop Resolution Request. If the field's length as specified in ar$sstl is 0 then no storage is allocated for this address at all. Client Protocol Address This field identifies the protocol address of the client whose state is being kept in the NHSs' cache. CSA Originator Protocol Address This field contains the protocol address of the NHS which originated the CSA record. Designated NHS State This field/record exists if and only if the S bit is set. See Section A.2.1.2 for more details. A.2.1.1 Client State This field/record contains an octet string which identifies the current state of the client. The field/record is broken down as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | State | Maximum Transmission Unit | Holding Time | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Hold Time(Cntd)| +-+-+-+-+-+-+-+-+ State This field contains a value which represents the change in state of the client. For example: 0 - Client is registered and available. 1 - Holding timer expired for client. Luciani [Page 27] INTERNET-DRAFT SCSP-NBMA Expires September 1996 2 - Client reregistered. 3 - Client has been purged. Maximum Transmission Unit This field represents the acceptable Maximum MTU for connections to the client. Holding Time The Holding Time field specifies the number of seconds for which the client information specified is valid. Cached information SHALL be discarded when the holding time expires. A.2.1.2 Designated NHS State This field/record exists if and only if the S bit is set in the CSA record. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | DS Proto Len | Pref DSP | unused | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Preferred Designated NHS Protocol Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ DS Proto Len This field holds the length in octets of the Preferred Designated NHS's Protocol Address. Pref DSP: This field contains the priority of the preferred Designated NHS as seen from the perspective of the server creating the CSA record. This field does not exist in a record when the C/S bit is zero. Preferred DSID: This field contains the ID of the preferred designated as seen from the perspective of the server creating the CSA record. This field does not exist in a record when the C/S bit is zero. A.3 NHRP Client State Update Reply (NHRP CSU Reply) The NHRP Client State Update Reply (CSU Reply) message is used to acknowledge the reception of NHRP Client State Update Request. A NHRP CSU Reply message is sent from one NHS (the DCS) to the Luciani [Page 28] INTERNET-DRAFT SCSP-NBMA Expires September 1996 NHS (the LS) which sent the original NHRP CSU Request. The NHRP CSU Reply message has a type code of 12. The Mandatory Part of the NHRP CSU Reply is the same as that of the NHRP CSU Request so that when an NHS receives an NHRP CSU Request all that needs to be done to reply to it is to change the type code to 12 and send the packet back. However, the Mandatory part may be truncated from the NHRP CSU Request to as little as the following: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Src Proto Len | unused | No. of CSAs | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Request ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Src Proto Len This field holds the length in octets of the Source Protocol Address. No. of CSAs This field contains the number of Client State Advertisements (CSAs) contained in the NHRP CSU message. This field is set to zero when the Reply is merely a truncated version of the request. Request ID A value which, when coupled with the protocol address of the source, provides a unique identifier which can be used to match the reply to original request. This value is equivalent to the CSU Sequence Number in SCSP. A "larger" sequence number means a more recent advertisement. Larger is defined in the well known lollipop fashion. Source Protocol Address This is the protocol address of the NHS which sent the original NHRP CSU Request. This value is equivalent to the CSU Originator ID in SCSP. A.4 NHRP Client State Update Solicit Message (NHRP CSUS message) The NHRP CSUS message contains a client state update header and zero or more CSAS records. This message allows one NHS (LS) to solicit the entirety of CSA data stored in the cache of a directly connected NHS (DCS). The DCS responds with CSUs containing the appropriate Luciani [Page 29] INTERNET-DRAFT SCSP-NBMA Expires September 1996 CSAs. The NHRP CA message has a type code of 14. The Mandatory Part has the following format: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Src Proto Len | Dst Proto Len |M|I|O| unused | No. of CSASs | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CSUS Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CSAS Record | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ....... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CSAS Record | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Src Proto Len This field holds the length in octets of the Source Protocol Address. Dst Proto Len This field holds the length in octets of the Destination Protocol Address. M This bit is part of the negotiation process for the cache alignment. When this bit is set then the sender of the CSUS message is indicating that it wishes to lead the alignment process. This bit is equivalent to the Master/Slave (MS) bit in SCSP. I When set, this bit indicates that the sender of the CSUS message believes that it is in a state where it is negotiating for the status of master or slave. This bit is equivalent to the Initialization bit in SCSP. O bit This bit indicates that the sender of the CSUS message has more CSAS records to send. This implies that the cache alignment process must continue. This bit is equivalent to the More bit in SCSP. No. of CSASs Luciani [Page 30] INTERNET-DRAFT SCSP-NBMA Expires September 1996 This field contains the number of Client State Advertisements Summaries (CSASs) contained in the NHRP CSU message. CSUS Sequence Number A value which provides a unique identifier to aid in the sequencing of the cache alignment process. The slave NHS always copies the sequence number from the master NHS's previous NHRP CSUS message into its current NHRP CSUS message thus acknowledging the master's CSUS message. When the slave receives a "higher" sequence number then the number that the slave previously sent then the slave's previous NHRP CSUS message is acknowledged. A "larger" sequence number means a more recent CSUS message. Larger is defined in the well known lollipop fashion. Source Protocol Address This is the protocol address of the NHS which is sending the NHRP CSUS message. This value is equivalent to the CSUS Originator ID in SCSP. Destination Protocol Address This is the protocol address of the NHS which is to receive the NHRP CSUS message. This value is equivalent to the Receiver ID in SCSP. CSAS record See Section A.1.1. A.5 NHRP Hello Request: The NHRP Hello Request message is used to check connectivity between the sending NHS (the LS) and one of its directly connected neighbor NHSs (the DCSs). The NHRP Hello Request packet has a type code of 9. The Mandatory Part has the following format: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Src Proto Len | Dst Proto Len | unused | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Request ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | HelloInterval | DeadFactor | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Luciani [Page 31] INTERNET-DRAFT SCSP-NBMA Expires September 1996 Src Proto Len This field holds the length in octets of the Source Protocol Address. Dst Proto Len This field holds the length in octets of the Destination Protocol Address. Request ID A value which, when coupled with the address of the source, provides a unique identifier for the information contained in a NHRP Hello Request and its associated NHRP Hello Reply. This value can be used by the source to aid in matching a NHRP Hello Request with a NHRP Hello Reply. This value is equivalent to the Hello Sequence Number in SCSP. A "larger" sequence number means a more recent advertisement. Larger is defined in the well known lollipop fashion. HelloInterval The hello interval advertises the time between sending of consecutive Hello Packets by an LS. If the time between Hello packets exceeds the HelloInterval then the Hello is to be considered late by the DCS. On the other hand, if the LS does not receive a Hello Reply within its HelloInterval then the LS resends the same Hello message it sent previously DeadFactor This is a multiplier to the HelloInterval. If a DCS does not receive a Hello packet within the interval HelloInterval*DeadFactor from an LS that advertised the HelloInterval then the DCS MUST consider the LS to be stalled at which point the DCS should transition to the Waiting State. On the other hand, if the LS does not receive a Hello Reply within DeadFactor*HelloInterval then one of two things happens: 1) if the LS has received Hello messages from the DCS during this time then the LS transitions to the Unidirectional State; otherwise, 2) the LS transitions to the Waiting State. Source Protocol Address This is the protocol address of the NHS which is sending the NHRP Hello Request. This value is equivalent to the Sender ID in SCSP. Destination Protocol Address This is the protocol address of the NHS which is to Reply to the NHRP Hello Request. This value is equivalent to the Receiver ID in SCSP. If the sender does not know this address then the sender sets it to zero and it will be filled in the subsequent reply. Luciani [Page 32] INTERNET-DRAFT SCSP-NBMA Expires September 1996 A.6 NHRP Hello Reply: The NHRP Hello Reply message is used to check connectivity between the NHS sending the NHRP Hello Request (the LS) and one of its directly connected neighbor NHSs (the DCSs). The DCS sends the NHRP Hello Reply to the LS as a result of having received an NHRP Hello Request message. The NHRP Hello Reply message has a type code of 10. The Mandatory Part has the following format: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Src Proto Len | Dst Proto Len | unused | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Request ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Protocol Address (variable length) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Src Proto Len This field holds the length in octets of the Source Protocol Address. Dst Proto Len This field holds the length in octets of the Destination Protocol Address. Request ID This value must be the same as that sent in the NHRP Hello Request that precipitates the NHRP Hello Reply. Source Protocol Address This is the protocol address of the NHS which is sent the NHRP Hello Request. This value is equivalent to the Sender ID in SCSP. Destination Protocol Address This is the protocol address of the NHS which is Replying to the NHRP Hello Request. This value is equivalent to the Receiver ID in SCSP. If the NHRP Hello Request has set this field to zero then the replying NHS will fill this field with the appropriate value. Appendix B: Packet Formats For ATMARP Luciani [Page 33] INTERNET-DRAFT SCSP-NBMA Expires September 1996 Work in progress. Appendix C: Packet Formats For IPMC Work in progress. Appendix D: Packet Formats For LECS Work in progress. References [1] "Classical IP and ARP over ATM", Laubach, RFC 1577. [2] "NBMA Next Hop Resolution Protocol (NHRP)", Katz, Piscitello, Cole, Luciani, draft-ietf-rolc-nhrp-07.txt. [3] "OSPF Version 2", Moy, RFC1583. [4] "PNNI Draft Specification", Dykeman, Goguen, ATM Forum 94-0471R15 (Straw Vote), 1996. Acknowledgments This I-D is a distillation of issues raised during private discussions, on the IP-ATM mailing list, and during the Dallas IETF (12/95). Thanks to all who have contributed but particular thanks to Andy Malis, Raj Nair, and Matthew Doar of Ascom Nexion. Author's Address James V. Luciani Ascom Nexion 289 Great Road Acton, MA 01720-4379 USA Phone: +1 508 266 3450 Email: luciani@nexen.com Luciani [Page 34]