Internet Draft Alex Audu Expiration: December 2002 Alcatel USA Inc. File: draft-gopal-forces-fact-01.txt Working Group: ForCES Ram Gopal Nokia June 2002 ForwArding and Control ElemenT protocol (FACT) draft-gopal-forces-fact-01.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as ``work in progress.'' The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [2]. Abstract This document defines a FACT protocol that is suitable for communicating between Forwarding Element and Control Elements inside a network element. This protocol addresses all the requirements described in Forces [3] requirements document. This document also describes some of the architecture that FACT may leverage during the protocol operation. Gopal,Audu Expires December 2002 [Page 1] Internet Draft Forwarding and Control Element protocol June 2002 Table of Content 1. Definitions.....................................................3 2. Introduction....................................................5 3. Protocol Overview...............................................5 3.1. 3.1 Reliability...............................................7 4. Message Overview................................................9 4.1. Protocol Message Header structure.............................9 4.1.1. Version.....................................................9 4.1.2. Message Classes and Types...................................9 4.1.3. Length or Reserved.........................................11 4.1.4. CE Tag.....................................................11 4.1.5. FE Identifier..............................................12 4.1.6. Reserved Bits..............................................12 4.1.7. Priority (P) Bits..........................................12 4.1.8. Transaction Sequence Number (TSN)..........................12 4.2. Service Data or Payload Structure............................12 5. FACT Messages..................................................13 5.1. Association and Connection (CAM) Messages....................14 5.1.1. Join Request...............................................14 5.1.2. Join Response..............................................15 5.1.3. Leave Request..............................................15 5.1.4. Leave Response.............................................16 5.1.5. Release Request............................................16 5.1.6. Release Response...........................................17 5.2. Capabilities Control (CAPCO) Messages........................17 5.2.1. Capabilities Request.......................................17 5.2.2. Capabilities Response......................................18 5.2.3. Configure Logic Components.................................19 5.2.4. Configure Logic Components Acknowledgement.................20 5.2.5. Topology Request...........................................20 5.2.6. Topology Response..........................................20 5.2.7. Port Status request - CE requesting a FE for physical configuration.....................................................21 5.2.8. Port Status response - FE exporting the physical configuration to CE...............................................21 5.3. PE Maintenance Messages......................................22 5.3.1. Protocol Element Active....................................22 5.3.2. Protocol Element Active Acknowledgement....................23 5.3.3. Protocol Element Inactive..................................24 5.3.4. Protocol Element Inactive Acknowledgement..................24 5.3.5. Heartbeat (or Health check)................................24 5.3.6. Heartbeat Acknowledgement (FE response to heart beat request) ..................................................................25 5.3.7. Status Change and Event Notify message.....................25 5.4. PE Traffic Maintenance (PETM) Messages.......................25 5.4.1. Control packet Redirect to CE for processing...............25 5.4.2. Control Packet Redirect Acknowledgement....................26 Gopal,Audu Expires August 2002 [Page 2] Internet Draft Forwarding and Control Element protocol June 2002 5.4.3. Control Packet Forwarding through a specific output port...26 5.4.4. Control Packet Forwarding Acknowledgement..................27 5.5. Event Notification Messages..................................27 5.5.1. Event Register.............................................27 5.5.2. Event Register Acknowledgement.............................27 5.5.3. Event De-Register..........................................27 5.5.4. Event De-Register Acknowledgement..........................27 5.5.5. Status change in configuration (Asynchronous event).......27 5.5.6. FE notifying the change in port status to CE...............28 5.5.7. Notification of CE status..................................29 5.5.8. Dynamic configuration of FEÆs..............................29 6. Architecture support for FACT protocol.........................30 6.1. Security.....................................................30 6.2. High availability support....................................30 6.3. Access Control to FE.........................................30 6.4. Configurable parameters......................................31 6.5. Management Interface to FE and CE through FE-Manager and CE- Manager...........................................................31 7. References.....................................................31 8. Acknowledgments................................................32 9. Authors' Addresses.............................................32 1. Definitions The following definitions are taken from [3] Forwarding Element (FE) - A logical entity that implements the ForCES protocol. FEs use the underlying hardware to provide per- packet processing and handling as directed by a CE via the ForCES protocol. FEs may use PFE partitions, whole PFEs, or multiple PFEs. Control Element (CE) - A logical entity that implements the ForCES protocol and uses it to instruct one or more FEs how to process packets. CEs handle functionality such as the execution of control and signaling protocols. CEs may consist of PCE partitions or whole PCEs. Pre-association Phase - The period of time during which a FE Manager (see below) and a CE Manager (see below) are determining which FE and CE should be part of the same network element. Post-association Phase - The period of time during which a FE does know which CE is to control it and vice versa, including the time during which the CE and FE are establishing communication with one another. Gopal,Audu Expires August 2002 [Page 3] Internet Draft Forwarding and Control Element protocol June 2002 ForCES Protocol - While there may be multiple protocols used within the overall ForCES architecture, the term "ForCES protocol" refers only to the ForCES post-association phase protocol (see below). ForCES Post-Association Phase Protocol - The protocol used for post- association phase communication between CEs and FEs. This protocol does not apply to CE-to-CE communication, FE-to-FE communication, or to communication between FE and CE managers. The ForCES protocol is a master-slave protocol in which FEs are slaves and CEs are masters. This protocol includes both the management of the communication channel (e.g., connection establishment, heartbeats) and the control messages themselves. The term ForCES protocol may refer to a suite of protocols that are used to exchange control information as well as redirect data packets between the CEs and FEs. FE Model - Modeling of logical functions in a Forwarding element line card. FE Manager - A logical entity that operates in the pre-association phase and is responsible for determining to which CE(s) a FE should communicate. This process is called CE discovery and may involve the FE manager learning the capabilities of available CEs. A FE manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which CE(s) to use. Being a logical entity, a FE manager might be physically combined with any of the other logical entities mentioned in this section. CE Manager - A logical entity that operates in the pre-association phase and is responsible for determining to which FE(s) a CE should communicate. This process is called FE discovery and may involve the CE manager learning the capabilities of available FEs. A CE manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which FE to use. Being a logical entity, a CE manager might be physically combined with any of the other logical entities mentioned in this section. Pre-association Phase Protocol - A protocol between FE managers and CE managers that is used to determine which CEs or FEs to use. A pre-association phase protocol may include a CE and/or FE capability discovery mechanism. Note that this capability discovery process is wholly separate from (and does not replace) that used within the ForCES protocol (see Section 7, requirement #1). However, the two capability discovery mechanisms may utilize the same FE model (see Section 6). Pre-association phase protocols are not discussed further in this document (see Section 11.3). Gopal,Audu Expires August 2002 [Page 4] Internet Draft Forwarding and Control Element protocol June 2002 ForCES Network Element (NE) - An entity composed of one or more CEs and one or more FEs. To entities outside a NE, the NE represents a single point of management. Similarly, a NE usually hides its internal organization from external entities. ForCES Protocol Element - A FE or CE. Logical Component - Components in the forwarding data path like filter, meter, forwarder, shaper etc. FE Model - Organization of logical components in the Forwarding plane. 2. Introduction Network elements such as routers play an important role in transporting IP packets in the Internet. QoS aware router, policy based routing and middle-box functions such as firewall, NAT, proxies put heavy requirements on per-hop behavior treatment for IP packets. This complicates network element. Routers have emerged from simple monolithic software to a distributed routing complex that interconnects different networks and distributes the routing and forwarding logic to line cards and control cards. A line card does basic forwarding function and a control card runs all the control and management functions. Forces [3][5] defines both architectural and protocol requirements for the communication between CE and FE. Forwarding and Control ElemenT (FACT) protocol addresses all the requirement of Forces protocol and is described in this document. 3. Protocol Overview ForCES is a framework consisting of set of protocols and data structure representing the forwarding and control elements in the form of an extensible model [6][5][4]. CEs handle control, signaling and management protocols, while FEs perform forwarding functions. CEs control the behavior of FEs in a master/slave fashion. FACT protocol is designed to communicate between FE to CE and/or between CE to CE. Since CE-CE communication is outside the scope of Forces activity, we will not be discussing CE-CE communication in Gopal,Audu Expires August 2002 [Page 5] Internet Draft Forwarding and Control Element protocol June 2002 this document FACT protocol satisfies all the Forces protocol requirements. The FACT protocol is logically separated into base control protocol and logical components service functions. This is similar to SNMP where SNMP protocol provides a set of message to exchange messages between SNMP manager and agent(s)_ MIB is the actual payload communicated in the form of OID between them. Similarly FACT protocol has fixed header and a variable size payload field; the payload field carries data that may contain one of the following * Control packets which are received by FE through external port or Packets that are sent to the external egress port * Logical components details which are represented in FE Modeling Document [4] (a) Base control functions FACT protocol depends upon certain entities to start communicating between FE and CE; this information is sent to either FE or CE by FE-Manager and CE-Manager [sec 6.4] components respectively. After the pre-association phase, FACT protocol provides mechanism that carries traffic between FE and CE components. This includes support for dynamic association, high availability, security, topology discovery of FE and CE, Control packet redirection etc. (b) Service specific functions Using base forces protocol, any logical components inside the FE can be configured or managed. FE modeling is flexible enough to allow any arbitrary queries. These queries are in the form of OID (TLC also can be used and it is the responsibly of FE to interpret the payload). However, FE Modeling is based on MIB and standard OID representation is easy and does not require any explicit standardization. Through out this document we describe the Parameters of each logical component in the OID format. FACT protocol is independent of type of encoding. FACT protocol supports different types of messages and includes both synchronous messages like simple request/reply, asynchronous Gopal,Audu Expires August 2002 [Page 6] Internet Draft Forwarding and Control Element protocol June 2002 messages to notify the status and transaction oriented synchronous messages that to support 2 phase command bundling functions operations. The FACT protocol provides a notion of distributed IPC mechanism by providing support functions required for replication, high availability and fail-over support. All these features are optional and can be configured through FE-Manager and CE-Managers for FEs and CEs respectively during the pre-association phase. 3.1.3.1 Reliability FACT protocol supports FE and/or CE fail-over functions in order to support a high availability of the network element. All control or data messages exchanged between a CE and a FE are assigned a tag for identification purposes. The CE-SET is a list of CEs configured within a Network Element (NE) as a cooperating unit. Likewise, the FE-SET is a list of FEs configured in an NE as a cooperating unit. The following is a list of high-availability mechanisms, which can be supported by FACT protocol. Note this list not exhaustive and is provided for illustrative purpose. (1) Strong Consistency: One or more CE are active and only one CE is engaged in communicating with a FE. The other CEs may be simply snooping the traffic and update their internal states (assume shared LAN). If CEs are far away may be few hops, it is then responsibility of the CEs to synchronous the states. (2) Weak Consistency (Fail-over): FE can communicate directly to the designated CE and if the designated CE fails, FE will start communicating to the backup CE. (3) Load Sharing: Different CE can be configured to support part of the functions for example; OSPF may be running in one CE, BGP may be running in another CE, the functionality is distributed across the CE in a CE-set. FEs can forward the packets to different CE based on the CE's capability and functions. In this way the load and processing on the CEs is distributed evenly. In all the above cases, CE and FE are pre-configured to perform such activities and are done as part of pre-association phase. The fail-over model supports n+k redundancy, where n FEs are the Gopal,Audu Expires August 2002 [Page 7] Internet Draft Forwarding and Control Element protocol June 2002 Minimum number of redundant FEs required to handle configured traffic and k FEs are available to take over for a failed or unavailable FE. This is also true for CEs. Note that 1+1 active/standby redundancy is a subset of this model, as is a simplex 1+0 (no redundancy). To avoid a single point of failure, it is recommended that a minimum of a pair of two FEs be in the list, resident in separate hosts and available over different associations with the CEs. ******** ************** * *-----------------------------------------* ******** * * * * * FE1 * * * CE1 * Associations ---------* ******** * * *------------------------ | * ******** * ******** | | * * FE2 * * | | * ******** * ******** | | ************** * *------------------------------- * * | * CE2 * Associations | * *------------- | * * | | ******** | | ************** | |-----------------* ******** * | * * FE1 * * -----------------------------* ******** * * ******** * * * FE2 * * * ******** * ************** . Figure 1 - Illustration of Logical Assoication between CE-FE Gopal,Audu Expires August 2002 [Page 8] Internet Draft Forwarding and Control Element protocol June 2002 4. Message Overview This section describes the details of each message and its format. 4.1. Protocol Message Header structure FACT protocol Header contains the following fields; some of the fields are optional and it is interpreted based on the Type. All these messages are based on TLV format and is conforms to network byte ordering. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version |MsgCls|Msg-Type| Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CE-Tag | FE-Identifier |P| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Transaction sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Service data (Payload) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 4.1.1. Version The version field contains the version of the FACT protocol supported by the implementation. The current supported versions are : value - 0x01, Version - Rlease 1.0 4.1.2. Message Classes and Types The messages can be grouped into classes, with each class consisting Of a set of related message types. The valid message classes are: Message Class: 3 bits (unsigned integer) 0 Reserved 1 PE Connection and association (CA) Messages 2 Capabilities Control (CAPCO) Messages 3 PE State Maintenance (PESM ) Message 4 PE Traffic Maintenance (PETM) Messages 5 Event Noticiation (EN) Messages 6 Vendor Specific (VS) Messages Gopal,Audu Expires August 2002 [Page 9] Internet Draft Forwarding and Control Element protocol June 2002 The message names for the defined message classes are as follows: Message Type for Connection and Association (CA) Message Class 0 Reserved 1 Join Request 2 Join Response 3 Leave Request 4 Leave Response 5 Release Request 6 Release Response 7-31 Reserved by IETF Message Type for Capabilities Control (CAPCO) Message Class 0 Reserved 1 Capabilities Request 2 Capabilities Response 3 Configure Logic Components 4 Configure Logic Components Ack. 5 Topology Request 6 Topology Respons 7 Port Status Request 8 Port Status Response Message Types for PE State Maintenance (PESM) Message Class 0 Reserved 1 Error 2 PE Active 3 PE Inactive 4 PE Active ACK 5 PE Inactive ACK 6 Heartbeat 7 Heartbeat Ack 9-31 Reserved by IETF Message Types for PE Traffic Maintenance (PETM) Message Class 0 Reserved 1 Control Packet Redirect 2 Control Packet Redirect Acknowledgement 3 Control Packet Forward 4 Control Packet Forward Acknowledgement 5-31 Reserved by IETF Gopal,Audu Expires August 2002 [Page 10] Internet Draft Forwarding and Control Element protocol June 2002 Message Types for Event Notification (EN) Message Class 0 Reserved 1 Event Register 2 Event Register Acknowledgement 3 Event De-register 4 Event De-register Acknowledgement 5 Status Change and Event Notify from FE to CE 6 Port Status change notification from FE to CE 7-31 Reserved by IETF Message Types for Vendor Specific Function (VSF) Message Class Vendor specific function types specification is beyond the scope of this protocol. 4.1.3. Length or Reserved Depending on the message type this field either contains a length of the packet in bytes or it is reserved. If this field is reserved it MUST be set to Zero 4.1.4. CE Tag During a pre-association phase, CE can be configured using CE- Manager component. In a network element, there may be many CEs; one or more CEs can be grouped together to form a CE- set. CE-set is unique in one network element and is identified by an 8-bit number. To identify a CE inside a CE-set, the CE identifier is used which is also a 8-bit field. Figure 1 shows the CE-Tag fields, CE-Tag is a 16-bit integer which has two portions, the higher order 8-bit describes the CE-set number, and the lower 8-bit describes the CE- identification number. The main advantage of having such naming is to uniquely identify the Communication elements and their logical association inside a network element. This field is optional for certain message types. +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CE-Set Number | CE-Identifier | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 1 CE-Tag fields Gopal,Audu Expires August 2002 [Page 11] Internet Draft Forwarding and Control Element protocol June 2002 4.1.5. FE Identifier This is a 16-bit field to uniquely identify the FE in one network element. The main advantage is that FE or CE may use different interconnect technologies, they can be identified by using the address itself (for example IP address or MAC address etc). This unique naming scheme helps to manage the CE and FEs together and support some features, which are required for back-up recovery, configuration update process, and high availability support. This identifier is always present in all type of messages. 4.1.6. Reserved Bits 7-bit field MUST be set to zero. 4.1.7. Priority (P) Bits If this bit is set then the message should be treated as a high-priority message by the receiving end-point. If this is set to zero then the message is of normal priority. 4.1.8. Transaction Sequence Number (TSN) This 32-bit field uniquely identifies the transaction between the FE and CE. When a request is made by one endpoint (Say CE) it generates TSN number that is sent in the request message; the other endpoint (Say FE) can copy this same TSN number in its reply message. 4.2. Service Data or Payload Structure FACT protocol messages consist of the Common Message Header described in the previous section, followed by zero or more variable length parameters, as defined by the message type. This constitutes the Payload or Service Data. This service data represents one of the following (1) Logical component configuration (2) Logical component statistics (3) Logical component status and events (4) FE capabilities and topology information (5) Command data which FE wants to send through a particular logical component output interface Gopal,Audu Expires August 2002 [Page 12] Internet Draft Forwarding and Control Element protocol June 2002 The variable length parameters in the payload are defined in a Tag- Length-Value (TLV) format as shown below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Parameter Tag | Parameter Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ \ \ / Parameter Value / \ \ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Mandatory parameters MUST be placed before optional parameters in a message. Parameter Tag: 16 bits (unsigned integer) The Tag field is a 16-bit identifier of the type of parameter. It takes a value of 0 to 65534. The value of 65535 is reserved for IETF-defined extensions. Values other than those defined in specific parameter description are reserved for use by the IETF. Parameter Length: 16 bits (unsigned integer) The Parameter Length field contains the size of the parameter in bytes, including the Parameter Tag, Parameter Length, and Parameter Value fields. The Parameter Length does not include any padding bytes. Parameter Value: variable-length The Parameter Value field contains the actual information to be transferred in the parameter. The total length of a parameter (including Tag, Parameter Length and Value fields) MUST be a multiple of 4 bytes. If the length of the parameter is not a multiple of 4 bytes, the sender pads the parameter at the end (i.e., after the Parameter Value field) with all zero bytes. The length of the padding is NOT included in the parameter length field. A sender SHOULD NEVER pad with more than 3 bytes. The receiver MUST ignore the padding bytes. 5. FACT Messages This section defines the messages and their parameter contents. Gopal,Audu Expires August 2002 [Page 13] Internet Draft Forwarding and Control Element protocol June 2002 5.1. Association and Connection (CAM) Messages 5.1.1. Join Request After the pre-association phase [6.4], the FEs can join and leave with any CE in a CE-set. During the join request FE reports their capabilities to CE. Example of such capabilities include whether it can support features like command bundling, 2 phase command operation, or support for high availability etc.[section 6] At a given point, CEs from one CE set can communicate with a FE. FE has to know which CE's request it can accept. This information is configured during the pre-association phase. FE uses this CE-list to send the join request. It first tries one of the CEs in the list and if it is not successful then it tries the next CE in the list. The format of the JOIN Message payload is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | HAS |C| Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 2 Join request HAS bits: These 3 bits are optionally used to communicate whether the FE can participate in high availability mechanisms. [section 6] HAS - High availability Support -------------------------------- 000 - No support 001 - Fail over support (Weak Consistency) 011 - Replicate the control packets to designated CEÆs in a CE set (Strong consistency). NOTE: This may also depend upon the type of interconnects technology being used. For example, if FE and CEÆs are connected through the shared bus or segment, this feature all the CEÆs may receive the copies of the packet. Bit C: This bit set to zero means it does not provide Gopal,Audu Expires August 2002 [Page 14] Internet Draft Forwarding and Control Element protocol June 2002 Persistence to command sets which is required to perform 2 phase command operations. This bit set to ONE means that this FE can participate in 2-phase command operation. 5.1.2. Join Response CE after receiving a join request message performs following operations. (1) Checks the FE identifier in the request message and if it equal to zero, then the CE generates a unique for that FE. I (2) If the FE identifier field is not zero, then the CE checks up its previously stored configuration for that FE. If it has any, it will upload that configuration to FE in the reply message. This is warm restart operation. The format for the JOIN RESPONSE message payload is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Previous Configuration Data | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Previous configuration data: This is optional field and length field in the FACT protocol header determines the size of this field. If FE is performing warm restart, CE may send the FE previous configuration information in this field. 5.1.3. Leave Request The FE or CE uses this to inform the NE that it intends to leave the NE group. The LEAVE message contains the following parameters: Reason Info String (optional) The format for the LEAVE Message parameters is same as follows: Gopal,Audu Expires August 2002 [Page 15] Internet Draft Forwarding and Control Element protocol June 2002 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Tag (0xa) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reason | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Tag (0x4) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | INFO String* | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The format and description of the optional Info String parameter is the same as for the PE Up message (See Section 4.3.2.1.). The reason parameter indicates the reason the PE is leaving the NE. Valid values are as follows: Value Description 0x1 Management Inhibit (Manual Removal) 0x2 Invalid NE group 5.1.4. Leave Response The CE uses this to acknowledge a LEAVE Request message, or to relay an unsuccessful join attempt by the requesting FE (or CE). The LEAVE Response message contains the following parameters: Reason Info String (optional) The format for the LEAVE Message parameters is same as in LEAVE Request message (See 5.1.3). 5.1.5. Release Request A CE or FE uses this message to request the start of the termination of the logical connection established with its master (CE) or slave (FE). The RELEASE message contains the following parameters: Gopal,Audu Expires August 2002 [Page 16] Internet Draft Forwarding and Control Element protocol June 2002 Reason The format for the RELEASE Message parameters is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Tag (0xf) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reason | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Valid values for Reason are as follows: Define Value Description RELEASE_MGMT 0x0 Management layer generated release. RELEASE_PHYS 0x1 Physical layer alarm generated release. RELEASE_EST_INV 0x2 Establish request invalid RELEASE_OTHER 0x3 Other reasons 5.1.6. Release Response A CE or FE uses this message to acknowledge the request by its slave FE or master CE to start of the termination of the logical connection established between them. The RELEASE Response message contains the following parameters: Reason The format for the RELEASE ACK Message parameters is the same as in RELEASE request message (see 5.1.7) 5.2.Capabilities Control (CAPCO) Messages 5.2.1. Capabilities Request FE may have one or more logical components on the forwarding plane like meter, shaper, egress port etc. CE may configure or query these components and their status at any time. In order to do this, CE needs to know the logical components placement and sequence in the forwarding data path. FE Model [4] describes the arrangement and the relationship of those components. For understanding purpose, we provide the summary as follows: Gopal,Audu Expires August 2002 [Page 17] Internet Draft Forwarding and Control Element protocol June 2002 A FE identifier identifies FE; FE may contain one or more parallel data path. On each parallel data path, there may be one or more logical components and each one is connected to another either in series or in parallel fashion. The logical component is uniquely identified by a number (which needs to be IANA assigned). Examples of logical components are filter, shaper, egress port etc. Each logical component attributes can be assessed by their OID values. Vendor specific components can also be added. For example, the following are the few queries that may be of interest to CE. (1) To get list of All logical components (2) To get list of Attributes of one logical components (3) To configure or modify one logical components Some of these queries are useful to take a snapshot of logical components configuration, if FE needs to restart, they are uploaded as part of join request message. The FE model is flexible and any type of queries (similar to SNMP) are possible. For illustration we use OID as data representation, but we can Specify using XML or other data format. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Query Information +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Query Message: This can be list of attributes that are part of logical functions (components) controlled and managed by a FE. These attributes can be referenced using XML or OID or any other suitable data format. NOTE: Response to this message is sent with message type value = 0x07. This message type is used to convey not only the port values it can convey any logical component attribute value pairs. 5.2.2. Capabilities Response Gopal,Audu Expires August 2002 [Page 18] Internet Draft Forwarding and Control Element protocol June 2002 This is used by the FE to report its capabilities to the CE as per CE's request. The format of this payload is still TBD. 5.2.3. Configure Logic Components The CE might have received a command (either through CLI or through SNMP etc) to setup some tunnels or path or some configuration. This may sometimes involve configuring more than one FE. That is, this packet may come through one line card and may leave through another line card. In this situation, CE has to configure both the FE logical components. If one of the logical components fails, it has to perform rollback operation and issues a command failure notification to the management station or CLI or SNMP etc. This operation is called command bundling; to perform this CE sends series of command to each FE with command bundling bit set. Each FE after receiving the command will have to save the current configuration and should check whether it can program the requested configuration. The status message is sent back to the CE; once FE receives all the status messages, it can then send an execute command with same transaction sequence number, and FE can now switch to the new configuration. This operation will get too complicated when more than one CE try to generate different configurations on the logical components; If FE tries to do parallel activities, it may get into a deadlock situation. It is up to the FE to react with appropriate status messages. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | C-Operation| Configuration Command +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ C-Operation: FEs and CEs may engage in two-phase commit o operation. This field provides the stage of such transaction. 0x00 - Command is single operation no need to generate a status message to FE 0x01 - This command is a two-phase operation, FE needs to save the previous state if rollback operation may be performed later by FE. 0x10 - Rollback to the previous state. Gopal,Audu Expires August 2002 [Page 19] Internet Draft Forwarding and Control Element protocol June 2002 0x11 - Execute and complete the command. During this operation the TSN value is same and used to identify the transaction. The same CE should generate this TSN otherwise, FE will treat this as different query from other CE. Configuration command: This field is a list of attributes which are to be set on a logical component(function) managed by a FE. The formats of the operation and configuration command are yet TBD. 5.2.4. Configure Logic Components Acknowledgement This is sent by the FE to CE to acknowledge Logic Components configuration as requested by CE. The format of this payload is still TBD. 5.2.5. Topology Request CE wants to know how each FE is connected or configured during the pre-association phase. This may be used by the CE to coordinate their activities for high availability, health check etc. 5.2.6. Topology Response FE manages the logical components and it requires some minimal information to communicate to CE, during pre-association phase the FE-Manager configure the FE capabilities this includes: (1) List of CEs with whom it can communicate and their IP address or MAC address etc. (2) May also designate one CE as primary and other have backup for asynchronous notification of events. (3) May also configures other parameters which are need for high availability operations All these are FE capabilities they are represented in the FE Model as OID or XML or any other suitable format in the FE table. FE looks into the CE-Table [FEM], which is part of FE object, and reports to the CE. The format of the payload is as follows: Gopal,Audu Expires August 2002 [Page 20] Internet Draft Forwarding and Control Element protocol June 2002 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Topology Information +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Topology Information: This is a sequence of CE list that are configured during pre-association phase. FE keeps this information in the CE-Table [4] model. This provides information about how each FE is connected (established) with other CEs. The topology information is not about the network interfaces, it provides information to CE how the FEs are configured and how they are communicating with the other CEs. The information can be represented using XML or OID or any other suitable data formats. The final format of the topology information is till TBD. 5.2.7.Port Status request - CE requesting a FE for physical configuration FE maintains the status of physical port and their configuration information. These model using logical components [4]. CE at any given point can request either one or more of the egress or ingress port status. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seq. of Port Number +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Port Status Request: Logical components can be referenced inside a FE can be referenced using OID or XML or any other format. 5.2.8. Port Status response - FE exporting the physical configuration to CE. Gopal,Audu Expires August 2002 [Page 21] Internet Draft Forwarding and Control Element protocol June 2002 FE reports the requested port status by picking up the port status value from the respective logical component (say ingress or egress port). There is another message that is asynchronous notification of change in port status.See Event Monitoring section [sec 5.5.5]. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seq. of Port and Status +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Sequence of Port number and Status: This field carries information about port and their status. The information can be conveyed in any suitable data formats for example OID, XML or any other representation. The choice of data representation should be supported by both CEs an FEs. The selection of such representation is done during the pre-association phase and it is implementation or operation dependent. 5.3. PE Maintenance Messages 5.3.1. Protocol Element Active The ACT message is sent by a PE to indicate to its master or slave that it is Active and ready to be used. The ACT message contains the following parameters Traffic Mode Type (Mandatory) INFO String (Optional) Gopal,Audu Expires August 2002 [Page 22] Internet Draft Forwarding and Control Element protocol June 2002 The format for the ACT message using integer formatted Association Identifiers is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Tag (0xb) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Traffic Mode Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Tag (0x4) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | INFO String* | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The Traffic Mode Type parameter identifies the traffic mode of operation of the PE within an NE. The valid values for Type are shown in the following table: Value Description 0x1 Over-ride-strong-consistency 0x2 Over-ride-weak-consistency 0x3 Load-share Within a particular Association Identifier, only one Traffic Mode Type can be used. The Over-ride value indicates that the PE is operating in Over-ride mode, where the PE takes over all traffic in an NE i.e., primary/back-up operation), over-riding any currently active PEs in the NE. In Load-share mode, the PE will share in the traffic distribution with any other currently active PEs. The format and description of the optional Info String parameter is the same as for the PE Up message (See Section 5.2.1.). 5.3.2. Protocol Element Active Acknowledgement The ACT Acknowledgement message is used to acknowledge a PE-Active message received from a remote PE master or slave peer. The ACT Acknowledgement message contains the following parameters: Traffic Mode Type (Mandatory) INFO String (Optional) The format for the ACT Acknowledgement message is the same as in PE Active Message (See 5.2.5) Gopal,Audu Expires August 2002 [Page 23] Internet Draft Forwarding and Control Element protocol June 2002 5.3.3. Protocol Element Inactive The INACT message is sent by a PE to indicate to its master/slave that it is no longer an active PE to be used from within a list of PEs. The receiver will respond with a INACT Ack message and either discard incoming messages meant for the outgoing PE, or buffer them for a timed period and then discard (if no other PEs become active). The INACT message contains the following parameters Traffic Mode Type (Mandatory) INFO String (Optional) The format for the PE Inactive message parameters is as shown for PE Active Message(See 5.2.5) 5.3.4. Protocol Element Inactive Acknowledgement The PE Inactive (INACT) Acknowledgement message is used to acknowledge a PE Inactive message received from a remote master or slave PE peer. The INACT Acknowledgement message contains the following parameters: Traffic Mode Type (Mandatory) INFO String (Optional) The format for the PE Inactive Acknowledgement message parameters is as shown for PE Active Message (See 5.2.5) 5.3.5. Heartbeat (or Health check) CE periodically polls each FEs to ensure that they are operational. A CE generates this message. There may be more than one CE in a CE- set; to avoid duplicate requests, one of the ACTIVE CEs in the CE set can generate this request. Then it is up to the CEs to update the status of the FE. Note: - By default, the ACTIVE CE should initiate the health check. An optional Heartbeat Data parameter may be sent in the heart beat message. Its contents are defined by the sending node. The Heartbeat Data could include, for example, a Heartbeat Sequence Number and, or Timestamp. The receiver of a Heartbeat message does not process this field as it is only of significance to the sender. The receiver MUST respond with a Heartbeat Acknowledgement message. Gopal,Audu Expires August 2002 [Page 24] Internet Draft Forwarding and Control Element protocol June 2002 5.3.6. Heartbeat Acknowledgement (FE response to heart beat request) After verifying the CE-Tag FE simply echo's the original heartbeat message. 5.3.7.Status Change and Event Notify message CE or FE may generate this status message to inform one of the following events. This list is not exhaustive (to be added later) Status Code Description ----------------------------------------------------------------- 0x100 FE sending a command failure notification to CE 0x101 FE information failure to communicate to CE 0x102 FE receiving request from another CE set 0x103 FE is in two-phase command lock 0x104 FE could not complete the 2-phase command 0x400 success 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Status Code | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Status code: Describing the result of status of transaction 5.4. PE Traffic Maintenance (PETM) Messages 5.4.1. Control packet Redirect to CE for processing Router receives both control and data packets through a physical port, the following action may occur: (a) Forwarding blade receives IP packet that is not destined for it, these packets are forwarded by the forwarding plane component. (b) Forwarding blade receives IP packet that is destined for it. These packets are not forwarded to the Control plane rather they are processed by the forwarding plane control logic (stack in the forwarding plane). Example of such packet is, ping request is a best example. (c) Forwarding blade received IP packets that may be routing protocol packets or which cannot be processed by the stack in the line card, these packets have to be forwarded to the control plane by the FE. FE encapsulates the control packet along with logical component information refer FE Modeling [4] in the FACT header and forwards it to CE. FE may be pre-configured during the pre-association phase or during the Join response message Gopal,Audu Expires August 2002 [Page 25] Internet Draft Forwarding and Control Element protocol June 2002 from the FE. For example, consider a scenario where each CE is running one routing protocol say OSPF, BGP, IS-IS. When a FE receives routing protocol packet, FE encapsulates that packet inside the Forces protocol and forward it to the respective CEÆs. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Control Packet +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Control Packet: The control packet that the network element received through a particular IP interface. 5.4.2. Control Packet Redirect Acknowledgement This may be used by the CE to acknowledge the Control Packet Redirect message received from the source FE. The format of this message is still TBD. 5.4.3. Control Packet Forwarding through a specific output port CE may request a packet and wants FE to forward that packet through a particular egress port. Example of such packets are routing protocol updates etc. Before generating such request, CE has to know the FE's logical components and the list of available port and the configuration status. (reference snapshot of Logical components ) 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet to be forwarded +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Packet to be forwarded: This field has two other sub fields namely the egress port through which the packet has to be forwarded, the actual packet. FE after receiving has to parse this packet and extracts the egress port and details, then extract the packet which has to be forwarded as it is to the through the egress port. Gopal,Audu Expires August 2002 [Page 26] Internet Draft Forwarding and Control Element protocol June 2002 The data representation of this field can be either OID or XML or any other suitable data format. Type of data format should be supported by both FE and CE and is part of pre-association phase. The exact format of this field is yet TBD. 5.4.4. Control Packet Forwarding Acknowledgement This is used by the FE to acknowledge the Control Packet Forwarding Request initiated by the CE. The exact format of this message is TBD. 5.5. Event Notification Messages 5.5.1. Event Register This is sent by the CE to the FE to request that FE notify the CE when the indicated events occur on the FE. The format of the payload is still TBD at this time. 5.5.2. Event Register Acknowledgement This is used by the FE to acknowledge CEÆs event registration request. The format of this payload is still TBD at this time. 5.5.3. Event De-Register This is sent by the CE to the FE to indicate that it no longer is interested in receiving notifies for the events indicated in message. The format of this payload is still TBD. 5.5.4. Event De-Register Acknowledgement The FE sends this message to the CE to acknowledge CEÆs request not get any more notifies for events indicated in message. The format for this payload is still TBD. 5.5.5. Status change in configuration (Asynchronous event) FE can be configured to report the activities that are done through either CLI or SNMP or any other mechanism. This is optional and may be implementation dependent. Gopal,Audu Expires August 2002 [Page 27] Internet Draft Forwarding and Control Element protocol June 2002 Either FE can provide the list of OID that are changed or it can simply redirect the copy of the information that it received from SNMP agent or CLI. How FE receives, this information is beyond the scope of this protocol. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Configuration Changes +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Configuration change: This field contains the list of command that was issued either through CLI or through any other interfaces. The data representation of the field may be OID or XML or any other suitable format supported by FE and CE. The choice of data format is implementation and operation dependent. 5.5.6. FE notifying the change in port status to CE ((Asynchronous event) FE may report the change in port status to CE; the only difference between configuration change and this port message is that, FE generates this message upon detecting a failure of port functions. It is the responsibility of the FE to keep track of the port status and it is implementation dependent. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence of Port No and status +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Sequence of Port number and Status: This field represents a sequence of one or more port status and is generated by FE to CE(s). The information can be conveyed in any suitable data formats for example OID, XML or any other representation. The choice of data representation should be supported by both CEs an FEÆs. The selection of such representation is Gopal,Audu Expires August 2002 [Page 28] Internet Draft Forwarding and Control Element protocol June 2002 done during the pre-association phase and it is implementation or operation dependent. 5.5.7.Notification of CE status (or association change to FEs) CE in CE-set may be added or deleted from CE-set or shutdown for some maintenance reasons. Other CEs in the CE set may either get notification from CE-Manager (if there is a change in configuration) or by CE-CE communication protocol. This is beyond the scope of this protocol. In all the above cases, the CEs in the CE-set (one of the CE) may need to inform such configuration change to all the affected FEs. This message describes the changes in pre-association configuration information. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | CE Configuration Change +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ CE Configuration Change: This field describes the CE information and change in status of the CE. If one of the CE is configured as Designated CE and if due to some reason the backup CE detects that the designated CE is not active or failed, the backup CE ca generate such information and inform about the status of the designated CE. How CE-to-CE communication is performed is beyond the scope of this protocol. This field can be represented using any suitable data format and the formats have to be agreed during the pre-association phase. 5.5.8.Dynamic configuration of FEs FE can be cascaded and made to form some mesh. That is, one of the FEs' logical function can be connected to other FE's logical function. This can be configured dynamically. << Details to TBW >> Gopal,Audu Expires August 2002 [Page 29] Internet Draft Forwarding and Control Element protocol June 2002 6. Architecture support for FACT protocol Pre-association phase is used to configure certain key attributes. FE-Manager and CE-Manager are responsible for providing that information. 6.1.Security If FE or CE is going to communicate over the untrust domain, then it is the responsibility of the administrator to choose appropriate lower layer security protocols. For example if the FE and CE are communicating in a shared LAN or Ethernet etc, then layer 2 encryption is used. If it is going be above IP layer either TLS or IPSec can be used. FACT protocol is just a payload for those protocols and does not carry any explicitly fields for authentication or encryption. 6.2. High availability support Fail-over, load sharing etc, are part of high-availability treatment. Since CE-CE communication is out of scope (but FACT protocol can be applied there also). FE-Manager can configure FE to perform one of the following functions - Strong consistency: Send all asynchronous notification to the entire CE in the CE set. - Weak consistency: Send all the asynchronous notification events only to one CE in the CE set, that CE may be acting as primary CE, when that CE is not active use the next one in the list or a backup CE. This type of requirement does not affect the protocol; the protocol has provision and bit fields to communicate the FE capabilities to CE during the join message request. 6.3. Access Control to FE It is up to the implementer to configure the number of CE that can access FE, as such protocol is flexible and has 16-bit fields to identify a CE. Gopal,Audu Expires August 2002 [Page 30] Internet Draft Forwarding and Control Element protocol June 2002 6.4.Configurable parameters The following are the currently identified configurable parameters that can be done through FE-Manager and CE-Manager for FE and CEÆs respectively (1) Load Sharing (optional) (2) Fail over configuration (optional) (3) Load Balancing (optional) (4) Security Keys (optional) (5) Number of CE to which it has to communicate (6) Maximum number of CE (7) Whether FE should notify to CEÆs regarding the change in configuration (which happened via SNMP or CLI) (8) Timer for health check (9) Data format support. If FE and CE can support different data formats say OID, XML etc. Then this has to established during this phase. 6.5.Management Interface to FE and CE through FE-Manager and CE- Manager. Previous section describes the FE and CE minimal configuration and information that are to be supported as part of pre-association phase. During the normal operation, it may be required to know the association and dynamic change of such attributed and capability configuration of FE and CE. This can be done either through CLI or SNMP or any other suitable mechanism. This information can be modeled as MIB for Forces protocol and their control elements. The following are the key attributes that are needed to support such operation. (1) Add new CE endpoint to the FE (2) Add new FE endpoint to the CE (3) (Re)Establish an association with CE or FE (4) Leave (Terminate) an association with CE or FE (5) Update the capabilities of FE or CE 7. References 1. S. Bradner, "The Internet Standards Process -Revision 3", RFC 2026, October 1996. 2. S. Bradner, "Keywords for use in RFCs to Indicate Requirement Levels", RFC2119 (BCP), IETF, March 1997. Gopal,Audu Expires August 2002 [Page 31] Internet Draft Forwarding and Control Element protocol June 2002 3. Anderson, et. al., "Requirements for Separation of IP Control and Forwarding", work in progress, February 2002,,IETF. 4. Ram Gopal, "Forwarding Element Modelling",work in progress, February 2002, , IETF 5. L. Yang, et. al, " ForCES Architectural Framework",work in progress", June 2002, 6. L. Yang, et. al, " ForCES Forwarding Element Functional Model", work in progress", June 2002,< draft-yang-forces-model-00.txt> 8. Acknowledgments I would like to thank Man Li, Nokia Research Center for her suggestions and comments. 9. Authors' Addresses Ram Gopal Nokia Research Center 5, Wayside Road, Burlington, MA 01803 Phone: 1-781-993-3685 Email: ram.gopal@nokia.com Alex Audu Alcatel R&I 1000 Coit Road Plano, TX 75075 Phone: 1-972-477-7809 Email: alex.audu@alcatel.com Gopal,Audu Expires August 2002 [Page 32]