Internet DRAFT - draft-anderson-forces-arch

draft-anderson-forces-arch





        Internet Draft                                 L. Yang 
        Expiration: November 2002                            Intel Labs 
        File: draft-anderson-forces-arch-01.txt        R. Dantu 
        Working Group: ForCES                               Netrake Corp.   
                                                       T. Anderson 
                                                            Intel Labs   
                                                       May 2002                      
         
         
                          ForCES Architectural Framework 
         
         
         
                        draft-anderson-forces-arch-01.txt 
         
         
         
         
        Status of this Memo 
         
        This document is an Internet-Draft and is in full conformance with 
        all provisions of Section 10 of RFC2026.  Internet-Drafts are 
        working documents of the Internet Engineering Task Force (IETF), its 
        areas, and its working groups.  Note that other groups may also 
        distribute working documents as Internet-Drafts. 
         
        Internet-Drafts are draft documents valid for a maximum of six 
        months and may be updated, replaced, or obsoleted by other documents 
        at any time.  It is inappropriate to use Internet-Drafts as 
        reference material or to cite them other than as ``work in 
        progress.'' 
         
        The list of current Internet-Drafts can be accessed at 
        http://www.ietf.org/ietf/1id-abstracts.txt. 
         
        The list of Internet-Draft Shadow Directories can be accessed at  
        http://www.ietf.org/shadow.html. 
         
     Conventions used in this document  
             
        The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",  
        "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in 
        this document are to be interpreted as described in [RFC-2119]. 
         
     1. Abstract 
      
        This document defines the architectural framework for ForCES network 
        elements (NE), and identifies the associated entities and the 
        interaction among them.  This framework is intended to satisfy the 
        requirements specified in the ForCES requirements draft [FORCES-
        REQ]. 
         
     2. Definitions 
      
      
     Internet Draft         ForCES Framework                  May 2002 
      
      
         
         
        A set of terminology associated with the ForCES requirements is 
        defined in [FORCES-REQ] and we only include the ones that are most 
        relevant to this document here.  
         
        Forwarding Element (FE) - A logical entity that implements the 
        ForCES protocol.  FEs use the underlying hardware to provide per-
        packet processing and handling as directed by a CE via the ForCES 
        protocol.  
         
        Control Element (CE) - A logical entity that implements the ForCES 
        protocol and uses it to instruct one or more FEs how to process 
        packets.  CEs handle functionality such as the execution of control  
        and signaling protocols.   
             
        ForCES Network Element (NE) - An entity composed of one or more CEs 
        and one or more FEs.  To entities outside a NE, the NE represents a 
        single point of management.  Similarly, a NE usually hides its 
        internal organization from external entities.  
             
        Pre-association Phase - The period of time during which a FE Manager 
        (see below) and a CE Manager (see below) are determining which FE 
        and CE should be part of the same network element. Any partitioning 
        of PFEs and PCEs occurs during this phase.  
             
        Post-association Phase - The period of time during which a FE does 
        know which CE is to control it and vice versa, including the time 
        during which the CE and FE are establishing communication with one 
        another.  
             
        ForCES Protocol - While there may be multiple protocols used within 
        the overall ForCES architecture, the term "ForCES protocol" refers 
        only to the ForCES post-association phase protocol (see below).  
             
        ForCES Post-Association Phase Protocol - The protocol used for post-
        association phase communication between CEs and FEs.  This protocol 
        does not apply to CE-to-CE communication, FE-to-FE communication, or 
        to communication between FE and CE managers.  The ForCES protocol is 
        a master-slave protocol in which FEs are slaves and CEs are masters.  
        This protocol includes both the management of the communication 
        channel (e.g., connection establishment, heartbeats) and the control 
        messages themselves. This protocol could be a single protocol or 
        could consist of multiple protocols working together.  
             
        FE Manager - A logical entity that operates in the pre-association 
        phase and is responsible for determining to which CE(s) a FE should 
        communicate.  This process is called CE discovery and may involve 
        the FE manager learning the capabilities of available CEs.  A FE 
        manager may use anything from a static configuration to a pre-
        association phase protocol (see below) to determine which CE(s) to 
        use.  Being a logical entity, a FE manager might be physically 
      
     Anderson, et. al.      Expires November 2002                  [Page 2] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
        combined with any of the other logical entities mentioned in this 
        section.  
             
        CE Manager - A logical entity that operates in the pre-association  
        phase and is responsible for determining to which FE(s) a CE should    
        communicate.  This process is called FE discovery and may involve 
        the CE manager learning the capabilities of available FEs.  A CE 
        manager may use anything from a static configuration to a pre-
        association phase protocol (see below) to determine which FE to use.  
        Being a logical entity, a CE manager might be physically combined 
        with any of the other logical entities mentioned in this section.  
             
        Pre-association Phase Protocol - A protocol between FE managers and 
        CE managers that is used to determine which CEs or FEs to use.  A 
        pre-association phase protocol may include a CE and/or FE capability 
        discovery mechanism.  Note that this capability discovery process is 
        wholly separate from (and does not replace) that used within the 
        ForCES protocol.  However, the two capability discovery mechanisms 
        may utilize the same FE model.   
             
        FE Model - A model that describes the logical processing functions 
        of a FE.  
             
        ForCES Protocol Element - A FE or CE.  
             
     3. Introduction to Forwarding and Control Element Separation (ForCES)  
      
        An IP network element (NE) appears to external entities as a 
        monolithic piece of network equipment, e.g., a router, NAT, 
        firewall, or load balancer. Internally, however, an IP network 
        element (NE) (such as a router or switch) is composed of numerous 
        logically separated entities that cooperate to provide a given 
        functionality (such as routing or IP switching).  Two types of 
        network element components exist: control element (CE) in control 
        plane and forwarding element (FE) in forwarding plane (or data 
        plane).  Forwarding elements typically are ASIC, network-processor, 
        or general-purpose processor-based devices that handle data path 
        operations for each packet.  Control elements are typically based on 
        general-purpose processors that provide control functionality like 
        routing and signaling protocols.   
         
        ForCES aims to define a framework and associated protocol(s) to 
        standardize the exchange of information between the control plane 
        and the forwarding plane. Having standard mechanisms between the CEs 
        and FEs allow these components to be physically separated. This 
        physical separation accrues several benefits to the ForCES 
        architecture. Separate components would allow component vendors to 
        specialize in one component without having to become experts in all 
        components. It also allows CEs and FEs from different component 
        vendors to interoperate with each other and hence it becomes 
        possible for system vendors to integrate together CEs and FEs from 
        different component vendors. This translates into a lot more design 
        choices and flexibility to the system vendors. Overall, ForCES will 
      
     Anderson, et. al.      Expires November 2002                  [Page 3] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
        enable rapid innovation in both the control and forwarding planes 
        while maintaining interoperability. Scalability is also easily 
        provided by this architecture in that additional forwarding or 
        control capacity can be added to existing network elements without 
        the need for forklift upgrades. 
         
        One example of such physical separation is at the blade level. 
        Figure 1 shows an example configuration of a router, with two 
        control blades and multiple router (forwarding) blades, all 
        interconnected into a switch fabric backplane. In such chassis 
        configuration, the control blades are the CEs while the router 
        blades are FEs, and the switch fabric backplane provides the 
        physical interconnect for all the blades. Routers today with this 
        kind of configuration use proprietary interface for messaging 
        between CEs and FEs. The goal of ForCES is to replace such 
        proprietary interface with a standard protocol. With a standard 
        protocol like ForCES implemented on all blades, it becomes possible 
        for control blades from vendor X and routing blades from vendor Y to 
        work seamlessly together in one chassis.  
         
         
             -------------------------       ------------------------- 
             |  Control Blade A      |       |  Control Blade B      | 
             |       (CE)            |       |          (CE)         | 
             -------------------------       ------------------------- 
                     ^   |                           ^    | 
                     |   |                           |    | 
                     |   V                           |    V 
             --------------------------------------------------------- 
             |               Switch Fabric Backplane                 | 
             --------------------------------------------------------- 
                    ^  |            ^  |                   ^  |         
                    |  |            |  |                  |  |         
                    |  V            |  V                   |  V         
                ------------    ------------           ------------     
                |Router    |    |Router    |           |Router    | 
                |Blade #1  |    |Blade #2  |           |Blade #N  | 
                |   (FE)   |    |   (FE)   |           |   (FE)   | 
                ------------    ------------           ------------     
                    ^  |            ^  |                   ^  |         
                    |  |            |  |                  |  |         
                    |  V            |  V                   |  V         
              
             Figure 1. A router configuration example with separate blades. 
      
        Another level of physical separation between the CEs and FEs can be 
        at the box level. In such configuration, all the CEs and FEs are 
        physically separated boxes, all interconnected together over high 
        speed LAN connection (e.g., Gigabit Ethernet). These separated CEs 
        and FEs are only one hop away from each other within a local area 
        network. The CEs and FEs communicate to each other by running 
        ForCES, and the collection of these CEs and FEs together become one 
        routing unit to the external world. 
      
     Anderson, et. al.      Expires November 2002                  [Page 4] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
      
             -------         ------- 
             | CE1 |         | CE2 | 
             -------         ------- 
                ^               ^ 
                |               | 
                V               V 
         ============================================ Ethernet 
             ^       ^              ^ 
             |       |               | 
             V       V               V 
          -------  -------         -------- 
          | FE#1|  | FE#2|         | FE#n | 
          -------  -------         -------- 
            ^  |     ^  |            ^  |         
            |  |     |  |            |  |         
            |  V     |  V            |  V         
              
             Figure 2. A router configuration example with separate boxes. 
         
      
             ------------------------------------------------- 
             |       |       |       |       |       |       | 
             |OSPF   |RIP    |BGP    |CAC    |LDP    |      | 
             |       |       |       |       |       |       | 
             ------------------------------------------------- 
             |               ForCES Interface                | 
             ------------------------------------------------- 
                                     ^   ^                    
                             ForCES  |   |data        
                             control |   |packets     
                             messages|   |(e.g., routing packets)     
                                     v   v 
             ------------------------------------------------- 
             |               ForCES Interface                | 
             ------------------------------------------------- 
             |       |       |       |       |       |       | 
             |LPM Fwd|Meter  |Shaper |NAT    |Classi-|      | 
             |       |       |       |       |fier   |       | 
             ------------------------------------------------- 
             |               FE resources                    | 
             ------------------------------------------------- 
         
                     Figure 3. Examples of CE and FE functions 
         
        Some examples of control functions that can be implemented in the CE 
        include routing protocols like RIP, OSPF and BGP, control and 
        signaling protocols like CAC (Call Admission Control), LDP (Label 
        Distribution Protocol) for MPLS, etc. Examples of forwarding 
        functions in FE include LPM (longest prefix match) forwarder, 
        classifiers, traffic shaper, meter, NAT, etc. Figure 3 shows a 
        diagram with examples in both CE and FE. Any given NE may contain 
        one or many of these CE and FE functions in it. The diagram also 
      
     Anderson, et. al.      Expires November 2002                  [Page 5] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
        shows that ForCES protocol is used to transport both the control 
        messages for ForCES itself and the data packets that are 
        originated/destined from/to the control functions in CE (e.g.,
        routing packets). Section 5.2 provides more detail on this.  
         
        A set of requirements for control and forwarding separation is 
        identified in [FORCES-REQ]. This document describes a ForCES 
        architecture that satisfies the architectural requirements of that 
        document and defines a framework for ForCES network elements and 
        associated entities to facilitate protocol definition. 
         
     4. Architecture 
      
        This section defines the ForCES architectural framework and 
        associated logical components.  This ForCES framework defines 
        components of ForCES NEs and also includes several ancillary 
        components.  These components can be connected in different kinds of 
        topologies for flexible packet processing.  
         
                                --------------------------------------- 
                                | ForCES Network Element              | 
         --------------   Fc    | --------------      --------------  | 
         | CE Manager |---------+-|     CE 1   |------|    CE 2    |  | 
         --------------         | |            |  Fr  |            |  | 
               |                | --------------      --------------  | 
               | Fl             |         |  |    Fp       /          | 
               |                |       Fp|  |----------| /           | 
               |                |         |             |/            | 
               |                |         |             |             | 
               |                |         |     Fp     /|----|        | 
               |                |         |  /--------/      |        | 
         --------------     Ff  | --------------      --------------  | 
         | FE Manager |---------+-|     FE 1   |  Fi  |     FE 2   |  | 
         --------------         | |            |------|            |  |               
                                | --------------      --------------  | 
                                |   |  |  |  |          |  |  |  |    | 
                                ----+--+--+--+----------+--+--+--+----- 
                                    |  |  |  |          |  |  |  | 
                                    |  |  |  |          |  |  |  | 
                                      Fi/f                   Fi/f 
         
                     Figure 4. ForCES Architectural Diagram 
         
        The diagram in Figure 4 shows the logical components of the ForCES 
        architecture and their relationships.  There are two kinds of 
        components inside a ForCES network element: control element (CE) and 
        forwarding element (FE). The framework allows multiple instances of 
        CE and FE inside one NE. Each FE contains one or more physical media 
        interfaces for receiving and transmitting packets from/to the 
        external world. The aggregation of these FE interfaces becomes the 
        NEs external interfaces. In addition to the external interfaces, 
        there must also exist some kind of interconnect within the NE so 
        that the CE and FE can communicate with each other, and one FE can 
      
     Anderson, et. al.      Expires November 2002                  [Page 6] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
        forward packets to another FE. The diagram also shows two entities 
        outside of the ForCES NE: CE Manager and FE Manager. These two 
        entities provide configuration to the corresponding CE or FE in the 
        pre-association phase (see Section 5.1). There is no defined role 
        for FE Manager and CE Manager in post-association phase, thus these 
        logical components are not considered part of the ForCES NE.  
         
        For convenience, the logical interactions between these components 
        are labeled by reference points Fp, Fc, Ff, Fr, Fl, and Fi. The FE 
        interface is labeled as Fi/f. More detail is provided in Section 4 
        and 5 for each of these reference points. It is important to 
        understand all these reference points from the architecture point of 
        view, however, the ForCES protocol is only defined for one reference 
        point (Fp).  
         
        The interface between two ForCES NEs is identical to the interface 
        between two conventional routers and these two NEs exchange the 
        protocol packets through the interfaces at Fi/f. ForCES NEs connect 
        to existing routers transparently. 
      
     4.1. Control Elements and Fr Reference Point 
         
        It is not necessary to define any protocols across the Fr reference 
        point to enable control and forwarding separation for simple 
        configurations like single CE and multiple FEs. However, this 
        architecture permits multiple CEs to be present in a network 
        element. In cases where an implementation uses multiple CEs, it is 
        expected the invariant that the CEs and FEs together appear as a 
        single NE MUST be maintained. 
         
        Multiple CEs may be used for redundancy, load sharing, distributed 
        control, or other purposes.  Redundancy is the case where one or 
        more CEs are prepared to take over should an active CE fail.  Load 
        sharing is the case where two or more CEs are concurrently active 
        and where any request that can be serviced by one of the CEs can 
        also be serviced by any of the other CEs.  In both redundancy and 
        load sharing, the CEs involved are equivalently capable.  The only 
        difference between these two cases is in terms of how many active 
        CEs there are.  Distributed control is the case where two or more 
        CEs are concurrently active but where certain requests can only be 
        serviced by certain CEs. 
      
        When multiple CEs are employed in a ForCES NE, their internal 
        organization is considered an implementation issue that is beyond 
        the scope of the ForCES effort. CEs are wholly responsible for 
        coordinating amongst themselves via the Fr reference point to 
        provide consistency and synchronization. However, ForCES does not 
        define the implementation or protocols used between CEs, nor does it 
        define how to distribute functionality among CEs. Nevertheless, 
        ForCES will support mechanisms for CE redundancy or fail over, and 
        it is expected that vendors will provide redundancy or fail over 
        solutions within this framework. 
      
      
     Anderson, et. al.      Expires November 2002                  [Page 7] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
      
     4.2. Forwarding Elements and Fi reference point 
        
        FEs perform per-packet processing and handling as directed by CEs. 
        FEs have no initiative of their own.  Instead, FEs are slaves and 
        only do as they are told.  FEs may communicate with one or more CEs 
        concurrently across reference point Fp.  FEs have no notion of CE 
        redundancy, load sharing, or distributed control.  Instead, FEs 
        accept commands from any CE authorized to control them, and it is up 
        to the CEs to coordinate among themselves to achieve redundancy, 
        load sharing or distributed control. The idea is to keep FEs as 
        simple and dumb as possible so that FEs can focus its resource on 
        the packet processing functions.  
         
         
                     -------   Fr  ------- 
                     | CE1 | ------| CE2 | 
                     -------       ------- 
                       |   \      /   | 
                       |    \    /    | 
                       |     \  /     | 
                       |      \/Fp    | 
                       |      /\      | 
                       |     /  \     | 
                       |    /    \    | 
                     -------       ------- 
                     | FE1 |<----->| FE2 | 
                     -------       ------- 
      
                         Figure 5. CE redundancy example. 
         
        For example, in Figure 5, FE1 and FE2 can be configured to accept 
        commands from both the primary CE (CE1) and the backup CE (CE2). At 
        the beginning, CE1 issues commands to FEs while CE2 silently remains 
        in sync with CE1 via CE to CE protocol over Fr reference point. When 
        CE1 fails, CE2 detects it and starts to take over and issue commands 
        to the FEs.  
         
        Distributed control can be achieved in the similar fashion, without 
        much intelligence on the part of FEs. For example, FEs can be 
        configured to detect RSVP and BGP protocol packets, and forward RSVP 
        packets to one CE and BGP packets to another CE. Hence, FEs may need 
        to do packet filtering for forwarding packets to specific CEs. 
      
        This architecture permits multiple FEs to be present in a NE. When 
        multiple FEs are present, ForCES requires that packets MUST be able 
        to arrive at the NE by one FE and leave the NE via a different FE 
        (See [FORCES-REQ], Section 5, Requirement #3).  Packets that enter 
        the NE via one FE and leave the NE via a different FE are 
        transferred between FEs across the Fi reference point.  The Fi 
        reference point is a separate protocol from the Fp reference point 
        and is not currently defined by the ForCES architecture. 
         
      
     Anderson, et. al.      Expires November 2002                  [Page 8] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
        [FORCES-REQ] dictates that the ForCES protocol MUST be able to scale 
        to at least hundreds of FEs (see [FORCES-REQ] Section 5, requirement 
        #11). Each of these FEs may potentially have a different set of 
        capabilities, with different media interfaces.  FEs express their 
        capabilities to CEs using the ForCES FE model (see [FORCES-REQ] 
        Section 6). FEs are responsible for establishing and maintaining 
        layer-2 connectivity with other FEs and with entities external to 
        the NE.  Thus, FEs are also responsible for any signaling required 
        at layer-2. 
         
        A packet processing operation may need multiple FE capabilities. 
        These operations may be pipelined or parallel. In order to 
        accomplish full capabilities, one FE may feedback to another FE 
        after partial processing. FEs are connected in different kinds of 
        topologies and packet processing may spread across several FEs in 
        the topology. Hence, logical packet flow may be different from 
        physical FE topology. It is possible that one FE communicate to 
        another FE via CE, but it is also possible that direct communication 
        exists between FEs without CE in the way to allow better performance 
        and scalability. Figure 6 provides some topology examples, but it is 
        not a complete list of all possible topologies. ForCES protocol does 
        not make any assumption on the internal topology among FEs, as long 
        as each FE can communicate to its CE and each FE can forward packets 
        to another FE. Since the topology can vary a lot, it is also 
        important for FEs to convey the topology information to their CEs as 
        part of the ForCES protocol. Such topology information can help CEs 
        understand how to forward packets between FEs most efficiently.  
                             -----------             
                             |   CE    |       
                             -----------         
                              ^       ^ 
                              |       | 
                              v       v 
                          -------   ------- 
                          | FE1 |   | FE2 | 
                          -------   ------- 
                           ^  |       ^  | 
                           |  |       |  | 
                           |  v       |  v 
                              
                     (a) Simple topology: no direct link between FE1 and FE2  
         
                             -----------             
                             |   CE    |       
                             -----------         
                              ^       ^ 
                              |       | 
                              v       v 
                          -------   ------- 
                          | FE1 |<->| FE2 | 
                          -------   ------- 
                           ^  |       ^  | 
                           |  |       |  | 
      
     Anderson, et. al.      Expires November 2002                  [Page 9] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
                           |  v       |  v 
         
                     (b) There exists direct link between FE1 and FE2 
         
                             -----------             
                             |   CE    |       
                             -----------         
                            ^ ^       ^ ^ 
                           /  |       |  \ 
                    /------   |       |   ------\ 
                    v         v       v          v 
                -------   -------   -------   ------- 
                | FE1 |<->| FE2 |<->| FE3 |<->| FE4 | 
                -------   -------   -------   ------- 
                  ^  |     ^  |       ^  |     ^  | 
                  |  |     |  |       |  |     |  | 
                  |  v     |  v       |  v     |  v 
         
                     (c) Multiple FEs in a daisy chain 
      
      
                                ^ | 
                                | v 
                             -----------             
                             |   FE1   |<-----------------------|      
                             -----------                        | 
                               ^    ^                           | 
                              /      \                          |  
                       | ^   /        \   ^ |                   V 
                       v |  v          v  | v                ---------- 
                     ---------        ---------              |        | 
                     | FE2   |        |  FE3  |<------------>|   CE   | 
                     ---------        ---------              |        | 
                         ^  ^          ^                     ---------- 
                         |   \        /                        ^  ^ 
                         |    \      /                         |  | 
                         |    v     v                          |  | 
                         |   -----------                       |  | 
                         |   |   FE4   |<----------------------|  |    
                         |   -----------                          | 
                         |      |  ^                              | 
                         |      v  |                              | 
                         |                                        | 
                         |----------------------------------------| 
         
                     (d) Multiple FEs connected by a ring 
         
                     Figure 6. Some examples of FE topology. 
          
     4.3. CE Managers 
         
        CE managers are responsible for determining which FEs a CE should 
        control.  It is legitimate for CE managers to be hard-coded with the 
      
     Anderson, et. al.      Expires November 2002                  [Page 10] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
        knowledge of with which FEs its CEs should communicate. A CE manager 
        may also be physically embedded into a CE and be implemented as a 
        simple keypad or other direct configuration mechanism on the CE.  
        Finally, CE managers may be physically and logically separate 
        entities that configure the CE with FE information via such 
        mechanisms as COPS-PR [RFC3084] or SNMP [RFC1157]. 
         
     4.4. FE Managers 
         
        FE managers are responsible for determining to which CE any 
        particular FE should initially communicate.  Like CE managers, no 
        restrictions are placed on how a FE manager decides to which CEs its 
        FEs should communicate, nor are restrictions placed on how FE 
        managers are implemented. 
         
     5. Operational Phases 
         
        Both FEs and CEs require some configuration in place before they can 
        start information exchange and function as a coherent network 
        element. Two operational phases are identified in this framework -- 
        pre-association and post-association.  
         
     5.1.Pre-association Phase 
      
        Pre-association phase is the period of time during which a FE 
        Manager and a CE Manager are determining which FE and CE should be 
        part of the same network element. The protocols used during this 
        phase may include all or some of the message exchange over Fl, Ff 
        and Fc reference points. However, all these may be optional and none 
        of this is within the scope of ForCES protocol. 
         
     5.1.1. Fl Reference Point 
         
        
             FE Manager      FE               CE Manager     CE 
              |              |                 |             | 
              |              |                 |             | 
              |(security exchange)             |             | 
             1|<------------------------------>|             | 
              |              |                 |             | 
              |(a list of CEs and their attributes)          | 
             2|<-------------------------------|             | 
              |              |                 |             | 
              |(a list of FEs and their attributes)          | 
             3|------------------------------->|             | 
              |              |                 |             | 
              |              |                 |             | 
              |<----------------Fl------------>|             | 
         
          Figure 7. An example of message exchange over Fl reference point 
         
        CE managers and FE managers may communicate across the Fl reference 
        point in the pre-association phase in order to determine which CEs 
      
     Anderson, et. al.      Expires November 2002                  [Page 11] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
        and FEs should communicate with each other.  Communication across 
        the Fl reference point is optional in this architecture.  No 
        requirements are placed on this reference point. 
      
        CE managers and FE managers may be operated by different entities.  
        The operator of the CE manager may not want to divulge, except to 
        specified FE managers, any characteristics of the CEs it manages.  
        Similarly, the operator of the FE manager may not want to divulge FE 
        characteristics, except to authorized entities.  As such, CE 
        managers and FE managers may need to authenticate one another.  
        Subsequent communication between CE managers and FE managers may 
        require other security functions such as privacy, non-repudiation, 
        freshness, and integrity. 
         
        Once the necessary security functions have been performed, the CE 
        and FE managers communicate to determine which CEs and FEs should 
        communicate with each other.  At the very minimum, the CE and FE 
        managers need to learn of the existence of available FEs and CEs 
        respectively.  This discovery process may or may not entail one or 
        both managers learning the capabilities of the discovered ForCES 
        protocol elements. Figure 7 shows an example of possible message 
        exchange between CE manager and FE manager over Fl reference point. 
         
     5.1.2. Ff Reference Point 
         
             FE Manager      FE               CE Manager     CE 
              |              |                 |             | 
              |              |                 |             | 
              |(security exchange)             |(security exchange) 
             1|<------------>|authentication  1|<----------->|authentication
              |              |                 |             | 
              |(FE ID, attributes)             |(CE ID, attributes) 
             2|<-------------|request         2|<------------|request 
              |              |                 |             | 
             3|------------->|response        3|------------>|response 
              |(corresponding CE ID)           |(corresponding FE ID) 
              |              |                 |             | 
              |              |                 |             | 
              |<-----Ff----->|                 |<-----Fc---->| 
         
                          Figure 8. Examples of message exchange 
                             over Ff and Fc reference points. 
                                              
        The Ff reference point is used to inform forwarding elements of the 
        association decisions made by FE managers in pre-association phase.  
        Only authorized entities may instruct a FE with respect to which CE 
        should control it.  Therefore, privacy, integrity, freshness, and 
        authentication are necessary between FE manager and FEs when the FE 
        manager is remote to the FE.  Once the appropriate security has been 
        established, FE manager instructs FEs across this reference point to 
        join a new NE or to disconnect from an existing NE. Figure 8 shows 
        example of message exchange over Ff reference point. 
         
      
     Anderson, et. al.      Expires November 2002                  [Page 12] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
        Note that when the FE manager function may be co-located with the FE 
        (such as by manual keypad entry of the CE IP address), in which case 
        this reference point is reduced to a built-in function. 
         
     5.1.3. Fc Reference Point 
         
        The Fc reference point is used to inform control elements of the 
        association decisions made by CE managers in pre-association phase.  
        When the CE manager is remote, only authorized entities may instruct 
        a CE to control certain FEs.  Privacy, integrity, freshness and 
        authentication are also required across this reference point in such 
        a configuration.  Once appropriate security has been established, 
        the CE manager instructs CEs as to which FEs they should control and 
        how they should control them. Figure 7 shows example of message 
        exchange over Fc reference point. 
      
        As with the FE manager and FEs, configurations are possible where 
        the CE manager and CE are co-located and no protocol is used for 
        this function. 
         
     5.2. Post-association Phase and Fp reference point 
         
        Post-association phase is the period of time during which a FE and 
        CE have been configured with information necessary to contact each 
        other and includes both communication establishment and steady-state 
        communication. The communication between CE and FE is performed 
        across the Fp ("p" meaning protocol) reference point. ForCES 
        protocol is exclusively used for all communication across the Fp 
        reference point.  
         
     5.2.1. Association Establishment 
         
        As an example, figure 9 shows some of the message exchange that need 
        to happen before the association between CE and FE is fully 
        established. Typically, FE would need to inform the CE of its own 
        capability and its topology in relation to other FEs. The capability 
        of FE is represented by FE model, described in another separate 
        document. The model would allow FE to describe what kind of packet 
        processing functions it contains, in what order these processing 
        happen, what kind of configurable parameters it allows, what 
        statistics it collects and what events it might throw, etc. Once 
        such information is available to CE, CE sends all the necessary 
        configuration to FE so that FE can start receiving and processing 
        packets. For example, CE might need to send a snapshot of the 
        current routing table to FE so that FE can start routing packets 
        correctly. Once FE starts accepting packets for processing, we say 
        the association of this FE with its CE is now established. From then 
        on, CE and FE enter steady-state communication as described in 
        5.2.2. 
         
         
         
         
      
     Anderson, et. al.      Expires November 2002                  [Page 13] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
                     FE                      CE 
                     |                       | 
                     |(Hello, are you there?)| 
                    1|<----------------------| 
                     |                       | 
                     |(Yes. let me join the NE please.)       
                    2|---------------------->| 
                     |                       | 
                     |(Security exchange.)   | 
                    3|<--------------------->| 
                     |                       | 
                     |(What kind of FE are you? -- capability query) 
                    4|<----------------------| 
                     |                       | 
                     |(Here is my capability: use model to describe) 
                    5|---------------------->| 
                     |                       | 
                     |(How are you connected with others? -- topology query) 
                    6|<----------------------| 
                     |                       | 
                     |(Here is the topology info) 
                    7|---------------------->| 
                     |                       | 
                     |(Config for FE initialization, e.g. routing table) 
                    8|<----------------------| 
                     |                       | 
                     |(I am ready to go. Shall I?)     
                    9|---------------------->| 
                     |                       | 
                     |(Go ahead!)            | 
                   10|<----------------------| 
                     |                       | 
                      
               Figure 9. Example of message exchange between CE and FE 
                         over Fp to establish NE association 
         
         
     5.2.2. Steady-state Communication 
      
        Once an association is established between the CE and the FE, the 
        ForCES protocol is used by the CE and the FE over Fp reference point 
        to exchange information to facilitate packet processing.  
         
                     FE                      CE 
                     |                       | 
                     |(Add these new routes.)| 
                    1|<----------------------| 
                     |                       | 
                     |(Successful.)          | 
                    2|---------------------->| 
                     |                       | 
                     |                       | 
                     |                       | 
      
     Anderson, et. al.      Expires November 2002                  [Page 14] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
                     |(Query some stats.)    | 
                    1|<----------------------| 
                     |                       | 
                     |(Reply with stats collected.)           
                    2|---------------------->| 
                     |                       | 
                     |                       | 
                     |                       | 
                     |(My port is down, with port #.)                     
                    1|---------------------->| 
                     |                       | 
                     |(Route to this port instead...)                 
                    2|<----------------------| 
                     |                       | 
                     |                       | 
         
              Figure 10. Examples of message exchange between CE and FE 
                      over Fp during steady-state communication 
         
        Based on the information acquired through CEs' control processing, 
        CEs will frequently need to manipulate the packet-forwarding 
        behaviors of their FE(s) by sending instructions to FEs. For 
        example, Figure 10 shows one such message exchange in which CE sends 
        new routes to FE so that FE can add them to its routing table. CE 
        can also query FE for statistics collected by FE and FE can also 
        notify CE of some important events, like interface up and down, etc. 
        Figure 8 also shows such examples. 
         
     5.2.3. Data Packets across Fp reference point 
         
        Control packets (such as RIP, OSPF messages) addressed to any of 
        NE's interfaces are typically redirected by the receiving FE to its 
        CE, and CE may originate packets and have its FE deliver them to 
        other NEs. Therefore, the communication across the Fp reference 
        point includes not only the control messages from CEs to FEs and the 
        status or statistics report from FEs to CEs, but also the data 
        packets that are redirected between them. Moreover, one FE may be 
        controlled by multiple CEs. In this configuration, the control 
        protocols supported by the FORCES NEs may spread across multiple 
        CEs. For example, one CE may support OSPF and another supports BGP. 
        FEs are configured to recognize these protocol packets and forward 
        them to the corresponding CE.  
         
        Figure 11 shows one example of how the OSPF packets originated by 
        router A are passed to router B. In this example, ForCES protocol is 
        used to transport the packets from CE to FE inside router A, and 
        then from FE to CE inside router B. In light of the fact that ForCES 
        protocol is responsible to transport both the control messages and 
        the data packets between CE and FE over Fp reference point, it is 
        possible to use either a single protocol or multiple protocols to 
        achieve that. 
         
         
      
     Anderson, et. al.      Expires November 2002                  [Page 15] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
             ---------------------           ---------------------- 
             |                   |           |                    | 
             |    +--------+     |           |     +--------+     | 
             |    |CE(OSPF)|     |           |     |CE(OSPF)|     | 
             |    +--------+     |           |     +--------+     | 
             |        |          |           |          ^         | 
             |        |Fp        |           |          |Fp       | 
             |        v          |           |          |         | 
             |    +--------+     |           |     +--------+     | 
             |    |  FE    |     |           |     |   FE   |     | 
             |    +--------+     |           |     +--------+     | 
             |        |          |           |          ^         | 
             | Router |          |           | Router   |         | 
             | A      |          |           | B        |         | 
             ---------+-----------           -----------+---------- 
                      v                                 ^ 
                      |                                 | 
                      |                                 | 
                      ------------------->---------------     
         
            Figure 11. Example to show data packet flow between two NEs. 
         
     5.2.4. Interconnect and Proximilty between CE and FE 
         
        CEs and FEs can be connected by a variety of interconnect 
        technologies, including Ethernet connections, backplanes, ATM (cell) 
        fabrics, etc. ForCES should be able to support each of these 
        interconnects (see [FORCES-REQ] Section 5, requirement #1). The 
        proximity between CEs and FEs can range from very close (same 
        box/one hop) to close (same room/small number of hops) (see [FORCES-
        APP]), but the initial focus for ForCES protocol would be on very 
        close proximity scenario. When CEs and FEs are separated beyond a 
        single hop, ForCES will make use of an existing RFC2914 compliant L4 
        protocol with adequate reliability, security and congestion control 
        (e.g. TCP, SCTP) for transport purposes. 
         
     5.2.5. Proxy FE 
         
        In the case where a physical FE cannot implement (e.g., due to the 
        lack of a general purpose CPU) the ForCES protocol directly, a proxy 
        FE can be used in the middle of Fp reference point. This allows the 
        CE communicate to the physical FE via the proxy by using ForCES, 
        while the proxy manipulates the physical FE using some intermediary 
        form of communication (e.g., a non-ForCES protocol or DMA). In such 
        an implementation, the combination of the proxy and the physical FE 
        becomes one logical FE entity.  
         
     5.3. Association Re-establishment 
         
        FEs and CEs may join and leave NEs dynamically (see [FORCES-REQ] 
        Section 5, requirements #12 and #13). When a FE or CE leaves the NE, 
        the association with the NE is broken. If the leaving party rejoins 
        a NE later, to re-establish the association, it may or may not need 
      
     Anderson, et. al.      Expires November 2002                  [Page 16] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
        to re-enter the pre-association phase. Loss of association can also 
        happen unexpectedly due to loss of connection between the CE and the 
        FE. Therefore, the framework allows the bi-directional transition 
        between these two phases, but the ForCES protocol is only applicable 
        for the post-association phase. However, the protocol should provide 
        mechanisms to support association re-establishment (see [FORCES-REQ] 
        Section 5, requirement #7). 
         
        Let's use the example in Figure 5 to see what happens when the 
        association is broken and later re-established again. Section 4.2 
        already explains what happens if CE1 fails and how CE2 can take 
        over. Note that if no CE redundancy is provided, FEs need to be told 
        at the association establishment time what to do in the case of CE 
        failure. FEs may be told to stop packet processing all together if 
        its CE fails. Or, FEs may be told to continue forwarding packets 
        even in the face of CE failure. No matter what, it needs to be part 
        of the configuration when the association is established.  
         
        Let's now look at the case when FE1 leaves the NE temporarily, 
        assuming CE1 is the working CE for the moment. FE1 may voluntarily 
        decides to leave the association. Or, it is more likely that FE1 
        stops functioning simply due to unexpected failure. In former case, 
        CE1 receives a "leave-association request" from FE1. In the latter, 
        CE1 detects the failure of FE1 by some other mean. In both cases, 
        CE1 would keep a note of such event for FE1 while continue 
        commanding FE2. When FE1 decides to rejoin again, or when it is back 
        up again from the failure, FE1 would need to re-discover its master 
        (CE). This can be achieved by several means. It may re-enter the 
        pre-association phase and get that information from its FE manager. 
        It may retrieve the previous CE information from its cache, if it 
        decides that the information is still valid. Or, that information 
        can be simply hard-coded or pre-configured into it. Once it 
        discovers its CE, it starts message exchange with CE to re-establish 
        the association just as outlined in Figure 9, with the possible 
        exception that it might be able to bypass the transport of the 
        complete initialization information. Suppose that FE1 still have its 
        routing table and other state information from the last association, 
        instead of sending all the information again from scratch, it can 
        choose to use more efficient mechanism to re-sync up the state with 
        its CE. For example, a checksum of the state might give a quick 
        indication of whether or not the state is in-sync with its CE. By 
        comparing its state with CE first, it sends information update only 
        if it is needed. 
         
         
     6. Applicability to RFC1812  
      
        [FORCES-REQ] Section 5, requirement #9 dictates that "All proposed 
        ForCES architecture MUST explain how that architecture may be 
        applied to support all of a router's functions as defined in 
        [RFC1812]." RFC1812 discusses many important requirements for IPv4 
        routers from the link layer to the application layer. This section 
        addresses the relevant requirements for implementing IPv4 routers 
      
     Anderson, et. al.      Expires November 2002                  [Page 17] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
        based on ForCES architecture and how ForCES satisfies these 
        requirements. 
         
        IPv4 routers must implement IP to support its packet forwarding 
        function, which is driven by its FIB (Forwarding Information Base). 
        This Internet layer forwarding (see [RFC1812] Section 5) 
        functionality naturally belongs to FEs in the ForCES architecture.  
         
        A router may implement transport layer protocols (like TCP and UDP) 
        that are required to support application layer protocols (see 
        [RFC1812] Section 6). One important class of application protocols 
        is routing protocols (see [RFC1812] Section 7). In ForCES 
        architecture, routing protocols are naturally implemented by CEs. 
        All modern routing protocols require routers communicate with each 
        other. This communication between CEs in different routers is 
        supported in ForCES by the FEs' ability to redirect data packets  
        addressed to routers (i.e., NEs) and CEs' ability to originate 
        packets and have them delivered by their FEs. This communication 
        occurs across Fp reference point inside each router and between 
        neighboring routers' interfaces, as illustrated in Figure 11.  
         
        After routing protocol update its routing tables, ForCES is used to 
        send the new routing table entries from CEs to FEs. It is also 
        required that certain events like interface up/down must be reported 
        back to upper layer application. ForCES provides the mechanism for 
        FEs to report such events back to CE. 
      
        RFC1812 also dictates "Routers MUST be manageable by SNMP." (see 
        [RFC1157] Section 8) In general, for post-association phase, most 
        external management tasks (including SNMP) SHOULD be done through 
        interaction with the CE in order to support the appearance of a 
        single functional device. Therefore, it is recommended that SNMP 
        management agent be implemented by CEs and the SNMP messages sent to 
        FEs be redirected to their CEs. In certain conditions (e.g. CE/FE 
        disconnection), it may be useful to allow SNMP to be used to 
        diagnose and repair problems. However, care should be taken when 
        exercising such mechanisms and guidelines are provided in [FORCES-
        REQ], Section 5, requirement #4. 
          
     7. Security Considerations 
         
        The security necessary across each reference point except Fp is 
        discussed throughout the document. In general, the physical 
        separation of two entities usually requires much stricter security 
        measurement in place. For example, we pointed out in Section 5.1 
        that authentication becomes necessary between CE manager and FE 
        manager, between CE and CE manager, between FE and FE manager in 
        some configuration.  The physical separation of CE and FE also 
        imposes serious security requirement for ForCES protocol. The 
        security requirements for reference point Fp (i.e., ForCES protocol) 
        are discussed in detail in [FORCES-REQ] Section 8. 
         
     8. Intellectual Property Right 
      
     Anderson, et. al.      Expires November 2002                  [Page 18] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
         
        The authors are not aware of any intellectual property right issues 
        pertaining to this document. 
         
     9. Normative References 
         
        [RFC2914] S. Floyd, "Congestion Control Principles", RFC2914, 
        September 2000. 
         
        [RFC1157] J. Case, et. al., "A Simple Network Management Protocol 
        (SNMP)", RFC1157, May 1990. 
         
        [RFC3084] K. Chan, et. al., "COPS Usage for Policy Provisioning 
        (COPS-PR)", RFC3084, March 2001. 
         
        [RFC1812]  F. Baker, "Requirements for IP Version 4 Routers", 
        RFC1812, June 1995. 
         
     10. Informative References 
         
        [FORCES-REQ] T. Anderson, et. al., "Requirements for Separation of 
        IP Control and Forwarding", work in progress, February 2002, <draft-
        ietf-forces-requirements-02.txt>. 
         
        [FORCES-APP] A. Crouch, et. al., "ForCES Applicability Statement", 
        work in progress, February 2002, <draft-ietf-forces-applicability-
        00.txt>. 
         
     11. Acknowledgments 
         
        The authors would like to thank Alan Crouch, Patrick Droz, Joel M. 
        Halpern, David Putzolu and Hormuzd M. Khosravi for their invaluable 
        comments and suggestions. 
         
     12. Authors' Addresses 
      
        Lily L. Yang 
        Intel Labs 
        2111 NE 25th Avenue 
        Hillsboro, OR 97124 USA 
        Phone: +1 503 264 8813 
        Email: lily.l.yang@intel.com 
         
        Ram Dantu 
        Netrake Corporation 
        3000 Technology Drive 
        Plano, Texas 75074 
        Phone: +1 214 291 1111 
        Email: ramd@netrake.com 
         
        Todd A. Anderson 
        Intel Labs 
      
     Anderson, et. al.      Expires November 2002                  [Page 19] 
     Internet Draft         ForCES Framework                  May 2002 
      
      
        2111 NE 25th Avenue 
        Hillsboro, OR 97124 USA 
        Phone: +1 503 712 1760 
        Email: todd.a.anderson@intel.com 
         
         
         
      
        1. Abstract........................................................1 
        2. Definitions.....................................................1 
        3. Introduction to Forwarding and Control Element Separation 
        (ForCES)...........................................................3 
        4. Architecture....................................................6 
           4.1. Control Elements and Fr Reference Point....................7 
           4.2. Forwarding Elements and Fi reference point.................8 
           4.3. CE Managers...............................................10 
           4.4. FE Managers...............................................11 
        5. Operational Phases.............................................11 
           5.1. Pre-association Phase.....................................11 
              5.1.1. Fl Reference Point...................................11 
              5.1.2. Ff Reference Point...................................12 
              5.1.3. Fc Reference Point...................................13 
           5.2. Post-association Phase and Fp reference point.............13 
              5.2.1. Association Establishment............................13 
              5.2.2. Steady-state Communication...........................14 
              5.2.3. Data Packets across Fp reference point...............15 
              5.2.4. Interconnect and Proximilty between CE and FE........16 
              5.2.5. Proxy FE.............................................16 
           5.3. Association Re-establishment..............................16 
        6. Applicability to RFC1812.......................................17 
        7. Security Considerations........................................18 
        8. Intellectual Property Right....................................18 
        9. Normative References...........................................19 
        10. Informative References........................................19 
        11. Acknowledgments...............................................19 
        12. Authors' Addresses............................................19 
            
      
     Anderson, et. al.      Expires November 2002                  [Page 20]