The Client Interface
Geoff Huston, July 1994
Copyright (c) Geoff Huston, 1994
The Client InterfaceThe client interface is the interface between the network service structure and the client network, and is specified both in a physical sense in terms of equipment interconnection and in terms of delineation of protocol and service operational responsibility. The following sections examine this interface in more detail.
The Client Interface
- Use of RIP as Network / client boundary routing protocol
simple
widely implemented
adequate
In terms of protocol interaction for Internet services, the client interface is a routing interface.
Routing information passes in both directions across this interface:
In referencing RIP the author is aware that this is not perhaps the "best" advice from the Internet perspective. While the Internet continues to use either class A, B, C addresses, or aggregates of Class C addresses the use of RIP in this context as an interface protocol between service provider and client is not particularly dangerous - perhaps inefficient, but readily configured and reliable to operate in such a configuration.
However the author strongly cautions the reader that this advice is based on three major premises - all of which may be wrong now, or in the future!
The Client Interface
- Use of inbound routing filters to preserve network integrity
prevent advertising bogus routes
integrity of client network
- Use of outbound static default route to simplify client routing
stability of presented service
simplicity of presented service
The proposed routing model of the provider client interface is based on a model of explicit protection.
The provider should explicitly set up an exclusion filter in client originated routing advertisements which ensure that only those networks that the client and provider agree are to be advertised by the client are passed into the provider's internal routing tables for advertisement throughout the network. Not only does this routing configuration ensure that the client is only capable of advertising those networks which have been agreed as being the basis of the service consumer role, but effectively isolates any routing faults on the client side from causing havoc within the provider's routing space.
From the client side the provider is in a position, in many cases (where the provider is operating in the role of sole provider of external Internet services, or where the provider is operating in the role of preferred external provider), to simply advertise to the client the default route, and not to provide an explicit enumeration within the routing interface the exact composition of this default route.
Both of these measures are intended to drastically simplify the routing interface between the client and provider. The incoming routing filters place the provider in the position of being able to only "believe" those routing advertisements from the client which are within the scope of the service agreement, allowing the provider to operate the network with a considerable level of inherent integrity, in a routing sense. The synthesis of a default route, and passing just this route to the client allows the client to immediately disseminate this route within the client network, again enhancing the overall stability and integrity of the end to end service.
The Client InterfaceThis configuration can be illustrated by considering the provider's interface configuration within an example situation.
The example chosen is where a client is routing internally network 150.10.0.0, and wishes to connect this network into the provider's service.
The provider configuration is such that for incoming routing updates from the client-connected physical interface of the router all routing information other than that pertaining to network 150.10.0.0 is filtered out. reachability to this network is passed onward within the provider's network, based on this reachability information being received from the client.
The provider generates statically a default route and passes this route to the client, effectively asserting Internet reachability as the provided service.
This default advertisement is effectively a route of the last resort. If the client undertakes any form of backdoor connection to an external site, the explicit routing information that would probably accompany such a connection would ensure that traffic between the two sites would travel preferentially along this backdoor connection, on the basis of a more specific route taking precedence over a default route.
The Client Interface
- Clear demarcation of boundary between client and network is required
for consistency of service
- Single demarcation model is required for the network to ensure
manageability and operability
- The network service should never transit a client network
There are a number of architectural issues surrounding the design of the client interface.
The most important aspect of this interface is the requirement for a clear demarcation of the boundary defining the client and provider networks. The intent of this clear definition of a boundary is to eliminate the issue of shared or (worse) overlapping operational responsibility for equipment and its operation, or gaps in the interface where operational responsibility for equipment is not claimed by either provider or client.
In addition it is noted that the most efficient means of defining this demarcation point with each client is to define the network service boundary as a uniform definition of the network attachment architecture and as a uniform definition of the service interface, and explicitly state that in all cases it is the responsibility of the client to provide equipment and services to interface to this defined attachment. This uniform interface definition also assists in manageability and operability of the network by allowing the client interface to be assumed as a constant factor in operational procedures.
It is also worth noting that the network interface to the client is absolute - it is certainly most unwise to contemplate a design where the network provider must transit across a client's network to provide service to another client. Such designs inevitably involve compromising delivered service quality metrics and reliability control on the basis that the provider has no direct control over the complete path to the network's client in such situations.
In those cases where the desired topology for the network service provider is a chain-like topology is appropriate to interface the first client in a single "drop-off", and then provide a direct connection to the next client, and so on. It is highly inadvisable to contemplate a topology where the network traverses a client's network in order to reach other direct clients of the service.
The Client InterfaceThe general architecture of the client interface is that of two routers, interconnected by a dedicated tail loop.
The first router is an integral component of the internal structure of the provider's network, and is used to terminate a number of dedicated tail loops to local clients (as well as connecting to other provider's core routers through the internal provider carriage network). This router defines the boundary of the core carriage network, and is the effective "point of presence" (POP) for this network.
It is appropriate to place routing filters and configuration commands which generate a statically defined default route within this POP router. It is noted that this router is of course owned and operated by the network provider exclusively. No client requires (or should have) access to this router in any management capacity, other than operational hosting of the equipment under the terms of a remote hosting agreement. It is also noted that this router need not function in a dedicated mode for a single client, but is most effectively configured as a client concentrator, terminating a number of client circuits simultaneously.
The dedicated tail loop should be terminated with a dedicated interface router, located on the client's site. This router performs the non-routing service definition functions, as well as acting as the point of demarcation between the client's network and the provider's carriage network.
If there are service definition configurations sets, host access filters, and other router configuration defined functions it is most effective to place such functions within this router, at the edge of the network. It is also noted that it is not advisable to use this router to interconnect distinct components of a client's network as the primary (or sole) means of interconnection. For the same reasons why it is highly inadvisable to use a client's network as a transit for the service provider's network, it is equally inadvisable to use the service provider's network as the primary or sole means of interconnecting a client's network.
Such shared configurations cause continual operational problems through the conflicting service and operational requirements of the client and the external network service provider.
The Client Interface
- The POP Access Model
Client is responsible for CPE router and tail loop
Network Provider provides router attachment points at a number of locations
Network Boundary located at POP interface
There are a number of client interface models which can be used as the basis of service provision.
One method used is the Point of Presence (POP) access model. In this model the network interface is defined at the client access points, and the exact connection interface point is defined as access ports on the POP access routers.
The client is responsible for the routing equipment and configuration on the client's site, and is responsible for the provision of the circuit between the client's site and the POP location. The exact location of the POP is not that important, although it is reasonable in terms of marketing effectiveness to locate POPs within targeted service provision areas.
The exact choice of the POP site depends on a number of factors, including ease of access by operational support staff, client tail loop tariffs, capability to house client provided circuit termination equipment, and so on. It is often the case that the most effective POP arrangement is within centrally located client sites, where the client staff and the network service provide enter into a formal contract for the provision of hosting equipment and associated operational support (that is of course unless the network service provider is a part of a telephone enterprise, in which case co-location at telco POPs is typically a very attractive proposition).
The Client Interface
- The Comprehensive Service Model
Network provider installs and operates CPE router and tail loop
Network provider attaches to client LAN
Network Boundary located at LAN attachment point
A different interface model is one where the service provider offers a comprehensive connection service, and provides service right up to the interface with the client site's networking infrastructure on the client's site.
Within this model the network service provider leases the tail loop, and installs and operates the customer premises equipment which interfaces directly onto the client's local network.
Within this model the client interface is the point of attachment to the local network, and, in Internet Protocol terms the interface is engineered through the advertisement of a default route onto the client's network, and the advertisement of the client's network numbers throughout the Internet.
This model of service is very much closer to a very traditional telephone service model, where the tail loop and the customer's handset are provided by the service provider. Within the networking domain there is little doubt that there is a very significant area of demand for such services, and indeed a demand for superset of these services which also cover the operation of the name service and management of the configuration of various services, including electronic mail.
Some caution is advocated however in this area, that covering this area of requirement can be very expensive in terms of resource expenditure. Within the initial phase of establishment of a national network service it may be the case that this service model is the only way in which sites can be connected, and the additional cost of supporting such a service model is partially offset by the expanding pool of network service expertise which accompanies such efforts. It is noted that typically however the core public network service model generally shrinks slightly following the initial startup phase of operation, and value added service providers then occupy a niche in the market, providing additional connection services and associated support for those clients who choose not to provide such support in house. This service infill by third parties appears to be a natural progression for the Internet market within many countries.
The Client Interface
- The Confused Model
Network Provider installs tail loop
Network Provider installs router interface card in client router
Client and network provider operate client router simultaneously
Two client network interface models have been described above, either of which will operate in a stable fashion within a national network service environment.
What cannot operate stably is a client network interface which is inconsistently defined, if defined at all. The major source of operational issues is where both the network service provider and the client attempt to operate a single resource simultaneously, and the conflicting requirements of both operations become the source of constant problems.
Such issues can range from the issue of tracking vendor-supplied software and hardware upgrades to change control of router configurations and service management issues which threaten the continued viability of the service offering to the client.
What is seen as being perhaps the worst case in poor interface definition is where the interface is effectively the backplane of a router, where the service provider and the client both operate interface cards within the single unit. This in an environment which was never intended to operate stably, and caution is urged on the part of the network designed to avoid such a situation of shared responsibility at all costs!
The Client Interface
- POP or end-to-end model depends on:
telco bulk purchase tariff discounting
router vendor bulk purchase discounting
staff availability
client expertise levels
defined service level
The choice of whether to offer an access structure based on the client connecting to POPs or a structure based on service connection directly to the client site does depend on a number of local factors which must be considered.
The Internet Service Provider business is typically structured as a value added bandwidth reselling business, where the purchase price of the bandwidth resource can depend on the total volume of business. The Provider purchase of the tail loops may significantly increase the total business volume, and can act as a level on tariffs. Similar considerations may apply with router purchasing.
Additionally staff availability may be an issue. The POP model does entail higher staffing support levels than a full service model (due predominantly to the higher load on consultation and client education, as the interface model of the POP requires significantly more action on the part of the client when compared to the client site access model.
Client expertise is also an issue within this area of consideration, as it is of marginal relevance to offer a service which is effectively unusable by the potential client base.
The Client Interface
- You can do both POP and end-to-end
as long as all routing integrity is maintained within the POP
locations for all clients
Finally it is noted that the service provider does not need to stick to a single interface model, and both POP access and client site access models can be offered if there is a perception of as market need for both.
If the tail loop and the client site router are priced at a level which is revenue neutral, then overall pricing for either model is roughly equivalent - the additional operational costs of the client site access method are balanced by the higher level of consultant advisory costs associated with a POP access service model.
If offering both forms of access it is very important to maintain a clear picture of the boundary of the service provider's network, and ensure that the service provider is in a position to ensure the integrity of routing, and the integrity of various service offerings by using appropriate configuration management and network design principles.
The Client Connection
- Routers provide:
security capability
management capability
routing management
traffic management
service management
efficiency
integration
There is an increasingly visible demand for low cost Internet connections, and the service provider is faced with the design issue of how to service such connections.
The choices are generally between router based environments, and lower cost host systems running routing software.
In general routers provide greater levels of functionality in traffic and service management, and greater levels of service availability through the additional management capabilities normally found on dedicated routing platforms. Such functions include:
The Client Connection
- SLIP / PPP implementations in hosts
cheap!
Capital price differential between hosts and router is small
Operating cost is higher using hosts as routers
use as single end host access system
An alternative approach is to use general purpose computing platforms as routing managers, using interface ports and routing software to manage the interconnection function.
Such an approach is, in capital terms, generally cheaper than the use of a dedicated routing platform (although with the entry of low price access routers onto the market this statement may not necessarily be true in all cases today). However host-based software systems suffer from a number of performance and operational inefficiencies and functional deficiencies which leads to the general observation that the cost of ownership of such systems, when deployed in the role of connection interface units, are significantly higher than the cost of ownership of dedicated routing platforms.
Such software typically uses CPU-based switching, which impacts of overall throughput under conditions of high asynchronous or synchronous serial traffic, and the additional management and security functions are also CPU-based software functions (if present). For all but dial-up asynchronous traffic such platforms are typically not a cost effective design choice for the network provider or the client.
Routing to the Client
- Multiple client interfaces
bifurcation of client and provider network - multiple default paths
asymmetric routes can be generated
client network internal breakage causes black hole routing
- Requires careful management and clear understanding of the routing issues
The consideration of the client interface above is within the implicit assumption of a configuration of a single external service interface. The situation does become more involved in the case where a client connects to the service provider in multiple locations, or uses a number of external service providers, or uses the service provider as a connectivity provider for the client's corporate network. These cases are considered, in design terms, in the following sections.
The first typical case is where a single, internally interconnected client network attaches to the service provider in a number of locations. If the client chooses to advertise the same routes through both service interface points, and if the service provider is offering the default route through both interfaces the overall result is effectively a traffic bifurcation of both the client's network and the service provider's network with respect to the client's traffic.
From the client's perspective the default route is being offered through two distinct locations, and the propagation of these routes through the client's network will be on the basis of the incremental adjustment of the metric of each default route an accordance with the propagation. At any point within the client's network one or the other default route interface will be seen with a lower metric, and traffic will flow to that interface unconditionally. Care must be taken where the two metrics are of equal value, as the routing protocols may choose to use load balancing and attempt to traffic share, causing session traffic to be generated across both interfaces, causing potential end to end traffic efficiency implications through the consequent requirement to reorder packets, and the interplay between the two end to end path round trip times and the reordering TCP packet timers.
Within the service provider's network there is a similar situation when propagating the routes associated with the client's network, where the service provider will preferentially route through one interface or the other depending on the relative metrics, and again there is a potential issue with traffic efficiency at those points where equal weight metrics are seen for both routes. It is also noted that asymmetric routes are a typical outcome of such configurations, where an end to end session may well be set up such that incoming traffic is directed over one interface by the service provider, and outgoing traffic is directed to another interface by the client network. While such asymmetric configurations are readily supported, they tend to present significant operational challenges in the event of partial failure within the total networking environment, and also may cause service implications in the event that the service application is making implicit assumptions about symmetry of traffic delays and related traffic characteristics.
The client must also exercise considerable care with deployment of networks, and subnets. In the situation where the client's network loses internal connectivity between the two interface points the service provider will not normally be in a position to act as a bridge and traffic will fall down a routing black hole within the client's network during such periods of outage (this is a conditional statement, as it is in general possible to configure such a backup connectivity service, although this does entail significant levels of customised configuration on the part of both the client and the service provider).
The overall conclusions from a design and architecture perspective are that multiple client interfaces do not in general achieve optimal load balancing outcomes (the typical motivation for such connections). Where such configurations are contemplated considerable care must be exercised by both the client and the service provider to ensure that the resultant operational environment is a stable configuration, and such care involves very precise handling of network routing advertisements and associated propagation of routes and metrics through both networks.
It should be noted as a postscript to this text that the previous statements about the applicability of RIP as a client interface protocol within qualified circumstances of simplicity and operational reliability do not readily transfer into this, or similar more complex client interface scenarios.
Routing to the Client
- Multiple providers
Only one provider can provide "default"
other connected providers must resort to explicit provision of routes
to enumerated networks
All providers must ensure that the client is not used as a transit
facility through explicit route management on the part of all providers
The second configuration area is where the client connects to more than one external service provider. Within such a configuration the general guideline is that the client can only accept a default route from one such external provider, and therefore must accept only explicit routes from other external providers.
This quickly becomes a policy implementation domain, as the default route is effective the service of last resort, and the explicit network advertisements from other service providers will take precedence over the default route. It is often the case that all external providers will be offering a common subset of routes, and the client may need to synthesise additional explicit routes in conjunction with the default route, or explicitly filter routes from an explicit list in order to implement the interaction policies that form part of the external connection service.
Again here the typical intention of the client in making such multiple external connections is to enhance the reliability of the external connectivity service, intending to configure one provider as a backup in the event of outage from the primary provider. Considerable care must be exercised by both the service providers and the clients to ensure that this is indeed the case in such situations.
The second area of care is the issue of the client's resultant advertisements back to the service provider, as route leakage, where one provider's network advertisements are readvertised to a second provider by the client. Here again the service provider has to exercise considerable care in the filtering of client's advertisements to the provider's network, to ensure that only those networks which form part of the client's domain are accepted as valid routes by the service provider.
Distributed Client support
- VPN architecture issues
VPNs via filtering
VPNs via tunnelling
- Why Support VPNs?
The third area is that of a distributed client who wishes to use the service provider as a transit facility between components of the client's network - generally termed Virtual Private Network support (VPN).
There are a number of ways in which VPN support can be configured as a provided service. The most readily supportable model is where each distinct serviced element of the client's network uses distinct network numbers, and through the use of routing filters at each interface point only internal corporate traffic can be directed down each of the client's interfaces (variants of this are also readily possible, where, with the management of routing advertisements and associated metrics, multiple external interface points can be constructed.)
Where the client is using subnets of a common network within the distributed configuration more complex routing configurations are required to support this environment, and here the use of encapsulation (using a technique of adding an additional transport header to tunnel a packet through the provider's network) is an alternative design choice. While disconnected subnet routing does pose greater levels of configuration complexity on the provider, there is the advantage that all traffic is within the providers scope of operational visibility, and the provider is in a position to assist the client in the identification and rectification of operational problems. If the traffic uses encapsulation (tunnels) to connect the disparate client sites the traffic is essentially invisible to the provider, and the provider becomes a virtual dynamic channel switching fabric to the client. For some clients this may indeed be the desired configuration, where the client is responsible for the VPN routing mechanism and operational support - for other clients they may wish to pass bot the transport and routing support to the service provider as an outsourced function.
VPN services are not the most readily supportable service option. Clients who request VPN support with associated quality of service conditions cannot be readily configured into a general datagram delivery infrastructure, and the advantages of dedicated bandwidth as the VPN support mechanism generally dictate that VPN services are typically provided on a best effort basis by a shared access Internet service provider, and as such do not readily lend themselves to be a major service offering from the service provider's perspective.
However distributed corporate entities who are testing the marketplace for cost effective support for their requirements are a constant factor, and the VPN market is evidently a growing activity sector.
3. Engineering the Network - The Client Interface
4. Engineering the Network - External Peering
5. Engineering the Network - Infrastructure
7. The Operational Environment
GH, July 1994