INTERNET-DRAFT Roland Bless Expires: January 2003 Klaus Wehrle Internet Draft Universitaet Karlsruhe (TH) July 2002 Document: draft-bless-diffserv-multicast-04.txt IP Multicast in Differentiated Services Networks Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Distribution of this document is unlimited. Abstract This document presents some of the problems which will arise when IP Multicast is used in DiffServ networks without taking special precautions into account for supplying multicast services. Although the basic DS forwarding mechanisms also work with IP Multicast, some facts have to be considered which are related to the provisioning of multicast resources. The presented problems mainly lead to situations in which other service users are affected adversely in their experienced quality. In order to retain the benefits of the DiffServ approach, a quite simple and scalable solution for those problems is required, not resulting in additional complexity or costs related to forwarding mechanisms in a DiffServ domain. Internet-Draft IP Multicast in DiffServ Networks July 2002 The proposed solution in this document is as scalable as IP Multicast itself, and, a simple implementation requires only an additional field for the DiffServ Codepoint in the multicast routing table and some support by management mechanisms. Table of Contents 1 Introduction..................................................3 1.1 Management of Differentiated Services.........................4 2 Problems of IP Multicast in DS Domains........................4 2.1 Neglected Reservation Subtree Problem (NRS Problem)...........5 2.2 Heterogeneous Multicast Groups...............................12 2.3 Dynamics of Any-Source Multicast.............................13 3 Solutions for Enabling IP-Multicast in Differentiated Services Networks.....................................................13 3.1 Solution for the NRS Problem.................................14 3.2 Solution for Supporting Heterogeneous Multicast Groups.......18 3.3 Solution for Any-Source Multicast............................18 4 Scalability Considerations...................................19 5 Security Considerations......................................19 6 References...................................................20 7 Acknowledgements.............................................21 8 Authors' Addresses...........................................21 A Proof of the Neglected Reservation Subtree Problem...........23 A.1 Implementation of the proposed solution......................23 A.2 Test Environment and Execution...............................24 B Simulative Study of the NRS Problem and Limited Effort PHB...27 B.1 Simulation Scenario..........................................27 Bless & Wehrle Expires: January 2003 [Page 2] Internet-Draft IP Multicast in DiffServ Networks July 2002 B.2 Simulation Results for different router types................29 B.2.1 Interior Router............................................29 B.2.1.1 Case I:..................................................30 B.2.1.2 Case II:.................................................30 B.2.1.3 Case III:................................................31 B.2.1.4 Case IV:.................................................31 B.2.2 Boundary Router............................................32 B.2.2.1 Case I:..................................................32 B.2.2.2 Case II:.................................................32 B.2.2.3 Case III:................................................33 B.2.2.4 Case IV:.................................................34 1 Introduction Services in the Internet offering a better quality than the current best-effort service are increasingly required. Many advanced applications need certain assurances from the network layer, e.g., a maximum delay, a minimum packet loss rate or guaranteed transmission rate. The currently used IP mechanisms are not able to offer such guarantees, especially, if group communication is additionally required. The "Differentiated Services" (DiffServ or DS) approach [1, 2, 3] defines certain building blocks and mechanisms to offer qualitatively better services than the usual “normal” best-effort delivery service in an IP network. In the DiffServ Architecture [2] scalability is achieved by avoiding complexity and maintenance of per-flow state information in core nodes and by pushing unavoidable complexity to the network edges. Therefore, individual flows belonging to the same service are aggregated, thereby eliminating the need for complex classification or managing state information per flow in interior nodes. On the other hand, the reduced complexity in DS nodes makes it more complex to use those "better" services together with IP Multicast (i.e., point-to-multipoint or multipoint-to-multipoint communication). Problems emerging from this fact are described in section 2. Although the basic DS forwarding mechanisms also work Bless & Wehrle Expires: January 2003 [Page 3] Internet-Draft IP Multicast in DiffServ Networks July 2002 with IP Multicast, some facts have to be considered which are related to the provisioning of multicast resources. However, it is important to integrate IP Multicast functionality right from the beginning into the architecture, and, to provide simple solutions for those problems not defeating the gained advantages so far. The EF PHB [4] shows also very interesting properties within a multicast context. The very low packet loss characteristic makes it suitable as a basis for a highly (but not absolute) reliable multicast service. Packet loss cannot be fully precluded, because of aggregation effects which may nevertheless lead to packet losses. However, in reality packet losses should occur so infrequently that many applications can tolerate these losses, or if this is not the case, that at least very simple retransmission schemes can be applied. 1.1 Management of Differentiated Services At least for Per-Domain Behaviors and services based on the EF PHB admission control and resource reservation are required. Furthermore, installation and updating of traffic profiles in boundary nodes is necessary. Most network administrators will not accomplish this task manually, even for long term service level agreements (SLAs). Furthermore, offering services on demand requires some kind of signaling and automatic admission control procedures. Therefore, the concept of Bandwidth Brokers was already suggested by Van Jacobson at a very early stage of DiffServ research [5]. In this concept, the Bandwidth Broker (BB) is a dedicated node in each DS domain, which keeps track of the amount of available and reserved bandwidth for services, and, which processes admission control requests from customers or BBs of adjacent domains. Additionally, it installs or alters traffic profiles in boundary nodes. Protocols for signaling a reservation request to a Differentiated Services Domain are required. For accomplishing end-system signaling to DS domains RSVP [6] may be used with new DS specific reservation objects [7]. RSVP is mainly designed for use in multicast scenarios and is already supported by many operating systems. However, when RSVP is applied to a DiffServ network some problems will arise which are described in the next section. 2 Problems of IP Multicast in DS Domains Although potential problems and the complexity of providing multicast with Differentiated Services are considered in a separate section of [2], both aspects have to be discussed in greater detail. Bless & Wehrle Expires: January 2003 [Page 4] Internet-Draft IP Multicast in DiffServ Networks July 2002 The simplicity of the DiffServ Architecture and its router models is necessary to reach high scalability, but it causes also fundamental problems in conjunction with the use of IP Multicast in DS domains. The following subsections describe these problems for which a simple solution is proposed in section 3. This solution is as scalable as IP Multicast and needs no resource separation by using different codepoint values for unicast and multicast traffic. Because Differentiated Services are unidirectional by definition, we also consider the point-to-multipoint communication being of unidirectional nature. In traditional IP Multicast any node can send packets spontaneously and asynchronously to a multicast group, respectively to their multicast group address (therefore, traditional IP Multicast offers a multipoint-to-multipoint service, also referred to as Any-Source Multicast). This feature is discussed in section 2.3. For subsequent considerations we assume, unless stated otherwise, at least a unidirectional point-to-multipoint communication scenario in which the sender generates packets which experience a "better" Per- Hop Behavior than the traditional default PHB, resulting in a service of better quality compared to the default best-effort service. In order to accomplish this, a traffic profile corresponding to the traffic conditioning specification has to be installed in the sender's first-hop router (the first boundary node of the first DS domain receiving those packets). Furthermore, it must be assured that the corresponding resources are available on the path from the sender to all the receivers, possibly requiring adaptation of traffic profiles at involved domain boundaries. Note that the latter process may also be initiated on demand of a receiver. 2.1 Neglected Reservation Subtree Problem (NRS Problem) Typically, resources for Differentiated Services must be reserved before actually using them. But in a multicast scenario group membership is often highly dynamic, therefore limiting the use of a sender-initiated resource reservation in advance. Unfortunately, dynamic addition of new members of the multicast group using Differentiated Services can adversely affect existing other traffic, if resources were not explicitly reserved before use. A practical prove of this problem is given in appendix A.3. IP Multicast packet replication usually takes place when the packet is handled by the "routing" core (cf. Fig. 1), i.e., when it is forwarded according to the routing table. Thus, a DiffServ capable node would also copy the content of the DS field [1] into the IP packet header of every replicate. Consequently, replicated packets get exactly the same DS codepoint (DSCP) as the original packet, Bless & Wehrle Expires: January 2003 [Page 5] Internet-Draft IP Multicast in DiffServ Networks July 2002 and, therefore experience the same forwarding treatment as the incoming packets of this multicast group (see Fig. 1, in this case the egress interface comprises functions for (BA-) classification, traffic conditioning and queueing). Interface A IP Routing Interface C +-----------+ +--------------+ +-----------+ MC-flow | | | replication | | egress | ---->| ingress |---->|------+-------|----->|(class.,TC,|----> | | | | | | queueing) | +-----------+ | | | +-----------+ | | | Interface B | | | Interface D +-----------+ | | | +-----------+ | | | | | | egress | | ingress | | +-------|----->|(class.,TC,|----> | | | | | queueing) | +-----------+ +--------------+ +-----------+ Figure 1: Multicast packet replication in a DS router Normally, the replicating node cannot test whether a corresponding reservation exists for a particular flow of replicated packets on an output link (resp. its corresponding interface), because a flow- specific traffic profile is usually not available in boundary (except in first-hop nodes) and interior nodes. When a new receiver joins an IP Multicast group, the corresponding multicast routing protocol (e.g., DVMRP [8, 9], PIM-DM [10] or PIM- SM [11]) accomplishes that the multicast tree is expanded by a new branch which connects the new receiver to the already existing multicast tree. As a result of tree expansion and missing per-flow classification and policing mechanisms, the new receiver will implicitly use the service of better quality, because of the copied "better" DSCP. If the additional amount of resources which are consumed by the new part of the multicast tree are not taken into account by the domain management (cf. section 1.1), the currently provided level of quality of service of other receivers (with correct reservations) will be affected adversely or even violated. This negative effect on existing traffic contracts by a neglected resource reservation -- in the following designated as Neglected Reservation Subtree Problem (NRS Problem) -- must be avoided under any circumstances. One can distinguish two distinct major cases of the NRS Problem. In order to compare their different effects a simple example of a share of bandwidth is illustrated in Fig. 2. Bless & Wehrle Expires: January 2003 [Page 6] Internet-Draft IP Multicast in DiffServ Networks July 2002 40% 40% 20% +--------------------+---------------------+------------+ |Expedited Forwarding| Assured Forwarding | Best-Effort| +--------------------+---------------------+------------+ ----------------------------------------------------------> output link bandwidth Figure 2: An example bandwidth share of different behavior aggregates Three types of services (respectively their corresponding behavior aggregates) share the bandwidth of the considered output link: Expedited Forwarding, Assured Forwarding and the traditional Best- Effort service. In this example we assume that routers perform simple priority queueing, where EF has the highest and Best-Effort the lowest assigned priority. When Weighted Fair Queueing (WFQ) would be used, the described effects would essentially also occur, only with minor differences. The Neglected Reservation Subtree problem appears in two different cases: o Case 1: If the branching point of the new subtree (at first only a branch) and the previous multicast tree is an (egress) boundary router, as shown in Fig. 3, the additional multicast flow now increases the amount of used resources for the corresponding aggregate and will be greater than the originally reserved amount on the affected output link. Consequently, the policing component in the egress boundary router drops packets until the traffic aggregate is in accordance to the traffic contract. But during dropping packets, the router can not identify the responsible flow (because of missing flow classification functionality), and, thus randomly discards packets, whether they belong to a correctly reserved flow or not. As a result, there will be no longer any service guarantee for the reserved flows. Bless & Wehrle Expires: January 2003 [Page 7] Internet-Draft IP Multicast in DiffServ Networks July 2002 Sender +---+ | S | DS domains ....... +---+ ..... / \ . .. || . . .. / \ .. . ..||.. .... .<- ->... .. . || . . . . +---+ +--+ +--+ *) +--+ +--+ +--+ +------+ . |FHN|===|IN|=====|BN|###########|BN|####|IN|######|BN|####|Recv.B| . +---+ +--+ +--+\\ +--+ +--+ +--+ +------+ . \\ \ . \\ . \ . . +--+ +--+ . \\ . \ . . |IN|-----|IN| . \\ .. +--+ . . +--+ +--+ . \\ ... ....|BN|.. . || \ ... +------+ ... +--+ . || \ . |Recv.A| .+--+ ...+--+ +------+ |BN|..... |BN| +--+ +--+ || FHN: First-Hop Node S: Sender BN: Boundary Node Recv.x: Receiver x IN: Interior Node ===: Multicast branch with reserved bandwidth ###: Multicast branch without reservation *) Bandwidth of EF aggregated on the output link is higher than actual reservation, EF aggregate will be limited in bandwidth without considering the responsible flow. Figure 3: The NRS Problem (case 1) Bless & Wehrle Expires: January 2003 [Page 8] Internet-Draft IP Multicast in DiffServ Networks July 2002 +------------------+---------------------+--------------+------+ | Expedited Forw. | Expedited Forw. | Assured Forw.| BE | | | | | | | with reservation | excess flow | with reserv. | | | | without reservation | | | +------------------+---------------------+--------------+------+ | | | | | EF with and without reservation share | 40 % | 20% | | 40% of reserved EF aggregate. | | | | -> EF packets with reservation and | | | | without reservation will be | | | | discarded! | | | | | | | +------------------+---------------------+--------------+------+ (a) Excess flow has EF codepoint +------------------+---------------------+--------------+------+ | Expedited Forw. | Assured Forwarding | Assured Forw.| BE | | | | | | | with reservation | excess flow | with reserv. | | | | without reservation | | | +------------------+---------------------+--------------+------+ | | | | | | AF with & without reservation share| 20 % | | | 40% of reserved EF aggregate. | | | 40% | -> EF packets with reservation and | | | | without reservation will be | | | | discarded! | | | | | | +------------------+---------------------+--------------+------+ (b) Excess flow has AF codepoint Figure 4: Resulting share of bandwidth in a egress boundary router with a neglected reservation of (a) an Expedited Forwarding flow or (b) an Assured Forwarding flow. Fig. 4 shows the resulting share of bandwidth in cases when (a) Expedited Forwarding and (b) Assured Forwarding is used by the additional multicast branch causing the NRS Problem. Assuming that the additional traffic would use another 30% of the link bandwidth, Fig. 4 (a) illustrates that the resulting aggregate of Expedited Forwarding (70% of the outgoing link bandwidth) is throttled down to its originally reserved 40%. In this case, the amount of dropped EF bandwidth is equal to the amount of excess bandwidth. Consequently the original Expedited Forwarding aggregate (which had 40% of the link bandwidth reserved) is Bless & Wehrle Expires: January 2003 [Page 9] Internet-Draft IP Multicast in DiffServ Networks July 2002 affected by packet losses, too. The other services, e.g., Assured Forwarding or Best-Effort, are not disadvantaged. Fig. 4 (b) shows the same situation for Assured Forwarding. The only difference is that now Assured Forwarding is solely affected by discards, the other services will still get their guarantees. In either case, packet losses are restricted to the misbehaving service class by the traffic meter and policing mechanisms in boundary routers. Moreover, the latter problem (case 1) occurs only in egress boundary routers, because they are responsible, that not more traffic is leaving the Differentiated Services domain, than the following ingress boundary router will accept. Therefore, those violations of SLAs will be already detected and processed in egress boundary routers. Sender +---+ | S | DS domains ....... +---+ ..... / \ . .. || . . .. / \ .. . ..||.. .... .<- ->... .. . || . . . . +---+ +--+ +--+ +--+ +--+ +--+ +-------+ . |FHN|===|IN|=====|BN|===========|BN|====|IN|======|BN|===| Recv.B| . +---+ +--+ +--+\\ +--+ +--+ +--+ +-------+ . \\ \ . \\ . # . . +--+ +--+ . \\ . # *) . . |IN|-----|IN| . \\ .. +--+ . . +--+ +--+ . \\ ... ....|BN|.. . || \ ... +------+ ... +--+ . || \ . |Recv.A| # .+--+ ...+--+ +------+ # |BN|..... |BN| +------+ +--+ +--+ |Recv.C| || +------+ FHN: First-Hop Node S: Sender BN: Boundary Node Recv.x: Receiver x IN: Interior Node ===: Multicast branch with reserved bandwidth ###: Multicast branch without reservation *) Bandwidth of EF aggregated on the output link is higher than actual reservation, EF aggregate will be limited in bandwidth without considering the responsible flow Figure 5: Neglected Reservation Subtree problem case 2 after join of receiver C Bless & Wehrle Expires: January 2003 [Page 10] Internet-Draft IP Multicast in DiffServ Networks July 2002 o Case 2: The Neglected Reservation Subtree problem can also occur, if the branching point between the previous multicast tree and the new subtree is located in an interior router (as shown in Fig. 5). Because the router is not equipped with metering or policing functions it will not recognize any amount of excess traffic and will forward the new multicast flow. If the latter belongs to a higher priority service, such as Expedited Forwarding, bandwidth of the aggregate is higher than the aggregate's reservation and will steal bandwidth from lower priority services. The additional amount of EF without a corresponding reservation is forwarded together with the aggregate which has a reservation. This results in no packets losses for Expedited Forwarding as long as the resulting aggregate is not higher than the output link bandwidth. Because of its higher priority, Expedited Forwarding gets as much bandwidth as needed and as is available (strictly speaking, it is implementation dependent whether interior routers have something like a maximum configured service rate). +------------------+---------------------+--------------+------+ | Expedited Forw. | Expedited Forw. | Assured Forw.| BE | | | | | | | with reservation | excess flow | with reserv. | | | | without reservation | | | +------------------+---------------------+--------------+------+ | | | | | | 40% | 30% | 30% | 0% | | | | | | +------------------+---------------------+--------------+------+ EF with reservation and the excess flow use together 70% of the link bandwidth, because EF (with or without reservation has the highest priority. (a) Excess flow has EF codepoint +------------------+---------------------+--------------+------+ | Expedited Forw. | Assured Forw. | Assured Forw.| BE | | | | | | | with reservation | excess flow | with reserv. | | | | without reservation | | | +------------------+---------------------+--------------+------+ | | | | | 40% | 60% | 0% | | | | | | | 10% loss | | | | | | +------------------+---------------------+--------------+------+ AF with reservation and the excess flow use together 60% of the link bandwidth, because EF has the highest priority (-> 40%). 10% of AF packets will be lost. Bless & Wehrle Expires: January 2003 [Page 11] Internet-Draft IP Multicast in DiffServ Networks July 2002 (b) Excess flow has AF codepoint Figure 6: Resulting share of bandwidth in an interior router with a neglected reservation of (a) a Expedited Forwarding flow or (b) an Assured Forwarding flow The result is, that there is no restriction for Expedited Forwarding, but as Fig. 6 (a) shows, other services will be extremely disadvantaged by this use of non-reserved resources. Their bandwidth is stolen by the new additional flow. In this case, the additional 30% Expedited Forwarding traffic preempts resources from the Assured Forwarding traffic, which in turn preempts resources from the best-effort traffic, resulting in 10% packet losses for the Assured Forwarding aggregate and complete loss of best-effort traffic. The example in Fig. 6 (b) shows that this can also happen with lower priority services like Assured Forwarding. When a reservation for a service flow with lower priority is neglected, other services (with even lower priority) can be reduced in their quality (in this case the best-effort service). As shown in the example, the service's aggregate causing the problem can itself be affected by packet losses (10% of the Assured Forwarding aggregate is discarded). Besides the described problems of case 2, case 1 will arise in the next boundary router, that performs traffic metering and policing for flows of the service aggregate. Directly applying RSVP to Differentiated Services would also result in an NRS Problem, because a receiver has to join the IP multicast group BEFORE sending a resource reservation request (RESV message), in order to receive the sender's PATH messages at first. Thus, the join for receiving PATH messages will already cause an NRS Problem if this situation is not handled in a special way (e.g., by marking all PATH messages with codepoint 0). 2.2 Heterogeneous Multicast Groups Heterogeneous multicast groups contain one or more receivers, which would like to get another service or quality of service as the sender provides or other receiver subsets currently use. A very important characteristic which should be supported by Differentiated Services is that participants requesting a best-effort quality only should also be able to participate in a group communication which otherwise utilizes a better service class. The next better support for heterogeneity provides concurrent use of more than two different service classes within a group. Things tend to get even more complex Bless & Wehrle Expires: January 2003 [Page 12] Internet-Draft IP Multicast in DiffServ Networks July 2002 when not only different service classes are required, but also different values for quality parameters within a certain service class. A further problem is to support heterogeneous groups with different service classes in a consistent way. It is possible that some services will not be comparable to each other so that one service cannot be replaced by the other and both services have to be provided over the same link within this group. Because an arbitrary new receiver that wants to get the different service can be grafted to any point of the current multicast delivery tree, even interior routers may have to replicate packets with the different service. At a first glance, this seems to be a contradiction with respect to simplicity of the interior routers, because they do not even have any profile available and should now convert the service quality of individual receivers. Consequently, in order to accomplish this, interior routers have to change the codepoint value during packet replication. 2.3 Dynamics of Any-Source Multicast Basically, within an IP multicast group any participant (actually, it can be any host not even receiving packets of this multicast group) can act as a sender. This is an important feature which should also be available in case a specific service other than best- effort is used within the group. Differentiated Services possess conceptually a unidirectional character. Therefore, for every multicast tree implied by a sender resources must be reserved separately if simultaneous sending should be possible with a better service. This is even true if shared multicast delivery trees are used (e.g., with PIM-SM or Core Based Trees). If not enough resources are reserved for a service within a multicast tree allowing simultaneous sending of more than one participant, the NRS problem will occur again. The same argument applies to half-duplex traffic which would share the reserved resources by several senders, because it cannot be ensured by the network that exactly one sender sends packets to the group. Accordingly, the corresponding RSVP reservation styles "Wildcard Filter" and "Shared-Explicit Filter" [6] cannot be supported within Differentiated Services. The IntServ approach is able to ensure the half-duplex nature of the traffic, because every router can check each packet for conformance with the installed reservation state. 3 Solutions for Enabling IP-Multicast in Differentiated Services Networks Bless & Wehrle Expires: January 2003 [Page 13] Internet-Draft IP Multicast in DiffServ Networks July 2002 The problems described in the previous section are mainly caused by the simplicity of the Differentiated Services Architecture. Solutions have to be developed which do not introduce an additional amount of complexity which diminishes the scalability of this approach. In this document, a simple solution is suggested for most of the problems. In order to keep things simple, we restrict this first solution for supporting heterogeneous groups to the case in which only two different services within a multicast group can be used simultaneously. 3.1 Solution for the NRS Problem The proposed solution consists conceptually of the following three steps that are described in more detail later. 1. A new receiver joins a multicast group that is using a DiffServ service. Multicast routing protocols accomplish the connection of the new branch to the (possibly already existing) multicast delivery tree as usual. 2. The unauthorized use of resources is avoided by re-marking at branching nodes all additional packets leaving downwards the new branch. At first the new receiver will get all packets of the multicast group without QoS. The management entity of the correspondent DiffServ domain is informed about the extension of the multicast tree. 3. If a pre-issued reservation is available for the new branch or somebody (receiver, sender or a third party) issues one, the management entity instructs the branching router to set the corresponding codepoint for the demanded service. Usage of resources which where not reserved before must be precluded. In the following discussed example, the case is considered when the join of a new receiver to a DS multicast group requires grafting of a new branch to an already existing multicast delivering tree. The connecting node which joins both trees converts the codepoint (and therefore the Per-Hop Behavior) to a codepoint of a PHB which is similar to the default PHB (see (1) and (3) in Fig. 7) in order to provide a best-effort-like service for the new branch. More specifically, this particular PHB can be different from the default PHB providing a service which is even worse than the best-effort service of the default PHB. The conversion to this specific PHB could be necessary in order to avoid unfairness being introduced otherwise within the best-effort service aggregate, and, which results from the higher amount of Bless & Wehrle Expires: January 2003 [Page 14] Internet-Draft IP Multicast in DiffServ Networks July 2002 resource usage of the incoming traffic belonging to the multicast group. If the rate at which re-marked packets are injected into the outgoing aggregate is not reduced, those re-marked packets will probably cause discarding of other flow's packets in the outgoing aggregate if resources are scarce. Therefore, the re-marked packets from this multicast group should be discarded more aggressively than other packets in this outgoing aggregate. This could be accomplished by using an appropriate configured PHB (and a related DSCP) for those packets. In order to distinguish this kind of PHB from the default PHB, it is referred to as Limited Effort (LE) PHB (which can be realized by an appropriately configured AF PHB [12]). Merely dropping packets more aggressively at the re-marking node is not sufficient, because there may be enough resources in the outgoing BA to transmit every re- marked packet and not requiring discarding any other packets within the same BA. However, resources in the next node may be short for this particular BA. Therefore, those "excess" packets must be identifiable at this node. [EF| ] || || || JOIN_INDICATION \/ +------+ (2) +---------------+ Management | |<----------| | Entity | ME | | Router | | |---------->| | +------+ (4) +---------------+ SET_CODEPOINT // ^ \ // | \ // \ \ // \ \ || | | || (1) JOIN| | || | | \/ | V [EF| ] (3) [LE| ] (5) [EF| ] ===: Multicast branch with reserved resources for Expedited Forwarding ---: New Multicast branch [x| ]: IP packet with DSCP of PHB x Figure 7: Sequence of the proposed solution Re-marking packets is only required at branching nodes, whereas all other nodes of the multicast tree (such with outdegree 1) replicate packets as usual. Because a branching node may also be an interior Bless & Wehrle Expires: January 2003 [Page 15] Internet-Draft IP Multicast in DiffServ Networks July 2002 router of a domain, re-marking of packets requires conceptually per- flow classification. Though this seems to be in contradiction to the DiffServ philosophy of a core that avoids per-flow states, IP multicast flows are different from unicast flows: traditional IP multicast forwarding and multicast routing require to install states per multicast group for every outgoing link anyway. Therefore, re- marking in interior nodes is to the same extent scalable as IP multicast is (cf. section 4). Re-marking with standard DiffServ mechanisms [13] for every new branch requires activation of a default traffic profile. The latter accomplishes re-marking by using a combination of an MF-classifier and a marker at an outgoing link that constitutes a new branch. The classifier will direct all replicated packets to a marker that sets the new codepoint. The better service will be only provided if a reservation request was processed by the management (e.g., Bandwidth Brokers) in order to perform a required admission control test before resources are actually used. In case the admission test is successful, the re- marking node will be instructed by the management entity to stop re- marking and to use the original codepoint again (conceptually by removing the profile). Because reservation requests may also be initiated by the sender, an incoming JOIN-Request of a new receiver branch should be also signaled by a boundary node to the management node (indicated by the JOIN_INDICATION message in step (2) in Fig. 7), so that the re- marking node can be instructed (via the SET_CODEPOINT message in step (4)) to immediately use the same codepoint value for replicated packets belonging to this group as for incoming packets (EF in (5) of Fig. 7). A simple alternative implementation is also possible and is described in the following. This particular implementation does not require any additional classification of multicast groups within an aggregate. Because every multicast packet has to be handled by the multicast routing process (in this context, this notion signifies the multicast forwarding part and not the multicast route calculation and maintenance part, see Fig. 1), addition of an extra byte in each multicast routing table entry for containing the DS field, and, thus its DS codepoint value, per output link (resp. virtual interface, see Fig. 8) results in nearly no additional cost. Packets will be replicated by the multicast routing process, so this is also the right place for setting the correct DSCP values of the replicated packets. Their DSCP values are not copied from the incoming original packet, but from the additional DS field in the multicasting routing table entry for the corresponding output link (only the DSCP value must be copied, while the two remaining bits are ignored and are present for simplification reasons only). This Bless & Wehrle Expires: January 2003 [Page 16] Internet-Draft IP Multicast in DiffServ Networks July 2002 field contains initially the codepoint of the LE PHB if incoming packets for this specific group do not carry the codepoint of the default PHB. When a packet arrives with the default PHB, the outgoing replicates should also get the same codepoint in order to retain the behavior of todays common multicast groups using the default PHB. The SET_CODEPOINT message changes the DSCP values in the multicast routing table and may also carry the new DSCP value which should be set in the replicated packets. It should be noted that although re-marking may also be performed by interior nodes, the forwarding performance will not be decreased, because the decision when and what to re-mark is made by the management (control plane). Furthermore, there must be a mechanism for DiffServ nodes to inform a management entity about the join request of a new branch (something like the JOIN_INDICATION message). In order to keep the complexity of interior nodes low, this task may be preferably handled by boundary nodes. Additionally, a mechanism must be supplied for instructing a router to change the DSCP value for a specific branch of a multicast group (something like the SET_CODEPOINT message). This mechanism may be also incorporated into an existing multicast routing protocol as an extension. Multicast Other List Destination Fields of Address virtual Inter- DS interfaces face ID Field +--------------------------------+ +-------------------+ | X | .... | *-------------------->| C | (DSCP,CU) | |--------------------------------| +-------------------+ | Y | .... | *-----------+ | D | (DSCP,CU) | |--------------------------------| | +-------------------+ | ... | .... | ... | | . . . . | +-------------------+ . ... . .... . ... . +-------->| B | (DSCP,CU) | +--------------------------------+ +-------------------+ | ... | .... | ... | | D | (DSCP,CU) | +--------------------------------+ +-------------------+ | ... | ... | . . . . . . Figure 8: Multicast routing table with additional fields for DSCP values In summary, only those receivers will obtain a better service within a DiffServ multicast group, which previously reserved the Bless & Wehrle Expires: January 2003 [Page 17] Internet-Draft IP Multicast in DiffServ Networks July 2002 corresponding resources in the new branch with assistance of the management. Otherwise they get a quality which might be even lower than best-effort. 3.2 Solution for Supporting Heterogeneous Multicast Groups In this document considerations are currently limited to provision different service classes, but not different quality parameters within a certain service class. The proposed concept from section 3.1 provides also a limited solution of the heterogeneity problem. Receivers are allowed to obtain a Limited Effort service without a reservation, so that at least two different service classes within a multicast group are possible. Therefore, it is possible that any receiver may participate in the multicast session without getting any quality of service. This is useful if a receiver just wants to see whether the content of the multicast group is of interest for it, before requesting a better service which must be paid for (like snooping into a group without prior reservation). Alternatively, a receiver might not be able to receive this better quality of service (e.g., because it is mobile and uses a wireless link), but it may be satisfied with the reduced quality, instead of getting no content at all. Additionally, applying the RSVP concept of listening for PATH messages before sending any RESV message is now possible again. Without using the proposed solution this would have caused the NRS Problem. Theoretically, the proposed approach also supports more than two different services within one multicast group, because the additional field in the multicast routing table can store any DSCP value. However, this would work only if PHBs can be partially ordered, so that the "best" PHB among different required PHBs downstream is chosen to be forwarded on a specific link. This is mainly a management issue and out of scope for this document. 3.3 Solution for Any-Source Multicast Every participant would have to initiate an explicit reservation if it wants to make sure that it is possible to send with a better service quality to the group, regardless whether other senders within the group already use the same service class simultaneously. Bless & Wehrle Expires: January 2003 [Page 18] Internet-Draft IP Multicast in DiffServ Networks July 2002 This would require a separate reservation for each sender-rooted multicast tree. However, in the specific case of best-effort service (the default PHB), it is nevertheless possible for participants to send packets anytime to the group without requiring any additional mechanisms. The reason for this is that the first-hop router will mark those packets with the DSCP of the default PHB because of a missing traffic profile for this particular sender. First hop routers should therefore always classify multicast packets in dependence of the sender's address and multicast group address. 4 Scalability Considerations The proposed solution does not the extend the DS architecture or a DS router with additional complexity. Re-marking of packets in interior nodes is not considered as a scalability problem or to be in contradiction with the DiffServ approach itself, because a router has to manage and hold information about multicast flows anyway. Moreover, the decision when to change a re-marking policy is not performed by the router, but by some management entity at a time scale which is different from the time scale at the packet forwarding level. However, one may argue that there exists a scalability problem in holding necessary routing information for each multicast flow. But this problem applies to the nature of IP multicast itself. When a router is not capable of holding and managing a multicast routing table then IP multicast (in its current implementation) itself leads to a scalability problem. Thus, as long as IP multicast is considered to be scalable the herein proposed solution is also scalable. 5 Security Considerations Basically, the security considerations in [1] apply. The proposed solution does not really imply new security aspects. If it is not wanted that arbitrary end-systems can join a multicast group anytime (thereby receiving a lower than best-effort quality) the application has usually to preclude these participants by using authentication, authorization or ciphering techniques at application level just as for traditional IP multicast scenarios. On the one hand, instructing the router to set the codepoint value to a specific entry is naturally a powerful function which could be an objective for theft of service attacks. On the other hand, its security depends on the management mechanisms which are used to Bless & Wehrle Expires: January 2003 [Page 19] Internet-Draft IP Multicast in DiffServ Networks July 2002 realize this functionality. This management should generally be protected against unauthorized use, therefore preventing those attacks. 6 References [1] F. Baker, D. Black, S. Blake, and K. Nichols. Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers. RFC 2474, Dec. 1998. [2] S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, and W. Weiss. An Architecture for Differentiated Services. RFC 2475, Dec. 1998. [3] K. Nichols, B. Carpenter. Definition of Differentiated Services Per Domain Behaviors and Rules for their Specification. RFC 3086, Apr. 2001. [4] B. Davie, A. Charny, J.C.R. Bennett, K. Benson, J.Y. Le Boudec, W. Courtney, S. Davari, V. Firoiu, D. Stiliadis, An Expedited Forwarding PHB. RFC 3246, March 2002. [5] V. Jacobson, K. Nichols, and L. Zhang. A Two-bit Differentiated Services Architecture for the Internet. RFC 2638, July 1999. [6] R. Braden, S. Berson, S. Herzog, S. Jamin, and L. Zhang. Resource ReSerVation Protocol (RSVP) -- Version 1. RFC 2205, Sept. 1997. [7] Y. Bernet, Format of the RSVP DCLASS Object, RFC 2996, November 2000. [8] D. Waitzman, C. Partridge, and S. Deering. Distance Vector Multicast Routing Protocol. RFC 1075, Nov. 1988. [9] S. Deering, D. Estrin, D. Farinacci, V. Jacobson, A. Helmy, D. Meyer, and L. Wei. Protocol independent multicast version 2 dense mode specification. Internet-Draft -- draft-ietf-pim-v2- dm-03.txt, June 1999, work in progress. [10] D. Estrin, D. Farinacci, A. Helmy, D. Thaler, S. Deering, M. Handley, V. Jacobson, C. gung Liu, P. Sharma, and L. Wei. Protocol Independent Multicast-Sparse Mode (PIM-SM): Protocol Specification. RFC 2362, June 1998. [11] S. Deering, D. Estrin, D. Farinacci, V. Jacobson, A. Helmy, D. Meyer, and L. Wei. Protocol independent multicast version 2 Bless & Wehrle Expires: January 2003 [Page 20] Internet-Draft IP Multicast in DiffServ Networks July 2002 dense mode specification. Internet-Draft -- draft-ietf-pim-v2- dm-03.txt, June 1999, work in progress. [12] F. Baker, J. Heinanen, W. Weiss, and J. Wroclawski. Assured Forwarding PHB Group. RFC 2597, June 1999. [13] Y. Bernet, S. Blake, D. Grossman, A. Smith: An Informal Management Model for DiffServ Routers. RFC 3290, May 2002 [14] R. Bless, K. Wehrle: "Evaluation of Differentiated Services using an Implementation und Linux"; Proceedings of the Intern. Workshop on Quality of Service (IWQOS'99), London, 1999 [15] R. Bless, K. Wehrle. Group Communication in Differentiated Services Networks, Internet QoS for the Global Computing 2001 (IQ 2001), IEEE International Symposium on Cluster Computing and the Grid), May 2001, Brisbane, Australia, IEEE Press [16] K. Wehrle, J. Reber, V. Kahmann. A simulation suite for Internet nodes with the ability to integrate arbitrary Quality of Service behavior, Proceedings of Communication Networks And Distributed Systems Modeling And Simulation Conference (CNDS 2001), Phoenix (AZ), January 2001 7 Acknowledgements The authors wish to thank all the people from the Institute of Telematics (University of Karlsruhe) and those from the DiffServ community who contributed to the discussion of all the topics related to this document. Special thanks go to Milena Neumann for her extensive efforts in performing the simulations. We would also like to thank the KIDS simulation team [16]. 8 Authors' Addresses Comments and questions related to this document can be addressed to one of the authors listed below. Roland Bless Institute of Telematics Universitaet Karlsruhe (TH) Zirkel 2 D-76128 Karlsruhe, Germany Bless & Wehrle Expires: January 2003 [Page 21] Internet-Draft IP Multicast in DiffServ Networks July 2002 Phone: +49 721 608 6413 Email: bless@tm.uka.de Klaus Wehrle Institute of Telematics Universitaet Karlsruhe (TH) Zirkel 2 D-76128 Karlsruhe, Germany Phone: +49 721 608 6414 Email: wehrle@tm.uka.de Bless & Wehrle Expires: January 2003 [Page 22] Internet-Draft IP Multicast in DiffServ Networks July 2002 Appendix A Proof of the Neglected Reservation Subtree Problem In the following, it is shown that the NRS problem actually exists and occurs in reality. Hence, we investigated the problem and its solution using a standard Linux Kernel (v2.3.49) and the Linux-based implementation KIDS, which is described in an early version detailed in [14]. Furthermore, we implemented the proposed solution for the NRS problem by enhancing the multicast routing table as well as the multicast routing behavior in the Linux kernel. In the following section, the modification is briefly described. Additional measurements with the simulation model simulatedKIDS [16] will be presented in appendix B. They show the effects of the NRS problem more detailed and also the behavior of the BAs using or not using the Limited Effort PHB for re-marking. A.1 Implementation of the proposed solution As described in section 3.1, the proposed solution for avoiding the NRS Problem is just adding one byte to the routing table entries in each Multicast router. In the Linux OS the multicast routing table is implemented by the "Multicast Forwarding Cache (MFC)". The MFC is a hash table consisting of an "mfc-cache"-entry for each combination of the following three parameters: sender's IP address, multicast group address and incoming interface. The routing information in a "mfc-cache"-entry is kept in an array of TTLs for each virtual interface. When the TTL is zero, a packet matching to this "mfc-cache"-entry will not be forwarded on this virtual interface. Otherwise, if the TTL is less than the packet's TTL, the latter will be forwarded on the interface with a decreased TTL. In order to set an appropriate codepoint if bandwidth is allocated on an outgoing link, we added a second array of bytes -- similar to the TTL array -- for specifying the codepoint that should be used on a particular virtual interface. The first six bits of the byte contain the DSCP that should be used and the seventh bit indicates, whether the original codepoint in the packet has to be changed to the specified one (=0) or has to be left unchanged (=1). The Bless & Wehrle Expires: January 2003 [Page 23] Internet-Draft IP Multicast in DiffServ Networks July 2002 default entry of the codepoint byte is zero, so initially all packets will be re-marked to the default DSCP. Furthermore, we modified the multicast forwarding code for considering this information while replicating multicast packets. To change an "mfc-cache"-entry we implemented a daemon for exchanging the control information (e.g. JOIN-INDICATION - and SET-CODEPOINT- messages) with a management entity (e.g., a bandwidth broker). Currently, the daemon uses a proprietary protocol, but it is planned to migrate to the COPS protocol (RFC 2748). A.2 Test Environment and Execution Sender +---+ FHN: First Hop Node | S | BN: Boundary Node +---+ +# +# +# +---+ +--+ +------+ |FHN|++++++++++++|BN|+++++++++++| host | | |############| |***********| B | +---+ +--+## +------+ +# # +# # +# # +------+ +------+ |host A| |host C| +------+ +------+ +++ EF flow (group1) with reservation ### EF flow (group2) with reservation *** EF flow (group2) without reservation Figure A.1: Evaluation of NRS-Problem described in Figure 3 In order to prove NRS problem case 1, as described in section 2.1, a testbed shown in Figure A.1 was built. It is a reduced version of the network shown in Figure 5 and consists of two DS-capable routers, a first-hop router and an egress boundary router. The absence of interior routers does not have any effects on to the proof of the described problem. The testbed comprises of two Personal Computers (Pentium III at 450 Mhz, 128 MB Ram, 3 network cards Intel eepro100) used as DiffServ routers, as well as one sender and three receiver systems (also Bless & Wehrle Expires: January 2003 [Page 24] Internet-Draft IP Multicast in DiffServ Networks July 2002 PCs). On the routers KIDS has been installed and a mrouted (Multicast Routing Daemon) was used to perform multicast routing. The network was completely built of separate 10BaseT Ethernet segments in full-duplex mode. In [14] we evaluated the performance of the software routers and found out that even a PC at 200Mhz had no problem to handle up to 10Mbps DS traffic on each link. Therefore, the presented measurements are not a result of performance bottlenecks caused by these software routers. The sender generated two shaped UDP traffic flows of 500kbps (packets of 1.000 byte constant size) each and sends them to multicast group 1 (233.1.1.1) and 2 (233.2.2.2). In both measurements receiver A had a reservation along the path to the sender for each flow, receiver B has reserved for flow 1 and C for flow 2. Therefore, two static profiles are installed in the first- hop router with 500kbps EF bandwidth and a token bucket size of 10.000byte for each flow. In the egress boundary router one profile has been installed for the output link to host B and one related for the output link to host C. Each of them permits up to 500kbps Expedited Forwarding, but only the aggregate of Expedited Forwarding traffic carried on the outgoing link is considered. In measurement 1 the hosts A and B joined to group 1 and A, B and C joined to group 2. Those joins are using a reservation for the group towards the sender. Only the join of host B to group 2 has no admitted reservation. As described in section 2.1 this will cause the NRS problem (case 1). Metering and policing mechanisms in the egress boundary router throttle down the EF aggregate to the reserved 500kbps, no depending on whether individual flows have reserved or not. +--------+--------+--------+ | Host A | Host B | Host C | +---------+--------+--------+--------+ | Group 1 | 500kbps| 250kbps| 500kbps| +---------+--------+--------+--------+ | Group 2 | 500kbps| 250kbps| | +---------+--------+--------+--------+ Figure A.2: Results of measurement 1 (without the proposed solution): Average bandwidth of each flow. --> Flows of group 1 and 2 on the link to host B share the reserved aggregate of group 1. Bless & Wehrle Expires: January 2003 [Page 25] Internet-Draft IP Multicast in DiffServ Networks July 2002 Figure A.2 shows the obtained results. Host A and C received their flows without any interference. But host B received data from group 1 only with half of the reserved bandwidth, so one half of the packets have been discarded. Figure A.2 also shows that receiver B got the total amount of bandwidth for group 1 and 2, that is exactly the reserved 500kbps. Flow 2 got Expedited Forwarding without actually having reserved any bandwidth and additionally violated the guarantee of group 1 on that link. For measurement 2 the previously presented solution (cf. section 3.1) has been installed in the boundary router. Now it checks during duplicating the packets, whether the codepoint has to be changed to Best-Effort (or Limited Effort) or whether it can be just duplicated. In this measurement it changed the codepoint for group 2 on the link to Host B to Best-Effort. +--------+--------+--------+ | Host A | Host B | Host C | +---------+--------+--------+--------+ | Group 1 | 500kbps| 500kbps| 500kbps| +---------+--------+--------+--------+ | Group 2 | 500kbps| 500kbps| | +---------+--------+--------+--------+ Figure A.3: Results of measurement 1 (with the proposed solution): Average bandwidth of each flow. --> Flow of group 1 on the link to host B gets the reserved bandwidth of group 1. The flow of group 2 has been re-marked to Best-Effort. Results of this measurement are presented in Figure A.3. Each host received its flows with the reserved bandwidth and without any packet loss. Packets from group 2 are re-marked in the boundary router so that they have been treated as best-effort traffic. In this case, they got the same bandwidth as the Expedited Forwarding flow, because there was not enough other traffic on the link present, and thus no need to discard packets. The above measurements confirm that the Neglected Reservation Subtree problem is to be taken seriously and that the presented solution will solve it. Bless & Wehrle Expires: January 2003 [Page 26] Internet-Draft IP Multicast in DiffServ Networks July 2002 B Simulative Study of the NRS Problem and Limited Effort PHB This section shows some results from a simulative study which shows the correctness of the proposed solution and the effect of re- marking the responsible flow to Limited Effort. A proof of the NRS problem has also been given in appendix A and in [15]. This section shows the benefit for the default Best Effort traffic when Limited Effort is used for re-marking instead of Best Effort. The results strongly motivate the use of Limited Effort. B.1 Simulation Scenario In the following scenario Boundary Routers had a link speed of 10 Mpbs and Interior Routers had a link speed of 12 Mbps. In Boundary Routers a 5 Mbps aggregate for EF has been reserved. When Limited Effort was used, LE got 10% capacity (0.5Mpbs) from the original BE aggregate and BE 90% (4.5Mbps) of the original BE aggregate capacity. The bandwidth between LE and BE is shared by using WFQ scheduling. The following topology was used, where Sx is a sender, BRx a Boundary Router, IRx an Interior Router and Dx a destination/receiver. +--+ +--+ +---+ +--+ |S1| |S0| /=|BR5|=====|D0| +--+ +--+ // +---+ +--+ \\ || // \\ || // +--+ \+---+ +---+ +---+ +---+ +--+ |S2|===|BR1|=====|IR1|=====|IR2|======|BR3|=====|D1| +--+ +---+ /+---+ +---+ +---+ +--+ // \\ +--+ // \\ /=|D2| +--+ +---+ // \\ // +--+ |S3|===|BR2|=/ +---+/ +--+ +---+ /=|BR4|=\ || +--+ // +---+ \\ +--+ +--+ |D4|=/ \=|D3| |S4| +--+ +--+ +--+ Figure B.1: Simulation Topology The following table shows the flows in the simulation runs, e.g., EF0 is sent from Sender S0 to Destination D0 with a rate of 4 Mbps using an EF reservation. Bless & Wehrle Expires: January 2003 [Page 27] Internet-Draft IP Multicast in DiffServ Networks July 2002 In the presented cases (I to IV) different amounts of BE traffic were used to show the effects of Limited Effort in different cases. The intention of these four cases is described after the table. In all simulation models EF sources generated constant rate traffic with constant packet sizes using UDP. The BE sources also generated constant rate traffic, where BE0 used UDP and BE1 used TCP as transport protocol. +----+--------+-------+----------+-----------+-----------+---------+ |Flow| Source | Dest. | Case I | Case II | Case III | Case IV | +----+--------+-------+----------+-----------+-----------+---------+ | EF0| S0 | D0 | 4 Mbps | 4 Mbps | 4 Mbps | 4 Mbps | +----+--------+-------+----------+-----------+-----------+---------+ | EF1| S1 | D1 | 2 Mbps | 2 Mbps | 2 Mbps | 2 Mbps | +----+--------+-------+----------+-----------+-----------+---------+ | EF2| S2 | D2 | 5 Mbps | 5 Mbps | 5 Mbps | 5 Mbps | +----+--------+-------+----------+-----------+-----------+---------+ | BE0| S3 | D3 | 1 Mbps | 2.25 Mbps | 0.75 Mbps |3.75 Mbps| +----+--------+-------+----------+-----------+-----------+---------+ | BE1| S4 | D4 | 4 Mbps | 2.25 Mbps | 0.75 Mbps |3.75 Mbps| +----+--------+-------+----------+-----------+-----------+---------+ Table B.1: Direction, amount and Codepoint of flows in the four simulation cases (case I to IV) The four cases (I to IV) used in the simulation runs had the following characteristics: Case I: In this scenario the BE sources sent together exactly 5 Mbps so there is no congestion in the BE queue. Case II: BE is sending less than 5 Mbps, so there is space available in the BE queue for re-marked traffic. BE0 and BE1 are sending together 4.5 Mbps, which is exactly the share of BE, when LE is used. So when multicast packets are re-marked to LE because of the NRS problem, then LE should get 0.5 Mbps and BE 4.5 Mbps, which is still enough for BE0 and BE1. LE should not show a greedy behavior and should not use resources from BE. Case III: In this case BE is very low. BE0 and BE1 use together only 1.5 Mbps. So when LE is used, it should be able to use the unused bandwidth resources from BE. Case IV: BE0 and BE1 send together 7.5 Mbps so there is congestion in the BE queue. In this case LE should get 0.5 Mbps (not more and not less). Bless & Wehrle Expires: January 2003 [Page 28] Internet-Draft IP Multicast in DiffServ Networks July 2002 In each scenario loss rate and throughput of the considered flows and aggregates have been metered. B.2 Simulation Results for different router types B.2.1 Interior Router When the branching point of a newly added multicast subtree is located in an Interior Router the NRS problem can occur as described in section 2.1 (Case 2). In the simulation runs presented in the following four subsections D3 joins to the multicast group of sender S0 without making any reservation or resource allocation. Consequently a new branch is added to the existing multicast tree. The branching point issued by the join of D3 is located in IR2. On the link to BR3 no bandwidth was allocated for the new flow (EF0). The metered throughput of flows on the link between IR2 and BR3 in the four different cases is shown in the following four subsections. The situation before the new receiver joins is shown in the second column. The situation after the join without the proposed solution is shown in column three. The fourth column presents the results when the proposed solution of section 3.1 is used and the responsible flow is re-marked to LE. Bless & Wehrle Expires: January 2003 [Page 29] Internet-Draft IP Multicast in DiffServ Networks July 2002 B.2.1.1 Case I: +--------+-----------------+-----------------+------------------+ | | before join | after join |after join, | | | | (no re-marking) |(re-marking to LE)| +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: 4.007 Mbps | LE0: 0.504 Mbps | |achieved| EF1: 2.001 Mbps | EF1: 2.003 Mbps | EF1: 2.000 Mbps | |through-| EF2: 5.002 Mbps | EF2: 5.009 Mbps | EF2: 5.000 Mbps | |put | BE0: 1.000 Mbps | BE0: 0.601 Mbps | BE0: 1.000 Mbps | | | BE1: 4.000 Mbps | BE1: 0.399 Mbps | BE1: 3.499 Mbps | +--------+-----------------+-----------------+------------------+ |BA | EF: 7.003 Mbps | EF: 11.019 Mbps | EF: 7.000 Mbps | |through-| BE: 5.000 Mbps | BE: 1.000 Mbps | BE: 4.499 Mbps | |put | LE: --- | LE: --- | LE: 0.504 Mbps | +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: 0 % | LE0: 87.4 % | |packet | EF1: 0 % | EF1: 0 % | EF1: 0 % | |loss | EF2: 0 % | EF2: 0 % | EF2: 0 % | |rate | BE0: 0 % | BE0: 34.8 % | BE0: 0 % | | | BE1: 0 % | BE1: 59.1 % | BE1: 0 % | +--------+-----------------+-----------------+------------------+ (*) EF0 is re-marked to LE and signed as LE0 B.2.1.2 Case II: +--------+-----------------+-----------------+------------------+ | | before join | after join |after join, | | | | (no re-marking) |(re-marking to LE)| +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: 4.003 Mbps | LE0: 0.500 Mbps | |achieved| EF1: 2.000 Mbps | EF1: 2.001 Mbps | EF1: 2.001 Mbps | |through-| EF2: 5.002 Mbps | EF2: 5.005 Mbps | EF2: 5.002 Mbps | |put | BE0: 2.248 Mbps | BE0: 0.941 Mbps | BE0: 2.253 Mbps | | | BE1: 2.252 Mbps | BE1: 0.069 Mbps | BE1: 2.247 Mbps | +--------+-----------------+-----------------+------------------+ |BA | EF: 7.002 Mbps | EF: 11.009 Mbps | EF: 7.003 Mbps. | |through-| BE: 4.500 Mbps | BE: 1.010 Mbps | BE: 4.500 Mbps | |put | LE: --- | LE: --- | LE: 0.500 Mbps | +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: 0 % | LE0: 87.4 % | |packet | EF1: 0 % | EF1: 0 % | EF1: 0 % | |loss | EF2: 0 % | EF2: 0 % | EF2: 0 % | |rate | BE0: 0 % | BE0: 58.0 % | BE0: 0 % | | | BE1: 0 % | BE1: 57.1 % | BE1: 0 % | +--------+-----------------+-----------------+------------------+ (*) EF0 is re-marked to LE and signed as LE0 Bless & Wehrle Expires: January 2003 [Page 30] Internet-Draft IP Multicast in DiffServ Networks July 2002 B.2.1.3 Case III: +--------+-----------------+-----------------+------------------+ | | before join | after join |after join, | | | | (no re-marking) |(re-marking to LE)| +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: 3.998 Mbps | LE0: 3.502 Mbps | |achieved| EF1: 2.000 Mbps | EF1: 2.001 Mbps | EF1: 2.001 Mbps | |through-| EF2: 5.000 Mbps | EF2: 5.002 Mbps | EF2: 5.003 Mbps | |put | BE0: 0.749 Mbps | BE0: 0.572 Mbps | BE0: 0.748 Mbps | | | BE1: 0.749 Mbps | BE1: 0.429 Mbps | BE1: 0.748 Mbps | +--------+-----------------+-----------------+------------------+ |BA | EF: 7.000 Mbps | EF: 11.001 Mbps | EF: 7.004 Mbps | |through-| BE: 1.498 Mbps | BE: 1.001 Mbps | BE: 1.496 Mbps | |put | LE: --- | LE: --- | LE: 3.502 Mbps | +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: 0 % | LE0: 12.5 % | |packet | EF1: 0 % | EF1: 0 % | EF1: 0 % | |loss | EF2: 0 % | EF2: 0 % | EF2: 0 % | |rate | BE0: 0 % | BE0: 19.7 % | BE0: 0 % | | | BE1: 0 % | BE1: 32.6 % | BE1: 0 % | +--------+-----------------+-----------------+------------------+ (*) EF0 is re-marked to LE and signed as LE0 B.2.1.4 Case IV: +--------+-----------------+-----------------+------------------+ | | before join | after join |after join, | | | | (no re-marking) |(re-marking to LE)| +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: 4.001 Mbps | LE0: 0.500 Mbps | |achieved| EF1: 2.018 Mbps | EF1: 2.000 Mbps | EF1: 2.003 Mbps | |through-| EF2: 5.005 Mbps | EF2: 5.001 Mbps | EF2: 5.007 Mbps | |put | BE0: 2.825 Mbps | BE0: 1.000 Mbps | BE0: 3.425 Mbps | | | BE1: 2.232 Mbps | BE1: --- | BE1: 1.074 Mbps | +--------+-----------------+-----------------+------------------+ |BA | EF: 7.023 Mbps | EF: 11.002 Mbps | EF: 7.010 Mbps | |through-| BE: 5.057 Mbps | BE: 1.000 Mbps | BE: 4.499 Mbps | |put | LE: --- | LE: --- | LE: 0.500 Mbps | +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: 0 % | LE0: 75.0 % | |packet | EF1: 0 % | EF1: 0 % | EF1: 0 % | |loss | EF2: 0 % | EF2: 0 % | EF2: 0 % | |rate | BE0: 23.9 % | BE0: 73.3 % | BE0: 0 % | | | BE1: 41.5 % | BE1: --- | BE1: 0 % | +--------+-----------------+-----------------+------------------+ (*) EF0 is re-marked to LE and signed as LE0 NOTE: BE1 has undefined throughput and loss in situation "after join Bless & Wehrle Expires: January 2003 [Page 31] Internet-Draft IP Multicast in DiffServ Networks July 2002 (no re-marking)", because TCP is going into retransmission back-off timer phase and closes the connection after 512 seconds. B.2.2 Boundary Router When the branching point of a newly added multicast subtree is located in a Boundary Router the NRS problem can occur as described in section 2.1 (Case 1). In the simulation runs presented in the following four subsections D3 joins to the multicast group of sender S1 without making any reservation or resource allocation. Consequently, a new branch is added to the existing multicast tree. The branching point issued by the join of D3 is located in BR3. On the link to BR4 no bandwidth was allocated for the new flow (EF1). The metered throughput of the flows on the link between BR3 and BR4 in the four different cases is shown in the following four subsections. The situation before the new receiver joins is shown in the second column. The situation after the join but without the proposed solution is shown in column three. The fourth column presents results when the proposed solution of section 3.1 is used and the responsible flow is re-marked to LE. B.2.2.1 Case I: +--------+-----------------+-----------------+------------------+ | | before join | after join |after join, | | | | (no re-marking) |(re-marking to LE)| +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: --- | EF0: --- | |achieved| EF1: --- | EF1: 1.489 Mbps | LE1: 0.504 Mbps | |through-| EF2: 5.002 Mbps | EF2: 3.512 Mbps | EF2: 5.002 Mbps | |put | BE0: 1.000 Mbps | BE0: 1.000 Mbps | BE0: 1.004 Mbps | | | BE1: 4.000 Mbps | BE1: 4.002 Mbps | BE1: 3.493 Mbps | +--------+-----------------+-----------------+------------------+ |BA | EF: 5.002 Mbps | EF: 5.001 Mbps | EF: 5.002 Mbps | |through-| BE: 5.000 Mbps | BE: 5.002 Mbps | BE: 4.497 Mbps | |put | LE: --- | LE: --- | LE: 0.504 Mbps | +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: --- | EF0: --- | |packet | EF1: --- | EF1: 25.6 % | LE1: 73.4 % | |loss | EF2: 0 % | EF2: 29.7 % | EF2: 0 % | |rate | BE0: 0 % | BE0: 0 % | BE0: 0 % | | | BE1: 0 % | BE1: 0 % | BE1: 0 % | +--------+-----------------+-----------------+------------------+ (*) EF1 is re-marked to LE and signed as LE1 B.2.2.2 Case II: Bless & Wehrle Expires: January 2003 [Page 32] Internet-Draft IP Multicast in DiffServ Networks July 2002 +--------+-----------------+-----------------+------------------+ | | before join | after join |after join, | | | | (no re-marking) |(re-marking to LE)| +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: --- | EF0: --- | |achieved| EF1: --- | EF1: 1.520 Mbps | LE1: 0.504 Mbps | |through-| EF2: 5.003 Mbps | EF2: 3.482 Mbps | EF2: 5.002 Mbps | |put | BE0: 2.249 Mbps | BE0: 2.249 Mbps | BE0: 2.245 Mbps | | | BE1: 2.252 Mbps | BE1: 2.252 Mbps | BE1: 2.252 Mbps | +--------+-----------------+-----------------+------------------+ |BA | EF: 5.003 Mbps | EF: 5.002 Mbps | EF: 5.002 Mbps | |through-| BE: 4.501 Mbps | BE: 4.501 Mbps | BE: 4.497 Mbps | |put | LE: --- | LE: --- | LE: 0.504 Mbps | +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: --- | EF0: --- | |packet | EF1: --- | EF1: 24.0 % | LE1: 74.8 % | |loss | EF2: 0 % | EF2: 30.4 % | EF2: 0 % | |rate | BE0: 0 % | BE0: 0 % | BE0: 0 % | | | BE1: 0 % | BE1: 0 % | BE1: 0 % | +--------+-----------------+-----------------+------------------+ (*) EF1 is re-marked to LE and signed as LE1 B.2.2.3 Case III: +--------+-----------------+-----------------+------------------+ | | before join | after join |after join, | | | | (no re-marking) |(re-marking to LE)| +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: --- | EF0: --- | |achieved| EF1: --- | EF1: 1.084 Mbps | LE1: 2.000 Mbps | |through-| EF2: 5.001 Mbps | EF2: 3.919 Mbps | EF2: 5.000 Mbps | |put | BE0: 0.749 Mbps | BE0: 0.752 Mbps | BE0: 0.750 Mbps | | | BE1: 0.749 Mbps | BE1: 0.748 Mbps | BE1: 0.750 Mbps | +--------+-----------------+-----------------+------------------+ |BA | EF: 5.001 Mbps | EF: 5.003 Mbps | EF: 5.000 Mbps | |through-| BE: 1.498 Mbps | BE: 1.500 Mbps | BE: 1.500 Mbps | |put | LE: --- | LE: --- | LE: 2.000 Mbps | +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: --- | EF0: --- | |packet | EF1: --- | EF1: 45.7 % | LE1: 0 % | |loss | EF2: 0 % | EF2: 21.7 % | EF2: 0 % | |rate | BE0: 0 % | BE0: 0 % | BE0: 0 % | | | BE1: 0 % | BE1: 0 % | BE1: 0 % | +--------+-----------------+-----------------+------------------+ (*) EF1 is re-marked to LE and signed as LE1 Bless & Wehrle Expires: January 2003 [Page 33] Internet-Draft IP Multicast in DiffServ Networks July 2002 B.2.2.4 Case IV: +--------+-----------------+-----------------+------------------+ | | before join | after join |after join, | | | | (no re-marking) |(re-marking to LE)| +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: --- | EF0: --- | |achieved| EF1: --- | EF1: 1.201 Mbps | LE1: 0.500 Mbps | |through-| EF2: 5.048 Mbps | EF2: 3.803 Mbps | EF2: 5.004 Mbps | |put | BE0: 2.638 Mbps | BE0: 2.535 Mbps | BE0: 3.473 Mbps | | | BE1: 2.379 Mbps | BE1: 2.536 Mbps | BE1: 1.031 Mbps | +--------+-----------------+-----------------+------------------+ |BA | EF: 5.048 Mbps | EF: 5.004 Mbps | EF: 5.004 Mbps | |through-| BE: 5.017 Mbps | BE: 5.071 Mbps | BE: 4.504 Mbps | |put | LE: --- | LE: --- | LE: 0.500 Mbps | +--------+-----------------+-----------------+------------------+ | | EF0: --- | EF0: --- | EF0: --- | |packet | EF1: --- | EF1: 40.0 % | LE1: 68.6 % | |loss | EF2: 0 % | EF2: 23.0 % | EF2: 0 % | |rate | BE0: 30.3 % | BE0: 32.1 % | BE0: 0 % | | | BE1: 33.3 % | BE1: 32.7 % | BE1: 0 % | +--------+-----------------+-----------------+------------------+ (*) EF1 is re-marked to LE and signed as LE1 Bless & Wehrle Expires: January 2003 [Page 34]