Inter-Domain Routing Internet Draft D. Turk Document: draft-turk-ertb-00.txt Bell Canada Expires: December 2002 June 2002 Enhanced Remotely Triggered Black holing using BGP Communities Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 except that the right to produce derivative works is not granted. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT","SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC-2119]. Abstract This document describes a technique that uses BGP communities to remotely trigger black holing of a particular destination network. Black holing can be applied on a selection of routers rather than all BGP speaking routers in the network. The document also describes a sinkhole tunnel technique using BGP communities and Tunnels to pull traffic into a sinkhole router for analysis. Turk Expires - December 2002 [Page 1] Enhanced Remotely Triggered Black holing June 2002 Table of Contents 1. Existing Remote Black holing Techniques 2 2. Enhanced Remotely Triggered Black holing Technique 3 3. Sinkhole tunnles 4 Security Considerations 6 Disclaimer 6 References 7 Acknowledgments 7 Author's Addresses 7 1. Existing Remote Black holing Techniques Current remotely triggered black holing techniques rely on announcing the target (destination) network address, experiencing a certain type of anomalies including "attack", into BGP. The BGP announcement is placed on one router participating in the BGP domain. The next hop address is altered in this announcement to point to an RFC1918 address. Most routers on the Internet, especially edge routers have static routes pointing RFC 1918 addresses to the null0 interface. When a BGP speaking router receives the update it will install the target prefix in its routing table with the next hop of one of the networks listed in RFC 1918. Consequently, the router will perform a route lookup to determine which interface the RFC 1918 address shall be forwarded to. Since the router has a static route pointing the RFC 1918 address to a null interface. Traffic destined to the targeted network gets dropped making the target unreachable to the attacker. This technique causes relief to the infrastructure from getting hit by the attack traffic load but on the other hand the targeted network becomes unreachable everywhere BGP is running. Even if a particular BGP speaking router does not have a static route pointing RFC 1918 to a null interface. The modified next hop makes the traffic un- routable to its legitimate destination. Granted, most ISPs usually do not leave the black hole on for the entire time of the attack. Instead, they enable black holing for a short period of time to drop traffic on all possible ingress points of a network. Relying on the fact that most routers are configured to send ICMP unreachable to the source(s) of the traffic informing the source that traffic was dropped. One of the source addresses would be hijacked to collect the ICMP unreachable packets. After the ICMP packets are gathered, sources address on the ICMP packets points out to border routers where traffic is entering. A network operator at this point may opt to stop the traffic on the routers where the traffic is entering. It is important to remember that only routers that receive actual traffic will send ICMP unreachable to the source of the network. Turk Expires - December 2002 [Page 2] Enhanced Remotely Triggered Black holing June 2002 There are several steps that an ISP might take to minimize the damage caused by the attack. This starts by black holing the target address and sucking back ICMP unreachable messages to identify ingress points. Then isolating interfaces and peer networks driving the attack into the Autonomous System. Finally installing an ACL, a rate- limiting policy or forwarding traffic to a null interface. Other techniques that might save time and effort are to utilize Netflow using an appliance that recognizes DoS attacks and identifies interfaces on the network where traffic is coming from. All those techniques relay on manually stopping attack traffic on respective routers. 2. Enhanced Remotely Triggered Black holing Technique This paper describes a technique developed to remotely trigger a selective set of routers to forward traffic destined to the victim network address to a null interface or, as discussed later to in this draft, to a sinkhole tunnel. This technique does not invoke an access list or Rate limiting policies to deal with attack traffic. The technique also does not involve changing the next hop of the victim network to an RFC1918 address all across the local AS like other techniques suggest. It will only change it on selected boxes with the aid of BGP communities for router selection. First, the ISP needs to assign a unique community value for each edge router that could potentially drive attack traffic to the victim. Taking an example of a small ISP network that consists of two border routers, R1 and R2. Assuming that the Autonomous System Number (ASN) for that ISP is 65001 then the ISP can assign community value 65001:1 to R1, community value 65001:2 for R2 and community value 65001:666 for both R1 and R2. After the assignment of communities has been done, the ISP has to do the following on all border boxes the attack could potentially come from, which in this example are R1 and R2. 1. Static route pointing an RFC1918 network to a null interface. 2. AS-Path access list that matches locally BGP generated prefix announcements. 3. BGP community access list to match the community value assigned by the ISP for a particular router (i.e. 65001:1 for R1). 4. BGP community access list to match the community value assigned by the IPS for all router (i.e. 65001:666 for R1 and R2) 5. Under the BGP process, an IBGP import route policy should be applied to do the following logic. (Statements are in a logical AND order) a. First statement to permit routes that match the following criteria and apply the following changes. Turk Expires - December 2002 [Page 3] Enhanced Remotely Triggered Black holing June 2002 i. Match for community specific to that router (i.e. 65001:1, for R1). ii. Match AS-Path to locally generated BGP announcements. iii. Set BGP next hop to an RFC1918 network. iv. Overwrite BGP community with the well-known community (no-advertise). b. First statement to permit routes that match the following criteria and apply the following changes. i. Match for community that covers all routers (i.e. 65001:666). ii. Match AS-Path to locally generated BGP announcements. iii. Set BGP next hop to an RFC1918 network. iv. Overwrite BGP community with the well-known community (no-advertise). After the policies have been configured on R1 and R2, the ISP can in the case of an attack, announce the targeted network in its BGP table with a community value associated with the router where the attack is arriving from and preserving the next hop to the actual next hop to the target. IBGP will then carry the announcement to all routers in the AS. All routers except the router that matches that community will be oblivious to the community value and will install the network address with its legitimate next hop. The router that matches the community will be the only one to install the network and alter the next hop to an RFC1918 network and then a null interface. The reason for matching for locally announced networks is to make sure that no EBGP customer can misuse this community to drive any network to a null interface. This technique stops traffic from getting forwarded to the legitimate destination on routers identified as transit routers for attack traffic. Thus, all other traffic on the network being (on network) or (off network) will still get forwarded to its legitimate destination. 3. Sinkhole tunnels Further to the Enhanced Remotely Triggered Black holing Technique, it may become a requirement to take a look at the attack traffic for further analysis. This requirement adds to the complexity of the exercise. Usually with broadcast interfaces, engineers install network sniffers on a spanned port and dump the traffic of the sniffer for analysis. Another method would be to send a network prefix that covers the attack host address(s) into BGP, altering the next hop to a sinkhole device that can log traffic for analysis. Those techniques result in taking down the services offered on the targeted IP addresses. Inter-AS traffic will be sucked into the sinkhole along with Intra-AS traffic. Turk Expires - December 2002 [Page 4] Enhanced Remotely Triggered Black holing June 2002 The concept of sinkhole tunnels becomes viable when the solutions usually involve taking the targeted IP address down when the need for packet level analysis arises. The idea of a sinkhole tunnel is like a garden hose or a short cut path. When traffic is pushed to enter one end of the tunnel, it will end up exiting at the other end. This concept is useful to forward traffic on a network where the targeted address next hop address is not to be tampered with. First, a sinkhole router is installed with optional sniffers attached to it, and then a tunnel is set up using for instance MPLS Traffic Engineering tunnel from all border routers a packet can potentially enter from (Inter-AS traffic) to the sinkhole router. This allows the use of the community technique of altering the next hop of the targeted prefix to a series of /30 subnets assigned to the tunnels with the tail end terminating at the sinkhole router. In other words, the border router alters the next hop of the targeted IP address to the address of the sinkhole router within the subnet of the tunnel that starts at the border router. All other routers within the AS are oblivious to the change made by announcing a prefix with the preset community described in section 2. If legitimate traffic is coming from a different part of the network, like another border router, the next hop would not be altered because of the community, since next hop altering route maps do not exist anywhere else. Attack traffic is then terminated at the sinkhole. If the requirement was not to kill the traffic but rather to analyze it and then send it back towards the router it was originally destined to, then by using a default network statement pointing to any interface attached to the network including the same physical interface the traffic came from. The traffic will be pushed out of an interface back to the network. Routing protocols will then take care of properly routing it to its original destination as if nothing had happened. This is possible because no router, other than the border router that has a match for the community, will take action based on that community. It becomes apparent that this technique can be used for purposes other than analyzing attack traffic. Legitimate traffic could be pulled out of normal routing into a slide or a tunnel and then reinserted onto the backbone without altering the addressing scheme. MPLS Traffic Engineering is a good way of sliding traffic to the sinkhole simply because of the many features you can have enabled for it. Features like QoS policies on the attack traffic so that it will not compete with legitimate traffic, hiding the extra hops caused by diverting the traffic by altering its TTL, and so on. Turk Expires - December 2002 [Page 5] Enhanced Remotely Triggered Black holing June 2002 To be able to alter the next hop on the border router, a subnet of an RFC1918 network is statically routed to the tunnel interface. Take for example 192.0.2.33/32 ip route 192.0.2.1 255.255.255.255 Tunnel0 Setting next hop of the target IP address to 192.0.2.1/32 will force the traffic to go through the tunnel. After the traffic is received on the sinkhole interface via TE tunnel, rate-limiting policies, QoS policies or access lists can be installed to rate limit or drop traffic classified as attack traffic. This could be done on the interface of the sinkhole. Another useful application for a sinkhole router is to pull in traffic via slide tunnels to an inbound interface and have a default route statement forwarding the traffic coming in from the tunnels to an Ethernet interface connected to the network where a packet sniffer is installed to further classify the attack traffic. This becomes very useful when it is not feasible to apply an Access list or a rate limiting statement on the border router or last hop router because of hardware or software limitations. Instead of upgrading interfaces where attack traffic will potentially enter from, traffic could be pulled into the sinkhole and treated on that box. Reduction in cost can be exercised if the sinkhole router is a powerful device. Security Considerations It is very important to practice tight control over BGP peering points before implementing this technique. BGP customers might be able to compromise a particular piece of the network using the Black holing communities. Making sure that a match for locally generated BGP announcement exist helps in limiting the ability to drive a network to a null interface to the local AS. Some other security considerations MUST be looked at based on the status of the AS those techniques are being implemented at. Disclaimer The views and specification here are those of the authors and are not necessarily those of their employers. The authors and their employers specifically disclaim responsibility for any problems arising from correct or incorrect implementation or use of this specification. Turk Expires - December 2002 [Page 6] Enhanced Remotely Triggered Black holing June 2002 References [1] B. Greene "ISP Security Essentials - Best Practice Cisco IOS and other Techniques to Help an ISP Survive in today's Internet, version 2.0", May, 2001 Acknowledgments The author of this document would like to acknowledge the developers of the remotely triggered black holing technique and the developers of the backscatter technique for collecting backscatter traffic. The author would also like to thank all members of the IP Engineering department at Bell Canada. Author's Addresses Doughan Turk Bell Canada 100 Wynford Drive Email: doughan.turk@bell.ca Turk Expires - December 2002 [Page 7]