INTERNET-DRAFT Kathleen M. Moriarty draft-moriarty-ddos-rid-02.txt MIT Lincoln Laboratory Expires: April 6, 2003 October 6, 2002 Distributed Denial of Service Incident Handling: Real-Time Inter-Network Defense Status of this Memo This document is an Internet-Draft and is subject to all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract One of the latest trends attacking Internet security is the increasing prevalence of Denial of Service (DoS) attacks. DoS attacks target system and/or network resources and seek to prevent valid access by consuming resources. Internet Service Providers (ISPs) need to be equipped and ready to assist in tracing these attacks with tools and procedures in place before the occurrence of a DoS attack. This paper proposes a proactive inter-network communication method to integrate existing tracing mechanisms across ISP boundaries to identify the source(s) of an attack. The various methods implemented to trace attacks must be coordinated both on the ISPs network as well as provide a communication mechanism across network borders. It is imperative that ISPs have quick communication methods defined to enable neighboring ISPs to assist in tracking a security incident across the Internet. This proposal makes use of current tracing practices for traffic and performance management, which could be extended for DoS incident handling. Policy guidelines for handling these incidents are recommended and can be used by Internet Service Providers (ISPs) and extended to their clients in conjunction with the technical recommendations. Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 TABLE OF CONTENTS Status of this Memo ................................................ 1 Abstract ........................................................... 1 1.1 Introduction ............................................... 3 2.1 Recommended Internet Service Provider (ISP) Technologies ... 4 3.1 Tracing Denial of Service Incidents ........................ 5 3.2 Tracing a Single Source Attack with Filters ................ 5 3.3 Tracing a Distributed attack ............................... 6 3.4 Trace Approaches ........................................... 7 3.4.1 Trace approach via Traffic Flow Analysis ............. 8 3.4.2 Trace Approach via Hash-Based IP Traceback ........... 11 3.5 Correlating Collected Data ................................. 12 4.1 Communication amongst ISPs ................................. 13 4.2 Inter-ISP NMS Messaging .................................... 14 4.3 Message Formats ............................................ 15 4.3.1 Trace Request ........................................ 15 4.2.2 Trace Authorization Message .......................... 16 4.3.3 Source Found Message ................................. 17 4.3.4 Example Upstream Trace ............................... 18 4.4 Message Structure: ......................................... 18 4.5 Message Delivery and Security Considerations ............... 21 5.1 Security Considerations .................................... 21 6.1 Summary .................................................... 22 7.1 References ................................................. 23 8.1 Acknowledgements ........................................... 24 8.2 Author Information ......................................... 24 Moriarty Expires: April 6, 2003 [Page 2] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 1.1 Introduction The CERT Coordination Center published a paper in October, 2001 entitled, "Trends in Denial of Service Attack Technology"[6]. The paper outlined the behavior of Denial of Service attacks of both single-source and multiple-source origins. Denial of Service attacks attempt to consume bandwidth, processing power, or system resources. The bandwidth-based attacks flood systems with TCP, UDP or ICMP packets. The bandwidth or processing power based attacks may use variations on these packets, such as altering the source address, port numbers, or TCP options. The source address might be altered in an ICMP attack where packets are sent through a site with the IP direct broadcast option enabled on the site's router. The IP directed broadcast would amplify the number of packets sent To the victim, thus flooding the network and consuming bandwidth. Port numbers might be altered to enable packets to bypass packet filters. Other approaches are to use packets, which are targeting valid services hosted by the network since they must be permitted and the attack traffic cannot be deciphered from valid network traffic. Another type of attack is aimed at consuming system resources, as in the SYN flood attack. A SYN flood attack is a TCP attack where the initiator of the attack begins the TCP handshake sequence by sending a SYN packet to the destination server of the attack with a spoofed source address. The server responds with an acknowledgement of the SYN packet, but the response packet is sent to the spoofed source address, which does not accept the unexpected acknowledgment packet. Thus leaving the connection open on the server and consuming system resources. There are a limited number of connections a server can maintain, and once this limit is reached, valid connections are denied. DoS attacks are characterized by large amounts of traffic destined for particular Internet locations and can originate from multiple sources. An attack from multiple sources is known as a Distributed Denial of Service attack (DDoS). Because DDoS attacks can originate from multiple sources, tracing such an attack can be extremely difficult or nearly impossible. These attacks can be launched from systems across the Internet unified in their efforts or by compromised systems enlisted as 'zombies' that are controlled by servers, thereby providing anonymity to the controlling server of the attack. DDoS attacks do not necessarily spoof the source of an attack since there are a large number of source addresses, which make it difficult to trace anyway. Current approaches to mitigating the effects of DoS and DDoS attacks are aimed at attackers who seek to hide the origin of their attack by source address spoofing from multiple locations. Measures can be taken at ISP border routers providing ingress, egress, and broadcast filtering as a recommended best practice in RFC2827. These filters ensure that the traffic leaving and entering client locations contain valid source and destination addresses. Moriarty Expires: April 6, 2003 [Page 3] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 2.1 Recommended Internet Service Provider (ISP) Technologies ISPs typically manage and monitor their networks through a centralized network management system (NMS). This system usually performs trend analysis for bandwidth utilization and reports communication problems. As a part of the bandwidth trend analysis performed, denial of service traffic or unusually unexpected increases in bandwidth could also be reported. With the capability of seeing the entire network under management, the NMS could be utilized to trace to the origin of network traffic. This would enable ISPs to become proactive in handling denial of service attacks through the use of tools already in existence on their networks. If trend analysis is not already being performed for bandwidth utilization, this may require the use of an additional server to perform this function. Such a server would need to account for traffic redirection events resulting in bandwidth fluctuations due to networking problems in other areas of an ISP's backbone. Ideally, this system would have the ability to perform a trend analysis on the network to determine if there was an unusually large increase in traffic without explanation, such as a network event elsewhere on the backbone. Client events, such as the launch of a new web site or streaming video of a noteworthy news event should be programmable exceptions to the alert. Once it can be determined that the event may be significant, a trace back for the source of the increased traffic should be performed through analysis on the neighboring connections on the network. If possible, the trace may need to inspect packets to determine a pattern, which could assist reverse path identification. This may be accomplished by inspecting packet header information such as the source and destination IP addresses, ports, and protocol flags to determine if there is a way to distinguish them from other packets. A description of the incident along with any available automated trace data should trigger an alert to the ISPs security team for further investigation. The proactive monitoring for bandwidth related attacks could use trending analysis as a guideline to determine acceptable levels of traffic across the network. Unexplained and extended spikes in traffic would be a signal that a DoS attack may be in progress. Resource related attacks would be more difficult to detect. Methods such as trending the packet size of traffic to and from networks may be an indicator of this type of attack. If there is a large increase in small packets sent to and from a network, that may be an indicator of a SYN flood attack. TCP connections begin with small packets, but the session data over the established connection may be a mixture of large and small packets. A change in the size of packets to or from a network or host may be an indicator of a DoS attack using different types of traffic than normally seen in the monitored network. Moriarty Expires: April 6, 2003 [Page 4] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 3.1 Tracing Denial of Service Incidents DDoS attacks are difficult or nearly impossible to trace because of the nature of the attack. Some of the difficulties in tracing these attacks include O the attack originates from multiple sources, O the attack may include various types of traffic meant to consume server resources, such as a SYN flood attack without a significant increase in bandwidth utilization, O the type of traffic could include valid destination services, which cannot be blocked since they are essential services to business, such as DNS servers at an ISP or HTTP requests sent to an organization connected to the Internet, O the attack may utilize varying types of packets including TCP, UDP, ICMP, or other IP protocols. O the attack may use a very small number of packets from any particular source, thus making a trace after the fact nearly impossible. If the source(s) of the attack cannot be determined from tracing the increased bandwidth utilization, it may be possible to trace the traffic based on the type of packets seen by the client. This paper discusses methods that can be used to trace back to the source of an attack, the methods include filtering, packet hash comparisons, and packet flow analysis. 3.2 Tracing a Single Source Attack with Filters In 1997, the MCI Security team wrote DoSTrack [2], a program used to trace back through a network in order to identify the origin of a DoS attack, such as the SYN flood attack. The program started its tracing efforts at the ISP border router for a client and determined if the router was seeing the attack. If the router was passing the attack traffic, DoSTrack would then identify the interface on a Cisco router from which the traffic was originating. Once the next hop was identified, DosTrack would log into the next router in the upstream path of the attack traffic and set a filter to monitor for the traffic. If the router did not see the attack traffic, DoSTrack would stop its tracing efforts at that router. If the attack traffic were seen, it would continue tracing until the source of the traffic or a bordering ISP was identified. The traces were performed through the use of access control filters implemented on each router in the upstream path of the traffic. The filters were used to permit the suspicious traffic and then log the packet information, including the source interface on the router that the packet used to identify the next hop. Router Moriarty Expires: April 6, 2003 [Page 5] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 access control lists and logging are both resource intensive operations. The additional resources required to implement filters across a network are not available at many ISPs. In many cases ISPs maintain routers close to maximum capacity for normal operation. CenterTrack [2] was an improvement on DoSTrack eliminating the need for filtering in the center of the network, however it is still taxing on router resources and lacks the ability for Inter-network communication. This approach could help identify the source of an attack, but a less resource intensive approach is needed for many ISPs. Several other limitations include: O DosTrack was primarily used for tracing Denial of Service attacks from a single-source, thus the solution would not scale to a trace a multiple-source attack. O DosTrack was written specifically for Cisco routers. Other vendors, for example high-end routers used in multi-gigabit networks, may not even provide similar filtering options. O The DoSTrack program can only be used within a single ISPs backbone because of access capabilities to neighboring network's equipment. A solution to address attacks of a distributed nature, crossing multiple network backbones, and various network equipment is needed. The approach should consider the resource limitations of many Internet Service Providers. 3.3 Tracing a Distributed attack Tracing a DDoS attack is a very difficult problem. Since DDoS attacks may involve multiple sources with spoofed addresses, there may only be a small amount of traffic from each of the originating hosts making it difficult to trace back. The sources may also alternate the type of traffic as well as vary the sources from within the pool of servers launching the attack. Immediate action would need to be taken to have any hopes of locating the origin(s) of the attack with a near real-time trace. In order to identify a DoS attack, a client may notify its ISP that it is currently under attack. Automated methods might include analysis of traffic, which looks at statistics such as unexpected fluctuations in bandwidth or the size and types of packets sent between networks or hosts could be used. There is ongoing research in the area of detecting DoS and DDoS, and any effective techniques could be integrated with the tracing techniques described in this paper. Current research approaches range from methods such as detecting backscatter traffic [3], using a data structure for bandwidth attack detection [4], and monitoring Moriarty Expires: April 6, 2003 [Page 6] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 congestion through packet retransmission information [5]. Another detection method may include monitoring any changes in the size of packets sent to and from a network. For example, if a site that normally receives small packets and replies with large packets experiences a change in traffic pattern such as the sending and receiving of large amounts of either small or large packets, that may indicate a DoS attack. Once an attack has been detected through traffic analysis and bandwidth usage statistics, traces would have to quickly identify the various sources of the attack. Once an attack is detected, a generalized approach to trace back connections might include packet information such as destination IP address and any distinguishing marks of the traffic seen by the site during the attack. If a trace can identify the sources of a distributed attack, blocking the sources at the ISP level close to the attacker could be an immediate action to stopping the attack. In the case of a DDoS attack, further information may be obtained from the attacking computers as to the controller of the attack sending the 'zombies' control information to carry out the attack. This additional trace is beyond the scope of this paper, but may use additional tracing mechanisms such as sniffing the network to locate the controllers of the attack. Finding a faster and more efficient way to trace multiple sources of an attack is essential to combating DDoS attacks. The ability to quickly relay and act upon the trace information gathered is imperative to stopping DDoS attacks. Tracing multiple attack paths can also cause additional stress on the network and does not scale well. 3.4 Trace Approaches There have been many separate research initiatives to solve the problem of tracing upstream packets to detect the true source of attack traffic. Upstream packet tracing is currently confined to the borders of a network or a ISPs network. Traces require access to network equipment and resources, which limit a trace to a specific network. Once a trace reaches the boundaries of a network, the network manager or ISP adjacent in the upstream trace must be contacted in order to continue the trace. ISPs have been working on individual solutions to accomplish upstream tracing within their own network environment. The tracing mechanisms implemented thus far have included proprietary solutions requiring specific information such as IP packet header data or hash values of the attack packets in the case of the 'Hash Based IP Traceback'[8]. Other research solutions involve marking packets as explained in 'ICMP Traceback Messages'[10] and 'Practical Support for IP Traceback' [9]. The following sections outline some available solutions for implementing traceback within the confines of a network managed by a single entity. Later in the paper the Moriarty Expires: April 6, 2003 [Page 7] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 focus will be on the information needed to accomplish the trace and to make possible the Inter-ISP communication specified. 3.4.1 Trace approach via Traffic Flow Analysis Traffic flow analysis is used to monitor individual network traffic streams, such as a single TCP session beginning with the SYN packet and up to the final FIN ACK in a session. There have been a few efforts to standardize flow analysis for network management, one through the traffic flow management mib and another through Cisco's NetFlow. The 'Traffic Flow Management' RFC [RFC2720] was designed to provide management information such as behavior models, capacity planning, network performance, quality of service, and attribution of network usage to system administrators. NetFlow from Cisco [7] provides similar capabilities to the traffic flow mib, except that it is specific to IP traffic and has already been implemented for traffic management in Cisco equipment. Although NetFlow was developed by Cisco, it is also an open standard. The flow analysis in both implementations can monitor with a capture filter on source and destination addresses, the number of packets and the count of bytes in each flow, the originating interface of the traffic, and the upstream peer information. The upstream peer information is essential to tracing a spoofed packet back to the true origin. During a DoS, the destination IP address and port information could be used to detect the event while the source information could be used to trace the path to the origin of the attack. The packet and byte count variables keep a count of the packets in the bi-directional flow as well as the number of octets in a flow. This information could be correlated and used in network analysis for attack detection. There are several differences in the implementations and the monitor and capture capabilities of the two flow analysis implementations. NetFlow collects all packets and maintains the following information on packet flows which can be analyzed later: O Source and destination IP address O Source and destination TCP/User Datagram Protocol (UDP) ports O Type of service (ToS) O Packet and byte counts O Start and end timestamps O Input and output interface numbers O TCP flags and encapsulated protocol (TCP/UDP) O Routing information (next-hop address, source autonomous system (AS) number, destination AS number, source prefix mask, destination prefix mask) The flow management in RFC2720 is based on an SNMP mib extension. The monitoring device in the network uses a capture filter to determine what packets should be buffered until retrieved by a central monitoring system. The capture filter or monitoring Moriarty Expires: April 6, 2003 [Page 8] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 information can include source and destination information for various protocols, a start and end time for the packet flows, packet count and byte information for the flow, as well as detailed address information for the source and destination of the flow. RFC2720 has not been implemented widely and may not be a feasible option for flow analysis at this point in time. NetFlow is based specifically on IP flow analysis. In addition to filtering on IP address information, NetFlow also looks into TCP and UDP packets for port information as well as TCP flags, which can be very useful in tracing attack traffic. NetFlow also provides detailed peer information such as the source interface of the flow through a router and routing information such as the next hop and the Autonomous System (AS) peer information. In either case, the structures defined for traffic flow management could be extended for use in tracing DoS or DDoS incidents. The traffic flow management information could first be used in trend analysis of network behavior in order to detect anomalies such as bandwidth fluctuations or variations in the types of packet flows. An ISP's central management station could be used to coordinate a trace of suspicious network traffic and may also be responsible for the traffic flow analysis of data gathered via devices on the network running the traffic flow captures. The monitoring devices could be existing network equipment or promiscuous listening devices. In order to expedite the trace(s) of an attack, pre- defined filters can be used to quickly trace traffic as described in the traffic flow management specifications. A pre-defined filter might generically capture all TCP port 23 traffic, but would require a ustom filter to include destination address information. An SNMP approach using the Traffic Flow MIB or the NetFlow approach to detect DDoS may help to reduce the necessary resources used by network routers in comparison to a filter based approach. The traffic flow MIB was designed to be used for network performance measurement and took into consideration system resources in the implementation. A coordinated SNMP approach can provide the dynamic framework necessary for tracing attacks in near real-time because a central management station would control trace attempts and quickly move through the network identifying the source of an attack. A similar system might be used to alter capture filters through a network using NetFlow or a similar technology to trace attack traffic. The design of NetFlow also sought to reduce the necessary resources consumed in existing network equipment. Options to offload the resources to promiscuous devices aids in the reduction of resource utilization of backbone equipment. Flow analysis for DDoS attacks would be performed on potentially congested network segments where router resources may be dangerously low. Using promiscuous devices may be the only feasible answer without impacting the network further to perform a trace. Moriarty Expires: April 6, 2003 [Page 9] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 NetFlow differs from the flow mib in its collection method. NetFlow uses an export model to collect information from monitoring devices. The devices periodically send flow information to a predefined FlowCollector via UDP packets. The data is sent to the FlowCollector when a flow has completed or when the flow continues past a certain length of time. The communication mechanism between ISPs and their network management systems described later, provide a framework for dynamically sending traces to bordering networks to continue the search for the attack origins. The 'Traffic Flow Measurement' RFC2720 and RFC2722 outline a structure for monitoring flows in a network through the use of capture filters deployed on network devices throughout the network. In the case of NetFlow, flow collection is enabled for all traffic on a specific device interface with all data sent to a FlowCollector. The device could be a router or switch already in the network or an appliance-like device promiscuously monitoring the network using either flow analysis method on a broadcast segment or a mirrored port on a switch to the router. Another option would be a device used to promiscuously monitor all interfaces of a router using network taps. The monitoring device would use a network tap on each of the connections to the device being monitored to ensure the capture of all data to and from the router or switch. The information collected could then identify the source interface of traffic flows. The RFC2720 monitoring is performed by a network device with a defined filter, which watches for a specific flow of traffic. The capture filter can be predefined on the devices and managed by a central device collecting the data captured. A central network management station taking periodic snapshots of the flows of traffic through network devices could be extended for use in detecting abnormal behavior on the network, such as the occurrence of a denial of service attack. The NMS could also perform any post-processing and analysis of the captured data. The filters might be set to trace certain packets in the event of an attack or set to monitor normal traffic flows with periodic collection for trending analysis. NetFlow could be used to send continuous flow data to a FlowCollector for post-processing. The flow analysis could be used to detect attacks or for tracing attack traffic after the flows from particular sources have already completed. The monitoring device would incur added overhead when monitoring a flow, especially in the case of a high bandwidth network. Most ISPs seem to prefer using separate monitoring devices for any approach to tracing attack traffic. Several ISPs have already implemented their own proprietary tracing mechanism. Moriarty Expires: April 6, 2003 [Page 10] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 3.4.2 Trace Approach via Hash-Based IP Traceback BBN implemented a trace back solution, which collects hashes of IP packets across the network. The Hash-Based IP Traceback was designed specifically to trace attack traffic and achieve the following objectives O trace attacks after specific flows of the attack have completed O reduce storage requirements needed to save traceable packet data O provide a secure method to store packet captures on the Internet Hash-based IP traceback is another solution to provide the ability to trace attack traffic. By capturing all packets across the network and saving hash values for the IP header information that does not get altered as it traverses the network, attacks can be traced after the fact. Since hashes of IP header information are stored instead of the straight header information, privacy concerns are no longer an issue as might be the case with packet captures across the Internet. If a system used to store the packet captures was compromised, the data could not be used to identify what entities are 'talking' to each other on the Internet. BBN also considered how traces could be performed across a single network, for example an ISPs backbone. The solution divides the network up into regions, each with its own collection station. The trace might be initiated at a particular collection station where data for a specific router is stored. When the collection station traces through its database for the matches of particular hashes of IP packets, it follows the trace through the network equipment for its own region. The collection station then determines what bordering region was the next upstream source of the attack and the trace is continued at the next collection agent. The trace can continue until the source is identified or a neighboring network is identified as the upstream source of the attack. The upstream network must then be notified in some way in order to continue the trace. The upstream network may require the hash information if they use hash-based IP Traceback or they may require IP header information if another solution is in use. The upstream provider will want to look at its network and resources and decide if it would like to initiate a trace across their network. A possible solution for the upstream trace between bordering networks is discussed later. Hash-based tracing has some definite advantages over other solutions, such as the ability to store a larger amount of packets for a greater period of time by using 32-bit digests. However, there are some disadvantages as well. Additional equipment is needed to capture and store the hashed-based IP header information. Some ISPs have made investments in their current tracing capabilities and do not have the resources to deploy a new solution. Moriarty Expires: April 6, 2003 [Page 11] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 3.5 Correlating Collected Data Integrating this extended monitoring capability into the NMS would allow for the formulation of network behavior statistics and thus the detection of anomalous usage behavior. The network trend analysis capability could be used to detect unexplained spikes in bandwidth usage on the network as a whole, as well as, on specific networks in an ISP backbone or servers on a local network. The NMS would be aware of network outages that may result in traffic being redirected through alternate paths of the network, which would be an example of an explained spike in bandwidth. A client connected to an ISP may be hosting an event, such as a web cast. An abundance of valid connection attempts can also result in bringing a web site down, targeting either bandwidth or system resources. If an ISP was aware of such an event, it could set higher thresholds for bandwidth usage alerting during that period of time to prevent false alarms. The anomalies could be used as methods of detecting Denial of Service attacks. Since a centralized server would be recording all of the capture information from the network devices, it may be easier to track the source of an attack. If the current capture filters on the monitoring devices do not provide the necessary data to trace the attack, pre-defined or new, filters could be set on the monitoring devices to watch for the traffic flows known to be part of the attack. The peer information could be used by the NMS to dynamically set the new filters upstream in the reverse path of the traffic flow until the central monitoring station discovers the source. However, this may limit the trace to the boundaries of a specific network. Since the trace may have begun on a specific company's network, its border with the ISP may be the boundary where the tracking must then become the responsibility of the ISP because of access limitations. This would also be true in the case of a boundary between ISPs. When tracing back an attack, the ISP could elect to enable this tracking feature, and then poll the routers on their network for the types of traffic seen during the attack. In order for this trace of traffic to be meaningful for a DDoS attack, the IP header information or the IP hash values of the packet would be necessary as the traffic is traced back through the network. Using the traffic flow MIB the IP destination and possibly protocol information could be set in the monitoring devices for tracking via a filter, this may be a viable means to trace back traffic quickly through a network. DDoS attacks have many origins and traces such as this may only lead back to a client that is a very small part of the attack. However, once a few clients are detected as being part of the attack, methods could be used to determine the location of the controller of the client to further trace the true source of the attack. That trace may involve intrusion detection or monitoring on Moriarty Expires: April 6, 2003 [Page 12] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 the client and is beyond the scope of this paper. 4.1 Communication amongst ISPs Expediting the communication between ISPs is essential when responding to a security related incident such as a Denial of Service attack, which may cross network access points, (Internet backbones) between providers. As a result of the urgency involved in this inter-ISP security incident communication, there must be an effective system in place to facilitate the interaction. This communication policy or system should involve multiple means of communication to avoid a single point of failure. Email may be the best way to transfer information about the incident, packet traces, etc. However, email may not be received in a timely fashion or be acted upon with the same urgency as a phone call. Each ISP should dedicate a phone number to reach a member of their security incident response team. The phone number could be dedicated to inter-ISP incident communications and must be a hotline that provides a 24x7 live response. The phone line should reach someone who would have either the authority and expertise or the means to expedite the necessary action to investigate the incident. This may be a difficult policy to establish at smaller ISPs due to resource limitations; so another solution may be necessary. An outside group may be able to serve this function given the necessary access to the ISPs network. The outside resource should be able to mitigate or alleviate the financial and experience resource limitations. A technical solution may be to implement traffic flow meters, as described in the 'Traffic Flow Measurement: Architecture' document [RFC2722], that can be read by other ISPs or a neutral third party to facilitate the investigation of a denial of service attack. Meters can be read by multiple meter readers, which may limit the expense and provide the ability for a quick response to an attack. The meter uses a single active capture filter and the buffered data can be retrieved by any of the configured meter readers. Each ISP may want to maintain their own management station used for network monitoring and analysis. A neutral third party may have access to a specific group of capture filters that can be implemented dynamically as an event is traced beyond network borders. This could be a function of a central organization operating as a computer response team for the Internet as a whole. An alternative to permitting access to other network's equipment would be to create a standard messaging mechanism to enable NMS systems to communicate trace information to neighboring networks' Network Management Systems. The third party mentioned above may be used in this technical solution to assist in facilitating traces through smaller ISPs. The messaging mechanism may be a logical or physical out-of-band network to ensure the communication is secure Moriarty Expires: April 6, 2003 [Page 13] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 and unaffected by the state of the network under attack. This would also enable the individual ISPs to involve any intervention by the network operations staff to authorize the continuance of a trace through their network via a notification and alerting system. The out-of-band logical solution may be permanent virtual circuits configured with a small amount of bandwidth dedicated to NMS communications between ISPs. Procedures for incident handling need to be established and well known by anyone that may be involved in incident response. The procedures should also contain contact information for internal escalation procedures as well as external assistance groups such as CERT, GIAC, and the FBI. 4.2 Inter-ISP NMS Messaging In order to implement a messaging mechanism between NMS systems, a standard protocol and format is required to ensure inter- operability between vendors. The messages would have several requirements in order to be meaningful as they traversed multiple networks. The Real-Time Inter-Network Defense against DoS (RID DoS) provides the framework necessary for communication between networks involved in the trace back and mitigation of a DoS attack. There are several types of messages that would be needed to facilitate a trace across multiple networks. A message sent between NMS systems to request the continuance of a trace through a bordering network would require the information enumerated below. 1. Messages sent between network management stations would have to contain enough information to enable the network administrators to make a decision about the importance of continuing the trace. 2. The trace request message should also contain the filter or IP packet hash information needed to carry out the trace. 3. Contact information of the origin of the trace should be included. The contact information could be provided through the autonomous system number [RFC1930] whois information listed in the American Registry for Internet Numbers (ARIN) database. 4. Network path information to help prevent any routing loops through the network from perpetuating a trace. If an NMS receives a trace request containing its own information in the path, the trace must cease and the NMS should generate an alert to inform the network operations staff that a routing loop exists. 5. A unique identifier for a single attack should be used to correlate traces to multiple sources in a DDoS attack. Moriarty Expires: April 6, 2003 [Page 14] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 4.3 Message Formats The following section describes the four message types used to facilitate the communication between ISPs tracing an incident. The messages would be generated and received on Network Management Systems on the ISPs network. The fields in the messages are described following the message descriptions. 4.3.1 Trace Request Description: This message is sent to the Network Management Station next in the upstream trace. Message Type 1 Time Stamp Incident Identifier ASN for originating ISP Incident number based on incremental tracking Trace number - used for multiple traces of a single incident Confidence rating of detected Denial of Service attack (0-100) Level Meaning ______________________ 1 Low probability the detected attack occurred 100 Attack detected with 100 percent confidence rating Filter used to trace incident across meters in the network Number of NMS in the path listed in message (count of items in following list) Path information of Network Management Systems used in the trace Autonomous System Number and NMS IP address of Next Network Autonomous System Number and NMS IP address Current Network ... Autonomous System Number and NMS IP address of Originating Network A DDoS attack can have many sources resulting in multiple traces to locate the sources of the attack. It may be valid to continue multiple traces for a single attack. The path information would enable the administrators to determine if the exact trace had already passed through a single network. Moriarty Expires: April 6, 2003 [Page 15] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 4.2.2 Trace Authorization Message Description: This message is sent to the initiating NMS from the next upstream ISP's NMS to provide information on the trace status in the current network. Message Type 2 Time Stamp Incident Identifier ASN for originating ISP Incident number based on incremental tracking Trace number - used for multiple traces of a single incident Confidence rating of detected Denial of Service attack (0-100) Filter used to trace incident across meters in the network Trace Status 0 Pending 1 Approved 2 Denied Number of NMS in the path listed in message (count of items in following list) Path information of Network Management Systems used in the trace Autonomous System Number and NMS IP address of Next Network Autonomous System Number and NMS IP address Current Network ... Autonomous System Number and NMS IP address of Originating Network A message is sent back to the NMS that initiated the trace as a notification means. This message verifies that the next NMS in the path has received the message from the previous NMS in the path. This message also verifies that the trace is now continuing, stopped or pending in the next upstream network in the path of the trace. The pending status would be automatically generated after a 2-minute timeout without system predefined or administrator action taken to approve or disapprove the trace continuance. Moriarty Expires: April 6, 2003 [Page 16] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 4.3.3 Source Found Message Description: This message indicates that the source of the attack in this trace was located and is sent to the initiating NMS through the network of out-of-band NMS systems in the path of the trace. Message Type 3 Time Stamp Incident Identifier ASN for originating ISP Incident number based on incremental tracking Trace number - used for multiple traces of a single incident Action Taken (multiple selections permitted) Bit Meaning ______________________________________________ 0 No action at this time 1 Switch port blocked 2 Network segment blocked 3 Host blocked from Internet access 4 Protocol port used in attack blocked 5 Alert generated 6 Site notified 7 Other Number of contacts listed in packet Value should always be 1 for this message type Autonomous System Number and NMS IP address of the system, which located the source of the trace [Text field for True Source address information of attack and any additional information for the action taken] A message sent back to initiating NMS to notify the originating network administrators that the source has been located. The actual source information may or may not be included depending on the policy of the network in which the client or host is attached. Any action taken by the ISP to act upon the discovery of the source of a trace should be included. The ISP may be able to automate the adjustment of filters at their border router to block outbound access for the machine(s) discovered as a part of the attack. The filters may be as comprehensive as to block all Internet access until the host has removed any Trojan or zombie on the system. Moriarty Expires: April 6, 2003 [Page 17] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 4.3.4 Example Upstream Trace The diagram below outlines the RID-DOS communication between NMS systems on different networks tracing an attack. Attack Dest ISP-1 ISP-2 ISP-3 Attack Src 1. Attack | Attack reported | detected 2. Initiate trace 3. Locate origin through upstream ISP. 4. o---Msg-Type-1------> 5. Trace Initiated 6. <-----Msg-Type-2----o 7. Locate origin through upstream ISP. 8. o---Msg-Type-1---> 9. Trace Initiated 10. <------------Msg-Type-2------------o 11. Locate attack source on network X 12. <------------Msg-Type-3------------o The ISP who detected the attack initiates the trace and the attack is traced to the source or the next upstream ISP. This process continues until the trace identifies the source of the attack. Any type 2 and 3 messages must pass through all NMS systems in the path back to the initiator of the trace because of the secure connections established between NMS systems of border networks. 4.4 Message Structure: The TCP protocol would be a good candidate to provide a reliable transport mechanism for the messages sent between network management systems. The application for RID DoS would operate on a port assigned by the Internet Assigned Numbers Authority (IANA). The message structure for the data portion of the packet is detailed below. Message types 1 and 2 share the same structure. Message type 3 differs after the incident identifier. The first 88 bytes include the consistent fields used in each packet type: ------------------------------------------------------------------- | Msg Type | Time Stamp | Incident Identifier | ------------------------------------------------------------------- 8-bit 32-bit 48-bit Field 1: 8-bit field for Message Type containing the number Moriarty Expires: April 6, 2003 [Page 18] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 associated with the message type given in the above descriptions Field 2: 32-bit field for Time Stamp containing the current time in seconds since January 1,1970 Greenwich Mean Time from the sender of the packet Field 3: 48-bit field for Incident Identifier 16-bit field Autonomous System Number (ASN) 16-bit field for Incident number, rolls back to zero 16-bit field for Trace number Field 4: 8-bit field depending on message type Percentage for Confidence rating in message type 1 and 2 Bit set for Action taken (bit set to 1 if selected option) in message type 3 Field 5: 424-bit field for Filter used in message type 1 and 2. Note: This field is unnecessary in message type 3. Must include space for all items listed below and unused items would be left blank for specific filters. All non-changing fields of the IP header are included as well as 8 bytes from the payload to meet the needs of the Hash-based IP Traceback. The protocol and port information is listed for flow analysis or filter based traces. 4-bit field for IP header version 4-bit field for the IP header length, usually 20 bytes In Version 6 packets, this field is used for priority 16-bit field for size of datagram (IP header plus payload listed in bytes) field used for payload length in version 6 packets 16-bit field for identification to uniquely identify packets Version 6 8-bit for next hop and 8-bit for hop limit 3-bit field for IP flags, pad unused fields 13-bit field for fragmentation offset, listed in bytes 8-bit field for protocol number 128-bit Destination Internet IP address given in IPv6 format 128-bit Source Internet IP address given in IPv6 format 16-bit field for destination port 16-bit field for source port 64-bit field for 8 bytes of payload Field 6: 2-bit field to indicate trace status in message type 2, Followed by 6-bit of padding Following 2 fields used in message type 1, 2, and 3: Field 6 in Msg 1: Field 7 in Msg 2: Field 5 in Msg 3: 8-bit field Number of hops or contacts along the path of the trace Next Field 7, 8, or 6: 144-bit field for each identifier. 16-bit for AS number Moriarty Expires: April 6, 2003 [Page 19] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 128-bit for NMS IP address in IPv6 format Note 1: If identifier is unavailable, the IPv6 formatted address should be listed and the AS number field null terminated and padded. The optional text box at the end of the packet should be used to provide ISP name and contact information. Note 2: A maximum of 80 identifiers may be listed within the size constraints of a TCP packet. 15 would be large enough considering that this is a count of the NMS systems used to trace the attack. If the number is exceeded, that may indicate a loop or other problem with the trace. In the case were it is valid to have more than 15 NMS systems listed, the originating NMS should be left in the packet and the rollover should begin with the second hop. Each NMS should also track the trace identifiers that have already passed through their systems to prevent loops. Used in Message Type 3: Field 7: 128-bit field given for true source address of attack Field 8: 8-bit field with size of optional text field or null. Field 9: Optional Text Field 0-1428-byte text field used in message type 3 to provide additional attack information or contact information. 0-1154-byte text field if the optional text field is used in message type 2 for NMS identification information. The text field can extend to the end of the TCP packet data unit size limit of 1460 bytes. The number of bytes used for the text field is listed in the first 8-bits or a null character must be used to indicate the text field was not used. The text field must be terminated with a null character to signify the end of the text portion of the packet. Note: The text is in English and is represented using ASCII format. A single language would reduce management complexity, particularly in the case where the event spans ISPs in multiple countries. Total size of Message Type 1 packet = 656 bit with 1 NMS listed, 2448-bit with 15 NMS listed Total size of Message Type 2 packet is 8-bits larger than Message Type 1 Message Type 3 size before the optional text field is 256-bit Moriarty Expires: April 6, 2003 [Page 20] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 4.5 Message Delivery and Security Considerations Security considerations should include the integrity and authorization of the messages sent between NMS systems. The communication between NMS systems should be authenticated and encrypted to ensure the integrity of the messages and the NMS systems involved in the trace. Using out-of band communications dedicated to ISP interaction would provide additional security as well as guaranteed bandwidth. This might be accomplished through logical paths defined over the existing network. It may be appropriate to use SNMP trap messages since Network Management Systems already use this messaging structure. However, RFC1215 discourages the use of traps for this type of application and attempts to discourage the creation of new trap types. The current trap messaging structure does not satisfy the information requirements in order to successfully carry out a trace. Trap messages are sent via UDP, which assist in the quick delivery of packets, however they do not guarantee delivery as in TCP. The RID DoS protocol for inter-NMS communication could provide the transport mechanism for message delivery. In order to address the integrity and authenticity of messages, IPSec could be used to authenticate and encrypt the traffic sent between NMS systems with pre-defined trust relationships. 5.1 Security Considerations Communication between ISPs NMS must be protected. An out of band network, either logical or physical would prevent outside attacks. Authenticated encryption tunnels between stations would protect the data in transit as well as provide integrity of the data. The NMS should be configurable to either require user input or automatically continue traces. This feature would enable a network manager to assess the available resources before continuing a trace. A trace may cause adverse affects on a network. If the severity rating is low it may not be in the ISPs best interest to continue a trace. Policy between ISPs must be established to provide guidelines for communication. The policy should include communication methods and fall back procedures. The Policy should establish a method to protect communications between NMS systems between all bordering ISPs. The trust relationships would have to extend to all bordering ISPs in order to be successful in tracing and stopping attacks. A fully meshed communication ability would provide the means for message type 3 to be sent to an initiating NMS. Other policy considerations include how the AS number and NMS IP address should be shared. This should also be coupled with any necessary pre-shared key or certificate (or trusted Security Authority) stored in the NMS for encryption negotiation. Moriarty Expires: April 6, 2003 [Page 21] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 Note: The AS number and corresponding IP address for a network NMS would be pre-shared in a table between cooperating networks. This information may be stored locally on NMS systems or a central database accessible on the secured network used for inter-ISP messaging on NMS. A Certificate Authority may be used to establish security associations between NMS systems. The method of passing the trace to subsequent networks eliminates the need for either granting access to remote entities to configure network equipment. This also prevents the need for sharing authentication information to the devices. Network administrators who have the ability to base the decision on the available resources and other factors of their network maintain control of the continuance of a trace. 6.1 Summary Denials of service attacks have always been difficult to trace as a result of the spoofed sources, resource limitations, and bandwidth utilization problems. Methods to identify and trace attacks near real time are essential to thwarting attack attempts. ISPs need automated methods as well as policies in place in order to attempt to combat the hacker's efforts. Proactive monitoring and alerting of backbone and client bandwidth with trend analysis is an approach that can be used to help identify and trace attacks quickly without resource intensive side effects. Subsequent more detailed analysis could be used to complement the bandwidth monitoring. Timely communication between ISPs is essential in incident handling. Moriarty Expires: April 6, 2003 [Page 22] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 7.1 References [RFC791] "Internet Protocol, DARPA Internet Program, Protocol Specification". Information Sciences Institute, University of Southern California. September 1981. [RFC1213] "Management Information Base for Network Management of TCP/IP-based Internets: MIB-II". K. McCloghrie, M . Rose. March 1991. [RFC1215] "A Convention for Defining Traps for use with the SNMP". M. Rose. March 1991. [RFC1930] "Guidelines for creation, selection, and registration of an Autonomous System (AS)". J. Hawkinson and T. Bates. March 1996. [RFC2720] "Traffic Flow Measurement: Meter MIB". N. Brownlee. October 1999. [RFC2722]"Traffic Flow Measurement: Architecture". N. Brownlee, C. Mills, G. Ruth. October 1999. [RFC2723] "SRL: A Language for Describing Traffic Flows and Specifying Actions for Flow Groups". N. Brownlee. October 1999. [RFC2827] "Network Ingress Filtering: Defeating Denial of Service Attacks Which Employ IP Source Address Spoofing". P. Ferguson and D. Senie. May 2000. [1] http://www.info-sec.com/denial/infosece.html-ssi [2] `CenterTrack: An IP Overlay Network for Tracing DoS Floods'. Robert Stone. 9th Usenix Security Symposium Proceedings, August 2000. [3] 'Inferring Internet Denial of Service Activity`. David Moore, Geoffrey M. Voelker and Stephan Savage. ;login. November 2001. [4] 'MULTOPS: A Data-Structure For Bandwidth Attack Detection'. Thomer M. Gil, Massimiliano Poletta. ;login. November 2001. [5] 'Network Congestion Monitoring and Detection using the IMI infrastructure'. Takeo Saitoh, Glenn Mansfield, and Norio Shiratori. Graduate School of Information Sciences, Tohoku University. [6] 'Trends in Denial of Service Attack Technology`. Kevin J. Houle and George M. Weaver. CERT Coordination Center. October 2001. [7] http://www.cisco.com/go/netflow [8] 'Hash Based IP Traceback'. A. Snoren, L. Sanchez, C. Jones, Moriarty Expires: April 6, 2003 [Page 23] Internet-Draft DDoS Incident Handling: RID-DoS October 6, 2002 F. Tchakountio, S. Kent, and W. Strayer. SIGCOMM'01. August 2001. [9] 'Practical Network support for IP Traceback'. S.Savage, D. Wetherall, A. Karlin and T. Anderson. SIGCOMM'00. August 2000. [10] 'ICMP Traceback Messages'. S. M. Bellovin. Internet Draft: draft-bellovin-itrace-00.txt, March 2000. 8.1 Acknowledgements Dr. Robert K. Cunningham, MIT Lincoln Laboratory David J. Fried, MIT Lincoln Laboratory Cynthia D. McLain, MIT Lincoln Laboratory Dr. William Streilein, MIT Lincoln Laboratory 8.2 Author Information Kathleen M. Moriarty MIT Lincoln Laboratory 244 Wood Street Lexington, MA 02420 Phone: 781-981-5500 Email: Moriarty@ll.mit.edu This work was sponsored by the Air Force under Air Force Contract Number F19628-00-C-0002. "Opinions, interpretations, conclusions, and recommendations are those of the author and are not necessarily endorsed by the United States Air Force." Moriarty Expires: April 6, 2003 [Page 24]