IPFIX Working Group A. Kobayashi Internet-Draft K. Ishibashi Intended status: Informational T. Kondoh Expires: April 26, 2007 NTT PF Lab. D. Matsubara Hitachi October 23, 2006 IPFIX Mediators draft-kobayashi-ipfix-mediator-01.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on April 26, 2007. Copyright Notice Copyright (C) The Internet Society (2006). Kobayashi, et al. Expires April 26, 2007 [Page 1] Internet-Draft IPFIX Mediators October 2006 Abstract An IPFIX Mediator is an intermediate node between IPFIX Devices and traffic Collectors. This node acts as a IPFIX proxy or IPFIX concentrator. Therefore, IPFIX Mediators enables cascade connection. This document defines the Option Template required by cascade connection and a reference internal component model and describes several solution scenarios that use IPFIX Mediators. IPFIX Mediator has several capabilities such as storage function of original Flow Records, flexible aggregation function, and proper distribution function for collector or analyzer. The combination of these capabilities can solve several problems related to traffic monitoring. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Mediator Option Template Presentation . . . . . . . . . . . . 5 3.1. Route Option Templates . . . . . . . . . . . . . . . . . . 5 3.2. Consideration of Aggregation Pattern . . . . . . . . . . . 7 3.2.1. Aggregation beyond Several Routers . . . . . . . . . . 7 3.2.2. Aggregation beyond Several Observation Domain IDs . . 8 3.3. Usage of Scope Field . . . . . . . . . . . . . . . . . . . 9 4. Internal Components Model . . . . . . . . . . . . . . . . . . 11 4.1. Collecting Process . . . . . . . . . . . . . . . . . . . . 11 4.2. Selection Process . . . . . . . . . . . . . . . . . . . . 11 4.3. Aggregation Process . . . . . . . . . . . . . . . . . . . 12 4.4. Exporting Process . . . . . . . . . . . . . . . . . . . . 12 4.5. Storing Process . . . . . . . . . . . . . . . . . . . . . 13 5. Solution Scenarios with IPFIX Mediators . . . . . . . . . . . 14 5.1. Flexible Aggregation . . . . . . . . . . . . . . . . . . . 14 5.2. Distributed Aggregation . . . . . . . . . . . . . . . . . 14 5.3. Duplication of Flow Records . . . . . . . . . . . . . . . 15 5.4. Distribution of Flow Records . . . . . . . . . . . . . . . 16 5.5. Adding Information Elements . . . . . . . . . . . . . . . 17 5.6. Extraction of Suspicious Flow . . . . . . . . . . . . . . 18 6. Security Considerations . . . . . . . . . . . . . . . . . . . 20 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 22 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 23 9.1. Normative References . . . . . . . . . . . . . . . . . . . 23 9.2. Informative References . . . . . . . . . . . . . . . . . . 23 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 25 Intellectual Property and Copyright Statements . . . . . . . . . . 26 Kobayashi, et al. Expires April 26, 2007 [Page 2] Internet-Draft IPFIX Mediators October 2006 1. Introduction Several problems regarding traffic monitoring have occurred in each network domain. As the sizes of networks become larger, the number of Flow Records becomes greater. Large numbers of Flow Records have been burdening management networks, and the Collecting Process. Maintaining scalability is difficult as a given network grows. On the other hand, networks such as IPv4, IPv6, and VPN on MPLS have recently become more multifaceted. These sorts of Flow Records need to be analyzed separately from a different perspective. However, handling them separately without improving the capability of Collector is difficult. First, IPFIX protocol has an extensive Template format and flexible functions. By defining IPFIX Mediators, we can increasingly take advantage of an extensive Template format, and handle Flow Records in accordance with our preference. This document describes some solutions using IPFIX Mediator that solve the above problem and components that are needed in the internal function process. An IPFIX Mediator is located between one or more Exporting Processes and one or more Collecting Processes. IPFIX Mediator has a main function such as storing original Flow Records, aggregating them flexibly just like IPFIX concentrator, and distributing them to an appropriate Collector or traffic analyzer in accordance with flow contents. The internal model of a Mediator is composed of Collecting Processes, Metering Processes, and Exporting Processes. IPFIX Mediator acts as a Collector by receiving Flow Records, and it acts as an Exporter by sending Flow Records. This dual-role architecture enables cascading Mediators and buiilding a combination of several solutions. This document proposes a solution by using IPFIX Mediator and a reference model for IPFIX Mediators and describes the key component of this architecture. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL","SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. Kobayashi, et al. Expires April 26, 2007 [Page 3] Internet-Draft IPFIX Mediators October 2006 2. Terminology The definitions of basic IPFIX and PSAMP terms are identical with those in the [I-D.ietf-psamp-framework] and [RFC3917]. Other than the above terminology, the following terminology is used in this document. IPFIX Mediator: An IPFIX Mediator hosts at least one pair of Exporting Process and Collecting Process. It may have Metering Process and Storing Process as option. It may host multiple combinations of each process. An IPFIX proxy and an IPFIX concentrator are one node of IPFIX Mediators. Selection Process: An IPFIX Mediator's Selection Process is similar to that of PSAMP Devices, which is described in [I-D.ietf-psamp-framework]. However, its Selection Process differs from the following functions. This process in PSAMP Devices has two types of selection functions: Filtering and Sampling. In addition, the filtering function has two types of filtering methods: field-match filtering and hash-based selection. This process in IPFIX Mediator has only the field-match filtering function. This filter selects Flow Records based on Flow Record content. Aggregation Process: This process creates aggregated Flow Records, in accordance with aggregation rules that are described in [I-D.dressler-ipfix-aggregation], from Flow Records that are the output of the selection process. Storing Process: This process selects specified Information Elements by the storing rules from the output of Collecting Process and stores these Flow Records in a storage system such as a database or flat-file system. Information Elements that are described in [I-D.ietf-ipfix-info] are specified in storing rules. In addition, the Observation Domain ID and Export Time in the header of IPFIX messages are specified in the storing rules. Distribute Function: This function distributes final Flow Records based on the Flow content. The final Flow Records are handled by Exporting Process to export it to Collector. Each classified Flow Record is exported to each Collector. Kobayashi, et al. Expires April 26, 2007 [Page 4] Internet-Draft IPFIX Mediators October 2006 3. Mediator Option Template Presentation This section describes Option Templates that are used by IPFIX Mediators. 3.1. Route Option Templates The cascade connection of IPFIX Mediator enables several solutions that are described in the section 5. Each IPFIX Mediator and final destination Collector needs to know the source IPFIX Device of an original Flow Record and route of aggregated Flow records. An original Flow Record is created by a IPFIX Device that has Observation Point. Therefore, each IPFIX Mediator should inform the next Collector about previous Exporter information. The final destination Collector can recognize the source IPFIX device, and route of each Mediator. Route Option Template specifies the source of the original Flow and the route of the Aggregated Flow. This template is composed of the following information elements. o templateId o observationDomainId o exporterIPv4Address or exporterIPv6Address o exporterTransportPort The IPFIX Router that creates original Flow Records is identified by specifying "exporter{IPv4|IPv6}Address", "exporterTransportPort", and "observationDomainId". On the other hands, the intermediate Mediator is identified by specifying "exporter{IPv4|IPv6}Address" and "exporterTransportPort". The "templateId" is used as a Scope field. The route of received flow records is identified by an Exporter information in the route. In that case, the order dependency that is described in [I-D.ietf-ipfix-implementation-guidelines] is crucial. The IPFIX Mediator inserts its own Exporter information into received Data Records specified by the Route Option Template, and sends that information to the next Collector. In this way, the route is maintained until the final destination Collector. Kobayashi, et al. Expires April 26, 2007 [Page 5] Internet-Draft IPFIX Mediators October 2006 The following example describes the cascade connection of IPFIX Mediators. Each Mediator informs the next node about previous Exporter information. Session#a Session#b Session#c Router --------> Mediator#1 --------> Mediator#2 -------->Collector IP:1.1.1.1 IP:2.2.2.2 IP:3.3.3.3 Port:10 Port:20 Port:30 ODID:10 Figure A: Cascade connection of IPFIX Mediators. The Mediator#1 or Mediator#2 sends a Data Record specified by the Route Option Template. The Option templates and Data records are shown in Session#b or Session#c, as follows. Session#b OptionTemplate: Set ID = 3 Template ID = 259 Field Count = 4 Scope Count = 1 templateId exporterIPv4Address exporterTransportPort observationDomainId Session#b Data Record: Set ID = 259 templateId = XXX exporterIPv4Address =1.1.1.1 exporterTransportPort = 10 observationDomainId = 10 Session#c OptionTemplate: Set ID = 3 Template ID = 259 Field Count = 6 Scope Count = 1 templateId exporterIPv4Address exporterTransportPort exporterIPv4Address exporterTransportPort observationDomainId Kobayashi, et al. Expires April 26, 2007 [Page 6] Internet-Draft IPFIX Mediators October 2006 Session#c Data Record: Set ID = 259 templateId = XXX exporterIPv4Address =2.2.2.2 exporterTransportPort = 20 exporterIPv4Address =1.1.1.1 exporterTransportPort = 10 observationDomainId = 10 3.2. Consideration of Aggregation Pattern The Route Option Template should be used in consideration of the aggregation pattern in Mediators. In two aggregation patterns, the Route Option Template SHOULD be modified to handle properly. 3.2.1. Aggregation beyond Several Routers If the number of source IPFIX devices in the largest set of Flow information aggregated by a Metering Process becomes greater than that of the source IPFIX devices, information about source IPFIX devices is meaningless. In that case, this Mediator does not send a Route Option Template, and next Mediator informs about received Observation Domain ID. As shown in the Figure B, Mediator#1 does not send Route Option Template and Mediator#2 exports the received Observation Domain ID by the Route Option Template. From the Collector viewpoint, the information seems to have come from the source IPFIX Device, which is Mediator#1. The following example describes cascade connection of IPFIX Mediators. By the aggregation in Mediator#1, the largest set of Flow information aggregated becomes whole routers that are Router#1 and Router#2. Session#a-1 Session#b Session#c Router#1 -------+--> Mediator#1 --------> Mediator#2 -------->Collector IP:1.1.1.1 | IP:2.2.2.2 IP:3.3.3.3 Port:10 | Port:20 Port:30 ODID:10 | | Session#a-2 Router#2 -------+ IP:4.4.4.4 Port:40 ODID:40 Figure B: Largest set of aggregated Flow information becomes several routers. Kobayashi, et al. Expires April 26, 2007 [Page 7] Internet-Draft IPFIX Mediators October 2006 Note: In an actual aggregation pattern, this pattern does not seem to be considered. The aggregation beyond several routers causes packet properties derived from IPFIX Routers to become meaningless values. The Mediator#1 does not send a Data Record of a specified Route Option Template. The Mediator#2 sends a Data Record of a specified Route Option Template. The Option templates and Data records are shown in Session#c, as follows. Session#c OptionTemplate: Set ID = 3 Template ID = 259 Field Count = 4 Scope Count = 1 templateId exporterIPv4Address exporterTransportPort observationDomainId Session#c Data Record: Set ID = 259 templateId = XXX exporterIPv4Address =2.2.2.2 exporterTransportPort = 20 observationDomainId = 0 3.2.2. Aggregation beyond Several Observation Domain IDs If the number of Observation Domains in the largest set of Flow information aggregated by a Metering Process becomes greater than that of the Observation Domains of source IPFIX device, information about original Observation Domain ID of source IPFIX devices is meaningless. In that case, the Mediator replaces the original Observation Domain ID with 0. The following example describes a cascade connection of IPFIX Mediators. By the aggregation in Mediator#1, the largest set of aggregated Flow information becomes Router#1, in other words, the whole system. Session#a Session#b Session#c Router#1 --------> Mediator#1 --------> Mediator#2 -------->Collector IP:1.1.1.1 IP:2.2.2.2 IP:3.3.3.3 Port:10 Port:20 Port:30 ODID:10 ODID:40 Figure C: Largest set of aggregated Flow information becomes Kobayashi, et al. Expires April 26, 2007 [Page 8] Internet-Draft IPFIX Mediators October 2006 whole system. Router#1 exports Flow Records from two Observation Domains through Session#a. Mediator#1 sends a Data Record of a specified Route Option Template. The Option templates and Data records are shown in Session#b, as follows. Session#b OptionTemplate: Set ID = 3 Template ID = 259 Field Count = 4 Scope Count = 1 templateId exporterIPv4Address exporterTransportPort observationDomainId Session#b Data Record: Set ID = 259 templateId = XXX exporterIPv4Address =1.1.1.1 exporterTransportPort = 10 observationDomainId = 0 3.3. Usage of Scope Field An IPFIX Mediator needs to forward Option Template from source IPFIX devices. However, it can not export an original Option Template without modification, because changing a session from an Exporting Process to a Collecting Process causes the scope fields to become a meaningless value. In that case, an IPFIX Mediator can use the Route Option Template. The Option Template that was created from a source IPFIX device as a scope of an Observation Domain ID uses the entire field of the Route Option template as the scope. The Option Template that was created from a previous Mediator as a scope of the Observation Domain ID uses some fields of the Route Option template as a scope. These fields are given previous Mediator information directly because the Mediator does not have Observation Domain, and the Exporting Process of the Mediator uses the Observation Domain ID with a value of 0. However, if each node uses another field except for the Observation Domain ID as the scope, the scope field should be considered on a case-by-case basis. If there is no Metering Process in intermediate Mediators, the scope field with the template id can be maintained untill the final destination Collector. If there is a Metering process, the possibility of exporting this Option Template depends on aggregation method. Kobayashi, et al. Expires April 26, 2007 [Page 9] Internet-Draft IPFIX Mediators October 2006 The following example describes the cascade connection of IPFIX Mediators. Router#1 and Mediator#1 export the Metering Process Statistics Option Template. Session#a Session#b Session#c Router --------> Mediator#1 --------> Mediator#2 -------->Collector IP:1.1.1.1 IP:2.2.2.2 IP:3.3.3.3 Port:10 Port:20 Port:30 ODID:10 Figure D: Cascade connection of IPFIX Mediators. Mediator#2 exports each Option Template and its Data Record with a suitable scope. Session#c Metering Process Statistics Data Records from Router: Set ID = 3 Template ID = 259 Field Count = 8 Scope Count = 5 exporterIPv4Address = 2.2.2.2 exporterTransportPort = 20 exporterIPv4Address = 1.1.1.1 exporterTransportPort = 10 observationDomainId = 10 exportedMessageTotalCount exportedFlowTotalCount exportedOctetTotalCount Session#c Metering Process Reliability Statistics Data Records from Mediator#1: Set ID = 3 Template ID = 259 Field Count = 5 Scope Count = 2 exporterIPv4Address = 2.2.2.2 exporterTransportPort = 20 exportedMessageTotalCount exportedFlowTotalCount exportedOctetTotalCount Kobayashi, et al. Expires April 26, 2007 [Page 10] Internet-Draft IPFIX Mediators October 2006 4. Internal Components Model The following figure indicates five components (Collecting, Selection, Aggregation, Exporting, and Storing) within the IPFIX Mediator. These components are mapped to three processes as the IPFIX architecture in the [RFC3917]. The Collecting Process matches the IPFIX Collecting Process and the chain of Selection Process; the Aggregation Process matches the IPFIX Metering Process. The Exporting Process matches the IPFIX Exporting Process. +----------------------------------------------------------+ | IPFIX Mediator | | <----Metering Process-----> | | .----------. .---------. .-----------. .---------. | Flow -->|Collecting|->|Selection|->|Aggregation|->|Exporting|--> Flow Records |Process | |Process | |Process | |Process | Records | '----------' '---------' '-----------' '---------' | | | | | V | | .----------. .---------. | | |Storing |->|Database | | | |Process | | | | | '----------' '---------' | | | +----------------------------------------------------------+ Figure E: Key components within IPFIX Mediator. Each process is associated with a common identifier in the IPFIX Mediator. This method is similar to PSAMP associations in [I-D.ietf-psamp-sample-tech]. 4.1. Collecting Process This process receives Flow Records from IPFIX devices or a previous IPFIX Mediator. The instance of this process is created according to the IPFIX session. This process has functions that are described in [I-D.ietf-ipfix-protocol]. The Collecting Process also forwards received Flow Records with IPFIX header information to multiple Selection Processes or Storing Processes. In other words, Flow Records can be duplicated by forwarding several Selecting Process. 4.2. Selection Process The Selection Process receives Flow Records from Collecting Processes. This process has a filtering function and selects Flow Records that are matched under given conditions. Prior to receiving Flow Records, this process has instruction pattern data that is Kobayashi, et al. Expires April 26, 2007 [Page 11] Internet-Draft IPFIX Mediators October 2006 defined by the user, which specifies how the Flow Records are treated by this process. If some fields in the Flow Record match the instruction pattern, this process selects Flow Records that have all fields and forwards these to the Aggregation Process. For example, this process selects Flow Records to prevent the next stage node from being counted twice within the same flow. 4.3. Aggregation Process This process gathers Flow Records within a given time interval and then distinguishes Flow Records that have common properties. If values of a given key field are the same, that means these Flow Records have common properties. This process merges Flow Records that have a common property and creates an aggregated Flow Record. Therefore, for example, aggregated Flow Records have an aggregated counter that indicates the number of packets. These functions are defined in accordance with the IPFIX aggregation rule in [I-D.dressler-ipfix-aggregation]. This process has instructions that are user defined prior to receiving Flow Records. The process indicates fields that should become aggregated flow keys and other fields that should be kept or discarded. In addition, these instruction rules include Information Elements that should be added to aggregated Flow Records. The aggregated flow MAY need to complement information that is discarded during the aggregation process. For example, the aggregated flow has "averageActiveTime" or "flowCount" elements. 4.4. Exporting Process This process forwards Flow Records to the next IPFIX Mediator or Collector. These processes manage the reporting template and make an IPFIX datagram. The "Export time" and "Observation Domain Id" of IPFIX header information can be either of the following methods. The time of "Export time" field depends on whether the IPFIX Mediator is a proxy or not. From the point of view of a Collector, if this IPFIX Mediator is a proxy, it becomes the most recent time of the originally received data. If it is not a proxy, "Export time" becomes the export time of the Mediator. In the first case, this process needs to obtain the originally received data from the Aggregating Process. If IPFIX Mediator is not a proxy, its "Observation domain ID" should be zero. These IPFIX Header information values are forwarded from the Aggregation Process. In addition, this process has the Distribution Function as option. If this function is enabled, this process distributes Flow Records based on Flow content, and then it exports each classified Flow Records to each Collector. Kobayashi, et al. Expires April 26, 2007 [Page 12] Internet-Draft IPFIX Mediators October 2006 The Exporting Process distributes Flow Records based on Peering AS, as shown in the following figure. Each classified Flow Record is exported to a dedicated Collector per Peering AS. +-------------------------------------------+ | IPFIX Mediator .----------. | | |Exporting | | | |Process | | | .----------. .---------. | | | PeerAS#100 Flow -->|Collecting|->|Metering |----->/------------> Collector#A Records |Process | |Process | | | | | PeerAS#200 | '----------' '---------' | /------------> Collector#B | | | | | PeerAS#300 | | /------------> Collector#C | | | | | PeerAS#400 | | /------------> Collector#D | | | | | '----------' | +-------------------------------------------+ Figure F: Exporting each classified Flow Record to dedicated Collector. 4.5. Storing Process The storing process selects specified Information Elements by the storing instruction from the input Flow Records. Prior to receiving Flow Records, this process has instructions that are configured by the user, which indicate whether each field should be stored or discarded. The field modifier indicates "keep" or "discard", which is similar to the instruction of the aggregation process. The field modifier specifies how these Information Elements are treated by the process. This field modifier is applied to Information Elements within Flow Record and header information of IPFIX messages. The header information of IPFIX messages are "Observation Domain ID" and "Export time". This header information MAY be used when IPFIX datagrams are made of past Flow Records. When another node retrieves past Flow Records, we can consider several specifications. One solution is that another node gets a specified flat-file from a Mediator, and decodes that flat-file by itself. Other solutions are that another node sends out the query command to the Mediator through the XML-RPC, SNMP, or NETCONF, and then the Mediator exports the specified past Flow Records. This function is out of the scope of this document. Kobayashi, et al. Expires April 26, 2007 [Page 13] Internet-Draft IPFIX Mediators October 2006 5. Solution Scenarios with IPFIX Mediators 5.1. Flexible Aggregation An IPFIX Mediator can aggregate Flow Records as IPFIX concentrator and reduce the number of Flow Records received by a traffic collector. The following figure indicates a cascade connection of IPFIX Mediators. If a Collector measures a traffic matrix to obtain traffic demand, it needs Flow Records of the whole network domain, but does not need detailed Flow Records. In the first step, a Mediator receives Flow Records from IPFIX Devices and then creates aggregated low-level Flow Records. For example, this step is prefix mask aggregation. Next, the Mediator receives aggregated Flow Records and aggregates them further. For example, the second step is the aggregation of the BGP next-hop address and exporter address. After this, the collector receives high-level aggregated Flow Records and then stores them. This method enables step-by-step aggregation of Flow Records without overloading a single node. .--------. .--------. |IPFIX | |IPFIX | |router#1|---->|Mediator|---. | | |*1 | | '--------' '--------' | .--------. .---------. '--->|IPFIX | |Traffic | .--------. .--------. .--->|Mediator|---->|Collector| |IPFIX | |IPFIX | | |*2 | | | |router#2|---->|Mediator|---' '--------' '---------' | | |*1 | '--------' '--------' Figure G: Flexible Aggregation with cascading IPFIX Mediators. 5.2. Distributed Aggregation As the network becomes widely global, the distances between PoPs become longer, and the maintenance of a dedicated management network is very expensive. Therefore, the huge number of Flow Records has been burdening the management networks of global ISPs. If we place Mediators at each PoP, the number of Flow Records exported from each PoP can be reduced. Mediators can minimize the number of Flow Records exported to the Collector. If the Collector needs detailed information, it can retrieve Flow Records from Mediators that store original Flow Records. Kobayashi, et al. Expires April 26, 2007 [Page 14] Internet-Draft IPFIX Mediators October 2006 A management networks of a global ISP is shown in the following figure. The Mediators are located at each PoP of the network, and they collect Flow Records from routers in each PoP domain. The Mediator can reduces the number of Flow Records by aggregating or filtering, so this system can reduces load of a management network. POP#Asia .--------. .--------. | .---------. .--------. | |----->|IPFIX | |IPFIX | |------->|Mediator |----. |router |---'----->|#1 | | |#1 |-' '---------' | '--------' | | POP#America | .--------. | .--------. | .---------. | .---------. .--------. | |----->|IPFIX | '---->|Traffic | |IPFIX | |------->|Mediator |--------->|Collector| |router |---'----->|#2 | .---->| | |#4 |-' '---------' | '---------' '--------' | | POP#Europe | .--------. | .--------. | .---------. | .--------. | |----->|IPFIX | | |IPFIX | |------->|Mediator |----' |router |---'----->|#3 | |#7 |-' '---------' '--------' Figure H: Traffic monitoring architecture in global network. 5.3. Duplication of Flow Records An IPFIX Mediator can duplicates Flow Records to achieve redundant storage or utilizes them for several purposes. The pair of Collecting Process and Metering Processes is similar to the pair of the Observation Point and Metering Process. The Collecting process duplicates Flow records by forwarding them to the multi-Metering Process. Kobayashi, et al. Expires April 26, 2007 [Page 15] Internet-Draft IPFIX Mediators October 2006 Several departments in an ISP want to use the same traffic information for each intended purpose. For example, the network design department measures the traffic matrix as traffic demand, and the customer service division uses traffic information for the accounting information of each customer while the network operation center uses traffic information for trouble shooting analysis. That case is shown in the following figure. A Mediator can distribute Flow Records to several Collectors that have appropriate aggregated granularity. In addition, when NOC conducts trouble shooting, they can retrieve past Flow Records from Mediators. Measurement traffic matrix. .--------. .---------. |IPFIX | |Traffic | |router#1|----. .---->|Collector| | | | | |#1 | '--------' | | '---------' | | Using Accounting info. .--------. | .---------. | .---------. |IPFIX | '---->|IPFIX |----' |Traffic | |router#2|--------->|Mediator |--------->|Collector| | | .---->| |----. |#2 | '--------' | '---------' | '---------' | | Using Trouble shooting. .--------. | | .---------. |IPFIX | | | |Traffic | |router#1|----' '---->|Collector| | | |#3 | '--------' '---------' Figure I: Duplication of Flow Records for several purposes. 5.4. Distribution of Flow Records An IPFIX Mediator can distribute Flow Records based on flow content. This function enables load-balancing of Collector and sorting out Flow Records without extra Collector functions. If the Flow Records are used as accounting information, this solution is useful. Kobayashi, et al. Expires April 26, 2007 [Page 16] Internet-Draft IPFIX Mediators October 2006 When we disclose traffic information to each customer, security or privacy policy should be considered. In that case, Mediator can hide private information about each customer. For example, Mediator can distribute them based on RD(Route Distinguisher), egress IF, Peering AS number, or BGP next-hop which identifies the customer. In the following figure, the Mediator distributes based on RD. It securely allows for each customer to access only their own records. .--------. .---------. |IPFIX | |Traffic | |router#1|----. .---->|Collector|<===> Customer#A | | | | |#1 | '--------' | | '---------' | RD=100:1 | .---------. | .--------. '---->|IPFIX |----' .---------. |IPFIX | |Mediator | RD=100:2 |Traffic | |router#2|--------->| |--------->|Collector|<===> Customer#B | | | | |#2 | '--------' .---->| |----. '---------' | '---------' | | RD=100:3 .--------. | | .---------. |IPFIX | | | |Traffic | |router#1|----' '---->|Collector|<===> Customer#C | | |#3 | '--------' '---------' Figure J: Distribution of Flow Records for each customer. 5.5. Adding Information Elements An IPFIX Mediator can add the following kinds of Information Elements. Complementing lost information by aggregation: A Mediator can add complementary information to replace lost information by aggregation. A main problem of aggregation is the loss of some Information Elements. This problem could be alleviated by adding complementary Information Elements. They are added by the aggregation process and they help traffic collector to analyze aggregated flow. * averageActiveTime: This Information Element indicates average time in milliseconds of difference between flow start time and end time of each flow included in the aggregated flow. It is created from flow time stamp Information Elements. There are "flowStartSeconds", Kobayashi, et al. Expires April 26, 2007 [Page 17] Internet-Draft IPFIX Mediators October 2006 "flowEndSeconds", "flowStartMilliSeconds", "flowEndMilliSeconds", "flowStartSysUpTime", and "flowEndSysUpTime". In addition, "minimumActiveTime" and "maxmumActiveTime" might be considered as well as this element. * synCount: It is the total number of Flow Records that has "tcpControlBits" which SYN bit set to 1 in aggregated flow. By this elements, we can grasp the number of SYN packets in whole network. And also, "ackCount", "finCount", "pshCount", "urgCount" and "rstCount" might be considered as well as this element. * flowCount: It is the number of Flow Records that is included in the aggregated flow. Adding information instead of router: The Information Elements that ca not be put in a original Flow Record by a router are added to Flow Records by Mediator. For example, those information elements are BGP next-hop, and RD. A Mediator can add these elements by referring to BGP route information and SNMP MIB. Hereby, traffic collectors do not need to recognize the difference between implementations of routers from several vendors. Adding information that becomes easy to analyze: These Information Elements enable nodes in the next stage to become easy to analyze. For example, "Biflow ID" could be quoted. "Biflow" is defined in [I-D.ietf-ipfix-biflow]. This indicates a common ID that each UniFlow Record has in both directions. These 4-tuple elements that are in "Directional Key Field", have this value as follows. There is a case by case consideration of which fields will be included in the Hash function. Biflow ID= Hash[src.IP+Dst.IP+Src.Port+Dst.Port] Hereby, the next stage node can easily be used to gather UniFlow Records. In an asymmetric routing condition, it is useful for the accounting or analysis of anomalies in a session. 5.6. Extraction of Suspicious Flow An IPFIX Mediator filters based on flow content. If filter conditions are set depending on the suspicious flow as follows, the next Collector receives the specified suspicious flow and detects an anomalous flow by simply monitoring the traffic volume of each suspicious flow. Kobayashi, et al. Expires April 26, 2007 [Page 18] Internet-Draft IPFIX Mediators October 2006 o TCP Flow Records whose "tcpControlBits" is set to "null" o TCP Flow Records whose "tcpControlBits" is set to SYN bit only and the packet counter is only 1. o ICMP Flow Records whose length is too long. Kobayashi, et al. Expires April 26, 2007 [Page 19] Internet-Draft IPFIX Mediators October 2006 6. Security Considerations The IPFIX concentrator uses the IPFIX protocol. Security considerations about flow information are described in [I-D.ietf-ipfix-protocol]. Kobayashi, et al. Expires April 26, 2007 [Page 20] Internet-Draft IPFIX Mediators October 2006 7. IANA Considerations This document has no actions for IANA. Kobayashi, et al. Expires April 26, 2007 [Page 21] Internet-Draft IPFIX Mediators October 2006 8. Acknowledgments Many thanks to J. Quittek and B. Trammel for providing valuable comments. Kobayashi, et al. Expires April 26, 2007 [Page 22] Internet-Draft IPFIX Mediators October 2006 9. References 9.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 9.2. Informative References [I-D.dressler-ipfix-aggregation] Dressler, F., Sommer, C., and G. Munz, "IPFIX Aggregation", draft-dressler-ipfix-aggregation-03.txt (work in progress) , June 2006. [I-D.ietf-ipfix-biflow] Trammell, B. and E. Boschi, "Bidirectional Flow Export using IPFIX", draft-ietf-ipfix-biflow-00.txt(work in progress) , August 2006. [I-D.ietf-ipfix-implementation-guidelines] Boschi, E., Mark, L., Quittek, J., Stiemerling, M., and P. Aitken, "IPFIX Implementation Guidelines", draft-ietf-ipfix-implementation-guideline-00.txt(work in progress) , August 2006. [I-D.ietf-ipfix-info] Quittek, J., Bryant, S., Claise, B., and J. Meyer, "Information Model for IP Flow Information Export", draft-ietf-ipfix-info-13.txt(work in progress) , June 2006. [I-D.ietf-ipfix-protocol] Claise, B., "IPFIX Protocol Specification", draft-ietf-ipfix-protocol-23 (work in progress) , October 2006. [I-D.ietf-psamp-framework] Duffield, N., "A Framework for Packet Selection and Reporting", draft-ietf-psamp-framework-10.txt , January 2005. [I-D.ietf-psamp-sample-tech] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F. Raspall, "Sampling and Filtering Techniques for IP Packet Selection", draft-ietf-psamp-sample-tech-07.txt , July 2005. [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, Kobayashi, et al. Expires April 26, 2007 [Page 23] Internet-Draft IPFIX Mediators October 2006 "Requirements for IP Flow Information Export(IPFIX)", October 2004. Kobayashi, et al. Expires April 26, 2007 [Page 24] Internet-Draft IPFIX Mediators October 2006 Authors' Addresses Atsushi Kobayashi NTT Information Sharing Platform Laboratories 3-9-11 Midori-cho Musashino-shi, Tokyo 180-8585 Japan Phone: +81-422-59-3978 Email: akoba@nttv6.net Keisuke Ishibashi NTT Information Sharing Platform Laboratories 3-9-11 Midori-cho Musashino-shi, Tokyo 180-8585 Japan Phone: +81-422-59-3407 Email: ishibashi.keisuke@lab.ntt.co.jp Kondoh Tsuyoshi NTT Information Sharing Platform Laboratories 3-9-11 Midori-cho Musashino-shi, Tokyo 180-8585 Japan Phone: +81-422-59-2419 Email: kondoh.tsuyoshi@lab.ntt.co.jp Daisuke Matsubara Hitachi, Ltd., Central Reseach Laboratory 1-280 Higashi-koigakubo Kokubunji-shi, Tokyo 185-8601 Japan Phone: +81-42-323-1111 Email: d-matuba@crl.hitachi.co.jp Kobayashi, et al. Expires April 26, 2007 [Page 25] Internet-Draft IPFIX Mediators October 2006 Full Copyright Statement Copyright (C) The Internet Society (2006). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Acknowledgment Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). Kobayashi, et al. Expires April 26, 2007 [Page 26]