PSAMP working group Internet Draft EDITOR: B. Claise draft-ietf-psamp-protocol-01.txt Cisco Systems Expires: August 2004 February 2004 Packet Sampling (PSAMP) Protocol Specifications Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsolete by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This document specifies the export of packet information from a PSAMP Exporting Process to a PSAMP Colleting Process. For export of packet information the IP Flow Information eXport (IPFIX) protocol is used. The IPFIX protocol is well suited for this purpose, because the IPFIX architecture matches the PSAMP architecture very well and the means provided by the IPFIX protocol are sufficient. The document specifies in detail how the IPFIX protocol is used for PSAMP export of packet information. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. Claise, et. al Standard Track [Page 1] PSAMP Protocol Specifications February 2004 Table of Contents 1. Open Issues..................................................2 1.1 Open Issues................................................2 1.2 Action Items...............................................3 2. Introduction.................................................3 3. Terminology..................................................4 4. Differences between PSAMP and IPFIX..........................4 4.1 Architecture Point of View.................................4 4.2 Protocol Point of View.....................................6 4.3 Information Model Point of View............................6 5. Using IPFIX for PSAMP........................................7 5.1 High Level View of the Integration.........................7 5.2 Partial or Entire IPFIX Protocol Specifications Support....7 6. PSAMP Requirements versus the IPFIX Solution.................8 6.1 IPFIX Solution for the PSAMP Requirements..................8 7. Low Level View of the Integration...........................11 7.1 Sampling Case, PSAMP Base Level of Functionality..........11 7.1.1 Example..............................................11 7.2 Sampling Case.............................................12 7.2.1 Example..............................................13 7.3 Filtering Case............................................13 7.3.1 Example..............................................13 8. Security Considerations.....................................13 9. IANA Considerations.........................................13 10. References.................................................13 10.1 Normative References.....................................13 10.2 Informative References...................................14 11. Acknowledgments............................................14 1. Open Issues 1.1 Open Issues This section covers the open issues, still to be resolved/updated in this draft: PROTO-01 Do we want to distinguish an IPFIX Flow Record export with one packet from a PSAMP export? PROTO-02 Need to fill in the examples section 7.1.1, 7.1.2 and 7.1.3 PROTO-03 in packet interpretation. Options Template FlowSet (SELECTOR_ID, SAMPLING_ALGO, SAMPLING PARAM, TIMESTAMP, OBSERVATION POINT) The packet reports MUST contain: - the input sequence number(s), denoted the SEQUENCE-NUMBER in [PSAMP-INFO] Claise, et. al Standard Track [Page 2] PSAMP Protocol Specifications February 2004 - some number of contiguous bytes from the start of the packet, denoted the PACKET-SAMPLE in [PSAMP-INFO] - the destination BGP AS , denoted destinationAS in [IPFIX- INFO] - the input interface, denoted ingressPort in [IPFIX-INFO] THIS IS NOT A GOOD EXAMPLE PROTO-04 Extend security considerations by a discussion on exported payload 1.2 Action Items This section covers the action items for this draft ACTION-01 For section 6 "PSAMP requirements versus the IPFIX solution", check if there are any other requirements in the [PSAMP- FRAMEWORK]. ACTION-02 Update the terminology section ACTION-03 A new section about the terminology comparison between [PSAMP-PROTO] (hence [IPFIX-PROTO]) and [PSAMP-FRAMEWORK] - Flow Data Records sent in Data FlowSet = packet report in [PSAMP-FRAMEWORK] - Options Data Record sent in Data FlowSet = packet interpretation n [PSAMP-FRAMEWORK] Exporting Process in IPFIX = Reporting Process in [PSAMP- FRAMEWORK] Note1: this is somehow explained in section 5.1 ACTION-04 Should briefly discuss the fact that PSAMP is OK with IPFIX requirements in terms of time (uSec precision) ACTION-05 Check for the existence of the Information Elements defined here in [PSAMP-INFO] and modify if appropriate. Example: Selector ID, packet-sample, sampling-algorithm, hash-value, etc… For example, the section 7.1 ACTION-06 In section 6.1 ‘‘An Options Templates MUST be sent on regular basis.’’ -> make the link with Metering Process Stats currently discussed in the IPFIX mailing list and in [IPFIX-PROTO] ACTION-07 Some text explanation the encoding of the new Information Elements. For example, the ‘‘packet-fragment’’ will use the Variable Length Data Type as described in [IPFIX-PROTO] ACTION-08 Section 6 about ‘‘PSAMP requirements’’: check if any changes with the version 5 of [PSAMP-FRAMEWORK] 2. Introduction The IP Flow information export (IPFIX) protocol specified in [IPFIX- PROTO] and [IPFIX-INFO] exports IP traffic information observed at Claise, et. al Standard Track [Page 3] PSAMP Protocol Specifications February 2004 network devices. This matches the general protocol requirements outlined in the Packet SAMPling (PSAMP) framework [PSAMP-FMWK]. However, there are some architectural differences between IPFIX and PSAMP and in the requirements for an export protocol. While in the IPFIX architecture [IPFIX-ARCH] packet sampling is just one out of many components considered, it is the focus of the PSAMP framework [PSAMP-FMWK]. This basic difference and a set of derived differences in protocol requirements are outlined in Section 4. Despite these differences, the IPFIX protocol is well suited as PSAMP protocol. Section 5 specifies how the IPFIX protocol is used for the export of packet samples. Required extensions of the IPFIX information model are specified in the PSAMP information model [PSAMP-INFO]. 3. Terminology EDITOR’S NOTE: - To be copied in from [PSAMP-FRAMEWORK]. - From [IPFIX-PROTO]: - need Flow Record, Flow, Information Element, Metering Process, Exporting Process, Collector, Scope - need all terms from the table in section 5.2. That is: FlowSet, Template Record, Data Record, Flow Data Record, Data FlowSet, Options Data Record, Template FlowSet, Template Record(s), Options Template FlowSet, Options Template Record - need PSAMP device - All the terms will have their initial letter in upper case 4. Differences between PSAMP and IPFIX The output of the IPFIX working group relevant for this draft, is structured into three documents: - IP Flow information architecture [IPFIX-ARCH] - IPFIX Protocol Specifications [IPFIX-PROTO] - IP Flow information export information model [IPFIX-INFO] 4.1 Architecture Point of View Traffic Flow measurement as described in the IPFIX requirements [IPFIX-REQ] and the IPFIX architecture [IPFIX-ARCH] can be separated into two stages: packet processing and Flow processing. The figure below illustrates these stages. On stage 1, all processing steps act on packets. Packets are captured, time stamped, selected by one or more selection steps and finally forwarded to packet classification that maps packets to Claise, et. al Standard Track [Page 4] PSAMP Protocol Specifications February 2004 Flows. The packets selection steps may include filtering and sampling functions. On stage 2, all processing steps act on Flows. After packets are classified (mapped to Flows), Flows are generated or updated if they exist already. Flow generation and update steps may be performed repeatedly for aggregating Flows. Finally, Flows are exported. Packet sampling as described in the PSAMP framework [PSAMP-FMWK] covers only stage 1 of the IPFIX architecture with the packet classification replaced by packet record export. IPFIX architecture PSAMP framework packet header packet header capturing \ capturing | | | timestamping | timestamping | | | v | v +------>+ | stage 1: +------>+ | | > packet | | | packet | processing | packet | selection | | selection | | | | | +-------+ | +-------+ | | | v | v packet / packet record classification \ export | | v | +------>+ | | | | | Flow generation | | and update | stage 2: | | > Flow | v | processing | Flow | | selection | | | | +-------+ | | | v | Flow Record / export Claise, et. al Standard Track [Page 5] PSAMP Protocol Specifications February 2004 Comparison of IPFIX architecture and PSAMP framework 4.2 Protocol Point of View Concerning the protocol, the major difference between IPFIX and PSAMP is that the IPFIX protocol exports Flow Records while the PSAMP protocol exports packet records. From a pure export point of view, IPFIX will not distinguish a Flow Record composed of several packets aggregated together, from a Flow Record composed of a single packet. So the PSAMP export can be seen as special IPFIX Flow Record containing information about a single packet. EDITOR’S NOTE: maybe we want to distinguish an IPFIX Flow Record export with one packet from a PSAMP export? Extensions of the IPFIX protocol needed by PSAMP are rather limited. A basic one is the need of a data type for protocol fields that has flexible length, such as an octet array. This is needed by the PSAMP protocol for reporting content of captured packets, for example the first 40 octets of a packet. 4.3 Information Model Point of View However, the overlap between both protocols is still quite large. Most of the data fields in the IPFIX protocol also apply to PSAMP, for example all fields reporting packet header fields. Only a few fields, such as flowCount, packetCount (whose value will always be one) etc., cannot be used in a meaningful way by the PSAMP protocol. Also, IPFIX protocol requirements concerning stage 2 do not apply to the PSAMP protocol. Further required extensions apply to the information model. The IPFIX information model is rather poor concerning sampling. Just two fields, one for the sampling method and one for the sampling rate, are not sufficient, as shown in [PSAMP-SLCT]. A set of several additional fields is required for satisfying the requirements for a PSAMP information model. Additional required extensions of the information model concern packet filtering, and the a field reporting content of a packet using the flexible length data type mentioned above. Claise, et. al Standard Track [Page 6] PSAMP Protocol Specifications February 2004 Exploiting the extensibility of the IPFIX information model, the required extension is covered by the PSAMP information model specified in [PSAMP-INFO]. 5. Using IPFIX for PSAMP 5.1 High Level View of the Integration The Template Record in the Template FlowSet is used to describe the different PSAMP Information Elements that will be exported to the Collector. The Collector decodes the Template FlowSet and knows which Information Elements to expect when it receives the Flow Data Records in the Data FlowSet, i.e. the PSAMP Packet Reports. Typically, in the base level of the PSAMP functionality, the Template FlowSet will contain the input sequence number, the packet fragment (some number of contiguous bytes from the start of the packet) and the selector ID. The Options Template Record in the Options Template FlowSet is used to describe the different PSAMP Information Elements that concern the Metering Process itself: sampling and/or filtering functions, plus the associated parameters. The Collector decodes the Options Template FlowSet and knows which Information Elements to expect when it receives the Options Data Records in the Data FlowSet, i.e. the PSAMP Report Interpretation. Typically, the Options Template would contain the Selector ID, the sampling or filtering functions, and the sampling or filtering associated parameters. 5.2 Partial or Entire IPFIX Protocol Specifications Support The "High level view of the integration" section 5.1 concludes that PSAMP requires all the different possibilities of the IPFIX protocol specifications [IPFIX-PROTO]. That is the 3 types of FlowSet (Data FlowSet, Template FlowSet and Options Templates FlowSet), the 2 types of Templates Records (Template Record and Options Template Record), and the 2 types of Data Record (Flow Data Record, Options Data Record), as described again in the table below. +------------------+---------------------------------------------+ | | Contents | | +--------------------+------------------------+ | FlowSet | Template Record | Data Record | +------------------+--------------------+------------------------+ | | | Flow Data Record(s) | Claise, et. al Standard Track [Page 7] PSAMP Protocol Specifications February 2004 | Data FlowSet | / | or | | | | Options Data Record(s) | +------------------+--------------------+------------------------+ | Template FlowSet | Template Record(s) | / | +------------------+--------------------+------------------------+ | Options Template | Options Template | / | | FlowSet | Record(s) | | +------------------+--------------------+------------------------+ As a consequence, PSAMP can't rely on a subset of the IPFIX protocol specifications are described in [IPFIX-PROTO]. The entire IPFIX protocol specifications MUST be implemented for the PSAMP export. 6. PSAMP Requirements versus the IPFIX Solution [PSAMP-FRAMEWORK] describes some requirements that affect directly the export protocol. Refer to the following sections: section 3.2 "Reporting Process Requirements" section 3.3 "Exporting Process Requirements" section 5 "Reporting Process" [PSAMP-FRAMEWORK] also describes in the section 3.1 one requirement that, if not directly related to the export protocol, will put some constraints on it: Selection Process Requirements: - Parallel Measurements: multiple independent measurement processes at the same entity." [PSAMP-FRAMEWORK] finally describes in the section 5 some requirements regarding the reporting process. This series of requirements specifies the different Information Elements that MUST and SHOULD reported to the collector. Nevertheless IPFIX, being a generic export protocol, can export any Information Elements as long as there are described in the information model. So these requirements are mainly targeted for the [PSAMP-INFO] document. 6.1 IPFIX Solution for the PSAMP Requirements Let's address the PSAMP requirements one by one. * Parallel Measurements: multiple independent measurement processes at the same entity. Refer to [PSAMP-FRAMEWORK] section 3.1 "Selection Process Requirements". Claise, et. al Standard Track [Page 8] PSAMP Protocol Specifications February 2004 This requirement is addressed by exporting the Selector ID Information Element in every packet report, so part of every Flow Data Records. Note that without this requirement, exporting the Scope part of every single packet report could have been sufficient. * Transparency: allow transparent interpretation of measurements as communicated by PSAMP reporting, without any need to obtain additional information concerning the observed packet stream. Refer to [PSAMP-FRAMEWORK] section 3.2 "Reporting Process Requirements". This requirement is addressed by exporting the Selector ID Information Element in every Flow Data Records (packet report) and exporting the associated SAMPLING_ALGORITHM and SAMPLING PARAMETERS Information Elements in the Options Data Record (packet interpretation). So the all the Metering Process parameters are linked to the Flow Data Records. * Robustness to Information Loss: allow robust interpretation of measurements with respect to reports missing due to data loss, e.g. in transport, or within the measurement, reporting or Exporting Processes. Inclusion in reporting of information that enables the accuracy of measurements to be determined. Refer to [PSAMP-FRAMEWORK] section 3.2 "Reporting Process Requirements". An Options Templates MUST be sent on regular basis. This Options Template contains for example the total number of packet report exported from the PSAMP device, the total number of packet observed, etc... Thus the Collector can compare the number of packet report received per selector ID with the number actually metered and/or sent. In case of discrepancy, a new sampling rate could be computed. * Faithfulness: all reported quantities that relate to the packet treatment MUST reflect the router state and configuration encountered by the packet at the time it is received by the measurement process. Refer to [PSAMP-FRAMEWORK] section 3.2 "Reporting Process Requirements". This requirement doesn't concern the export protocol itself but the Metering Process, even if described in the "Reporting Process Requirements" section. Claise, et. al Standard Track [Page 9] PSAMP Protocol Specifications February 2004 * Privacy: selection of the content of packet reports will be cognizant of privacy and anonymity issues while being responsive to the needs of measurement applications, and in accordance with RFC 2804. Full packet capture of arbitrary packet streams is explicitly out of scope. Refer to [PSAMP-FRAMEWORK] section 3.2 "Reporting Process Requirements". This requirement doesn't concern the export protocol itself, even if described in the "Reporting Process Requirements" section. * Timeliness: reports on selected packets MUST be made available to the collector quickly enough to support near real time applications. Specifically, any report on a packet MUST be dispatched within 1 second of the time of receipt of the packet by the measurement process. Refer to [PSAMP-FRAMEWORK] section 3.3 "Export Process Requirements". The IPFIX protocol specifications [IPFIX-PROTO] describe an inactivity timeout for the Flow expiration. This inactivity timeout is configurable, with a minimum value of 0 for immediate expiration. Note that this minimum value of 0 will force every single Flow Data Record to contain information about a single packet and not an aggregation of packets. * Congestion Avoidance: export of a report stream across a network MUST be congestion avoiding in compliance with RFC 2914. Refer to [PSAMP-FRAMEWORK] section 3.3 "Export Process Requirements". IPFIX, by its charter, MUST also respect this requirement. * Secure Export: - confidentiality: the option to encrypt exported data MUST be provided. - integrity: alterations in transit to exported data MUST be detectable at the collector - authenticity: authenticity of exported data MUST be verifiable by the collector in order to detect forged data. The motivation here is the same as for security in IPFIX export. Refer to [PSAMP-FRAMEWORK] section 3.3 "Export Process Requirements". Claise, et. al Standard Track [Page 10] PSAMP Protocol Specifications February 2004 7. Low Level View of the Integration 7.1 Sampling Case, PSAMP Base Level of Functionality EDITOR’S NOTE: LET'S ASSUME THAT THE [PSAMP-INFO] DEFINES THE FOLLOWING DATA TYPES SEQUENCE-NUMBER: the input sequence number, PACKET-SAMPLE: some number of contiguous bytes from the start of the packet SELECTOR-ID: SAMPLING-ALGORITHM: SAMPLING-PARAMETER1, SAMPLING-PARAMETERS2, ETC... As described in the section 5.1 "Mandatory Contents of Packet Reports" of [PSAMP-FRAMEWORK], the packet reports must contain: - the input sequence number(s), denoted the SEQUENCE-NUMBER in [PSAMP-INFO] - some number of contiguous bytes from the start of the packet, denoted the PACKET-SAMPLE in [PSAMP-INFO]. Thus the Template FlowSet defines a Template Record composed of SEQUENCE-NUMBER, PACKET-SAMPLE and SELECTOR-ID. The report interpretation must contain: - the sampling algorithm, denoted SAMPLING-ALGORITHM in [PSAMP-INFO] - the sampling parameters denoted SAMPLING-PARAMETER1, SAMPLING- PARAMETER2, etc... in [PSAMP-INFO] The Options Template FlowSet defines a Options Template Record composed of SELECTOR-ID, SAMPLING-ALGORITHM, SAMPLING-PARAMETERS. Finally the Data FlowSet is used to export the Flow Data Record(s) containing the real values of SEQUENCE-NUMBER, PACKET-SAMPLE and SELECTOR-ID. The Data FlowSet is also used to export the Options Data Record(s) containing the real values of SELECTOR-ID, SAMPLING- ALGORITHM, SAMPLING-PARAMETERS. By means of the SELECTOR-ID, the Collector can link any Flow Data Record to the corresponding Options Data Record. That is, any Flow Data Record to the Metering Process function and parameters. 7.1.1 Example EDITOR’S THIS MUST BE A FULL EXAMPLE LIKE IN SECTION 13 OF [IPFIX- PROTO]. THE [PSAMP-INFO] MUST BE FIRST PUBLISHED. Claise, et. al Standard Track [Page 11] PSAMP Protocol Specifications February 2004 7.2 Sampling Case The PSAMP reporting process SHOULD also report fields relating to the protocols used in the packets, to the packet treatment and to the selection state associated with the packet, as specified in [PSAMP-FRAMEWORK] section 5.2 "Recommended Contents for Packet Reports". Let's take the same example as in the section 7.1, but let's add the export of the destination BGP Autonomous System (AS) [1771] and of the input interface The packet reports MUST contain: - the input sequence number(s), denoted the SEQUENCE-NUMBER in [PSAMP-INFO] - some number of contiguous bytes from the start of the packet, denoted the PACKET-SAMPLE in [PSAMP-INFO] - the destination BGP AS , denoted destinationAS in [IPFIX-INFO] - the input interface, denoted ingressPort in [IPFIX-INFO] Thus the Template FlowSet defines a Template Record composed of SEQUENCE-NUMBER, PACKET-SAMPLE and SELECTOR-ID, destinationAS and ingressPort. The report interpretation will remain unchanged and must contain: - the sampling algorithm, denoted SAMPLING-ALGORITHM in [PSAMP-INFO] - the sampling parameters denoted SAMPLING-PARAMETER1, SAMPLING- PARAMETER2, etc... in [PSAMP-INFO] The Options Template FlowSet is used to define this template composed of SELECTOR-ID, SAMPLING-ALGORITHM, SAMPLING-PARAMETERS. Finally Data FlowSet is used to export the Flow Data Record(s) containing the real values of SEQUENCE-NUMBER, PACKET-SAMPLE and SELECTOR-ID, destinationAS and ingressPort. The Data FlowSet is also used to export the Options Data Record(s) containing the real values of SELECTOR-ID, SAMPLING-ALGORITHM, SAMPLING-PARAMETERS. As a consequence, the collector can link any Flow Data Record to the sampling algorithm and sampling parameters, by means of the SELECTOR-ID value. Claise, et. al Standard Track [Page 12] PSAMP Protocol Specifications February 2004 7.2.1 Example EDITOR’S NOTE: THIS MUST BE A FULL EXAMPLE LIKE IN SECTION 13 OF [IPFIX-PROTO]. THE [PSAMP-INFO] MUST BE FIRST PUBLISHED. 7.3 Filtering Case EDITOR’S NOTE: ACTUALLY THE EXAMPLE WILL BE QUITE SIMILAR TO 7.1 AND 7.2 BUT WILL DEPEND A LOT ON HOW WE WILL DEFINE THE FILTERING IN [IPFIX-INFO]. 7.3.1 Example EDITOR’S NOTE: THIS MUST BE A FULL EXAMPLE LIKE IN SECTION 13 OF [IPFIX-PROTO]. THE [PSAMP-INFO] MUST BE FIRST PUBLISHED. 8. Security Considerations As IPFIX has been selected as the PSAMP export protocol and as the PSAMP security requirements are not stricter than the IPFIX security requirements, refer to the IPFIX export protocol [IPFIX-PROTO] for the security considerations. 9. IANA Considerations The only IANA considerations in this document concerns the extension of Information Elements, FlowSet ID and Scope. Refer to the IANA considerations section in [IPFIX-PROTO] where those possible new assignments are specified. 10. References 10.1 Normative References [PSAMP-SAMPLE-TECH] T. Zseby, M. Molina, F. Raspall, N. Duffield "Sampling and Filtering Techniques for IP Packet Selection" draft- ietf-psamp-sample-tech-01.txt [PSAMP-MIB] T. Dietz, D. Romascanu, B. Claise "Definitions of Managed Objects for Packet Sampling" draft-ietf-psamp-mib-01.txt Claise, et. al Standard Track [Page 13] PSAMP Protocol Specifications February 2004 [PSAMP-INFO] T. Dietz, F. Dressler, G. Carle, B. Claise, "Information Model for Packet Sampling Exports", draft-ietf-psamp- info-00.txt [IPFIX-ARCH] G. Sadasivan, N. Brownlee "Architecture Model for IP Flow Information Export" draft-ietf-ipfix-arch-02.txt", June 2003 [IPFIX-INFO] P. Calato, J. Meyer, J. Quittek, "Information Model for IP Flow Information Export" draft-ietf-ipfix-info-02, August 2003 [IPFIX-PROTO] B. Claise, M. Fullmer, P. Calato, R. Penno, "IPFIX Protocol Specifications", draft-ietf-ipfix-protocol-02.txt, June 2003 [RFC1771] Y. Rekhter, T. Li, "A Border Gateway Protocol 4 (BGP- 4)", RFC 1771, March 1995. 10.2 Informative References [PSAMP-FRAMEWORK] N. Duffield, D. Chiou, B. Claise, A. Greenber, M. Grossglauser "A Framework for Passive Packet Measurement" draft- ietf-psamp-framework-04.txt [IPFIX-REQ] J. Quittek, T. Zseby, B. Claise, S. Zander, "Requirements for IP Flow Information Export" draft-ietf-ipfix-reqs- 10.txt, June 2003 11. Acknowledgments To be completed. Author’s Addresses Benoit Claise Cisco Systems De Kleetlaan 6a b1 1831 Diegem Belgium Phone: +32 2 704 5622 E-mail: bclaise@cisco.com Juergen Quittek NEC Europe Ltd. Network Laboratories Claise, et. al Standard Track [Page 14] PSAMP Protocol Specifications February 2004 Kurfuersten-Anlage 36 69115 Heidelberg Germany Phone: +49 6221 90511-15 Email: quittek@ccrle.nec.de Claise, et. al Standard Track [Page 15]