IPFIX Working Group B. Trammell Internet-Draft CERT/NetSA Expires: February 4, 2006 August 3, 2005 Simple IPFIX Files for Persistent Storage draft-trammell-ipfix-file-00.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on February 4, 2006. Copyright Notice Copyright (C) The Internet Society (2005). Abstract It is useful to be able to serialize network flow and network event data to a persistent data store for recording and analysis purposes with a minimal amount of translation overhead. However, persistent data stores are not constrained by the same limits on the IPFIX wire protocol. This document outlines a minimal set of simplifications to the IPFIX wire protocol for simple, efficient persistent storage of event data using IPFIX. Trammell Expires February 4, 2006 [Page 1] Internet-Draft Simple IPFIX Files August 2005 1. Introduction IPFIX provides a generic, extensible, templated binary format for the transfer of network flow and event data. This format can also be useful for persistent storage of such data; however, persistent storage applications are not subject to the length restrictions that constrain the IPFIX wire protocol. This document describes a file format based upon IPFIX, using undefined length messages and data sets to remove message and data set length limitations from the file representation. Though this leads to only minor gains in file size efficiency, it does simplify implementation. Specifically, existing applications which support fixed-length binary record formats can be made interoperable with applications built to support this IPFIX-derived format changing only information in the file header. 2. Terminology An IPFIX File is a single serialized IPFIX data stream containing the data from one or more IPFIX messages conforming to the rules in this draft. 3. IPFIX File structure An IPFIX File is structured as a single IPFIX Message of an undefined length. This message contains zero or more Template Sets or Options Template Sets followed by one or more Data Sets. The final Data Set of this file may itself have an undefined length. The details of each section of the file are outlined in the sections below. 3.1 Message Header IPFIX Files MUST begin with a single IPFIX Message Header, as in RFC XXXX section 3.1. The Length field of this Message Header MUST be zero, as the underlying serialization technology will indicate the length of the file. The Sequence Number of this Message Header SHOULD be zero, as the semantics of the Sequence Number field are not applicable to IPFIX Files. 3.2 Template Sets and Options Template Sets IPFIX Files SHOULD contain all the Template Sets and Options Template Sets required to parse every Data Set in the File. Template Sets and Options Template Sets MUST appear directly after the IPFIX File Message Header. Template Sets and Options Template Sets may be omitted if the Files Trammell Expires February 4, 2006 [Page 2] Internet-Draft Simple IPFIX Files August 2005 are intended for the private use of a single suite of tools (e.g., ephemeral storage by a store-and-forward IPFIX message cache); however, such is not recommended. If any Template Sets or Options Template Sets are present in an IPFIX File, it should be taken to mean that the file is not for private use, and all Template Sets and Options Template Sets referenced in the file MUST be present. 3.3 Data Sets Data Sets appear after the IPFIX File Message Header and any Template Sets and Options Template Sets, if present. Data Sets are represented in an IPFIX File identically as they are in IPFIX Messages, with a single extension. If all the data from a given Data Set Header to the end of the file conforms to the same Template, the Length field of that Data Set Header MAY be set to zero. This indicates that the remainder of the IPFIX File should be treated as an array of records of the length defined by that template. 4. Time Delta Information Elements Note that, since an IPFIX File has a single Message Header, it has a single Export Time. All Data Sets within the IPFIX File are scoped to this Export Time. Processes creating IPFIX files MUST therefore rewrite all time delta Information Elements in their input IPFIX Messages to the be based on the Export Time in the File's single Message Header. 5. Security Considerations None. 6. IANA Considerations None. 7. References [1] Claise, B., "IPFIX Protocol Specification", July 2005. Trammell Expires February 4, 2006 [Page 3] Internet-Draft Simple IPFIX Files August 2005 Author's Address Brian H. Trammell CERT Network Situational Awareness Software Engineering Institute 4500 Fifth Avenue Pittsburgh, PA 15213 US Phone: +1 412 268 9748 Email: bht@cert.org Trammell Expires February 4, 2006 [Page 4] Internet-Draft Simple IPFIX Files August 2005 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society (2005). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Trammell Expires February 4, 2006 [Page 5]