SIPPING Working Group H. Kaplan Internet Draft Acme Packet Intended status: Standards Track Expires: December 16, 2009 June 16, 2009 PCAP-compatible Binary Syntax for SIP Common Log File Format draft-kaplan-sipping-clf-pcap-00 Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on December 16, 2008. Copyright Notice Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license- info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Kaplan Expires December 16, 2009 [Page 1] PCAP-Compatible CLF Format June 2009 Abstract This document proposes a libpcap/PCAP-compatible binary syntax for the SIP common log format (CLF). It does not cover semantic issues, and is meant to be evaluated in the context of the other efforts discussing SIP CLF. Table of Contents 1. Introduction..........................................2 2. Terminology...........................................3 3. Applicability.........................................3 4. Definitions...........................................4 5. Format Overview.......................................4 5.1. CLF Parsed Record Size Limitations....................6 6. File Header Format....................................7 7. Parsed Record Format..................................7 7.1. Parsed Record Header Format...........................8 7.2. Parsed Record TLV Field Format.......................12 7.3. Vendor--specific TLV Fields..........................13 7.4. Defined CLF Fields...................................14 8. Raw Record Format....................................14 9. Handling Errors and Failures.........................16 10. Example..............................................16 11. Security Considerations..............................16 12. IANA Considerations..................................16 13. Acknowledgements.....................................16 14. Informative References...............................16 Author's Address............................................17 Appendix A: CLF RADIUS Dictionary...........................17 1. Introduction The Common Log File (CLF) format for the Session Initiation Protocol (SIP) [gurbani-clf] proposes a syntax for logging SIP messages received and sent by SIP clients, servers, and proxies. The syntax proposed by that document has been inspired by the common HTTP log format. However, experience with that format has shown that dealing with large quantities of log data can be very processor intensive, as doing so necessary requires reading and parsing every byte in the log file(s) of interest. An alternative "hybrid" format, defined in [roach-clf], proposes a pseudo-binary format, whereby each CLF record entry contains a fixed format header, with fixed length fields identifying the pointer-values and lengths of some mandatory SIP message values. The message values themselves are appended after the header, as are any non-mandatory/optional values which are encoded in a TLV Kaplan Expires - December 2008 [Page 2] PCAP-Compatible CLF Format June 2009 style. However, the actual pointers, lengths, and TLV encodings are the ASCII representation of the hex values. Furthermore, the SIP message values themselves are separated by TAB characters, to aid text-processing tools (e.g., sed, Perl, grep, etc.) to parse the CLF file. This document proposes an alternative format: a pure binary format, which is designed to be compatible with the PCAP file format [pcap-format]. The CLF record entries are also designed to follow a format which PCAP-file decoders can decode as RADIUS messages without any changes. A RADIUS dictionary file is also defined, which can be used by such decoders to provide filtering/searching and human-readable output. For example, the dictionary file in Appendix A can be just copied into the "radius" directory of Wireshark [wireshark], as described in Appendix A, and Wireshark can use it without re-compiling. There are several motivations for this format: 1) The number of PCAP-file-format decoders available is significantly large, with open source implementations; and they are widely popular. 2) PCAP-format decoding libraries are widely available, with numerous licensing models, including Open Source. 3) For troubleshooting purposes, this author feels nothing short of the entire SIP message will do. The format in this document supports recording the entire SIP message in a highly efficient manner, with the ability to retain as much of the original message as the recorder has available. 4) The encoding method proposed allows both extension by standardization, and extension by vendor-specific (proprietary) means, with separate type-number spaces. (because RADIUS supports Vendor-Specific-Attributes) 2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. The terminology in this document conforms to RFC 2828, "Internet Security Glossary". 3. Applicability This draft proposes a Common Log File (CLF) format for use in SIP- based equipment, using a RADIUS format. Kaplan Expires - December 2008 [Page 3] PCAP-Compatible CLF Format June 2009 4. Definitions Parsed Record: a CLF Record entry that contains portions of the SIP Message, as well as Layer-3 and other related information, in a specific format useful for direct parsing/searching of the semantic data. Raw Record: a CLF Record entry that contains the entire SIP Message unchanged/un-parsed, with as much of the IP and transport layer header information as possible. 5. Format Overview The Common Log File (CLF) format defined in this document defines a format which is designed to mimic the format of PCAP-file containing Ethernet packets. There is a single CLF File Header, followed by any number of CLF Record Entries. There are two types of CLF Record Entries defined in this document: Parsed and Raw. Parsed Record Entries are designed to appear to PCAP-decoders as RADIUS records, including their Ethernet, IP, and UDP headers. Raw Record Entries are designed to appear as SIP messages, including their Ethernet, IP, and UDP or TCP headers. Although the overhead of these pseudo-layers may seem processing intensive, they are themselves fixed-format, fairly small, and almost all of their content value is fixed as well. In other words, in actual code they should be low overhead. In order to distinguish CLF Record Entries from real on-the-wire captured packets, the CLF Record Header for both Parsed and Raw formats sets certain values which could not appear on-the-wire. This allows a mix of real on-the-wire captured packets and CLF Record Entries to co-exist in the same CLF file, if Ethernet is the interface medium. (This restriction is due to the PCAP file format, which does not allow a mix of different link-layer types in a single file) There is no specific end-of-file trailer. Furthermore, using this format allows vendors to add CLF fields themselves, by defining their vendor-specific fields using their IANA-assigned vendor codes (SMI Enterprise numbers) in their vendor-specific RADIUS dictionary files, and encoding them in the Parsed Record format along with the ones defined in this document. Kaplan Expires - December 2008 [Page 4] PCAP-Compatible CLF Format June 2009 A schematic of a typical CLF File is shown below: +=======================================+ | CLF File | | +===================================+ | | | CLF File Header | | | +===================================+ | | | CLF Parsed Record | | | | +-------------------------------+ | | | | | Parsed Record Header | | | | | | [Mandatory Data] | | | | | +-------------------------------+ | | | | +-------------------------------+ | | | | | TLV Field (Optional) | | | | | | . . . | | | | | +-------------------------------+ | | | | +-------------------------------+ | | | | | TLV Field (Optional) | | | | | | . . . | | | | | +-------------------------------+ | | | +===================================+ | | | CLF Parsed Record | | | | +-------------------------------+ | | | | | Parsed Record Header | | | | | | [Mandatory Data] | | | | | +-------------------------------+ | | | | +-------------------------------+ | | | | | TLV Field (Optional) | | | | | | . . . | | | | | +-------------------------------+ | | | | . . . | | | +===================================+ | | | CLF Raw Record (Optional) | | | | +-------------------------------+ | | | | | Raw Record Header | | | | | | [Mandatory Data] | | | | | +-------------------------------+ | | | | +-------------------------------+ | | | | | Raw SIP Message | | | | | | . . . | | | | | | . . . | | | | | +-------------------------------+ | | | | . . . | | | +===================================+ | | . . . | +=======================================+ Figure 1: Typical CLF File Structure Kaplan Expires - December 2008 [Page 5] PCAP-Compatible CLF Format June 2009 5.1. CLF Parsed Record Size Limitations Since the CLF Parsed Record Entries follow a virtual RADIUS format, there are certain restrictions on sizes and numbers of fields. In particular, each Record Field is a TLV with single octet Length fields, and thus once the overhead is included, there can be no more than 247 characters from any SIP message field recorded into a single CLF Parsed Record Field. Furthermore, the entire CLF Parsed Record cannot exceed 4140 Bytes, including the Parsed Record Header. This is so that the resulting RADIUS message itself is no larger than 4096 bytes, per the RADIUS RFC. [note: some generic decoders cannot parse it if it exceeds this size] This can be both a beneficial and detrimental property: it is beneficial because it creates field and record size constraints, which should in theory mean the time and processing overhead of generating and decoding records is bounded; it is detrimental because SIP header fields can technically exceed this length. However, the author feels the trade-off is worth it, and that truncating header field values is not a bad idea for the purposes of a CLF file parsed data. Otherwise, if extremely large records could be created, it would clearly open up the recorder and parser to resource exhaustion attacks. The 4140 byte limit still allows a minimum of 16 maximum length (247-character) SIP header field values to be recorded in a Parsed Record entry, which is a lot of field data to report in a log entry that is not meant to be a raw capture of the SIP Message. [TBD is this a real problem? If so we could use the DIAMETER format instead, but it has higher overhead compared with RADIUS format] Kaplan Expires - December 2008 [Page 6] PCAP-Compatible CLF Format June 2009 6. File Header Format The format of the CLF File Header is designed to mimic the PCAP- format file header. It is formatted as follows: 0 7 8 15 16 23 24 31 +--------+--------+--------+--------+ | Magic = 0xD4B3C2A1 | 0 - 3 +--------+--------+--------+--------+ | Version = 0x02000400 | 4 - 7 +--------+--------+--------+--------+ | Timezone_Correction = 0x00 | 8 - 11 +--------+--------+--------+--------+ | Timestamp_Accuracy = 0x00 | 12 - 15 +--------+--------+--------+--------+ | Saved_Len = 0xFFFF0000 | 16 - 19 +--------+--------+--------+--------+ | Link_Type = 0x01000000 | 20 - 23 +--------+--------+--------+--------+ Figure 2: CLF File Header Structure The values of each 32-bit field is shown in the figure above. As currently specified, an implementaiotn compliant with this document will use the exact values above, and they essentially have no meaning for CLF. See [pcap-format] for details on what each field represents for a native PCAP file-format decoder. It is TBD if the Magic, Timezone, and Timestamp fields will be dynamically usable for the CLF format (i.e., whether we should allow them to have values which actually mean what PCAP fields mean, for those purposes). 7. Parsed Record Format Kaplan Expires - December 2008 [Page 7] PCAP-Compatible CLF Format June 2009 The format of the CLF Parsed Record is designed to mimic a RADIUS Accounting-Status message, in an Ethernet/IP/UDP packet. As such, there is a fixed-format header, followed by any number of TLV fields, up to the maximum sizes which can be encoded. A Parsed Record is distinguishable from the Raw Record by the Record Type fields, which is where the UDP source and destination port numbers are in a real IP/UDP packet. 7.1. Parsed Record Header Format The Parsed Record Header is formatted as follows: 0 7 8 15 16 23 24 31 +--------+--------+--------+--------+ | Timestamp-Seconds | 0 - 3 +--------+--------+--------+--------+ | Timestamp-Microseconds | 4 - 7 +--------+--------+--------+--------+ | Full-Record-Length | 8 - 11 +--------+--------+--------+--------+ | Recorded-Record-Length | 12 - 15 +--------+--------+--------+--------+ | CLF-Record-Entry-Indicator1 | 16 - 19 +--------+--------+--------+--------+ | CLF-Record-Entry-Indicator2 | 20 - 23 +--------+--------+--------+--------+ | CLF-Record-Entry-Indicator3 | 24 - 27 +--------+--------+--------+--------+ | IP4-AF-Type | 0x4500 | 28 - 31 +--------+--------+--------+--------+ | Length-1 | Rec-Seq-Num | 32 - 35 +--------+--------+--------+--------+ | Unused (0x0000) | 0xFF11 | 36 - 39 +--------+--------+--------+--------+ | Unused (0x0000) | Rem-IP4-High | 40 - 43 +--------+--------+--------+--------+ | Rem-IP4-Low | 44 - 47 +--------+--------+--------+--------+ | Loc-IP4-High | 48 - 51 +--------+--------+--------+--------+ | Loc-IP4-Low | Record-Type | 52 - 55 +--------+--------+--------+--------+ | Record-Type | Length-2 | 56 - 59 +--------+--------+--------+--------+ | Unused (0x0000) | 0x0600 | 60 - 63 +--------+--------+--------+--------+ | Length-3 | Unused (0x0000) | 64 - 67 +--------+--------+--------+--------+ Kaplan Expires - December 2008 [Page 8] PCAP-Compatible CLF Format June 2009 | Unused (all 0x00) | 68 - 71 +--------+--------+--------+--------+ | Unused (all 0x00) | 72 - 75 +--------+--------+--------+--------+ | Unused (all 0x00) | 76 - 79 +--------+--------+--------+--------+ | Unused (0x0000) | 0x1A |Mand-Len| 80 - 83 +--------+--------+--------+--------+ | SIP-CLF-Parsed-Record-VID | 84 - 87 +--------+--------+--------+--------+ | 0x01 | 0x06 | 0x00 |Trns-Typ| 88 - 91 +--------+--------+--------+--------+ | Direct |Msg-Type| 0x02 | 0x06 | 92 - 95 +--------+--------+--------+--------+ | SIP-Response-Code | 96 - 99 +--------+--------+--------+--------+ | 0x03 | 0x06 | SIP-CSeq-High | 100 - 103 +--------+--------+--------+--------+ | SIP-CSeq-Low | 0x04 | 0x06 | 104 - 107 +--------+--------+--------+--------+ | 0x0000 | SIP-Remote-Port | 108 - 111 +--------+--------+--------+--------+ | 0x05 | 0x06 | 0x0000 | 112 - 115 +--------+--------+--------+--------+ | SIP-Local-Port | 0x06 |MethdLen| 116 - 119 +--------+--------+--------+--------+ | Method string... | 120 - n +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+ Figure 3: Parsed Record Header Structure The fields are defined as follows, in order of the file format shown above: Timestamp-Seconds: This is the timestamp in seconds since the Unix epoch (midnight, UTC, Jan 1, 1970). Note that in the year 2038 this value will wrap. Timestamp-Microseconds: this is the number of microseconds after the seconds above. Full-Record-Length: this is the size of the entire record in bytes, not counting the Timestamp fields above and the next Recorded_Record_Length. This is typically the same value as the Recorded_Record_Length, except if the CLF Record could not store all the data the recorder had for it, then this field MUST be used to indicate that by having it larger than the actual recorded length. Recorded-Record-Length: this is the size of the entire record in bytes, starting *after* this field (i.e., not counting the Kaplan Expires - December 2008 [Page 9] PCAP-Compatible CLF Format June 2009 previous Timestamp and Full_Record_Length, nor this field itself). CLF-Record-Entry-Indicator1/2/3: each of these fields is set to all 0x00 values, which indicates this Record is a CLF Record Entry (because it makes the Ethernet source/dest be all Zeroes). IP4-AF-Type: this field is set to either 0x0800 to indicate an IPv4-based CLF Record Header Format, or to 0x86DD to indicate an IPv6-based CLF Record Header Format. Only the IPv4 formatted one is shown in this document. This mimics the Ether-type field. Length-1: this field is set to the number of bytes in the CLF Record Entry, starting from octet 30 (the 0x4500) onwards. This mimics the IP Header Length field. Rec-Seq-Num: this field is used for an incrementing counter value for each Record in a file, wrapping at 65535. Rem-IP4-High/Low: these 4 octets are used to encode the IPv4 address of the remote device which sent the SIP message. This mimics the IPv4 source address field. Loc-IP4-High/Low: these 4 octets are used to encode the IPv4 address of the local device's interface which received the SIP message. This mimics the IPv4 dest address field. Record-Type: this field indicates the type of CLF Record this entry is; currently 0x0715 is being used. Note this field is duplicated twice. This mimics the UDP source/dest port number, and 0x0715 is the well-known port 1813 for RADIUS. Length-2: this field is set to the number of bytes in the CLF Record Entry, starting from octet 54 (the first Record-Type field) onwards. This mimics the UDP Length field. Length-3: this field is set to the number of bytes in the CLF Record Entry, starting from octet 62 (the 0x0600) onwards. This mimics the RADIUS Length field. Mand-Len: this field is set to the number of bytes starting from octet 82 onwards, up to and including the last octet of the Method string name. This is effectively a length field of a RADIUS TLV for a SIP-CLF VSA, which indicates the length of its members, which includes multiple TLV's. Kaplan Expires - December 2008 [Page 10] PCAP-Compatible CLF Format June 2009 SIP-CLF-Parsed-Record-VID: this field is set to an IANA-assigned RADIUS Vendor ID (SMI Enterprise Number) for the SIP-CLF file format. In the example dictionary in Appendix A, the decimal number 33800 is temporarily being used. TransTyp: this 1-octet field is set to the transport type of the original SIP message received: 0 for UDP, 1 for TCP, 2 for TLS/TCP, 3 for SCTP, and 4 for DTLS. More defined values may be added in the future. Direct: this 1-octet field is set to the direction of the message: 0 for received, 1 for sent. Msg-Type: this 1-octet field is set to the type of SIP message received: either 0 for unknown, 1 for Request, or 2 for Response. [and possibly 3 for STUN?] SIP-Response-Code: this field is set to the numeric value of the SIP response code, if the SIP Message was a response, or to 0 otherwise. SIP-CSeq-High/Low: this field is set to the high and low halves of a 32-bit integer representing the received CSeq in the SIP Message. SIP-Remote-Port: this field is set to the transport port number (e.g., UDP port) of the remote device which sent the SIP message. SIP-Local-Port: this field is set to the transport port number of the local device which received the SIP Message. MethdLen: this field is set to the number of bytes starting at octet 112 onwards, up to and including the last octet of the SIP Method string. Method-String: this is set to the UTF-8 characters of the SIP Method of the received SIP message, not including any whitespace nor a NULL termination. [TBD: we could make methods an enumerated integer value instead] The CLF Parsed Record Header Format described above is for an IPv4-based SIP Message. An IPv6-based one will be documented if this document goes forward, though it is probably obvious what such a format would look like. In summary, the following information from the received SIP Message is encoded in the mandatory Parsed Record Header portion described above: Kaplan Expires - December 2008 [Page 11] PCAP-Compatible CLF Format June 2009 IPv4 Addresses: the IP Address of the sender of the received SIP Message, and the local IP Address of the receiver of the SIP Message. Transport Protocol: the transport protocol (UDP/TCP/etc.). Transport Port Numbers: the local and remote transport port numbers of the SIP Message. Direction: whether the message was received or sent. SIP Message Type: whether the SIP Message is a request or a response. SIP Response Code: what the response code of the SIP Message is, if it's a response (0 otherwise). SIP Method: the method name of the SIP message. SIP CSeq: the CSeq value of the SIP Message. Other, optional CLF Fields are recorded in subsequent TLV fields as described later. 7.2. Parsed Record TLV Field Format After the Parsed Record Header section, zero or more Parsed Record Entry Fields can be appended to the Record. Although this is described as a TLV format, due to the mechanics of encoding RADIUS Vendor-Specific Attribute fields, the actual "TLV" has 6 octets of fixed-value which represent the RADIUS VSA TLV, followed by the octets for the specific CLF Record Entry Field Type and Length as a sub-TLV. Although in theory one could encode multiple sub-TLV's in one VSA TLV, this would require additional processing logic for both the encoder and decoder. SIP Message fields are of arbitrary size, such that encoding and efficient parsing of variable-length strings as sub-TLV's of variable-length VSA TLV's is simply too error-prone, and unlikely to yield much benefit in terms of size reduction. Thus this document trades memory for speed, by encoding all CLF Fields beyond those in the Parsed Record Entry Header, in individual VSA TLV's. The format of the Parsed Record TLV Field is: 0 7 8 15 16 23 24 31 +--------+--------+--------+--------+ | 0x1A | Len-1 | PR-VID_high | 0 - 3 +--------+--------+--------+--------+ | PR-VID_low |FieldTyp| Len-2 | 4 - 7 +--------+--------+--------+--------+ | Field-Value | 8 - n +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+ Figure 4: Parsed Record TLV Field Structure Kaplan Expires - December 2008 [Page 12] PCAP-Compatible CLF Format June 2009 The fields are defined as follows, in order of the format shown above: Len-1: This is set to the number of octets of the entire Record Field, starting from octet 0 onwards, up to and including the last octet of the Field-Value. PR-VID_high/low: This is set to the high and low halves of the IANA-assigned RADIUS Vendor ID (SMI Enterprise Number) for SIP CLF Parsed Record purposes. This is the same value as the similar-named field in the Parsed Record Header. FieldTyp: this is set to the specific Parsed Record Field Type shown in the Table 1 later. Len-2: this is set to the number of octets starting at octet 6 (the FieldTyp octet), up to and including the last octet of the variable-length Field-Value. Field-Value: this is set to the value of the Parsed Record Entry Field. This can be any length up to 247 octets, and no less than 1. Zero-length values should not be encoded, and their Record Field should simply not be included. Field-Values of representational-type "integer" MUST be 4 octets in length. Field-Values of representational-type "string" are not NULL terminated, and SHOULD be considered raw octet strings, not merely UTF-8. 7.3. Vendor--specific TLV Fields Because the format defined in this document follows a pseudo- RADIUS message format, every TLV is encoded using a Vendor- Specific-Attribute (VSA) RADIUS type. Therefore, it is possible for vendors using this format to define their own additional fields by simply using their own VSA number space. Kaplan Expires - December 2008 [Page 13] PCAP-Compatible CLF Format June 2009 7.4. Defined CLF Fields The following SIP CLF Fields are defined thus far: Name Type-Number Rep-Type ----------- ----------- --------- SIP-Message-Type 1 integer SIP-Response-Code 2 integer SIP-CSeq-Number 3 integer SIP-Remote-Port 4 integer SIP-Local-Port 5 integer SIP-Method 6 string SIP-Call-ID 10 string SIP-To-Value 11 string SIP-To-Tag 12 string SIP-From-Value 13 string SIP-From-Tag 14 string SIP-Request-URI 15 string SIP-Contact-Value 16 string SIP-Max-Forwards-Value 17 string SIP-Server-Transaction 20 string SIP-Client-Transaction 21 string SIP-Session-Id 22 string SIP-Session-Ingress-Realm 23 string SIP-Session-Egress-Realm 24 string SIP-Originating-Trunk-Group 25 string SIP-Terminating-Trunk-Group 26 string SIP-Originating-Trunk-Context 27 string SIP-Terminating-Trunk-Context 28 string SIP-P-Asserted-ID 29 string SIP-History-Info 30 string Table 1: Defined Record Field Types Note that Field Types 1-7 are mandatory, and included in the Parsed Record Header section, and thus MUST NOT be encoded as separate TLV's in Parsed Record Fields - only in the header (which itself follows a RADIUS VSA TLV format). Although the mandatory ones can only appear once (in the header), other fields MAY be repeated multiple times if multiple instances were in the SIP Message (e.g., multiple Contact URI's). 8. Raw Record Format The Raw Record entry is designed to mimic a real, full, SIP Message being captured in a PCAP file. The format follows a PCAP Ethernet packet format, except the Ethernet source and destination Kaplan Expires - December 2008 [Page 14] PCAP-Compatible CLF Format June 2009 MAC Addresses are set to all 0x00, which distinguishes it from real on-the-wire captured SIP Messages and in theory allows both to co-exist in a single CLF file. [TBD: do we wish to allow that?] The IP Header MUST contain the real remote/source and local/destination IP Addresses of the received SIP Message. Other fields or optional headers in the IP Header SHOULD be those from the received SIP Message if possible, but this may not be possible for all system types to do, and certainly not when TLS is used, for example. SIP Messages received over UDP have a UDP header after the IP Header, those over TCP have a TCP header, and those over SCTP have an SCTP header; each header type would include the correct received transport port numbers in the appropriate fields. SIP Messages received over TLS/TCP would only have the TCP header, and their clear-text content would be inside that header. Likewise for DTLS, a UDP header only is used. [Note: we could steal bits from lower layers if we wish to encode TLS/DTLS was used in flags] The UDP, TCP, or SCTP headers MUST at least provide correct length field values, in order for parsers to decode them properly; and they SHOULD use appropriate sequence numbers (if applicable) to avoid generic PCAP decoders from believing they are re- transmissions. Checksums MAY be set to zero, to avoid the performance penalty of calculating them. (checksum errors typically don't stop PCAP parsers) The full format of such a Raw Record is not shown here, for the sake of brevity. Only the PCAP record entry header format is shown, as a convenience: 0 7 8 15 16 23 24 31 +--------+--------+--------+--------+ | Timestamp-Seconds | 0 - 3 +--------+--------+--------+--------+ | Timestamp-Microseconds | 4 - 7 +--------+--------+--------+--------+ | Full-Record-Length | 8 - 11 +--------+--------+--------+--------+ | Recorded-Record-Length | 12 - 15 +--------+--------+--------+--------+ | First 4 octets of Ethernet hdr...| 16 - n +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+ Figure 5: Raw Record Header Structure Kaplan Expires - December 2008 [Page 15] PCAP-Compatible CLF Format June 2009 The fields above are the same as those of the same name from the Parsed Record Header Format. Note that the Ethernet source and destination MAC addresses would be set to all 0x00, and the Ether-Type would be set to 0x0800 for IPv4 and 0x86DD for IPv6. In that sense this record looks exactly like the Parsed Record Format until the Record-Type octets, which are both 0x0715 for Parsed Record Format, and if otherwise assumed to be Raw Record Format. Since the IP Header may contain optional/extension headers, parsers need to check the value of the IP header length and protocol type fields to determine where to find the transport port numbers (i.e., the Record Type Fields). 9. Handling Errors and Failures TBD. 10. Example Because the contents are binary, the contents of a CLF file following this format cannot be shown directly in this document. However, an example output CLF file, as well as a C-code library to generate such, is available at: https://sourceforge.net/projects/sip-clf 11. Security Considerations There are no specific security considerations for this document's mechanism, other than the usual caution for implementers in dealing with length field limits, treating octet strings as ASCII- strings, buffer overruns, etc. 12. IANA Considerations This document requests the Internet Assigned Numbers Authority (IANA) to allocate a new SMI Network Management Private Enterprise Code for the IETF SIP-CLF Working Group, for the purpose of creating Vendor-Specific RADIUS Attributes. 13. Acknowledgements Thanks to Richard Hamming for his inspiring quote: "It is better to do the right problem the wrong way than the wrong problem the right way." 14. Informative References Kaplan Expires - December 2008 [Page 16] PCAP-Compatible CLF Format June 2009 [gurbani-clf] Gurbani, V., et al, "The Common Log File (CLF) format for the Session Initiation Protocol", draft-gurbani- sipping-clf-01, March 2009. [roach-clf] Roach, A., "Binary Syntax for SIP Common Log Format", draft-roach-sipping-clf-syntax-01, May 2009. [pcap-format] The Libpcap/PCAP format is documented in: http://wiki.wireshark.org/Development/LibpcapFileFormat [SIP-CLF-Lib] https://sourceforge.net/projects/sip-clf Author's Address Hadriel Kaplan Acme Packet 71 Third Ave. Burlington, MA 01803, USA Email: hkaplan@acmepacket.com Appendix A: CLF RADIUS Dictionary Since the CLF format defined in this document follows a virtual RADIUS format, one can define the Record Fields using the well- known RADIUS Dictionary schema. Some PCAP file format decoders, such as Wireshark [wireshark], can decode RADIUS messages without needing to be re-compiled/upgraded. For example, putting the following file in the radius directory of the Wireshark program directory allows Wireshark to decode the CLF Record format automatically, because the Wireshark application implements a late-binding approach for RADIUS decoding/dissecting. To use this in Wireshark, perform the following steps: 1) Copy the following file into a file named "dictionary.sipclf", using a plain text editor (e.g., Notepad, etc.). [note: do not add a .txt extension, and remove it if one is auto-appended] 2) Put the file into the radius directory of Wireshark. 3) In that same directory there should be a file called "dictionary"; open it for editing using a plain text editor, and add the line "$INCLUDE dictionary.sipclf" (no quotes) in this file. This will make Wireshark look for and read the dictionary.sipclf file to use. 4) Restart Wireshark. You're done! You should be able to open any CLF formatted file and view the CLF Parsed Record entries. Kaplan Expires - December 2008 [Page 17] PCAP-Compatible CLF Format June 2009 The Dictionary file is shown below, between the two dashed lines for begin and end: --filename:dictionary.sipclf------begin---------------------- # # dictionary.sipclf # # # Version: dictionary.sipclf,v 1.0 2009/06/13 # # For documentation on IETF SIP CLF RADIUS attributes, see: # # NOTE: Vendor number 33800 is temporary (not allocated by IANA) # # $Id: dictionary.sipclf 12349 2009-06-13 16:32:12Z hkaplan $ # VENDOR SIP-CLF 33800 BEGIN-VENDOR SIP-CLF # Note: the following are mandatory and encoded in the Parsed # Record Header portion ATTRIBUTE SIP-Message-Type 1 integer SIP-CLF ATTRIBUTE SIP-Response-Code 2 integer SIP-CLF ATTRIBUTE SIP-CSeq-Number 3 integer SIP-CLF ATTRIBUTE SIP-Remote-Port 4 integer SIP-CLF ATTRIBUTE SIP-Local-Port 5 integer SIP-CLF ATTRIBUTE SIP-Method 6 string SIP-CLF # the following will be in separate, VSA-wrapped TLV's ATTRIBUTE SIP-Call-ID 10 string SIP-CLF ATTRIBUTE SIP-To-Value 11 string SIP-CLF ATTRIBUTE SIP-To-Tag 12 string SIP-CLF ATTRIBUTE SIP-From-Value 13 string SIP-CLF ATTRIBUTE SIP-From-Tag 14 string SIP-CLF ATTRIBUTE SIP-Request-URI 15 string SIP-CLF ATTRIBUTE SIP-Contact-Value 16 string SIP-CLF ATTRIBUTE SIP-Max-Forwards-Value 17 string SIP-CLF ATTRIBUTE SIP-Server-Transaction 20 string SIP-CLF ATTRIBUTE SIP-Client-Transaction 21 string SIP-CLF ATTRIBUTE SIP-Session-Id 22 string SIP-CLF ATTRIBUTE SIP-Session-Ingress-Realm 23 string SIP-CLF ATTRIBUTE SIP-Session-Egress-Realm 24 string SIP-CLF ATTRIBUTE SIP-Originating-Trunk-Group 25 string SIP-CLF ATTRIBUTE SIP-Terminating-Trunk-Group 26 string SIP-CLF ATTRIBUTE SIP-Originating-Trunk-Context 27 string SIP-CLF ATTRIBUTE SIP-Terminating-Trunk-Context 28 string SIP-CLF Kaplan Expires - December 2008 [Page 18] PCAP-Compatible CLF Format June 2009 ATTRIBUTE SIP-P-Asserted-ID 29 string SIP-CLF ATTRIBUTE SIP-History-Info 30 string SIP-CLF # this needs to be expanded on VALUE SIP-Response-Code Not-Applicable 0 VALUE SIP-Response-Code Trying 100 VALUE SIP-Response-Code Ringing 180 VALUE SIP-Response-Code Progress 183 VALUE SIP-Response-Code OK 200 VALUE SIP-Response-Code Unauthorized 401 VALUE SIP-Response-Code Forbidden 403 VALUE SIP-Response-Code Service-Unavailable 503 # the SIP-Message-Type 4-byte integer is actually a byte-flag, # of {0,transport,direction,type} - hence the following VALUE SIP-Message-Type UDP-Received-Unknown-Msg 0 VALUE SIP-Message-Type UDP-Received-SIP-Request 1 VALUE SIP-Message-Type UDP-Received-SIP-Response 2 VALUE SIP-Message-Type UDP-Received-STUN-Msg 3 VALUE SIP-Message-Type UDP-Sent-Unknown-Msg 256 VALUE SIP-Message-Type UDP-Sent-SIP-Request 257 VALUE SIP-Message-Type UDP-Sent-SIP-Response 258 VALUE SIP-Message-Type UDP-Sent-STUN-Msg 259 VALUE SIP-Message-Type TCP-Received-Unknown-Msg 65536 VALUE SIP-Message-Type TCP-Received-SIP-Request 65537 VALUE SIP-Message-Type TCP-Received-SIP-Response 65538 VALUE SIP-Message-Type TCP-Received-STUN-Msg 65539 VALUE SIP-Message-Type TCP-Sent-Unknown-Msg 65792 VALUE SIP-Message-Type TCP-Sent-SIP-Request 65793 VALUE SIP-Message-Type TCP-Sent-SIP-Response 65794 VALUE SIP-Message-Type TCP-Sent-STUN-Msg 65795 VALUE SIP-Message-Type TLS-Received-Unknown-Msg 131072 VALUE SIP-Message-Type TLS-Received-SIP-Request 131073 VALUE SIP-Message-Type TLS-Received-SIP-Response 131074 VALUE SIP-Message-Type TLS-Received-STUN-Msg 131075 VALUE SIP-Message-Type TLS-Sent-Unknown-Msg 131328 VALUE SIP-Message-Type TLS-Sent-SIP-Request 131329 VALUE SIP-Message-Type TLS-Sent-SIP-Response 131330 VALUE SIP-Message-Type TLS-Sent-STUN-Msg 131331 VALUE SIP-Message-Type SCTP-Received-Unknown-Msg 196608 VALUE SIP-Message-Type SCTP-Received-SIP-Request 196609 VALUE SIP-Message-Type SCTP-Received-SIP-Response 196610 VALUE SIP-Message-Type SCTP-Received-STUN-Msg 196611 VALUE SIP-Message-Type SCTP-Sent-Unknown-Msg 196864 VALUE SIP-Message-Type SCTP-Sent-SIP-Request 196865 VALUE SIP-Message-Type SCTP-Sent-SIP-Response 196866 VALUE SIP-Message-Type SCTP-Sent-STUN-Msg 196867 VALUE SIP-Message-Type DTLS-Received-Unknown-Msg 262144 VALUE SIP-Message-Type DTLS-Received-SIP-Request 262145 Kaplan Expires - December 2008 [Page 19] PCAP-Compatible CLF Format June 2009 VALUE SIP-Message-Type DTLS-Received-SIP-Response 262146 VALUE SIP-Message-Type DTLS-Received-STUN-Msg 262147 VALUE SIP-Message-Type DTLS-Sent-Unknown-Msg 262400 VALUE SIP-Message-Type DTLS-Sent-SIP-Request 262401 VALUE SIP-Message-Type DTLS-Sent-SIP-Response 262402 VALUE SIP-Message-Type DTLS-Sent-STUN-Msg 262403 END-VENDOR SIP-CLF --filename:dictionary.sipclf------end---------------------- Kaplan Expires - December 2008 [Page 20]