NMRG J.G. van den Broek Internet-Draft University of Twente Intended status: Informational J. Schoenwaelder Expires: October 25, 2008 Jacobs University Bremen A. Pras University of Twente M. Harvan ETH Zurich April 23, 2008 SNMP Trace Analysis Definitions draft-schoenw-nmrg-snmp-trace-definitions-02.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on October 25, 2008. Copyright Notice Copyright (C) The IETF Trust (2008). van den Broek, et al. Expires October 25, 2008 [Page 1] Internet-Draft SNMP Trace Analysis Definitions April 2008 Abstract The Network Management Research Group (NMRG) started an activity to collect traces of the Simple Network Management Protocol (SNMP) from operational networks. To analyze these traces, it is necessary to split potentially large traces into more manageable pieces that make it easier to deal with large data sets and simplify the analysis of the data. This document provides some common definitions that have been found useful for implementing tools to support trace analysis. This document mainly serves as a reference for the definitions underlying these tools and it is not meant to explain all the motivation and reasoning behind the definitions. Some of this background information can be found in other research papers. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3. Traces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4. Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 5. Slices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 6. Slice Prefix . . . . . . . . . . . . . . . . . . . . . . . . . 20 7. Slice Type . . . . . . . . . . . . . . . . . . . . . . . . . . 26 8. Walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 9. Concurrency . . . . . . . . . . . . . . . . . . . . . . . . . 30 10. Security Considerations . . . . . . . . . . . . . . . . . . . 31 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 33 13. References . . . . . . . . . . . . . . . . . . . . . . . . . . 34 13.1. Normative References . . . . . . . . . . . . . . . . . . 34 13.2. Informative References . . . . . . . . . . . . . . . . . 34 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 35 Intellectual Property and Copyright Statements . . . . . . . . . . 36 van den Broek, et al. Expires October 25, 2008 [Page 2] Internet-Draft SNMP Trace Analysis Definitions April 2008 1. Introduction The Simple Network Management Protocol (SMMP) was introduced in the late 1980s. Since then, several evolutionary protocol changes have taken place, resulting in the SNMP version 3 framework (SNMPv3) [RFC3410][RFC3411], published as full standard in 2002. Extensive use of SNMP has led to significant practical experience by both network operators and researchers. However, up until now only little research has been done on characterizing and modeling SNMP traffic. Since recently, network researchers are in the possession of network traces, including SNMP traces captured on operational networks. The availability of SNMP traces enables research on characterizing and modeling real world SNMP traffic. However, experience with SNMP traces has shown that the traces must be large enough in order to make proper observations. A more detailed motivation for collecting SNMP traces and guidelines on how to capture SNMP traces can be found in [ID-IRTF-NMRG-SNMP-MEASURE]. Unfortunately, the analysis of large SNMP traces can take a large amount of processing time. Therefore, it is often desirable to focus the analysis on smaller, relevant sections of a trace. This in turn requires a proper way to identify these smaller sections of a trace. This document describes a number of identifiable sections within a trace which make specific research on these smaller sections more practical. The following figure shows the various entities associated with SNMP traces and how they relate to each other. +---------+ 0..* 1 +-------+ 1 0..* +------+ | Message |------------->| Trace |----------->| Flow | +---------+ belongs_to +-------+ contains +------+ | 1 | | contains | v 0..* +------------+ 1 0..* +-------+ | Slice Type |<----------| Slice | +------------+ of_type +-------+ ^ 1 | | is_a | | 0..1 +-------+ | Walk | +-------+ van den Broek, et al. Expires October 25, 2008 [Page 3] Internet-Draft SNMP Trace Analysis Definitions April 2008 This document defines the various entities (boxes) shown in the above figure. These definitions can be implemented by tools that can split SNMP traces into smaller sections for further analysis. The most central entity in the figure above is an SNMP trace, consisting of a potentially large set of SNMP messages. An SNMP trace is the result of recording SNMP traffic on a specific network for a specific time duration. Such a trace may, depending on the number of hosts in the respective network, contain SNMP messages exchanged between possibly many different SNMP engines. The messages contained in a trace may be represented in different formats. For the purpose of this document, the simple comma separated values (CSV) format defined in [ID-IRTF-NMRG-SNMP-MEASURE] contains sufficient information to split a trace into smaller sections. The SNMP messages belonging to an SNMP trace may have been exchanged between many different SNMP engines running on different hosts. Therefore, a first obvious way of separating a trace into smaller sets of SNMP messages is the separation of a trace into flows. Each flow contains only those SNMP messages of an SNMP trace that have been exchanged between two network layer endpoints. Such a separation may be necessary in case one wants to analyze specific SNMP traffic characteristics (e.g., number of agents managed by a management station) and wants to rule out network endpoint specific behaviour (e.g., different SNMP management stations may have different polling configurations). Flows within traces can still be quite large in terms of the number of messages they contain. Therefore, it may be necessary to split a flow into even smaller sections called slices. A slice contains all SNMP messages of a given flow that are related to each other in time and referenced information. Splitting a flow into slices makes it possible to separate SNMP messages within traces that belong to each other. For example, a slice may contain the SNMP messages exchanged between an agent and a manager, which polls that agent in a single polling instance. The manager may be configured to poll that agent every once in a while. If the requested information from the agent remains unchanged, then the respective slices of SNMP traffic occurring between this manager and agent will be highly comparable. In such a case the slices will be of the same slice type. Similar slices will thus be considered of the same slice type and incomparable slices will not be of the same slice type. Besides the fact that each slice is of specific slice type, slices can also be of a specific form with respect to the messages encompassing a slice. For example, slices containing a sequence of van den Broek, et al. Expires October 25, 2008 [Page 4] Internet-Draft SNMP Trace Analysis Definitions April 2008 linked GetNext or GetBulk requests are commonly called an SNMP walk. Note that only a subset of all slices will be walks. van den Broek, et al. Expires October 25, 2008 [Page 5] Internet-Draft SNMP Trace Analysis Definitions April 2008 2. Messages SNMP messages carry PDUs associated with well defined specific protocol operations [RFC3416]. The PDUs can be used to classify SNMP messages. Following are a number of definitions that help to classify SNMP messages based on the PDU contained in them. These definitions will be used later on in this document. Notation: The properties of an SNMP message M are denoted as follows: M.type = operation type of message M (get, getnext, ...) M.class = class of message M (according to RFC 3411) M.tsrc = transport layer source endpoint of message M M.tdst = transport layer destination endpoint of message M M.nsrc = network layer source endpoint of message M M.ndst = network layer destination endpoint of message M M.reqid = request identifier of message M M.time = capture timestamp of message M M.oids = OIDs listed in varbind list of message M M.values = values listed in varbind list of message M These properties of an SNMP message can be easily extracted from the exchange formats defined in RFC XXXX [ID-IRTF-NMRG-SNMP-MEASURE]. Definition (read request message): A read request message is a message M containing a PDU of type GetRequest, GetNextRequest, or GetBulkRequest. Definition (write request message): A write request message is a message M containing a PDU of type SetRequest. Definition (notification request message): A notification request message is a message M containing a PDU of type InformRequest. Definition (notification message): A notification message is a message M containing a PDU of type Trap or InformRequest. Definition (request message): A request message is a message M which is either a read request message, a write request message, or a notification request message. Definition (response message): A response message is a message M containing a PDU of type Response or of type Report. Report messages are treated like Response messages since the SNMPv3 specifications currently use Report messages only as an error reporting mechanism, always triggered by the processing of some request messages. In case future SNMP versions or extensions use van den Broek, et al. Expires October 25, 2008 [Page 6] Internet-Draft SNMP Trace Analysis Definitions April 2008 Report messages without having a request triggering the generation of Report messages, the definition above may need to be revisited. Definition (non-response message): A non-response message is a message M which is either a read request message, a write request message, or a notification message. Definition (command message): A command message is a message M which is either a read request message or a write request message. Definition (command group messages): A set of command group messages consists of all messages M satisfying either of the following two conditions: (C1) M is a command message (C2) M is a response message and there exists a command message C such that the following holds: M.reqid = C.reqid M.tdst = C.tsrc M.tsrc = C.tdst (M.time - C.time) < t The parameter t defines a maximum timeout for response messages. This definition requires that the response message originates from the transport endpoint over which the request message has been received. This is not strictly required by SNMP transport mappings and in particular the UDP transport mapping allows to send responses from different transport endpoints. While sending response messages from a different transport endpoint is legal, it is also considered bad practice causing interoperability problems, since some management systems do not accept such messages. It was decided to require matching transport endpoints since doing so significantly simplifies the procedures below and avoids accidentally confusing requests and responses. Implementations responding from different transport endpoints will lead to (a) a larger number of requests without related responses (and likely no retries) and (b) a similarly large number of responses without a matching request. If such behavior can be detected, the traces should be investigated and if needed the transport endpoints corrected. The requirement for matching transport endpoints only affects request / response pairs. It is perfectly fine for a manager to use different transport layer endpoints in different polling instances or even for different operations (i.e., slices) within the same polling instance. van den Broek, et al. Expires October 25, 2008 [Page 7] Internet-Draft SNMP Trace Analysis Definitions April 2008 Definition (notification group messages): A set of notification group messages consists of all messages M satisfying either of the following two conditions: (N1) M is a notification message (N2) M is a response message and there exists a notification request message N such that the following holds: M.reqid = N.reqid M.tdst = N.tsrc M.tsrc = N.tdst (M.time - N.time) < t The parameter t defines a maximum timeout for response messages. This definition again requires matching transport endpoints for notification group messages. van den Broek, et al. Expires October 25, 2008 [Page 8] Internet-Draft SNMP Trace Analysis Definitions April 2008 3. Traces Traces are (large) sets of SNMP messages that are the result of recording SNMP traffic using a single traffic recording unit (e.g., using tcpdump) on a network segment carrying traffic of one or more managers and agents. Traces being used in the remainder of this document may be altered as a result of anonymization, which may result in some message information loss. Definition (trace): An SNMP trace (or short "trace") T is an ordered set of zero or more SNMP messages M. All messages M in T are chronologically ordered according to the capture timestamp M.time. van den Broek, et al. Expires October 25, 2008 [Page 9] Internet-Draft SNMP Trace Analysis Definitions April 2008 4. Flows Traces may contain SNMP messages that have been exchanged between possibly many different network layer endpoints. One way of making an initial separation of such a trace into more manageable pieces is by splitting the messages into flows. Each flow contains only messages that have occurred between two network layer endpoints. Definition (flow): A flow F with parameter t is the set of messages of an SNMP trace T with the following properties: (F1) All response messages originate from a single network endpoint. (F2) All non-response messages originate from a single network endpoint. (F3) All messages are either command group messages with parameter t or notification group messages with parameter t. Parameter t defines the maximum timeout for response messages and is the same for both the notification group messages and the command group messages which a flow may consist of. Parameter t is the same for both cases, because they each may contain non-response messages that originate from a single network endpoint with one timeout characteristic. [TODO: I do not understand the last two sentences.] The value of t should be chosen such that only response messages to the respective non-response messages are considered part of the same flow. Analysis of a large number of traces shows that 25 seconds is a proper default value for t. It is possible that response messages of a trace cannot be classified to belong to any flow. This can happen if request messages triggering the response messages were not recorded (for example due to asymmetric routing), or because response messages were originating from transport endpoints different from the endpoint used to receive the associated request message. Subsequently, flows containing only command group messages are called command flows. Similarly, flows containing only notification group messages are called notification flows. This definition of a flow indicates that it can be either unidirectional (e.g., a manager sending non-response messages to a non-responding agent), or bidirectional (e.g., a manager reading a table from an agent). This is different from other flow definitions, like the NetFlow definition [RFC3954]. van den Broek, et al. Expires October 25, 2008 [Page 10] Internet-Draft SNMP Trace Analysis Definitions April 2008 The flow definition is mostly consistent with the definition of an SNMP flow used in [SPHSM07]. The difference is that the tool used to generate the data reported in [SPHSM07] did only require that the network layer source endpoint of the response messages matches the destination network layer endpoint of the associated request messages. Definition (flow initiator): A flow initiator is the network layer endpoint of the two endpoints involved in a flow, which is responsible for sending the first non-response message. Notation: The properties of a flow F are denoted as follows: F.type = type of the flow F (command/notification) F.nsrc = network layer source endpoint of F F.ndst = network layer destination endpoint of F F.start = timestamp of the first message in F F.end = timestamp of the last message in F F.init = initiator of the flow F F.t = parameter t of F (maximum timeout for response messages) Following is an example of a trace consisting of SNMP messages that were exchanged between different network layer endpoints. The network layer endpoints are represented by A, B, C and D. This trace will be separated into flows. ---------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID. --------------------------------------------------------------- 0 0.0 A -> B GetNext Request 1 1 0.04 C -> D Get Request 10 2 0.05 B -> A Response 1 3 0.08 D -> C Response 10 4 0.11 A -> B GetNext Request 2 5 0.15 B -> A Response 2 6 0.18 A -> B GetNext Request 3 7 0.22 D -> C Trap 14 8 0.25 B -> A Response 3 ---------------------------------------------------------------- This trace contains SNMP messages that have been exchanged between different network layer endpoints. One flow will consist of SNMP messages that have been exchanged between network layer endpoints A and B, where all response messages originate from B and all non- response messages originate from A. van den Broek, et al. Expires October 25, 2008 [Page 11] Internet-Draft SNMP Trace Analysis Definitions April 2008 ---------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID. ---------------------------------------------------------------- 0 0.0 A -> B GetNext Request 1 2 0.05 B -> A Response 1 4 0.11 A -> B GetNext Request 2 5 0.15 B -> A Response 2 6 0.18 A -> B GetNext Request 3 8 0.25 B -> A Response 3 ---------------------------------------------------------------- The minimum value of parameter t of this flow is 0.07 seconds, since that is the longest time between a request and its subsequent response message. The second flow contains SNMP messages exchanged between network layer endpoints C and D, where all response messages originate from D and all non-response messages originate from C. ---------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID. ---------------------------------------------------------------- 1 0.04 C -> D Get Request 10 3 0.08 D -> C Response 10 ---------------------------------------------------------------- The minimum value of parameter t for this flow is 0.04 seconds. The third and last flow contains the remaining SNMP messages of the trace that occurred between network layer endpoints C and D. In this case the non-response message originates from D. ---------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID. ---------------------------------------------------------------- 7 0.22 D -> C Trap 14 ---------------------------------------------------------------- There is no parameter t applicable to this flow, because there are no response messages. van den Broek, et al. Expires October 25, 2008 [Page 12] Internet-Draft SNMP Trace Analysis Definitions April 2008 5. Slices Flows can still contain a large amount of SNMP messages. A flow should therefore be split up into even smaller sets of messages. One way of identifying meaningful subsets of messages of a flow would be by considering the behavior of managers and agents. In the case of managers, they are usually configured to perform regular polling instances. In such a polling instance, the manager might poll a number of agents. Since a flow contains only the messages exchanged between two network layer endpoints, a flow therefore probably consists of only a subset of the messages that are part of a polling instance. So, one option of finding smaller, meaningful subsets of messages within flows, would be by looking for messages that belong to a particular operation of a polling instance. Such a smaller set of messages is called a slice. Definition (slice): A slice S with parameter e is a subset of SNMP messages in a flow F for which the following properties hold: (S1) All messages are exchanged between the same two transport endpoints (a single transport endpoint pair). (S2) All non-response messages must have a PDU of the same type. (S3) All messages with a PDU of type Get, Set, Trap, or Inform must contain the same set of OIDs. (S4) Each GetNext or GetBulk message must either contain the same set of OIDs as the preceding request or it must be linked to a response of the last previously answered request (i.e., the request must contain at least one OID that has been contained in the (repeater) varbind list of a preceding response message of the last answered request message). (S5) All Response messages must follow a previous request message. (S6) For any two subsequent non-response messages Q1 and Q2 with Q1.time < Q2.time, the following condition must hold: (Q2.time - Q1.time) < e S1 requires that the messages of a single slice are exchanged between a single transport layer endpoint pair. This is different from the flow definition, which requires a single network layer endpoint pair. The choice of looking at the transport layer endpoints in the case of slices is based on the assumption that, for instance, multiple managers and agents might be operating from the same respective network layer endpoint. Another assumption is that a manager and an van den Broek, et al. Expires October 25, 2008 [Page 13] Internet-Draft SNMP Trace Analysis Definitions April 2008 agent will only use a single transport layer endpoint respectively when they communicate for the duration of a slice. A previous section already mentioned some issues when a manager or agent uses different transport layer endpoints within a single slice. The parameter e in S6 defines the maximum time between two non- response messages that belong to a slice. This parameter should be chosen such that unrelated non-response messages within a flow are not considered to be of the same slice (i.e., it is used to detect the end of a slice). Unrelated non-response messages are those that, for instance, belong to different polling instances. The parameter e should therefore be larger than the retransmission interval in order to keep retransmissions within a slice and smaller than the polling interval used by the slice initiator. The value of parameter e might be closely related to parameter t of the respective flow the slice is part of. For instance, if parameter e is very large, than t is likely to be also very large and vice versa. Also, if parameter e is very small, than t is probably also very small. However, it is not possible to strictly state that e and t are always closely related to each other, because parameter e is specific for a slice. This is in contrast with parameter t which is specific for a much larger set of messages, a flow. Definition (slice initiator): A slice initiator is one of the two transport layer endpoints involved in a slice, which is responsible for sending the chronologically first non-response message. Notation: The properties of a slice S are denoted as follows: S.type = type of non-response messages in S S.tsrc = transport layer endpoint of initiator of S S.tdst = transport layer endpoint of non-initiator of S S.start = timestamp of the chronologically first message in S S.end = timestamp of the chronologically last message in S S.init = initiator of slice S S.e = parameter e of S (maximum time between two non-response messages) Following is an example of a flow F consisting of SNMP messages that have been exchange place between transport layer endpoints A, B, C and D. Note that even though there are multiple transport layer endpoints, it still means that within this single flow only two network layer endpoints can be identified. van den Broek, et al. Expires October 25, 2008 [Page 14] Internet-Draft SNMP Trace Analysis Definitions April 2008 ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 0 0.0 A -> B GetNext Request 1 alpha 1 0.05 B -> A Response 1 alpha.1 2 0.11 A -> B GetNext Request 2 alpha.1 3 0.15 B -> A Response 2 alpha.2 4 0.18 A -> B GetNext Request 3 alpha.2 5 0.25 B -> A Response 3 beta.1 6 300.0 C -> D GetNext Request 4 alpha 7 300.06 D -> C Response 4 alpha.1 8 300.14 C -> D GetNext Request 5 alpha.1 9 300.21 D -> C Response 5 alpha.2 10 300.28 C -> D GetNext Request 6 alpha.2 11 300.32 D -> C Response 6 beta.1 ------------------------------------------------------------------- This flow contains SNMP messages that have been exchanged between two different pairs of transport layer endpoints. According to S1, this means that the set of messages in this flow can be separated into at least two sets of messages that might be slices: ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 0 0.0 A -> B GetNext Request 1 alpha 1 0.05 B -> A Response 1 alpha.1 2 0.11 A -> B GetNext Request 2 alpha.1 3 0.15 B -> A Response 2 alpha.2 4 0.18 A -> B GetNext Request 3 alpha.2 5 0.25 B -> A Response 3 beta.1 ------------------------------------------------------------------- ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 6 300.0 C -> D GetNext Request 4 alpha 7 300.06 D -> C Response 4 alpha.1 8 300.14 C -> D GetNext Request 5 alpha.1 9 300.21 D -> C Response 5 alpha.2 10 300.28 C -> D GetNext Request 6 alpha.2 11 300.32 D -> C Response 6 beta.1 ------------------------------------------------------------------- S2 does not result in a further separation of these sets into smaller sets of SNMP messages that might be slices. This, because all the listed non-response messages are of the same PDU type. Neither is S3 van den Broek, et al. Expires October 25, 2008 [Page 15] Internet-Draft SNMP Trace Analysis Definitions April 2008 affecting the current sets. S4 also does not result in a further subdivision of these two sets, because in each of the two sets the non-response messages, that have preceding response messages, have the same OID in their varbind list as the respective preceding response. For example, the second and third non-response message in both sets have the same OID compared to the respective preceding response message (i.e., these non-response messages are a part of a single polling instance). S5 also does not affect these two sets, because both sets already contain only response messages that are the result of a non-response message that occurred before each response message respectively (i.e., each response has a request identifier that is the same as the request identifier of a non-response message listed before that response message). S6 states that parameter e should be chosen such that unrelated requests within a flow are not considered to be of the same slice. Considering the given flow and its messages, a proper value for e should be 0.14 <= e < 299.82 seconds. Such a value for parameter e will separate the flow into two apparent polling instances, which each contain the same set of messages as those two listed above. As a result, the two listed sets are also slices. The given flow thus consists of the following two slices with the value of parameter e as defined above. Following is an example of a flow F consisting of SNMP messages that have been exchanged between transport layer endpoints A, B and C. Note again that even though there are multiple transport layer endpoints, it still means that within this single flow only two network layer endpoints can be identified. ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 0 0.0 A -> B Get Request 1 alpha.1 1 0.06 B -> A Response 1 alpha.1 2 0.12 A -> B Get Request 2 beta.1 3 0.17 B -> A Response 2 beta.1 4 300.0 A -> B Get Request 3 alpha.1 5 300.05 B -> C Set 10 gamma.1 6 300.07 B -> A Response 3 alpha.1 7 300.14 A -> B Get Request 4 beta.1 8 300.19 B -> A Response 4 beta.1 ------------------------------------------------------------------- van den Broek, et al. Expires October 25, 2008 [Page 16] Internet-Draft SNMP Trace Analysis Definitions April 2008 This flow contains messages that are exchanged between different pairs of transport layer endpoints. Two different pairs of transport layer endpoints are used within this flow. According to S1, this means that at least two sets of messages might be slices: ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 0 0.0 A -> B Get Request 1 alpha.1 1 0.06 B -> A Response 1 alpha.1 2 0.12 A -> B Get Request 2 beta.1 3 0.17 B -> A Response 2 beta.1 4 300.0 A -> B Get Request 3 alpha.1 6 300.07 B -> A Response 3 alpha.1 7 300.14 A -> B Get Request 4 beta.1 8 300.19 B -> A Response 4 beta.1 ------------------------------------------------------------------- ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 5 300.05 B -> C Set 10 gamma.1 ------------------------------------------------------------------- Applying S2 to each of the two sets does not result in more sets, because each set only contains non-response messages of one PDU type respectively. S3 requires separating the first set of SNMP messages into two sets, because the Get Request messages must have the same set of OIDs. This results in the following three sets of SNMP messages. ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 0 0.0 A -> B Get Request 1 alpha.1 1 0.06 B -> A Response 1 alpha.1 4 300.0 A -> B Get Request 3 alpha.1 6 300.07 B -> A Response 3 alpha.1 ------------------------------------------------------------------- van den Broek, et al. Expires October 25, 2008 [Page 17] Internet-Draft SNMP Trace Analysis Definitions April 2008 ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 2 0.12 A -> B Get Request 2 beta.1 3 0.17 B -> A Response 2 beta.1 7 300.14 A -> B Get Request 4 beta.1 8 300.19 B -> A Response 4 beta.1 ------------------------------------------------------------------- ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 5 300.05 B -> C Set 10 gamma.1 ------------------------------------------------------------------- S4 is not applicable to either of these sets. S5 does not affect these sets, because these sets already contain only response messages that are the result of a non-response message that occurred before each response message respectively. Finally, S6 states that parameter e should be chosen such that unrelated requests within a flow are not considered to be of the same slice. The messages in the given flow suggest that two polling instances might have occurred. Therefore, an appropriate value for parameter e would be < 300 seconds. The determination of parameter e results in the separation of the first two sets into two sets each. Following are the resulting slices. ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 0 0.0 A -> B Get Request 1 alpha.1 1 0.06 B -> A Response 1 alpha.1 ------------------------------------------------------------------- ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 2 0.12 A -> B Get Request 2 beta.1 3 0.17 B -> A Response 2 beta.1 ------------------------------------------------------------------- van den Broek, et al. Expires October 25, 2008 [Page 18] Internet-Draft SNMP Trace Analysis Definitions April 2008 ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 4 300.0 A -> B Get Request 3 alpha.1 6 300.07 B -> A Response 3 alpha.1 ------------------------------------------------------------------- ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 7 300.14 A -> B Get Request 4 beta.1 8 300.19 B -> A Response 4 beta.1 ------------------------------------------------------------------- ------------------------------------------------------------------- Message | Time [s] | Direction | PDU type | Req. ID | OIDs ------------------------------------------------------------------- 5 300.05 B -> C Set 10 gamma.1 ------------------------------------------------------------------- van den Broek, et al. Expires October 25, 2008 [Page 19] Internet-Draft SNMP Trace Analysis Definitions April 2008 6. Slice Prefix As noted in the beginning of this document, it is desirable that slices can be tested for equality/comparability. This is where the slice prefix comes in. The slice prefix provides one of the means to compare slices. Using the slice prefix and a few other parameters (which will be discussed later on in this document) of a number of slices, one can determine which slices should be considered "equal" and which of them are incomparable. This will assist in the process of finding potentially other relations. The slice prefix is a set of OIDs. This set is constructed from the messages that make up a single slice. So, for example, a slice that is the result of a manager requesting the contents of a particular table (with OID alpha) on an agent using a simple single varbind GetNext walk, starting at the table OID alpha, shall yield a slice prefix which consists of the OID alpha. Because the aim is to compare various slices using the slice prefix (along some other characteristics of a slice), this implicitly suggests the need to know whether a number of slices are the result of the same behaviour (i.e., specific configuration) of the initiating party of these slices. For example, one may want to know whether a number of slices that involve a single manager and a single agent were the result of just one specific configuration of that manager. Multiple slices, that may all be initiated by that same manager and each slice possibly occurred in different polling instances, may in fact be the result of the same specific configuration of that particular manager. So, since in this case the specific configuration of the manager is only relevant for determining the behaviour, the slice prefix should be constructed based on OIDs in messages originating from that manager only. More generally, only the messages within slices that are sent by the initiating party (the non-response messages) are considered for the determination of the respective slice prefix of a slice. The resulting set of OID prefixes will represent the behaviour of the respective initiating party of that slice. This allows us to compare different slices. Following is a short introductory example which depicts what a slice could consist of and how one could determine the slice prefix in such a general case. Consider the case of a single manager A polling a specific agent B. More specifically, the manager A is configured to retrieve the complete contents of two columns alpha and beta of a some table. The resulting slice may contain the following messages: van den Broek, et al. Expires October 25, 2008 [Page 20] Internet-Draft SNMP Trace Analysis Definitions April 2008 ------------------------------------------------------------------- Message | Direction | PDU type | OIDs ------------------------------------------------------------------- 0 A -> B GetNext Request alpha, beta 1 B -> A Response alpha.1, beta.1 2 A -> B GetNext Request alpha.1, beta.1 3 B -> A Response alpha.2, beta.2 4 A -> B GetNext Request alpha.2, beta.2 5 B -> A Response gamma.1, delta.1 ------------------------------------------------------------------- The manager starts with a GetNext request referencing two OIDs, alpha and beta. The agent B replies in message 1 with the first items of each of the referenced columns. The manager in turn goes on obtaining data from these two columns until it receives message 5, which indicates that the manager has received all of the data from the two columns. It can be easily concluded that the manager was configured to retrieve the contents of the two columns alpha and beta (the slice prefix). A different slice involving the same manager and agent and that is again the result of the same configuration of the manager, should be considered "equal" to this one because the two slices are the result of the same behaviour. It should however be mentioned that such a second slice might contain a different number of messages, since the contents of the tables on the agent side might have changed over time. This underlines the previously made remark that only the messages originating from the initiating party should be considered in this process, because they will (in such a scenario) always illustrate the same behaviour of the initiating party. The previous example now makes it possible to give a more formal definition of a slice prefix. This is accomplished by first defining a slice signature which then leads to the definition of a slice prefix. Definition (slice signature): A slice signature S.sig of a slice S is a set of OIDs derived from the OIDs contained in the non-response messages of a slice. Let r(S) denote the set of response messages of slice S and n(S) the set of non-response messages of S. Then the set S.sig consists of the following OIDs: van den Broek, et al. Expires October 25, 2008 [Page 21] Internet-Draft SNMP Trace Analysis Definitions April 2008 case S.type = GetNext or S.type = GetBulk: S.sig = U (n.oids) - U (r.oids) n in n(S) r in r(S) otherwise: S.sig = U (n.oids) n in n(S) Definition (OID prefix): An OID a = a_1.a_2...a_n is a prefix of OID b = b_1.b_2...b_m if and only if (P1) n < m and (P2) a_i = b_i for 1 <= i <= n. Definition (slice prefix): The slice prefix S.slice is the set of all OIDs o in S.sig for which there is no p in S.sig such that p is a prefix of o. Following is an example to illustrate the definitions just described: Consider the case that a single manager A polling an agent B. More specifically, the manager A is programmed to retrieve the complete contents of two single column tables alpha and beta. Besides that, the manager now also requests the sysUpTime in the first request the manager sends to B. A resulting slice may contain the following messages: ------------------------------------------------------------------- Message | Direction | PDU type | OIDs ------------------------------------------------------------------- 0 A -> B GetNext Request sysUpTime, alpha, beta 1 B -> A Response sysUpTime.0, alpha.1, beta.1 2 A -> B GetNext Request alpha.1, beta.1 3 B -> A Response alpha.2, beta.2 4 A -> B GetNext Request alpha.2, beta.2 5 B -> A Response gamma.1, delta.1 ------------------------------------------------------------------- The slice prefix is determined as follows: 1. The union of all OIDs in non-response messages is the following set: N = { sysUpTime, alpha, beta, alpha.1, beta.1, alpha.2, beta.2 } van den Broek, et al. Expires October 25, 2008 [Page 22] Internet-Draft SNMP Trace Analysis Definitions April 2008 2. The union of the OIDs in response messages is the following set: R = { sysUpTime.0, alpha.1, beta.1, alpha.2, beta.2, gamma.1, delta.1 } 3. Subtracting the two sets results in the slice signature S.sig: S.sig = N - R = { sysUpTime, alpha, beta } 4. Since none of the OIDs in S.sig is a prefix of any other OID in S.sig, the slice prefix is equal to the slice signature: S.prefix = { sysUpTime, alpha, beta } This set of OIDs describes the behavior of the initiating party of this particular slice. Within the context of the described configuration of the initiating party is it apparent that a subsequent polling instance of the mentioned manager A polling agent B will result in the same slice signature. Following is a more elaborate slice for which the slice prefix is determined. Consider again the case that a single manager A is set to poll a specific agent B. Manager A is programmed to retrieve some values from B. A single slice may contain the following messages: ------------------------------------------------------------------- Message | Direction | PDU type | OIDs ------------------------------------------------------------------- 0 A -> B GetNext Request alpha, beta 1 B -> A Response alpha.1, beta.1 2 A -> B GetNext Request alpha.1, beta.1 3 B -> A Response alpha.2, beta.3 4 A -> B GetNext Request beta.2, alpha.2, sysUpTime 5 B -> A Response beta.3, alpha.3 sysUpTime.0 6 A -> B GetNext Request beta.3, alpha.3 7 B -> A Response gamma.1, alpha.4 8 A -> B GetNext Response alpha.4 9 B -> A Response delta.1 ------------------------------------------------------------------- This slice has a number of interesting properties: o Not all columns in the table have an equal length. o The manager is set to request the sysUpTime on an irregular basis (i.e., every few requests). van den Broek, et al. Expires October 25, 2008 [Page 23] Internet-Draft SNMP Trace Analysis Definitions April 2008 o The manager attempts to fill "holes" in the table. o The order of referenced OIDs in GetNext messages changes. All of these properties do not influence the process for determining the slice signature. The slice prefix is constructed as follows: 1. The union of all OIDs in non-response messages is the following set: N = { alpha, beta, alpha.1, beta.1, beta.2, alpha.2, sysUpTime, beta.3, alpha.3, alpha.4 } 2. The union of the OIDs in response messages is the following set: R = { alpha.1, beta.1, alpha.2, beta.3, alpha.3, sysUpTime.0, gamma.1, alpha.4, delta.1 } 3. Subtracting the two sets results in the slice signature S.sig: S.sig = N - R = { alpha, beta, beta.2, sysUpTime } The element beta.2 exists, because the manager was trying to fill a "hole" in a table. Since these holes reside in the tables on the agent side and may change dynamically, they do not really help in describing the behavior of the initiating party. 4. Since beta is a prefix of beta.2, the slice prefix becomes the following set: S.prefix = { alpha, beta, sysUpTime } The slice prefix does not include beta.2 anymore and thus a manager retrieving the same columns alpha and beta with and without holes will produce slices with the same slice prefix. Finally, consider a slice where s.type is not GetBulk or GetNext. Manager A is programmed to poll some values from B. A single slice may contain the following messages: ------------------------------------------------------------------- Message | Direction | PDU type | OIDs ------------------------------------------------------------------- 0 A -> B Get Request alpha.1, beta.1 1 B -> A Response alpha.1, beta.1 ------------------------------------------------------------------- The slice prefix is determined as follows: van den Broek, et al. Expires October 25, 2008 [Page 24] Internet-Draft SNMP Trace Analysis Definitions April 2008 1. The slice signature is the union of all OIDs in non-response messages: S.sig = { alpha.1, beta.1 } 2. Since none of the OIDs in S.sig is a prefix of any other OID in S.sig, the slice prefix is equal to the slice signature: S.prefix = { alpha.1, beta.1 } van den Broek, et al. Expires October 25, 2008 [Page 25] Internet-Draft SNMP Trace Analysis Definitions April 2008 7. Slice Type As described previously, the slice type allows for comparing slices. This means that any number of slices that are of the same slice type may be considered an equivalence class and may therefore be considered to be the result of the same behaviour of the slice initiator. Definition (slice equivalence): Two slices A and B satisfy the binary slice equivalence relation A ~ B if the following properties hold: (EQ1) All messages in A and B have been exchanged between the same two network layer endpoints. (EQ2) All read request messages, write request messages, and notification messages in A and B originate from the same network layer endpoint. (EQ3) All non-response messages in A and B are of the same type. (EQ4) The slices A and B have the same prefix, that is A.prefix = B.prefix. It can be easily seen that the relation ~ is reflexive, symmetric, and transitive and thus forms an equivalence relation between slices. Definition (slice type): Let S be a set of slices, then all slices in the equivalence class [A] = {s in S | s ~ A} with A in S, are of the same slice type. Following are two slices that are of the same equivalence class and are therefore of the same slice type. The first slice contains messages that have been exchanged between transport layer endpoints A and B. ------------------------------------------------------------------- Message | Direction | PDU type | OIDs ------------------------------------------------------------------- 0 A -> B GetNext Request alpha, beta 1 B -> A Response alpha.1, beta.1 2 A -> B GetNext Request alpha.1, beta.1 3 B -> A Response alpha.2, beta.2 4 A -> B GetNext Request alpha.2, beta.2 5 B -> A Response gamma.1, delta.1 ------------------------------------------------------------------- The second slice contains messages that have been exchanged between van den Broek, et al. Expires October 25, 2008 [Page 26] Internet-Draft SNMP Trace Analysis Definitions April 2008 transport layer endpoints C and D. However, the network layer endpoints of this slice are the same as the first slice and all non- response messages in both slices originate from the same network layer endpoint. ------------------------------------------------------------------- Message | Direction | PDU type | OIDs ------------------------------------------------------------------- 0 C -> D GetNext Request alpha, beta 1 D -> C Response alpha.1, beta.1 2 C -> D GetNext Request alpha.1, beta.1 3 D -> C Response alpha.2, beta.2 4 C -> D GetNext Request alpha.2, beta.2 3 D -> C Response alpha.3, beta.3 4 C -> D GetNext Request alpha.3, beta.3 5 D -> C Response gamma.1, delta.1 ------------------------------------------------------------------- EQ1 requires that both slices have occurred between the same two network layer endpoints, which is the case here. EQ2 requires that all read request messages, write request messages, and notification messages (i.e., all non-response messages) originate from the same network layer endpoint, which is the case. EQ3 is met, because all non-response messages are of PDU type GetNext Request in both cases. Using the definition of a slice prefix as stated in the previous chapter, it can be determined that both slices have the prefix { alpha, beta }. As a result, also EQ4 is met. Both slices are therefore part of the same equivalence class and are therefore of the same slice type. van den Broek, et al. Expires October 25, 2008 [Page 27] Internet-Draft SNMP Trace Analysis Definitions April 2008 8. Walks Definition (walk): A walk W is a slice S with the following properties: (W1) The type S.type of the slice S is either GetNext request or GetBulk request. (W2) At least one OID in the sequence of requests at the same varbind index must be increasing lexicographically while all OIDs at the same varbind index have to be non-decreasing. Definition (strict walk): A walk W is a strict walk if all OIDs in the sequence of requests at the same varbind index are strictly increasing lexicographically. Furthermore, the OIDs at the same index of a response and a subsequent request must be identical. Definition (prefix constrained walk): A walk W is a prefix constrained walk if all OIDs at the same index in the request have the same OID prefix. This prefix is established by the first request within the walk. Following is an example of a slice that is also a walk. ------------------------------------------------------------------- Message | Direction | PDU type | OIDs ------------------------------------------------------------------- 0 A -> B GetNext Request alpha, beta 1 B -> A Response alpha.1, beta.1 2 A -> B GetNext Request alpha.1, beta.1 3 B -> A Response alpha.2, beta.2 4 A -> B GetNext Request alpha.2, beta.2 5 B -> A Response gamma.1, delta.1 ------------------------------------------------------------------- This slice is a walk, because it adheres to W1, since it contains only requests that are of PDU type GetNext Request. W2 is also met, because there are always two object identifiers at the same varbind index that are increasing lexicographically in the sequence of requests. Also, none of the object identifiers at the same varbind index is lexicographically decreasing. This slice is also a strict walk, because all object identifiers in the sequence of requests at the same varbind index are strictly increasing lexicographically. Secondly, the object identifiers at the same index of a response and a subsequent request are always identical. van den Broek, et al. Expires October 25, 2008 [Page 28] Internet-Draft SNMP Trace Analysis Definitions April 2008 Finally, this walk is also a prefix constrained walk, because all object identifiers at the same index in the request have the same object identifier prefix. The first varbind index in the requests always has alpha as prefix and the second varbind index always has beta as prefix. van den Broek, et al. Expires October 25, 2008 [Page 29] Internet-Draft SNMP Trace Analysis Definitions April 2008 9. Concurrency Definition (concurrency): Two slices A and B of a given flow F are concurrent at time t if A.start <= t <= A.end and B.start <= t <= B.end. Definition (flow concurrency level): The concurrency level of a flow F at time t is given by the number of concurrent slices of F at time t. Definition (slice initiator concurrency level): The concurrency level of a slice initiator I at time t is given by the number of concurrent slices initiated by the slice initiator I at time t. Definition (overlapping): Two slices A and B of a given flow F are called overlapping if there exists a time t where A and B are concurrent. Definition (delta time serial): Two slices A and B of a given flow F with B.start > A.end are called delta time serial if (B.start - A.end) < delta. van den Broek, et al. Expires October 25, 2008 [Page 30] Internet-Draft SNMP Trace Analysis Definitions April 2008 10. Security Considerations This document provides definitions for the analysis of SNMP traces and does not impact the security of the Internet. van den Broek, et al. Expires October 25, 2008 [Page 31] Internet-Draft SNMP Trace Analysis Definitions April 2008 11. IANA Considerations This document has no actions for IANA. van den Broek, et al. Expires October 25, 2008 [Page 32] Internet-Draft SNMP Trace Analysis Definitions April 2008 12. Acknowledgements This document was influenced by discussions within the Network Management Research Group (NMRG). Part of this work was funded by the European Commission under grant FP6-2004-IST-4-EMANICS-026854-NOE. van den Broek, et al. Expires October 25, 2008 [Page 33] Internet-Draft SNMP Trace Analysis Definitions April 2008 13. References 13.1. Normative References [RFC3411] Harrington, D., Presuhn, R., and B. Wijnen, "An Architecture for Describing Simple Network Management Protocol (SNMP) Management Frameworks", RFC 3411, December 2002. [RFC3416] Presuhn, R., "Version 2 of the Protocol Operations for the Simple Network Management Protocol (SNMP)", RFC 3416, December 2002. 13.2. Informative References [RFC3410] Case, J., Mundy, R., Partain, D., and B. Stewart, "Introduction and Applicability Statements for Internet Standard Management Framework", RFC 3410, December 2002. [RFC3954] Claise, B., "Cisco Systems NetFlow Services Export Version 9", RFC 3954, October 2004. [ID-IRTF-NMRG-SNMP-MEASURE] Schoenwaelder, J., "SNMP Traffic Measurements and Trace Exchange Formats", ID draft-irtf-nmrg-snmp-measure-03.txt, February 2008. [SPHSM07] Schoenwaelder, J., Pras, A., Harvan, M., Schippers, J., and R. van de Meent, "SNMP Traffic Analysis: Approaches, Tools, and First Results", IFIP/IEEE Integrated Management IM 2007, May 2007. van den Broek, et al. Expires October 25, 2008 [Page 34] Internet-Draft SNMP Trace Analysis Definitions April 2008 Authors' Addresses Jurgen Gijs van den Broek University of Twente P.O. BOX 217 7500 AE Enschede Netherlands Phone: +31 6 13506591 Email: j.g.vandenbroek@student.utwente.nl Juergen Schoenwaelder Jacobs University Bremen Campus Ring 1 28725 Bremen Germany Phone: +49 421 200-3587 Email: j.schoenwaelder@jacobs-university.de Aiko Pras University of Twente P.O. BOX 217 7500 AE Enschede Netherlands Phone: +31 53 4893778 Email: a.pras@cs.utwente.nl Matus Harvan ETH Zurich ETH Zentrum 8092 Zurich Switzerland Phone: +41 44 632 68 76 Email: mharvan@inf.ethz.ch van den Broek, et al. Expires October 25, 2008 [Page 35] Internet-Draft SNMP Trace Analysis Definitions April 2008 Full Copyright Statement Copyright (C) The IETF Trust (2008). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Acknowledgment Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). van den Broek, et al. Expires October 25, 2008 [Page 36]