IPSP Working Group W. Hardaker INTERNET-DRAFT Network Associates Laboratories draft-hardaker-eos-oops-01.txt September 2002 Object Oriented PDUs for SNMP draft-hardaker-eos-oops-01.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) The Internet Society (2002). All Rights Reserved. 1. Introduction This draft specifies some new PDU operations intended to optimize SNMP requests for performance. This draft is not a complete specification, but merely is intended to illustrate concepts which can then be discussed within the EoS working group. A complete specification will be completed after the agreed features and packet structure is fairly well solidified so the author doesn't have to rewrite large portions of stuff ;-). There are no error handling codes or elements of procedures defined yet. The concepts, however, have been thought out about a lot. Feedback on what you see within would be greatly appreciated at this point. Wes Hardaker [Page 1] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 Terminology in this draft is frequently inconsistent. Forgive the use of the old-school terminology (e.g., "agent") intermixed with new terminology (e.g., "command responder"). The other common error is the usage of the word "column" instead of "sub-identifier" or whatever the next draft for SMIv3 is going to call it (I forget and think it may be changing in the next draft anyway hence I didn't try to get it right in this document). This too will be fixed if a standardized specification on this work goes forward. Some of the stuff in this draft has yet to undergo debate. Particularly the Write operation is not solidified in my mind and will certainly be subject to debate as to whether they go to far into solving the problems that many people have pointed out with existing transactions. Please read the draft all the way through, including the section on "Design Considerations and Questions" before asking questions of the author or the working group. Finally, this is very much a working document. The descriptions described here-in may not even exactly match the author's thoughts, as the document hasn't been extensively proof-read yet. 1.1. Related work with the SMIng working group. In revamping this document and the contained PDUs, the direct correlations to the SMIng working group have since been dropped. They'll be reinstated shortly, but for now just know that all the PDUs contained within this document can be used with the new hierarchal structure of a newer SMIv3 MIB structure. To think about it, everywhere you see the word "BER" in a PDU definition, think "BER or recursive". IE, sub-elements within a SMIng working group will be encoded into a BER field in the same way that the containing request needed it. 1.2. Table of Contents 1. Introduction 1 1.1. Related work with the SMIng working group. 2 1.2. Table of Contents 2 2. Background and Motivations 3 2.1. GET-NEXT complexity 3 2.2. OID compression 4 2.3. Row operations 4 2.4. Complex object relationships 4 2.5. Retrieving of more data than is desired. 5 2.6. Row hole traversal and object grouping 5 2.7. Index parsing and OID length restrictions. 6 2.8. Transactions 6 Wes Hardaker [Page 2] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 2.9. Easy conversion to and from popular data formats 6 2.10. Human readable error strings 6 3. Transport Protocol Considerations 7 4. New PDU definitions 7 4.1. The Get-Object-PDU 7 4.2. The Get-Object-Response PDU 13 4.3. The Write-Object-PDU 15 4.3.1. Write-Object-PDU transaction semantics 18 4.4. The Write-Object-Response-PDU 19 5. ASN.1 definitions for the PDUs 21 6. Examples 22 6.1. Retrieve a specific row from the ifTable 22 6.2. A multiple-packet example with a double filter. 23 7. Elements of Procedure. 26 8. Design Considerations and Questions 26 8.1. Return-field ranges 26 8.2. Write-Object-PDU complexity 26 8.3. Get-Object/Write-Object commonality. 27 8.4. Concrete packet formats vs optional field formats. 27 8.5. Push Button Functionality. 28 9. Author's Addresses: 29 10. References 29 11. Full Copyright Statement 29 2. Background and Motivations Many recognized problems exist within the SNMP protocol [OPREQ,...]. Many of the known problems are addressed by the new protocol operations defined within this draft. The general problems trying to be solved are described in this section. 2.1. GET-NEXT complexity A large number of problems result from a devices inability to access it's data in a way that can be efficiently traversed when searching for the next object in a series of GET-NEXT or GET-BULK SNMP requests. Internally, many implementations differ and thus it would be impossible in many cases to define a MIB which would be efficient when implemented on every architecture. The operations defined in this document do not suffer from these problems, as the data can be returned in any order that the agent feels is appropriate. It MUST, however, return data in a consistent manner such that if the same data is requested twice the agent will return the data in the same order. The exact order in which objects are returned is implementation specific. Wes Hardaker [Page 3] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 It's the author's belief that every agent will benefit from this, and that most managers don't care in the first place. Manager's are likely to either need the data unsorted in the first place (e.g. may only care about the values each row and not the relationships between the rows) or the data will need to be sorted in a different fashion than the MIB indexing defines in the first place. If the manager does need the data sorted, it's more likely it'll have the power to more easily do sorting via utilization of database technologies that are not realistically available on many agent platforms. 2.2. OID compression SNMPv2 PDU consist of a sequence of varbinds and in many cases, when multiple objects are requested from the agent, the OIDs contained within have common prefixes. It is widely recognized as a waste of resources to duplicate this prefix repeatedly. The PDUs defined in this document only rely on a base OID (e.g., an object identifier pointing to a SNMP Table) once, and the sub- objects (e.g., a row in a SNMP Table) underneath it are defined as references from the base OID. Each index set which defines the proper reference for the remainder of the data components in a given "row" or "sub-object" is only included once as well. This allows large groups of data to be transmitted while only requiring a single OID to identify the top most grouping object (e.g., a table). 2.3. Row operations The ability to manipulate an entire row within a SNMP Table at once has long been a frustration of programmers implementing the protocol. The PDUs defined in this document are able to operate on entire groups of objects at once, where one of these groupings types is a row within a normal SNMP Table. 2.4. Complex object relationships Many people do not properly understand the interrelationships between SNMP Tables and have desired the ability to express their data in a more hierarchal fashion, such that SNMP tables can contain other SNMP tables. Although it is not possible to enable nested datatypes within SMIv2 today, the SMIng working group is trying to solve these problems and create the ability to define more complex relationships in SMIv3. The way object groupings are referenced within the PDUs defined in Wes Hardaker [Page 4] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 this document allows hierarchal object data to be transmitted efficiently and as a single group, thus allowing groups defined within SMIv3 to be efficiently transmitted within the PDUs defined in this document, new SMIv3-style objects be deployed. XXX: mention the intention to make the object definitions a bit extensible, so that new types requiring new parameters can be easily handled in the future without replacing the PDUs themselves. 2.5. Retrieving of more data than is desired. Extracting only the data needed from a SNMP Table has traditionally required pulling the entire table of data to the management station in many cases, when the indexes and their ordering was not sufficient for requesting subsets of the data desired. The request PDUs defined in this document allow more careful selection of indexing data and allow simple search criteria to be fulfilled. A delicate balance is required to ensure that the devices being polled for data are not taxed with expensive search requests, so the criteria imposed within these documents is limited to a limited set of operators which may or may not be supported by the command responder implementation. The intention is not to shift all the complex decision processing burden from the management station to the command responder, but to increase efficiency on the wire where possible. XXX: the agent will actually have the ability to gain improvements from this optimization since the manager will perform fewer requests as a whole. 2.6. Row hole traversal and object grouping When requests to GET, GET-NEXT, and GET-BULK data come back in a SNMP RESPONSE PDU, the data can be hard to organize back into a logical structure again. Holes of data (a row that contains no data for a particular column) within SNMP Tables make processing of the data collected more tedious. Additionally, GET-BULK responses interleave it's data, possibly with END-OF-MIB exceptions, which only adds further complexity to the data processing. Since the PDUs defined in this document deal directly with MIB objects as a whole, the data relationships within an object is preserved. This makes transferring large amounts of data efficient on both the sending and receiving side, as neither side needs to sort VarBinds properly. The data groupings are now appropriately marked within the packet itself. Wes Hardaker [Page 5] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 2.7. Index parsing and OID length restrictions. Although the encoding and decoding of Table indexes into and out out OIDs is algorithmic, many SNMP software packages and toolkits make mistakes in implementing the algorithm. Additionally, since the maximum length of an OID is limited to 128 sub-identifiers, it limits the size of the index data that can be contained within an OID. This limitation results in poorly designed and broken MIBs and MIBs with less-than-ideal table indexes. The indexes within the PDUs defined in this document are encoded directly into the packet as opposed to being encoded into the OID first (which is then encoded into the packet). This simplifies both management and agent code and reduces the chances of one side incorrectly encoding or decoding the indexes. Additionally, it provides the ability to use a set of indexes which is larger than the OID-encoded imposed length of 255 sub-identifies. 2.8. Transactions The limited set of SNMP transactions have been difficult to cope with when large sets of data must be pushed around, since all the objects within a SNMP SET PDU must be set as if simultaneously. In actuality, configuration data often contains independent segments of data. Typically, independent sets of data must be pushed inside individual PDUS when configuring an agent, which is inefficient. The PDUs defined in this document define a less restrictive and more flexible transaction model that lets large quantities of data be pushed efficiently across the network and will still allow individual groupings to specify whether they are inter-related and must be treated as a single transaction or not and whether or not ordering is important 2.9. Easy conversion to and from popular data formats A desire has been shown by network operators for SNMP objects to be easily accessible and convertable to and from more human friendly expression languages and storage systems, like XML or SQL Databases. The objects within the PDUs contained in this document are designed for such a purpose. It should be, for example, trivial to convert them to and from a hierarchial XML schema set describing the objects in question. For the example XML case, the BER hierarchal packet format can be quickly converted to XML by replacing the sequence tags with XML ascii delimiter tags. 2.10. Human readable error strings Wes Hardaker [Page 6] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 Integer error codes are extremely useful for machine parsibility and interoperability, but it's frequently nice to have an extra error string that may be passed to an operator to assist in extra debugging of problems. The response PDUs defined in this document contain an "error string" for exactly this purpose. 3. Transport Protocol Considerations The PDUs defined in this document allow the transmission of large data sets in a more compressed form than previous SNMP PDUs, it is still recommended that large requests and responses be transmitted over a SNMP transmission domain which provides for guaranteed network packet delivery (e.g., TCP). Large responses, containing many objects, carried over transmission domains (e.g., UDP) which can not guarantee delivery are still likely to be problematic. It is well beyond the scope of this document to re-implement reliable transmission mechanisms. 4. New PDU definitions This section defines the new PDUs. Right now, these PDUs are not in a ASN.1 standard format simply for clarity to the reader. Proper ASN.1 definitions will be published as the format of the PDUs are more stabilized (probably late 2002) and more widely accepted. ASN.1 is designed for machine readability, and this draft is currently still intended for primarily human consumption. In the mean time, here is a non-ASN.1 PDU breakdown. Any {} pairs below are essentially a BER sequence. At least the larger sequences are expected to be implemented using the non-deterministic sequence type (a beginning and ending sequence tag, rather than a single tag and a length) to ease the burden on encoders. [] means a sequence list (array), ie multiple objects. The author would like to delay the argument over coding semantics until after the PDU structure itself has been defined. Note that some of the fields below might be implemented using the OPTIONAL ASN.1 field tag as well. Again, this will be a discussion for later. 4.1. The Get-Object-PDU Here's the breakdown of a Get-Object-PDU (GOP), which can be thought of as a replacement for GET, GETNEXT and GETBULK. Get-Object-PDU ::= { request-id Integer32, Wes Hardaker [Page 7] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 option-field {} -- a sequence reserved for future use request-objects[] ::= { { max-return-objects Integer32, skip-objects Integer32, max-depth Integer32, cursor OCTET STRING, flags BITS { restartOnInvalidCursor(0), returnAllDataOnUnsupportedMatch(1), dontReturnObjectNumbers(2), reserved(3) }, option-field {} -- a sequence reserved for future use request-base-OID, -- (eg, OID for a table) index-request-list[] ::= { INDEX1, INDEX2, ... }, column-request-list[] ::= { COLNUM1, COLNUM2, ... }, index-search-objects[] ::= { { index-number Integer32, match-type MATCH-TYPE, index-search-value BER }, ... }, column-search-objects[] ::= { { column-number Integer32, match-type MATCH-TYPE, column-search-value BER }, ... } } ... -- possibly more request-objects } In the above, the following pieces need explaining (and this is not an exact elements of procedure, but should give a feel for how PDU processing will eventually be described): request-id: The standard request-id concept that exists in previous PDUs defined by the previous versions of SNMP max-return-objects: The maximum number of objects to return in one response message. A value of 0 indicates "all". Wes Hardaker [Page 8] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 skip-objects: Don't return the first skip-objects objects, but rather return objects numebering from (skip-objects + 1) to (max-return-objects + skip-objects + 1). A value of 0 indicates start returning with the first object. max-depth: The maximum depth of a SMIv3 structure to return. This will be explained in greater detail in a later draft. cursor: An OCTET STRING defining the starting point for collecting data. This MUST be a zero length string for the first request, and SHOULD contain the value in the response for chained requests which are expected to return future data. The value of the cursor is implementation dependent at the command responder. The manager MUST treat it as generic data and should use it when possible to obtain the next set of information in a series of requests. Command responders MUST refer to the value of the skip-objects field if the cursor field is a zero length string. Command responders MUST ignore the value of the skip-objects field if the cursor field is a non-zero length string and is deemed to be a valid cursor. If this object is specified, the skip-objects is ignored by the command responder. Cursors SHOULD be defined by the command responder in such a way as to avoid referencing an existing object in such a way that if the existing object disappears the cursor will become invalid. I.E., where possible, cursors should be data independent where possible and should be a reference into the appropriate place in the storage mechanism and not a reference to a data piece itself within the storage mechanism. If a cursor is deemed invalid by the command responder and it is impossible for it to determine where to start collecting data from, the command responder must do Wes Hardaker [Page 9] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 one of two things: a) if the restartOnInvalidCursor flag is set, the search operation should be restarted from the beginning of all possible data. b) if the restartOnInvalidCursor flag is not set, the search operation should start at the point indicated by the skip-objects field. As an example cursor, an agent which was merely performing internal Get-Object-PDU translations to a GETNEXT translations might return a cursor containing the BER encoding of the last OID returned within the response. The next request could merely continue processing using the encoded OID just as if a real GETNEXT had come in. flags: A flag list indicating particular features to be used when processing the PDU. The flag values are described elsewhere in this document. option-field: A sequence reserved for future use. Command responders which encounter unknown option types specified in this sequence MUST ignore the data and proceed as if the field itself didn't exist in the request. Future protocol extensions MUST define extensions to be implemented within this sequence in such a way that the option can be safely ignored by implementations which fail to understand it. This field exists in multiple places in the PDU definitions contained within this document. In all cases, the sequences MUST contain only an OPTION-ized list of parameters to be defined in future documents. The option index numbers for the sequence will be assigned and administered by IANA. Note: all option-field names in this Wes Hardaker [Page 10] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 document are identical, which is purely due to laziness. They should be different in each occasion within each new PDU type. request-base-OID: The base OID for a given object ot request information from. For SMIv2 objects, this is either the OID pointing to a table or a particular scalar. index-request-list & These two sequences contain a list of column-request-list: subcomponents to be returned in the response. IE, what particular data elements should be returned. If both elements are empty (2 zero-length sequences) then all data for the given object MUST be returned. Managers SHOULD request accessible indexes elements in the column-request-list if the indexes are accessible and not in the index-request-list. index-search-objects & These two sequences contain a list of column-search-objects: search criteria for the data being requested. If the object data to be returned does not match the criteria imposed by these restrictions in the request, then it shouldn't be returned. index-number: The index number or column-number of a column-number: given index or column object within a table. Indexes should count from 1 in the index list for the given object. The column number should be the assigned number underneath the request-base-OID object. If the dontReturnObjectNumbers is set in the request, the command responder MAY NOT include these fields in the Get-Object-Response-PDU, but the command responder MUST set the bit according to what it actually did. The command responder MUST include these fields if the request did not have the dontReturnObjectNumbers bit set. Implementations SHOULD use column matching in preference to index matching, and thus Wes Hardaker [Page 11] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 index matching SHOULD ONLY be used for external index values which don't have a column number assignment within the object. match-type: Specifies the criteria to be imposed for a given search value. The criteria types are listed below: MATCH-TYPE ::= Integer32 { -- any datatypes: equals(0), not-equals(1), -- numerical only: lessThan(10), lessThanOrEqual(11), greaterThan(12), greaterThanOrEqual(13), -- binary comparisons: regexp(20), not-regexp(21), -- ... IANA assigned up to 255 -- enterprise specific: -- 256*EnterpriseID to -- 256*EnterpriseID + 255 } match-type's are intended for simple expressions to help reduce the data sent over the wire. They are not intended for complex querying which might impose a large burden on the end device who's resources are likely needed for their primary functionality (routing, ...). Multiple match-types within a request are always logically ANDed together. If the returnAllDataOnSearchFailure bit is set, then if a search operation fails due to a condition like an unsupported match-type, all data will be returned as if the search-type operator had always returned "true". If the returnAllDataOnSearchFailure bit is not Wes Hardaker [Page 12] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 set, an unSupportedSearchOperation error condition will be returned in the error-status field of the Get-Object-Response-PDU and the error-index field will be set to the N if this is the Nth search criteria contained within the original Get-Object-PDU. 4.2. The Get-Object-Response PDU This PDU is sent in response to a Get-Object-PDU. Get-Object-Response-PDU ::= { request-id Integer32, response-objects[] ::= { { error-status Integer32, error-index Integer32, error-string OCTET STRING, cursor OCTET STRING, flags BITS { restartOnInvalidCursor(0), returnAllDataOnSearchFailure(1), dontReturnObjectNumbers(2), containedLastDataObject(3) }, option-field {} -- a sequence reserved for future use request-base-OID, -- (eg, OID for a table) data-list[] ::= { index-data-list[] ::= { { index-number Integer32, -- ** see below index-value BER }, ... }, column-data-list[] ::= { { column-number Integer32, -- ** see below column-value BER }, ... } } } ... -- possibly more response-objects } } Wes Hardaker [Page 13] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 In the above, the following pieces need explaining. Items in the above definition not listed here have been previously described. error-status An error status for the entire request operation, if an error occurred which affected the entire operation. Agent's SHOULD strive to put exceptions in the data field where possible instead. error-index An index into "something", where "something" is defined by what the error-status field contains (as an example, the unSupportedSearchRange error indicates it's an index into the search objects in the original request). error-string A string describing the particular error. This SHOULD be set to a human readable string describing the problem in greater detail than the error-status field alone can indicate. flags The flags field MUST be set according to how the response was handled within the responder. Ideally this should be an exact copy of the flags field from the request, assuming the flags were understood. Non-understood flags MUST be set to 0 in the response. If the command responder can determine that the last object to be returned within the requested search range is contained within the Get-Object-Response-PDU, then the containedLastDataObject SHOULD be set. index-number: This will be one of: a) the count of the object into the INDEX clause of the component for which the data is being returned if the dontReturnObjectNumbers flag bit in the response is 0. b) If the dontReturnObjectNumbers flag bit in the request was 1 and the agent response's sets the dontReturnObjectNumbers to 1, then this field is dropped from the response and only the next field (the Wes Hardaker [Page 14] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 value) is present. column-number: This will be one of: a) the assigned sub-component number of the object being returned if the dontReturnObjectNumbers flag bit in the response is 0. b) If the dontReturnObjectNumbers flag bit in the request was 1 and the agent response's sets the dontReturnObjectNumbers to 1, then this field is dropped from the response and only the next field (the value) is present. 4.3. The Write-Object-PDU This Write-Object-PDU is used to update a remote entity with new data. Write-Object-PDU ::= { request-id Integer32, flags BITS { returnDataOnlyOnError(0) }, option-field {} -- a sequence reserved for future use set-objects[] ::= { { transaction-flags BITS { needSuccess(0), needAll(1), notOrderDependent(2), }, option-field { } -- a sequence reserved for future use transaction-data-list[] ::= { choice { create-transaction, modify-transaction, delete-transaction, set-objects -- sub-transactions -- iana assigned extensions here? }, ... } } Wes Hardaker [Page 15] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 ... -- possibly more set-objects } } -- creates new rows. error condition if indexes or required -- columns aren't complete. create-transaction ::= { request-base-OID, -- (eg, OID for a table) index-data-list[] ::= { { index-number Integer32, index-value BER }, ... } column-data-list[] ::= { { column-number Integer32, column-value BER }, ... } } -- modifies existing rows based on search criteria. -- error condition on any modification failure. modify-transaction ::= { request-base-OID, -- (eg, OID for a table) index-search-objects[] ::= { { index-number Integer32, match-type MATCH-TYPE, index-search-value BER }, ... }, column-search-objects[] ::= { { column-number Integer32, match-type MATCH-TYPE, column-search-value BER }, ... } column-data-list[] ::= { { column-number Integer32, -- ** see below column-value BER }, ... } } Wes Hardaker [Page 16] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 -- deletes existing rows based on search criteria. -- error condition on any delete-transaction ::= { request-base-OID, -- (eg, OID for a table) index-search-objects[] ::= { { index-number Integer32, match-type MATCH-TYPE, index-search-value BER }, ... }, column-search-objects[] ::= { { column-number Integer32, match-type MATCH-TYPE, column-search-value BER }, ... } } In the above, the following pieces need explaining: flags Global flags to apply to the entire PDU. If the returnDataOnlyOnError bit is set, then the Write-Object-Response-PDU will only contain a duplicate set-objects portion of the message if there was an error somewhere in the processing of the message, otherwise the objects will be truncated. Command Responder implementations SHOULD support this flag, but if for any reason they decide to return the set-objects data portion of the message they MUST NOT set the returnDataOnlyOnError bit. transaction-flags: Flags which apply to each sub-element of the transaction to be performed. Currently there are 3 bits to define transactional processing semantics to be used when processing this message. These values are described in greater detail in the next section. operation: Indicates what type of operation is to be performed by this transaction set. Specifically: Wes Hardaker [Page 17] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 create: creates a new row. It is to be considered an error condition if either the row can not be created or if the row already existed. modify: modifies an existing row to replace some or all of the writable parameters within the row. It is an error condition if the row to be modified does not exist. delete: deletes a given row. If the row does not exist or can not be deleted for some reason, it is considered to be an error condition. The column-data-list MUST be empty when transmitting this operation. transaction-data-list: The transaction-data-list contains all of the data to be applied based on the operation choice type. The search criteria imposed by the index-search-objects and column-search-objects are identical in operation to the Get-Object-PDU equivalents, except that instead of requested information back, the column-data field objects should be applied in the case of a create or a modify, or the rows should simply be deleted in the case of a delete. Unlike the Get-Object-PDU match-type handling, an unsupported match-type will always trigger an error condition. 4.3.1. Write-Object-PDU transaction semantics In short, the flags defining Write-Object semantics can be summarized as follows: needSuccess If the needSuccess bit is turned on, it means that this trans¡ action is to be considered a failed transaction if during any time an error condition occurs. If at anytime a failure occurs when processing the transaction-data-list, the entire transaction must be rolled back. Wes Hardaker [Page 18] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 needAll If needAll is specified, it means each sub-component in the transaction-data-list must be attempted. notOrderDependent If the notOrderDependent bit is turned on, the transactions contained in the transaction-data-list MAY be executed in an order, or in parallel. Note that they must still revert to their previous values in an error condition requires a state rollback. In short the following bit combinations are described in combination: needSuccess = true, needAll = true: doAll [doUntilFailure] In this case, all contained transactions must succeed. If any failure occurs, the entire set must be rolled back. needSuccess = false, needAll = true: tryAll With this combination set, the command responder must try to achieve all the transactions, however if any fail it is not required to roll back the rest of the transactions. The failed transactions themselves, however, must individually revert to the before-write state. needSuccess = true, needAll = false: doAtLeastOne [doUntilSuc¡ cess] In this case, at least one must succeed for the transaction as a whole to succeed. More importantly, if any one transaction component does succeed, processing MUST be stopped and the transaction as a whole is considered successful. Another way to put it: at most one successful component is executed never more than one. needSuccess = false, needAll = false: tryAtLeastOne In this case, at least one must succeed for the transaction as a whole to succeed. More importantly, if any one transaction component does succeed, processing MUST be stopped and the transaction as a whole is considered successful. Another way to put it: at most one successful component is executed never more than one. However, in this case even if none of the transaction components succeeds, the transaction as a whole is still considered successful. It is impossible for a transac¡ tion of this type to be considered a failure in itself. Also, error reporting will always occur on failed objects even if they don't affect the containing transactions and surrounding transactions. 4.4. The Write-Object-Response-PDU Wes Hardaker [Page 19] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 This Write-Object-Response-PDU returns the results of the Write- Object-PDU operation. Note that more data can be returned in many cases when search operations required operations on multiple rows based on the search criteria. Write-Object-Response-PDU ::= { request-id Integer32, flags BITS { returnDataOnlyOnError(0) }, option-field {} -- a sequence reserved for future use set-objects-response[] ::= { { error-status Integer32, -- will include "tooComplex" error-index Integer32, error-string OCTET STRING, transaction-flags BITS { needSuccess(0), needAll(1), notOrderDependent(2), }, option-field { } -- a sequence reserved for future use transaction-response-list[] ::= { choice { create-transaction-response, modify-transaction-response, delete-transaction-response, set-objects-response -- sub-transactions -- iana assigned extensions here? }, ... } } ... -- possibly more set-objects } } -- duplication of the original requested creation. create-transaction-response ::= { request-base-OID, -- (eg, OID for a table) index-data-list[] ::= { { index-number Integer32, index-value BER }, ... } Wes Hardaker [Page 20] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 column-data-list[] ::= { { column-number Integer32, column-value BER }, ... } } -- returns all indexes for all rows affected by the modification request -- note that the return is a array, but the request was not. IE, -- all objects which were operated on are returned. modify-transaction-response[] ::= { { request-base-OID, -- (eg, OID for a table) index-data-list[] ::= { { index-number Integer32, -- ** see below index-value BER }, ... }, column-data-list[] ::= { { column-number Integer32, -- ** see below column-value BER }, ... } } } -- returns all indexes for all rows affected by the deletion request -- note that the return is a array, but the request was not. IE, -- all objects which were operated on are returned. delete-transaction ::= { request-base-OID, -- (eg, OID for a table) index-data-list[] ::= { { index-number Integer32, -- ** see below index-value BER }, ... } } 5. ASN.1 definitions for the PDUs To be defined at a later date. Wes Hardaker [Page 21] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 6. Examples Here are some example requests and responses of data retrieval (I didn't have time to put in Write examples). In each case, the curly brases indicate a sequence of some kind within BER (in the case of index lists, they'd be a application taged sequence as opposed to the normal BER sequence). 6.1. Retrieve a specific row from the ifTable This example retrieves the ifDescr and ifType columns for the 5th interface from the ifTable The Request: Get-Object-PDU { request-id 1 request-objects ::= { { max-return-objects 0 -- get all skip-objects 0 -- starting with the first cursor "" -- 0 length for first request flags 0 -- none specified. option-field {} request-base-object ::= OID:ifTable index-request-list ::= {} column-request-list ::= {2, 3} -- ifDescr, ifType index-search-objects ::= { { index-number = 1, -- ifIndex match-type = 0, -- equals index-search-value = 5 } -- the fifth } column-search-objects ::= { } -- none } } } The Response: Get-Object-Response-PDU { request-id 1 response-objects ::= { { error-status 0 -- noError error-index 0 -- must be 0 if noError error-string "" cursor "interface 5" -- agent implementation cursor. Wes Hardaker [Page 22] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 flags 0 -- none specified. option-field {} request-base-object ::= OID:ifTable data-list ::= { { index-data-list ::= { } -- none were requested column-data-list ::= { { column-number = 2, column-value = "interface 5" }, { column-number = 3, column-value = 6 } -- ethernetCsmacd } } } } } } 6.2. A multiple-packet example with a double filter. This example shows the retrieval of the elements of the ifTable which are of type "ethernetCsmacd" and have a ifSpeed >= 10,000,000 bps. Retrieve only 1 row at a time. Note that the response has the objects returned in the order the remote agent specified [interface 12, then interface 5], not in something that would map to the older- style lexographical ordering. The order returned must be consistent from one request to the next, but the manner in which the ordering is accomplished is ipmlementation specific. The cursor is used to indicate to the base agent where the request should restart from. Also, the dontReturnObjectNumbers bit is specified, which means the returned response should not include the component numbering fields and that the manager will remember what the original request was in order to understand the response. The first request: Get-Object-PDU { request-id 2 request-objects ::= { { max-return-objects 1 -- only return 1 skip-objects 0 -- starting with the first cursor "" -- 0 length for first request flags { dontReturnObjectNumbers } option-field {} request-base-object ::= OID:ifTable index-request-list ::= {} -- none Wes Hardaker [Page 23] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 column-request-list ::= {1, 2, 3} -- ifIndex, ifDescr, ifType index-search-objects ::= { } -- none column-search-objects ::= { { column-number = 3, -- ifType match-type = 0, -- equals column-search-value = 6 }, -- ethernetCsmacd { column-number = 5, -- ifSpeed match-type = 12, -- greaterThanOrEqual column-search-value = 10000000 }, -- 10 Mbps } } } } The first response: Get-Object-Response-PDU { request-id 2 response-objects ::= { { error-status 0 -- noError error-index 0 -- must be 0 if noError error-string "" cursor "interface 12" -- agent implementation cursor. flags { dontReturnObjectNumbers } option-field {} request-base-object ::= OID:ifTable data-list ::= { { index-data-list ::= { } -- none requested column-data-list ::= { { column-value = 12 } -- ifIndex = 12 { column-value = "interface 12" }, -- idDescr { column-value = 100000000 } -- ifSpeed = 100Mbs } } } } } } The second request to obtain the rest of the data: Get-Object-PDU { request-id 2 request-objects ::= { { max-return-objects 1 -- only return 1 skip-objects 0 -- functionally not used Wes Hardaker [Page 24] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 cursor "interface 12" -- cursor where we left off at flags { dontReturnObjectNumbers } option-field {} request-base-object ::= OID:ifTable index-request-list ::= {} -- none column-request-list ::= {1, 2, 5} -- ifIndex, ifDescr, ifSpeed index-search-objects ::= { } -- none column-search-objects ::= { { column-number = 3, -- ifType match-type = 0, -- equals column-search-value = 6 }, -- ethernetCsmacd { column-number = 5, -- ifSpeed match-type = 12, -- greaterThanOrEqual column-search-value = 10000000 }, -- 10 Mbps } } } } The second response: Get-Object-Response-PDU { request-id 2 response-objects ::= { { error-status 0 -- noError error-index 0 -- must be 0 if noError error-string "" cursor "interface 5" -- agent implementation cursor. flags { dontReturnObjectNumbers } option-field {} request-base-object ::= OID:ifTable data-list ::= { { index-data-list ::= { } -- none requested column-data-list ::= { { column-value = 5 } -- ifIndex = 5 { column-value = "interface 5" }, -- idDescr { column-value = 100000000 } -- ifSpeed = 100Mbs } } } } } } Wes Hardaker [Page 25] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 7. Elements of Procedure. To be done... 8. Design Considerations and Questions A lot of thought has gone into the PDUs defined in this document. I've tried to achieve a balance between flexibility and interoper¡ ability that will lean toward interoperability most of the time. Comments welcome. This section describes some of the things that I'm still thinking about and would like feedback on. 8.1. Return-field ranges Currently the list of columns/indexes to return contains only a list of integers, which could be long if you needed every column from 1-25 but not 26-30, for example. Should we support the notion of "1-25" and if we do, should we only support ranges or should we try and mix and match. IE, how do you expresss "1-10 and 15 and 20-30": a) 1,10,15,15,20,30 b) {1,10},15,{20,30} {}s would have to specially tagged sequences c) 1,2,3,...,10,15,20,21,...,30 8.2. Write-Object-PDU complexity In the long run, I made the Write-Object-PDU more powerful than I was originally thinking. It took me a while to decide what to do, but the more I thought about it the processing of the requests con¡ tained within will not be that difficult. Specifically, the opera¡ tions are very recursive in nature and the complexity when dealing with one specific component is isolated. Rollback to some extent is still going to be necessary, but the PDU likely reduces the amount of UNDO processing due to the fact that you can encode failure cases which may be known to be problematic and can be ignored. I debated for a while whether to put create/delete/modify transac¡ tions with the same request, but I've come to believe that it's very necessary. Specifically, it's likely that you'll only want to delete a row if and only if the creation of another row succeeds. If either of these fail, then the entire transaction should fail else the box may be left in an un-usable state. However, there are also likely times that they are completely unrelated transactions, and thus can be flagged as independent using the needSuccess bit. One possible simplification would be to remove the recursiveness of the transaction and to leave everything else. If I were to remove anything, this would likely be it. I don't actually think it'll remove processing code within the agent, but it may lead to greater Wes Hardaker [Page 26] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 interoperability. I'm not at all convinced that's the case, how¡ ever. 8.3. Get-Object/Write-Object commonality. The more I think about it, the more that I'm beginning to lean toward the idea that the Get-Object and Write-Object-PDUs should be similar in design. I debated for a long time whether the search criteria should go into the Write-Object-PDU or not, and I'm still not entirely decided (but it's there now). The danger comes in non-equal sign search criteria where you might think you know all the data on the far side that you're going to operate on, but it turns out that new rows have added by another source and thus you might affect a row you didn't otherwise intend to. However, I'm currently leaning toward the thought that the power to affect multiple objects with one transaction would be greatly beneficial. It will certainly help scaling with respect to management of extremely large networks containing lots of data which needs to be modified. In either case, it would be highly beneficial to retrieve a list of all the rows which were modified, thus the Write-Object transactions are designed such that the indexes of all the datasets which have been affected by the transaction are returned to the caller. This is one way to check for errors in the case of external modifica¡ tions. So, the real question is: Do we want search criteria in Write-Object operations? I'm still leaning toward yes (even more so after writ¡ ing this section). 8.4. Concrete packet formats vs optional field formats. Warning: Assumes a lot of knowledge about BER encoding beyond what SNMP uses today. Some background is given, but not as much as is probably needed. The PDUs defined in this document today aren't described in an ASN.1 notation yet. We have a few options when deciding how we want to encode them. First, ASN.1/BER has the ability to define "optional" fields. These fields simply wouldn't exist in the packet unless they are needed. SNMP has traditionally relied on a fixed packet format without optional fields. Breaking from this tradition may provide some packet compression and extensibility benefits. The down side is that sequences with enumerated data fields have an extra tag/length Wes Hardaker [Page 27] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 wrapper around each component within a sequence. (so data within becomes: sequence-number-tag length datatype-tag length DATA). Note that you essentially get a minimum of another 2-byte overhead on each field within each sequence. The optional-field structures within the packet, I think, must be defined using these enumerated sequences and everything within must be defined as OPTIONAL, or else they may not be expressible in an extensible way. However, there are some other cases where we may want to adopt this style of encoding. In the above PDUs, the following components are functionally "optional" in nature. Some are sequences, and having a empty sequence is functionally equivalent to an optional set of data. However, if truly marked as OPTIONAL then they wouldn't be in the packet at all (and the two byte "30 00" empty sequence portion wouldn't be encoded): Get-Object-PDU: index-request-list column-request-list index-search-objects column-search-objects option-field match-type -- missing is = Get-Object-Response-PDU: error-status -- if missing, no error! error-index error-string option-field index-number column-number Write-Object-PDU: index-search-objects column-search-objects option-field match-type -- missing is = I don't think we should mix and match encoding styles. IE, we should probably pick whether we going to do enumerated sequences for every portion of the packet (suffering the two-byte-minimum overhead on each portion) or not do it at all. Anyone have preferences? 8.5. Push Button Functionality. SNMP usage has proven that method invocation is needed within the Wes Hardaker [Page 28] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 protocol. The SET semantics and Write-Object semantics do not make this easy. Should a "invocation" write method be put in here now, even though we don't have a SMI that can define it's usage yet? 9. Author's Addresses: Wes Hardaker P.O. Box 382 Davis, CA 95617 Email: hardaker@tislabs.com 10. References [OPREQ] Woodcock, D., "Operator Requirements of Infrastructure Management Methods", Internet Draft draft-ops-operator-req-mgmt-02.txt, expired August 2002. 11. Full Copyright Statement Copyright (C) The Internet Society (2002). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of develop¡ ing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 12. Humor GOP GORP? WOP WORP. Wes Hardaker [Page 29] INTERNET-DRAFT Object Oriented PDU's for SNMP Sep. 2002 NOP NORP! Wes Hardaker [Page 30]