Network Working Group P. Shafer Internet-Draft Juniper Networks Expires: December 22, 2006 June 20, 2006 A SYSLOG Capability for NETCONF draft-shafer-netconf-syslog-00 Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on December 22, 2006. Copyright Notice Copyright (C) The Internet Society (2006). Shafer Expires December 22, 2006 [Page 1] Internet-Draft netconf-syslog June 2006 Table of Contents 1. Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Event Notifications . . . . . . . . . . . . . . . . . . . . . 3 2.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . 4 3. The ":syslog" Capability . . . . . . . . . . . . . . . . . . . 6 3.1. get-syslog-streams . . . . . . . . . . . . . . . . . . . . 6 3.1.1. Parameters . . . . . . . . . . . . . . . . . . . . . . 6 3.1.2. Response . . . . . . . . . . . . . . . . . . . . . . . 6 3.2. get-syslog-events . . . . . . . . . . . . . . . . . . . . 9 3.2.1. Parameters . . . . . . . . . . . . . . . . . . . . . . 10 3.2.2. Response . . . . . . . . . . . . . . . . . . . . . . . 13 4. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 14 5. Normative References . . . . . . . . . . . . . . . . . . . . . 14 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 15 Intellectual Property and Copyright Statements . . . . . . . . . . 16 Shafer Expires December 22, 2006 [Page 2] Internet-Draft netconf-syslog June 2006 1. Abstract SYSLOG is the most prevalent mechanism for delivering event notification data off networking devices. NETCONF is intended to provide a standard mechanism for performing configuration operations on networking devices. The two areas will overlap, giving rise to a need for receiving notification data over NETCONF. This document presents a simple mechanism for accessing SYSLOG data over NETCONF, implemented using the existing NETCONF RPC infrastructure. 2. Event Notifications Management of a network depends on timely access to information about the network, including status, alarms, faults, errors, and failures. As events of interest occur in the network, devices detect them and emit event notifications. The notifications are typically emitted as SNMP [1] traps or SYSLOG [2] messages. Both of these notification mechanisms require the destinations for notifications to be pre- configured, which represents an impediment for applications which require a more dynamic way to receive this data. This document defines a NETCONF [3] capability for retrieving SYSLOG data. Network management applications will have access to the rich volume of information currently instrumented in device software over the secure, private, connection-oriented, and reliable communication path offered by NETCONF. An application can see important notifications about the hardware which the user is attempting to configure, or see critical errors to which the user would otherwise be blind. NETCONF gives access to a device's information, both configuration and status, using a simple RPC-based protocol. It allows the definition of "capabilities", which define a contract of behavior and a set of RPCs used to implement that behavior. If a device reports a capability string in its initial message, it is indicating its ability to perform those RPCs and to behave according to the capability definition. These capabilities are the extension mechanism for NETCONF. NETCONF RPCs can be defined to return any data. The volume of data and the duration of the RPC are unlimited. Specifics about the nature of the RPC and the data returned are detailed in the capability definition. SYSLOG is an event notification mechanism widely used among network Shafer Expires December 22, 2006 [Page 3] Internet-Draft netconf-syslog June 2006 service providers. The breadth and depth of content is unmatched. Device vendors have existing embedded code for generating appropriate SYSLOG messages, and have documented the content and cause of these notifications. The integration of SYSLOG with management application seems natural. The SYSLOG protocol lacks security and reliability. But accessing SYSLOG data over NETCONF gives the richness of SYSLOG data over a secure, connection-oriented communications path, giving the application the information it needs about the network with a minimal impact to the device. 2.1. Overview SYSLOG event notifications are made available using NETCONF RPCs. The NETCONF client issues an RPC with a set of parameters detailing the information desired. The NETCONF server replies with the appropriate data. +----+ | c1 |---+ +----+ | +------+ (off-box) +----+ | | | | c2 | +--->|syslog| +-------+ +-------+ +----+ | |daemon|--------->|netconf|<--->|netconf| ... | | | >|server | |client | System | +------+ / +-------+ +-------+ Components| \ / ... | v / +----+ | (----------) | cn |---+ (historical) +----+ ( events ) (e.g. log files) (----------) The model requires that the device support a set of SYSLOG "event streams", which consist of a series of events which match the set of filters given in the streams definition. This allows the device to control what information is available to the client. The stream definition may be provided by the device vendor or be part of the device's configuration. Clients request data from one of these streams, and can supply additional filtering criteria to further constrain the data retrieved. Event notifications are generated by system components and are passed to a central syslog component, which inspects each event and matches the event against the set of stream definitions. When a match occurs, the notification is considered a member of that event stream. A notification may be part of multiple event streams. Shafer Expires December 22, 2006 [Page 4] Internet-Draft netconf-syslog June 2006 In addition to the filters defined on an event stream, the client may request filtering based on syslog header fields, names and value of structured data parameters within the notification, or message content. The client may use date and time ranges to trim recorded data, as needed, and may limit the volume of data in the response. When an event notification matches both the filters for a given stream and the filters for a particular outstanding RPC, it is encoded by the NETCONF server and forwarded to the NETCONF client. The data can be transported as simple SYSLOG messages or using the extended text format defined in the SYSLOG Working Group's Internet- Draft on the SYSLOG protocol [4]. Each notification is wrapped in an XML element. To receive notifications, a NETCONF client makes a RPC, supplying the name of the stream and any filtering to be performed in addition to any filters from the stream definition. The NETCONF server responds with a set of matching notifications. The client may request a continuous feed of SYSLOG events, allowing it to receive them as they are generated. During periods when no messages are being generated, the server will not send data, but will not close the NETCONF rpc-reply, giving a mechanism for a long-lived RPC with an open-ended data stream. The client may include termination criteria in the RPC, given a timestamp to stop, or a count of how many notifications to include in the reply. If no termination criteria are given, the RPC continues until the NETCONF session is terminated. The device may also retain event data for a stream, allowing clients to retrieve historic events. These events may be locally recorded, using locally available resources such as memory or disk. The volume of recorded events may be limited by these resources, allowing the device to keep a certain volume of events in a log file, or the fixed number of records in memory, or perhaps a set number of days worth of events. By defining the presence of historical data in the event stream definition, the control of local resource utilization is placed in the hands of the device. Historical data, if supported, is optional for any particular event stream. If historical events are available, the NETCONF server can include that data in the response. The response can include historical data, new data, or a combination. Shafer Expires December 22, 2006 [Page 5] Internet-Draft netconf-syslog June 2006 3. The ":syslog" Capability The :syslog capability will be defined with the capability string as follows: http://ietf.org/netconf/syslog/1.0 Operations and content defined in this capability definition should appear in a namespace with this URI. This capability defines the following set of RPCs. If this capability is accepted by the NETCONF working group, the namespace should be changed to follow the URN format. 3.1. get-syslog-streams The RPC gives the client the ability to discover the current set of SYSLOG streams the device supports. A stream is a series of SYSLOG events that match a particular set of criteria. The stream may include historic events, which may be recorded locally. 3.1.1. Parameters None. 3.1.1.1. Example The following RPC requests the list of streams supported by the NETCONF server. 3.1.2. Response The reply consists of the top-level element containing a set of elements. The element contains the description of an individual stream of SYSLOG data. This element can contain the following elements: gives the unique name of the stream. This field is mandatory. if the current NETCONF session lacks sufficient permission to read this stream. An implementation is free to omit unreadable streams from the list of streams. Shafer Expires December 22, 2006 [Page 6] Internet-Draft netconf-syslog June 2006 if the stream is recording events locally gives the format of the data available from the stream, and should be one of: o -- "traditional" for traditional RFC 3164 [2] SYSLOG data o -- "structured-data" for the Structured Data Parameter format defined in SYSLOG Working Groups's Internet-Draft [4]. contains two attributes used to match the priority field of the syslog notifications. The "facility" attribute describes the part of the system generating the message, and many are not appropriate to routing devices. The "level" attribute describes the severity of the event. If either attribute is missing, the priority matches all notifications, similar to the "any" match in most SYSLOG implementations. The full set of facilities is specified by SYSLOG. o "authorization" -- The authorization system o "authpriv" -- The same as "auth", but logged to a file readable only by selected individuals o "console" -- Messages written to /dev/console by the kernel console output driver o "cron" -- The unix cron daemon o "daemon" -- System daemons that are not provided for explicitly by other facilities (This is the most common facility for routing devices) o "ftp" -- The file transfer protocol daemon o "kernel" -- Messages generated by the kernel o "lpr" -- The line printer spooling system o "mail" -- The mail system o "news" -- The network news system o "security" -- Security subsystems o "syslog" -- Messages generated internally by the syslog daemon Shafer Expires December 22, 2006 [Page 7] Internet-Draft netconf-syslog June 2006 o "user" -- Messages generated by user processes o "uucp" -- The uucp system o "local0", "local1", .. "local7" -- Reserved for local use The full set of levels is specified by SYSLOG. Any event containing a level equal or greater than this value will be carried in the stream. The following is an ordered list of levels, sorted highest to lowest. o -" emergency" -- A panic condition o "alert" -- A condition that should be corrected immediately o "critical" -- Critical conditions, e.g., hard device errors o "error" -- Errors o "warning" -- Warning messages o "notice" -- Conditions that are not error conditions, but should possibly be handled specially o "info" -- Informational messages o "debug" -- Messages that contain information normally of use only when debugging a program Multiple elements can be used to match multiple conditions. An event notification matches if it matches any of the conditions. contains a regular expression (as defined by IEEE Std 1003.2/POSIX.2 [5]) which the message of the SYSLOG must match. Any event whose message string matches this expression is carried in the stream. contains the name of a system process on which to match. contains a regular expression which the name of an event must match. contains an expression of the form 'name=value', where the name is an SD-Param parameter name and value is a regular expression which the parameter value from the event must match. If multiple elements appear, the event must match all name=value pairs to be carried in the stream. If an event does not Shafer Expires December 22, 2006 [Page 8] Internet-Draft netconf-syslog June 2006 contain the named parameter, it is not a match. The parameter name can be prefix by an optional SD-ID and a colon (":", ABNF %d58) to distinguish the naming authority of the parameter name. This allows a stream to filter in conditions when parameter names are duplicated with different SD-IDs. 3.1.2.1. Example The following response lists three streams, one a typical log file such as /var/log/messages in unix, one a debug stream, and one a structured data stream matching on a specific set of criteria. messages traditional debug traditional today's hot routing issue structured-data rpd .*peer.* peer=10.1.2.3 3.2. get-syslog-events The RPC gives access to the events inside a stream. When a NETCONF server receives this RPC, it will respond with a set of SYSLOG events that match the criteria in the request. The RPC may be long-lived, in that it does not normally complete in a fixed time frame, but continues to forward matching events from the device until the session is terminated. This allows the client to view a continual stream of events coming from the box, similar to a Shafer Expires December 22, 2006 [Page 9] Internet-Draft netconf-syslog June 2006 traditional SYSLOG event stream, but within a secure, connection- oriented NETCONF session. 3.2.1. Parameters contains the name of the stream from which syslog notifications are matched, as given in the element from the reply from . The parameter is mandatory. messages gives the maximum number of events to include in the response, allowing the client to limit the time or memory consumed by the reply. If this parameter is provided, the NETCONF server will close the RPC response when the given number of events have been matched. 100 instructs the device to match only recorded events, and to complete the RPC as soon as the last recorded event is inspected. If this parameter is given, the NETCONF server will close the RPC response when no further recorded events are available. gives the timestamp for the earliest event to be inspected. Events generated before this time are not matched. The format of the time in restricted in accordance with Section 6.2.3 of the SYSLOG [4] Internet-Draft. 2006-04-01T23:20:50.52Z gives the timestamp for the last event to be inspected. Events generated after this time are not matched. If this parameter is provided, the NETCONF server will close the response when the given time passes. 2006-04-01T23:30Z If the is given, the NETCONF server responds with events which have occurred since the given time and match the request's filter criteria. If historical data is available and matches the time constraint, it is included in the response. The NETCONF server will then respond the new events as they occur. If the and parameters are given but the is not, the NETCONF server responds with the most recent events which match the request's filter criteria. If neither or are included in the request, the NETCONF server responds with new events as they occur. Historical data is not used. The RPC reuses a number of elements defined in the response from the RPC. contains the facility and level attributes to match against the SYSLOG notification. Multiple elements can be used to match multiple conditions. contains a regular expression which the message of the SYSLOG must match. Any event whose message string matches this expression is considered a match. ALARM CLEAR:.*fxp0 contains the name of a system process on which to match. rpd contains a regular expression which the name of an event must match. CHASSIS.*DOWN contains an expression of the form 'name=value', where the name is an SD-Param parameter name and value is a regular expression which the parameter value from the event must match. If multiple elements appear, the event must match all name=value pairs to be considered a match. If an event does not contain the named parameter, it is not a match. peer=10.* The parameter name can be prefix by an optional SD-ID and a colon (":", ABNF %d58) to distinguish the naming authority of the parameter name. This allows a stream to filter in conditions when parameter names are duplicated with different SD-IDs. junos@2636.1.1.1.2.13:username=regress The , , and parameters are only Shafer Expires December 22, 2006 [Page 11] Internet-Draft netconf-syslog June 2006 supported if the requested stream is in the "structured-data" format. They are ignored for streams using other formats. 3.2.1.1. Closing the Response The RPC response will be closed when one of the following conditions are met: o the parameter was given and events have been included in the response, or o the parameter was given and the specified time has past, or o the parameter was given and no further recorded events are available If one of these parameters was given, the NETCONF server will continue to emit events until the appropriate condition is fulfilled. If more than one of these parameters are provided, the RPC will complete when any condition is satisfied. If none of there parameters was given, the NETCONF server will continue to emit events until the session is terminated, providing a mechanism for getting ongoing notification data from the server. When an event occurs on the device, the device will forward the event notification to any outstanding RPC which the event matches. 3.2.1.2. Examples messages ^User 'r.*' logout$ mgd UI_CHILD_START username=regress input=configure authentication-level=super-user signal-name=Broken pipe 2006-04-01T23:20:50.52Z 100 Shafer Expires December 22, 2006 [Page 12] Internet-Draft netconf-syslog June 2006 3.2.2. Response The reply consists of a top-level element containing a stream of SYSLOG events. These events are encoded in text strings according to the rules contained in RFC3164 and the SYSLOG Internet- Draft. Each event from a stream that are using the "traditional" format is contained in a element. If the stream is using the "structured-data" format, each event is contained in a element. 3.2.2.1. Examples (Newlines and whitespace were inserted in the text data to make the output in this section more readable and to fit within the IETF document formatting guidelines.) The following example contains two events in "traditional" format. Jun 14 08:29:14 kitkat mgd[3993]: UI_CHILD_START: Starting child '/sbin/ifinfo' Jun 14 08:29:14 kitkat mgd[3993]: UI_CHILD_STATUS: Cleanup child '/sbin/ifinfo', PID 3996, status 0 The following example contains the same events recorded in "structured-data" format. Shafer Expires December 22, 2006 [Page 13] Internet-Draft netconf-syslog June 2006 1 2006-06-14T08:29:14.397+05:30 kitkat mgd 3993 UI_CHILD_START [junos@2636.1.1.1.2.13 command="/sbin/ifinfo"] Starting child '/sbin/ifinfo' 1 2006-06-14T08:29:14.417+05:30 kitkat mgd 3993 UI_CHILD_STATUS [junos@2636.1.1.1.2.13 command="/sbin/ifinfo" process-id="3996" status="0"] Cleanup child '/sbin/ifinfo', PID 3996, status 0 4. Acknowledgments The author wishes to thank for following people for their most excellent review of this document and the numerous additions, suggestions, and refinements they have made: Pallavi Mahajan, Andy Bierman, Rob Enns, Sri Ram Sankar, Kent Watsen, and Reid Wilson. Comments are solicited and should be addressed to the NETCONF Working Group's mailing list and/or the author. 5. Normative References [1] Case, J., Fedor, M., Schoffstall, M., and J. Davin, "Simple Network Management Protocol (SNMP)", STD 15, RFC 1157, May 1990. [2] Lonvick, C., "The BSD Syslog Protocol", RFC 3164, August 2001. [3] Enns, R., "NETCONF Configuration Protocol", draft-ietf-netconf-prot-07 (work in progress), June 2005. [4] Gerhards, R., "The syslog Protocol", draft-ietf-syslog-protocol-14 (work in progress), July 2005. [5] Institute of Electrical and Electronics Engineers, "Information Technology - Portable Operating System Interface (POSIX) - Part 2: Shell and Utilities (Vol. 1)", IEEE Standard 1003.2, 1992. Shafer Expires December 22, 2006 [Page 14] Internet-Draft netconf-syslog June 2006 Author's Address Phil Shafer Juniper Networks 940 Main Campus Drive Raleigh, NC 27606 US Email: phil@juniper.net Shafer Expires December 22, 2006 [Page 15] Internet-Draft netconf-syslog June 2006 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society (2006). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Shafer Expires December 22, 2006 [Page 16]