Network Working Group B. Claise Internet-Draft J. Clarke Updates: 7950 (if approved) Cisco Systems, Inc. Intended status: Standards Track K. D'Souza Expires: May 15, 2018 AT&T November 11, 2017 New YANG Module Update Procedure draft-clacla-netmod-yang-model-update-02 Abstract This document specifies a new YANG module update procedure in case of backward-incompatible changes, as an alternative proposal to the YANG 1.1 specifications. This document updates RFC 7950. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on May 15, 2018. Copyright Notice Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Claise, et al. Expires May 15, 2018 [Page 1] Internet-Draft YANG Catalog November 2017 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. The Problems . . . . . . . . . . . . . . . . . . . . . . . . 3 2.1. Slow Standardization . . . . . . . . . . . . . . . . . . 3 2.2. Some YANG Modules are Not Backward Compatible . . . . . . 3 2.3. A Zoo of YANG Modules . . . . . . . . . . . . . . . . . . 3 2.4. YANG Modules Obsolete Relationship . . . . . . . . . . . 4 2.5. YANG Module Transition Strategy . . . . . . . . . . . . . 5 3. The Solution . . . . . . . . . . . . . . . . . . . . . . . . 6 3.1. SEMVER Semantic Versioning . . . . . . . . . . . . . . . 6 3.2. Updating the YANG 1.1 Specifications . . . . . . . . . . 9 3.3. The Derived Semantic Version . . . . . . . . . . . . . . 9 4. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 10 5. Semantic Version Extension Module . . . . . . . . . . . . . . 11 6. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 12 7. Security Considerations . . . . . . . . . . . . . . . . . . . 12 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 13 9.1. Normative References . . . . . . . . . . . . . . . . . . 13 9.2. Informative References . . . . . . . . . . . . . . . . . 13 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13 1. Introduction The YANG data modeling language [RFC7950] specifies strict rules for updating YANG modules (see section 11 "Updating a Module"). Citing a few of the relevant rules for this document: 1. "As experience is gained with a module, it may be desirable to revise that module. However, changes to published modules are not allowed if they have any potential to cause interoperability problems between a client using an original specification and a server using an updated specification." 2. "Note that definitions contained in a module are available to be imported by any other module and are referenced in "import" statements via the module name. Thus, a module name MUST NOT be changed. Furthermore, the "namespace" statement MUST NOT be changed, since all XML elements are qualified by the namespace." 3. "Otherwise, if the semantics of any previous definition are changed (i.e., if a non-editorial change is made to any definition other than those specifically allowed above), then this MUST be achieved by a new definition with a new identifier." What are the consequences? Claise, et al. Expires May 15, 2018 [Page 2] Internet-Draft YANG Catalog November 2017 1. Ideally, the YANG module names should not be changed due the importance of not changing the automation code in case of import statements or service composition at the orchestration layer. 2. When the same YANG module name is kept, the new YANG module revision must always be updated in a backward-compatible way. 2. The Problems This section lists a series of problems, hopefully listed in a logical order, which leads to the solution in the next section. 2.1. Slow Standardization The conclusions drawn in the introduction lead to the logical conclusion that the standardized YANG modules have to be perfect on day one (at least the structure), which in turn might explain why all the IETF YANG modules take so long to standardize. Shooting for perfection (at least in structure) is obviously a noble goal, but if the perfect standard comes too late, it doesn't help the industry. 2.2. Some YANG Modules are Not Backward Compatible As we learn from our mistakes, we're going to face more and more backward-incompatible YANG modules. An example is the YANG data model for L3VPN service delivery [RFC8049], which, based on implementation experience, must be updated in a backward-incompatible way with draft-wu-l3sm-rfc8049bis [I-D.wu-l3sm-rfc8049bis]. While Standards Development Organization (SDO) YANG modules are obviously better for the industry, we must recognize that many YANG modules are actually generated YANG modules (for example, from internal databases), also known as native YANG modules, or vendor modules [RFC8199]. From time to time, the new YANG modules are not backward-compatible. In such cases, it would be better to indicate how backward-compatible a given YANG module actually is. 2.3. A Zoo of YANG Modules Even if we focus on the IETF, we have to observe that many SDOs, opensource fora, and vendors develop YANG modules develop YANG modules. This should be considered a success for an IETF developed technoogy. However, the operators are faced with this problem: how to select the YANG modules to take into account for their service developments. Claise, et al. Expires May 15, 2018 [Page 3] Internet-Draft YANG Catalog November 2017 The site (and the YANG catalog that it provides: YANG module for yangcatalog.org, [I-D.clacla-netmod-model-catalog]) is an attempt to become a reference for all YANG modules available in the industry, for both YANG developers to search on what exists already) and for operators (to discover the more mature YANG models to automate services). This YANG catalog should not only contain pointers to the YANG modules themselves, but also contain metadata related to those YANG modules: What is the module type (service model or not?); what is the maturity level? (e.g., for the IETF: is this an RFC, a working group document or an individual draft?); is this module implemented?; who is the contact?; is there open-source code available? And we expect many more in the future. The industry has begun to understand that the metadata related to YANG models become equally important as the YANG models themselves. The yangcatalog.org instantiation of the catalog provides a means for module authors and vendors implementing modules to upload their metadata, which is then searchable via an API, as well as using a variety of web-based tools. The instructions for contributing and searching for metadata can be found at . The issue is actually the number of YANG modules the operators are offered. At the time of writing this document, the number of unique YANG modules in the catalog is exactly 2596 (and that number keeps growing), while the IETF has standardized or is busy standardizing a small subset of those. Therefore, it's important to distinguish the relevant YANG modules with the pack and to understand the relationship between the YANG modules. 2.4. YANG Modules Obsolete Relationship So the operators use the yangcatalog.org to discover which YANG modules they can use NOW. They base their selection not only on the YANG module content, but also on the related metadata. When faced with the zoo of the YANG modules, it's difficult to understand the relationship between YANG modules. As an example: how could an operator discovers that a YANG-MODULE-B obsoletes YANG-MODULE-A? Indeed, both have different YANG module names. The only available information is an "obsolete" tag in the published RFC containing YANG-MODULE-B: this tag would point to YANG-MODULE-A. In the world of automation, going through a published RFC as a level of indirection to understand the YANG module obsolete relationship is a non starter. Food for thought: the IETF should stop thinking that the metric for success is an RFC number, as opposed to the contained YANG module(s). Claise, et al. Expires May 15, 2018 [Page 4] Internet-Draft YANG Catalog November 2017 We need an automatic way to discover that a YANG-MODULE-B obsoletes YANG-MODULE-A, so that YANG-MODULE-A should not be given any attention. The following example is not an automatic way. description "This YANG module defines a generic service configuration model for Layer 3 VPNs. This model is common across all vendor implementations. This obsoletes the RFC8049 YANG module, ietf-l3vpn-svc@2017-01-2"; revision 2017-09-14 { description "First revision of RFC8049."; reference "RFC xxxx: YANG Data Model for L3VPN Service Delivery"; Along the same lines, going through an out-of-band tool such as the yangcatalog.org in order to discover the obsolete relationship is a possible automatic way, but not ideal. 2.5. YANG Module Transition Strategy Let's assume for a moment that we change the YANG module, with the specific example of the ietf-routing, which some propose to update to ietf-routing-2. Here are all the ietf-routing dependent YANG modules (those modules that depend on ietf-routing) . So many YANG modules. Let's look at the difference for ietf-routing-2: . Changing the module name from ietf-routing to ietf-routing-2 implies that the we have to warn all draft authors of ietf-routing YANG dependent modules. Firstly, to make sure they are aware of ietf- routing-2 (publishing a RFC8022bis mentioning in the module description that this module is not compatible with the NMDA architecture, and providing a pointer to ietf-routing-2 ... is not an automatic way... so barely useful). And secondly, to ask them to change their import (or service composition) to ietf-routing-2. Hopefully, in the ietf-routing case, all dependent YANG modules are part of the IETF, so the communication is a manageable. Claise, et al. Expires May 15, 2018 [Page 5] Internet-Draft YANG Catalog November 2017 Changing the ietf-interfaces YANG module name would be a different challenge, as it's used throughout the industry: 3. The Solution The solution is composed of two mandator aspects, a semantic versioning YANG extension and an update to RFC7950. An optional additional check, validating the semantic versioning from a syntact point of view, can either assist in determining the correct semantic versioning values, or can help in determining the values for YANG modules that don't support this extension. 3.1. SEMVER Semantic Versioning The semantic versioning solution proposed here as already been proposed in [I-D.openconfig-netmod-model-catalog] (cut/paste here with the authors permission)which itself is based on [openconfigsemver]. The goal is to indicate the YANG module backwards (in)compatibility, following semver.org semantic versioning [semver]: MAJOR is incremented when the new version of the specification is incompatible with previous versions. MINOR is incremented when new functionality is added in a manner that is backward-compatible with previous versions. PATCH is incremented when bug fixes are made in a backward- compatible manner. Claise, et al. Expires May 15, 2018 [Page 6] Internet-Draft YANG Catalog November 2017 // extension statements extension openconfig-version { argument "semver" { yin-element false; } description "The OpenConfig version number for the module. This is expressed as a semantic version number of the form: x.y.z where: * x corresponds to the major version, * y corresponds to a minor version, * z corresponds to a patch version. This version corresponds to the model file within which it is defined, and does not cover the whole set of OpenConfig models. Where several modules are used to build up a single block of functionality, the same module version is specified across each file that makes up the module. A major version number of 0 indicates that this model is still in development (whether within OpenConfig or with industry partners), and is potentially subject to change. Following a release of major version 1, all modules will increment major revision number where backwards incompatible changes to the model are made. The minor version is changed when features are added to the model that do not impact current clients use of the model. The patch-level version is incremented when non-feature changes (such as bugfixes or clarifications to human-readable descriptions that do not impact model functionality) are made that maintain backwards compatibility. The version number is stored in the module meta-data."; } Along these lines, we propose the following YANG 1.1 extension for a more generic semantic version. The formal definition is found at the end of this document. extension module-version { argument "semver" { yin-element false; } } Claise, et al. Expires May 15, 2018 [Page 7] Internet-Draft YANG Catalog November 2017 The extension would typically be used this way: module yang-module-name { yang-version "1"; namespace "name-space"; prefix "prefix-name"; import ietf-semver-extension { prefix "semver-ext"; } organization "to be completed"; contact "to be completed"; description "to be completed"; semver-ext:module-version "1.1.2"; revision 2017-10-30 { description "Change the module structure"; reference "1.1.2"; } revision 2017-07-30 { description "Fixed unprintable character"; reference "0.1.2"; } revision 2017-04-03 { description "Update copyright notice."; reference "0.1.1"; } revision 2017-01-26 { description "Initial module for inet types"; reference "0.1.0"; } //YANG module definition starts here Claise, et al. Expires May 15, 2018 [Page 8] Internet-Draft YANG Catalog November 2017 See also "Semantic Versioning and Structure for IETF Specifications" [I-D.claise-semver] for a mechanism to combine the semantic versioning, the github tools, and a potential change to the IETF process. 3.2. Updating the YANG 1.1 Specifications RFC 7950 section 11, must be updated to express: "As experience is gained with a module, it may be desirable to revise that module. Changes to published modules are allowed, even if they have some potential to cause interoperability problems, at the condition that the semantic versioning change are clearly indicated based on the SEMVER YANG extension." 3.3. The Derived Semantic Version The YANG catalog contains not only the most up-to-date YANG module revision of a given module, but keeps all previous revisions as well. With APIs in mind, it's important to understand whether different YANG module revisions are backward compatible (this is specifically imported for native YANG modules, i.e. the ones where generated-from = native), even for the YANG modules that don't support the YANG extension specified in this document. Two distinct leaves in the YANG module [I-D.clacla-netmod-model-catalog] contain this semver notation: the semantic-version leaf contains the value embedded within a YANG module (if it is available). the derived-semantic-version leaf is established by examining the the YANG module themselves. As such derived-semantic-version only takes syntax into account as opposed to the meaning of various elements when it computes the semantic version. The algorithm used to produce the derived-semantic-version is as follows: 1. Order all modules of the same name by revision from oldest to newest. 2. If module A, revision N+1 has failed compilation, bump its derived semantic MAJOR version. 3. Else, run "pyang --check-update-from" on module A, revision N and revision N+1 to see if backward-incompatible changes exist. Claise, et al. Expires May 15, 2018 [Page 9] Internet-Draft YANG Catalog November 2017 4. If backward-incompatible changes exist, bump module A, revision N+1's derived MAJOR semantic version. 5. If no backward-incompatible changes exist, compare the pyang trees of module A, revision N and revision N+1. 6. If there are structural differences (e.g., new nodes), bump module A, revision N+1's derived MINOR semantic version. 7. If no structural differences exist, bump module A, revision N+1's derived PATCH semantic version. Note that the absolute numbers in the semantic-version and derived- semantic-version are actually meaningless by themselves. That is, one must compare two different semver values for a given module to understand the compatibility between them. 4. Open Issues More work is needed in order to fully flesh out the semantic version requirements for YANG modules. These include the following. A new kind of import is required. Today, we have import by revision (though this is seldomly used). There should also be an import by module-version (i.e., the semantic version). This import will not simply be a copy of import by revision. Consideration needs to be given to expressions such as, "version==X", "version>=X", "major version==X", "major version>=X", etc. Similarly to import-by-version, we may also require a new naming convention for modules. Today, modules are named in module@revision.yang notation. Re-using the '@' for version is not ideal. Perhaps a new character such as '%' is needed (i.e., module%version.yang). In this manner, both version and revision could be used. Taking another page from Openconfig, the notion of a module bundle should be considered. That is, there may need to be a way to enumerate modules that are part of a bundle and are known to interoperate. This may not be as critical if a rich import-by- version is defined. Similarly, the concept of a feature bundle should be considered. Typically, operators combine and test YANG modules to build value- add services. These bundles form releases for specific features or services, and it is critical to ensure as the modules evolve, the bundles can coherently evolve with them. Claise, et al. Expires May 15, 2018 [Page 10] Internet-Draft YANG Catalog November 2017 More work is needed in order to fully flesh out the semantic version requirements for YANG modules (ultimately, a choice needs to be made as to what the next item is on which to focus). 5. Semantic Version Extension Module The extension described in this module is defined in the YANG module below. file "ietf-semver-extension@2017-11-10.yang" module ietf-semver-extension { yang-version 1.1; namespace "urn:ietf:params:xml:ns:yang:ietf-semver-extension"; prefix semver-ext; organization "IETF NETMOD (Network Modeling) Working Group"; contact "WG Web: WG List: Author: Benoit Claise Author: Joe Clarke Author: Kevin D'Souza "; description "This module contains a defintion for a YANG 1.1 extension to express the semantic version of YANG modules."; revision 2017-11-10 { description "Initial revision."; reference "draft-clacla-netmod-yang-model-update: New YANG Module Update Procedure"; } extension module-version { argument semver; description "The version number for the module. This is expressed as a semantic version number of the form: x.y.z where: * x corresponds to the major version, * y corresponds to a minor version, Claise, et al. Expires May 15, 2018 [Page 11] Internet-Draft YANG Catalog November 2017 * z corresponds to a patch version. This version corresponds to the model file within which it is defined. A major version number of 0 indicates that this model is still in development, and is potentially subject to change. Following a release of major version 1, all modules will increment major revision number where backwards incompatible changes to the model are made. The minor version is changed when features are added to the model that do not impact current clients use of the model. The patch-level version is incremented when non-feature changes (such as bugfixes or clarifications to human-readable descriptions that do not impact model functionality) are made that maintain backwards compatibility. The version number is stored in the module meta-data. By comparing the module-version between two revisions of a given module, one can know if revision N+1 is backwards compatible or not relative to revision N, as well as whether or not new features have been added to revision N+1."; reference "http://semver.org/ : Semantic Versioning 2.0.0"; } } 6. Contributors Anees Shaikh, Google Rob Shakir, Google 7. Security Considerations To be completed 8. IANA Considerations No IANA action is requested. Claise, et al. Expires May 15, 2018 [Page 12] Internet-Draft YANG Catalog November 2017 9. References 9.1. Normative References [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", RFC 7950, DOI 10.17487/RFC7950, August 2016, . 9.2. Informative References [I-D.clacla-netmod-model-catalog] Clarke, J. and B. Claise, "YANG module for yangcatalog.org", draft-clacla-netmod-model-catalog-02 (work in progress), October 2017. [I-D.claise-semver] Claise, B., Barnes, R., and J. Clarke, "Semantic Versioning and Structure for IETF Specifications", draft- claise-semver-01 (work in progress), July 2017. [I-D.openconfig-netmod-model-catalog] Shaikh, A., Shakir, R., and K. D'Souza, "Catalog and registry for YANG models", draft-openconfig-netmod-model- catalog-02 (work in progress), March 2017. [I-D.wu-l3sm-rfc8049bis] Wu, Q., Litkowski, S., Tomotaki, L., and K. Ogaki, "YANG Data Model for L3VPN Service Delivery", draft-wu-l3sm- rfc8049bis-09 (work in progress), October 2017. [openconfigsemver] "Semantic Versioning for Openconfig Models", . [RFC8049] Litkowski, S., Tomotaki, L., and K. Ogaki, "YANG Data Model for L3VPN Service Delivery", RFC 8049, DOI 10.17487/RFC8049, February 2017, . [RFC8199] Bogdanovic, D., Claise, B., and C. Moberg, "YANG Module Classification", RFC 8199, DOI 10.17487/RFC8199, July 2017, . [semver] "Semantic Versioning 2.0.0", . Authors' Addresses Claise, et al. Expires May 15, 2018 [Page 13] Internet-Draft YANG Catalog November 2017 Benoit Claise Cisco Systems, Inc. De Kleetlaan 6a b1 1831 Diegem Belgium Phone: +32 2 704 5622 Email: bclaise@cisco.com Joe Clarke Cisco Systems, Inc. 7200-12 Kit Creek Rd Research Triangle Park, North Carolina United States of America Phone: +1-919-392-2867 Email: jclarke@cisco.com Kevin D'Souza AT&T 200 S. Laurel Ave Middletown, NJ United States of America Email: kd6913@att.com Claise, et al. Expires May 15, 2018 [Page 14]