Network Working Group J. Schoenwaelder, Ed. Internet-Draft Jacobs University Intended status: Standards Track June 10, 2016 Expires: December 12, 2016 A Revised Conceptual Model for YANG Datastores draft-schoenw-netmod-revised-datastores-01 Abstract Datastores are a fundamental concept binding the YANG data modeling language to protocols transporting data defined in YANG data models, such as NETCONF or RESTCONF. This document defines a revised conceptual model of datastores based on the experience gained with the initial simpler model and addressing requirements that were not well supported in the initial model. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on December 12, 2016. Copyright Notice Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of Schoenwaelder Expires December 12, 2016 [Page 1] Internet-Draft Revised Model for YANG Datastores June 2016 the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 2 3. Original Model of Datastores . . . . . . . . . . . . . . . . 3 4. Revised Model of Datastores . . . . . . . . . . . . . . . . . 5 5. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 7 5.1. Missing Resources . . . . . . . . . . . . . . . . . . . . 7 5.2. System-controlled Resources . . . . . . . . . . . . . . . 7 5.3. Auto-configured or Auto-negotiated Values . . . . . . . . 8 5.4. Operational State with Different Origins . . . . . . . . 8 6. Implications . . . . . . . . . . . . . . . . . . . . . . . . 8 6.1. Implications on NETCONF . . . . . . . . . . . . . . . . . 8 6.2. Implications on RESTCONF . . . . . . . . . . . . . . . . 9 6.3. Implications on YANG . . . . . . . . . . . . . . . . . . 9 6.4. Implications on Data Models . . . . . . . . . . . . . . . 9 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 8. Security Considerations . . . . . . . . . . . . . . . . . . . 10 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 10 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 10.1. Normative References . . . . . . . . . . . . . . . . . . 11 10.2. Informative References . . . . . . . . . . . . . . . . . 11 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 12 1. Introduction This document provides a revised architectural framework for datastores as they are used by network management protocols such as NETCONF [RFC6241], RESTCONF [I-D.ietf-netconf-restconf] and the YANG [RFC6020] data modeling language. Datastores are a fundamental concept binding management data models to network management protocols and agreement on a common architectural model of datastores ensures that data models can be written in a network management protocol agnostic way. This architectural framework identifies a set of conceptual datastores but it does not mandate that all network management protocols expose all these conceptual datastores. Furthermore, the architecture does not detail how data is encoded by network management protocols. 2. Background NETCONF [RFC6241] provides the following definitions: Schoenwaelder Expires December 12, 2016 [Page 2] Internet-Draft Revised Model for YANG Datastores June 2016 o datastore: A conceptual place to store and access information. A datastore might be implemented, for example, using files, a database, flash memory locations, or combinations thereof. o configuration datastore: The datastore holding the complete set of configuration data that is required to get a device from its initial default state into a desired operational state. YANG 1.1 [I-D.ietf-netmod-rfc6020bis] provides the following refinements when NETCONF is used with YANG (which is the usual case but note that NETCONF was defined before YANG did exist): o datastore: When modeled with YANG, a datastore is realized as an instantiated data tree. o configuration datastore: When modeled with YANG, a configuration datastore is realized as an instantiated data tree with configuration data. RFC 6244 defined operational state data as follows: o Operational state data is a set of data that has been obtained by the system at runtime and influences the system's behavior similar to configuration data. In contrast to configuration data, operational state is transient and modified by interactions with internal components or other systems via specialized protocols. Section 4.3.3 of RFC 6244 discusses operational state and among other things mentions the option to consider operational state as being stored in another datastore. Section 4.4 of this document then concludes that at the time of the writing, modeling state as a separate data tree is the recommended approach. Implementation experience and requests from operators indicate that the datastore model initially designed for NETCONF and refined by YANG needs to be extended. In particular, the notion of intended configuration and applied configuration has developed. Furthermore, separating operational state data from configuration data in a separate branch has been found operationally complicated. The relationship between the branches is not machine readable and filter expressions operating on configuration data and on related operational state data are different. 3. Original Model of Datastores The following drawing shows the original model of datastores as it is currently used by NETCONF [RFC6241]: Schoenwaelder Expires December 12, 2016 [Page 3] Internet-Draft Revised Model for YANG Datastores June 2016 +-------------+ +-----------+ | | | | | (ct, rw) |<---+ +--->| (ct, rw) | +-------------+ | | +-----------+ | | | | | +------------+ | +-------->| |<--------+ | (ct, rw) | +------------+ | v +--------------------------------+ | operational-state |<-- control plane | (cf, ro) | +--------------------------------+ ct = config true; cf = config false; rw = read-write; ro = read-only Note that read-only (ro) and read-write (rw) is to be understood at a conceptual level. In NETCONF, for example, support for the and datastores is optional and the datastore does not have to be writable. Furthermore, the datastore can only be modified by copying to in the standardized NETCONF datastore editing model. The RESTCONF protocol does not expose these differences and instead provides only a writable unified datastore, which hides whether edits are done through a datastore or by directly modifying the datastore or via some other implementation specific mechanism. RESTCONF also hides how configuration is made persistent. Note that implementations may also have additional datastores that can propagate changes to the datastore. NETCONF explicitly mentions so called named datastores. Some observations: o Operational state has not been defined as a datastore although there were proposals in the past to introduce an operational state datastore. o The NETCONF operation returns the content of the configuration datastore together with the operational state. It is therefore necessary that config false data is in a different branch than the config true data. This is in particular relevant if operational state data can have a different lifetime compared to configuration data or if configuration data is not immediately or successfully applied. Schoenwaelder Expires December 12, 2016 [Page 4] Internet-Draft Revised Model for YANG Datastores June 2016 o Several implementations have proprietary mechanisms that allow clients to store inactive data in the datastore; this inactive data is only exposed to clients that indicate that they support the concept of inactive data; clients not indicating support for inactive data receive the content of the datastore with the inactive data removed. Validation always happens on the datastore with inactive data removed. o Some operators have reported that it is essential for them to be able to retrieve the configuration that has actually been successfully applied, which may be a subset or a superset of the configuration. 4. Revised Model of Datastores Below is a new conceptual model of datastores extending the original model in order reflect the experience gained with the original model. Schoenwaelder Expires December 12, 2016 [Page 5] Internet-Draft Revised Model for YANG Datastores June 2016 +-------------+ +-----------+ | | | | | (ct, rw) |<---+ +--->| (ct, rw) | +-------------+ | | +-----------+ | | | | | +------------+ | +-------->| |<--------+ | (ct, rw) | +------------+ | (A) // e.g., removal of 'inactive' nodes v +------------+ | | // subject to validation | (ct, ro) | +------------+ | (B) // e.g., missing resources or delays v +------------+ | | | (ct, ro) | +------------+ | (C) // e.g., autodiscovery, control-plane | // protocols, control-plane datastores v +--------------------------------+ | | | (ct + cf, ro) | +--------------------------------+ ct = config true; cf = config false; rw = read-write; ro = read-only The main changes are: o The original configuration datastore has been split into the configuration datastore and the configuration datastore. The configuration datastore contains the configuration that is intended to be applied to the device. On a traditional NETCONF implementation, and are always the same. However, implementations that support inactive configuration usually expose to clients that understand inactive configuration and they expose to clients that do not understand inactive configuration. The introduction of an datastore makes this difference explicit. Schoenwaelder Expires December 12, 2016 [Page 6] Internet-Draft Revised Model for YANG Datastores June 2016 o A new configuration datastore has been introduced that reflects the configuration currently active on the device. This may be a subset or a superset of the configuration. Possible reasons for differences are situations where intended configuration can't be applied due to missing resources or where configuration changes take noticeable time to become applied. o The operational state is considered to reside in a conceptual datastore. This new read-only datastore consists of config true and config false nodes; in the original model the operational state only had config false nodes. The reason for incorporating config true nodes here is to be able to expose all operational settings without having to replicate definitions in the data models. o The model foresees control-plane datastores that are by definition not part of the persistent configuration of a device. In some contexts, these have been termed ephemeral datastores since the information is ephemeral, i.e., lost upon reboot. The control- plane datastores interact with the rest of the system like any other control-plane mechanisms (e.g., routing protocols, discovery protocols). Note that the ephemeral datastore discussed in I2RS documents maps to a control-plane datastore in the revised datastore model described here. 5. Discussion 5.1. Missing Resources Sometimes some parts of configuration refer to resources that are not present and hence parts of the configuration cannot be applied. A typical example is an interface configuration that refers to an interface that is not currently present. In such a situation, the interface configuration remains in but the interface configuration will not appear in . 5.2. System-controlled Resources Sometimes resources are controlled by the device and such system controlled resources appear in (and disappear from) the operational state dynamically. If a system controlled resource has matching configuration in when it appears, the system will try to apply the configuration, which causes the configuration to appear in eventually (if application of the configuration was successful). Schoenwaelder Expires December 12, 2016 [Page 7] Internet-Draft Revised Model for YANG Datastores June 2016 5.3. Auto-configured or Auto-negotiated Values Sometimes configuration leafs support special values that instruct the system to automatically configure a value. An example is an MTU that is configured to 'auto' to let the system determine a suitable MTU value. Another example is Ethernet auto-negotiation of link speed. In such a situation, the and configuration datastores report the value 'auto' while the corresponding leaf in the operational state datastore will report the actual MTU value or the auto-negotiated link speed. Since a config true leaf may be used both for configuration and for reporting operational state, the value set of a leaf allowed in a configuration datastore may be different from the value set of the corresponding leaf in the operational state datastore. 5.4. Operational State with Different Origins Sometimes a single list is used to report operational state values that originate from difference sources, i.e., configuration, control plane protocols, or internal processing. An example are IP addresses of an interface that can originate from configuration, from DHCP, or may be dynamically auto-configured. In this case, the operational state datastore will report all IP addresses that are assigned to an interface while the applied configuration datastore only lists the successfully configured addresses that have originated from the intended configuration datastore. 6. Implications 6.1. Implications on NETCONF o A mechanism is needed to announce support for and configuration datastores. o Support for and datastores should be a feature (optional to implement). o For systems supporting or configuration datastores, the operation may be used to retrieve data stored in these new datastores. o A new operation should be added to retrieve the operational state data store (e.g., ). (In principle could work but it would be a confusing name.) o The operation will be deprecated since it returns data from two datastores that may overlap in the revised datastore model. Schoenwaelder Expires December 12, 2016 [Page 8] Internet-Draft Revised Model for YANG Datastores June 2016 o Invoking on will return for backwards compatibility. [XXX: How to deal with for old and new clients with inactive nodes? XXX] 6.2. Implications on RESTCONF o The {+restconf}/data resource represents the combined configuration and state data resources that can be accessed by a client. This is effectively bundling together with , much like the operation of NETCONF. The RESTCONF design should change such that the {+restconf}/data resource does not return the content of multiple datastores. Instead, it should return the datastore by default. o The "content" query parameter can be used to select whether config, nonconfig or all data is returned. It defaults to all. The "content" query parameter should be changed to allow the selection of other datastores, e.g., . 6.3. Implications on YANG o Some clarifications may be needed if this revised model is adopted. YANG currently describes validation in terms of the configuration datastore while it really happens on the configuration datastore. 6.4. Implications on Data Models o Since the NETCONF operation returns the content of the configuration datastore and the operational state together in one tree, data models were often forced to branch at the top-level into a config true branch and a structurally similar config-false branch that replicated some of the config true nodes and added state nodes. With the revised datastore model, this is not needed anymore since the different datastores handle the different lifetimes of data objects and together with the deprecation of the operation it is not possible to write simpler models. o There may be some differences in the value set of some objects that are used for both configuration and state. At this point of time, these are considered to be rare cases that can be dealt with using text in description statements. Future versions of YANG may consider whether it is reasonable to allow value sets of schema nodes to be partitioned into config true and config false value sets. Schoenwaelder Expires December 12, 2016 [Page 9] Internet-Draft Revised Model for YANG Datastores June 2016 o It is desirable to be able to obtain information why a certain value exists in the operational state datastore. Metadata annotations [I-D.ietf-netmod-yang-metadata] should be defined that allow to report the origin of data in the operational state datastore. Note that the definition needs to be flexible and incrementally deployable since not all systems today maintain information about the origin with the operational state. 7. IANA Considerations None. 8. Security Considerations This document discusses a conceptual model of datastores for network management using NETCONF/RESTCONF and YANG. It has no security impact on the Internet. 9. Acknowledgments This document grew out of many discussions that took place since 2010. Several Internet-Drafts ([I-D.wilton-netmod-opstate-yang], [I-D.bjorklund-netmod-operational], [I-D.wilton-netmod-opstate-yang], [I-D.ietf-netmod-opstate-reqs], [I-D.kwatsen-netmod-opstate], [I-D.openconfig-netmod-opstate]) and [RFC6244] touched on some of the problems of the original datastore model. The following people were authors to these Internet-Drafts or otherwise actively involved in the discussions that led to this document: o Andy Biermann, YumaWorks, o Martin Bjorklund, Tail-f Systems, o Marcus Hines, Google, o Christian Hopps, Deutsche Telekom, o Acee Lindem, Cisco Systems, o Ladislav Lhotka, CZ.NIC, o Thomas Nadeau, Brocade Networks, o Anees Shaikh, Google, o Rob Shakir, Jive Communications, o Kent Watsen, Juniper Networks, Schoenwaelder Expires December 12, 2016 [Page 10] Internet-Draft Revised Model for YANG Datastores June 2016 o Robert Wilton, Cisco Systems, 10. References 10.1. Normative References [I-D.ietf-netconf-restconf] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF Protocol", draft-ietf-netconf-restconf-13 (work in progress), April 2016. [I-D.ietf-netmod-rfc6020bis] Bjorklund, M., "The YANG 1.1 Data Modeling Language", draft-ietf-netmod-rfc6020bis-13 (work in progress), June 2016. [I-D.ietf-netmod-yang-metadata] Lhotka, L., "Defining and Using Metadata with YANG", draft-ietf-netmod-yang-metadata-05 (work in progress), March 2016. [RFC6020] Bjorklund, M., Ed., "YANG - A Data Modeling Language for the Network Configuration Protocol (NETCONF)", RFC 6020, DOI 10.17487/RFC6020, October 2010, . [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., and A. Bierman, Ed., "Network Configuration Protocol (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, . 10.2. Informative References [I-D.bjorklund-netmod-operational] Bjorklund, M. and L. Lhotka, "Operational Data in NETCONF and YANG", draft-bjorklund-netmod-operational-00 (work in progress), October 2012. [I-D.ietf-netmod-opstate-reqs] Watsen, K. and T. Nadeau, "Terminology and Requirements for Enhanced Handling of Operational State", draft-ietf- netmod-opstate-reqs-04 (work in progress), January 2016. [I-D.kwatsen-netmod-opstate] Watsen, K., Bierman, A., Bjorklund, M., and J. Schoenwaelder, "Operational State Enhancements for YANG, NETCONF, and RESTCONF", draft-kwatsen-netmod-opstate-02 (work in progress), February 2016. Schoenwaelder Expires December 12, 2016 [Page 11] Internet-Draft Revised Model for YANG Datastores June 2016 [I-D.openconfig-netmod-opstate] Shakir, R., Shaikh, A., and M. Hines, "Consistent Modeling of Operational State Data in YANG", draft-openconfig- netmod-opstate-01 (work in progress), July 2015. [I-D.wilton-netmod-opstate-yang] Wilton, R., ""With-config-state" Capability for NETCONF/ RESTCONF", draft-wilton-netmod-opstate-yang-02 (work in progress), December 2015. [RFC6244] Shafer, P., "An Architecture for Network Management Using NETCONF and YANG", RFC 6244, DOI 10.17487/RFC6244, June 2011, . Author's Address Juergen Schoenwaelder (editor) Jacobs University Email: j.schoenwaelder@jacobs-university.de Schoenwaelder Expires December 12, 2016 [Page 12]