Thing-to-Thing Research Group K. Hartke Internet-Draft Ericsson Intended status: Experimental May 9, 2020 Expires: November 10, 2020 Publish/Subscribe over the Constrained Application Protocol (CoAP) using the Constrained RESTful Application Language (CoRAL) draft-hartke-t2trg-coral-pubsub-01 Abstract This document explores how the Constrained RESTful Application Language (CoRAL) might be used for enabling publish/subscribe-style communication over the Constrained Application Protocol (CoAP), which allows CoAP nodes with long breaks in connectivity and/or up-time to exchange data via a publish/subscribe broker. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on November 10, 2020. Copyright Notice Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of Hartke Expires November 10, 2020 [Page 1] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Preamble . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2.1. Brokers and Topics . . . . . . . . . . . . . . . . . . . 3 2.2. Publish and Subscribe . . . . . . . . . . . . . . . . . . 4 2.3. Notational Conventions . . . . . . . . . . . . . . . . . 5 3. Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.1. Topic Creation and Configuration . . . . . . . . . . . . 5 3.1.1. Configuration Properties . . . . . . . . . . . . . . 6 3.1.2. Status Properties . . . . . . . . . . . . . . . . . . 6 3.1.3. Topic Configuration Representation . . . . . . . . . 6 3.2. Topic Discovery . . . . . . . . . . . . . . . . . . . . . 6 3.2.1. Topic List Representation . . . . . . . . . . . . . . 6 3.2.2. Filter Query Representation . . . . . . . . . . . . . 7 3.3. Interactions . . . . . . . . . . . . . . . . . . . . . . 7 3.3.1. Getting All Topics . . . . . . . . . . . . . . . . . 7 3.3.2. Getting Topics by Properties . . . . . . . . . . . . 7 3.3.3. Creating a Topic . . . . . . . . . . . . . . . . . . 8 3.3.4. Reading the Configuration of a Topic . . . . . . . . 8 3.3.5. Updating the Configuration of a Topic . . . . . . . . 9 3.3.6. Deleting a Topic . . . . . . . . . . . . . . . . . . 9 4. Publish/Subscribe . . . . . . . . . . . . . . . . . . . . . . 10 4.1. Topic Lifecycle . . . . . . . . . . . . . . . . . . . . . 10 4.2. Rate Limiting . . . . . . . . . . . . . . . . . . . . . . 11 4.3. Interactions . . . . . . . . . . . . . . . . . . . . . . 11 4.3.1. Publishing to a Topic . . . . . . . . . . . . . . . . 11 4.3.2. Subscribing to a Topic . . . . . . . . . . . . . . . 12 4.3.3. Unsubscribing from a Topic . . . . . . . . . . . . . 13 4.3.4. Getting the Latest Published Data . . . . . . . . . . 13 5. Security Considerations . . . . . . . . . . . . . . . . . . . 14 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 14 7.1. Normative References . . . . . . . . . . . . . . . . . . 14 7.2. Informative References . . . . . . . . . . . . . . . . . 15 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 15 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 15 1. Preamble This document explores how CoAP Publish/Subscribe Broker [I-D.ietf-core-coap-pubsub] might look like if based on CoRAL [I-D.ietf-core-coral]. The exploration is done in the style of a self-contained specification rather than a description of changes. Hartke Expires November 10, 2020 [Page 2] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 2. Introduction Constrained RESTful Environments (CoRE) realize the Representational State Transfer (REST) architectural style [REST] in a suitable form for constrained nodes (e.g., 8-bit microcontrollers with limited RAM and ROM) and constrained networks [RFC7228]. CoRE technologies like the Constrained Application Protocol (CoAP) [RFC7252] are aimed at machine-to-machine (M2M) applications like smart energy and building automation. An important class of constrained nodes are devices that are intended to run for years on a small battery or by scavenging energy from their environment. These nodes have limited reachability, since they spend most of their time in a sleeping state with no network connectivity. Another important class of nodes are devices with limited reachability due to middle-boxes like Network Address Translators (NATs) and firewalls. For these nodes, the client/server-oriented architecture of REST can be challenging when interactions are not initiated by the devices themselves. A publish/subscribe-oriented architecture where nodes are separated by a broker and data is exchanged via topics might fit these nodes better. This document applies the idea of a "Publish/Subscribe Broker" to Constrained RESTful Environments. The broker enables store-and- forward data exchange between nodes, thereby facilitating the communication of nodes with limited reachability, providing simple many-to-many communication, and easing integration with other publish/subscribe systems. 2.1. Brokers and Topics In this specification, publishing and subscribing is facilitated by a CoAP server, called the "publish/subscribe broker" (see Figure 1). Publishers and subscribers are CoAP clients that interact with this broker using a RESTful API. Hartke Expires November 10, 2020 [Page 3] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 CoAP CoAP CoAP Clients Server Clients ___________ __________ observe ____________ | | publish | | .--------- | | | Publisher | ---------> | | |--------> | Subscriber | |___________| | | '--------> |____________| . | | . . | Broker | . ___________ | | observe ____________ | | publish | | .--------- | | | Publisher | ---------> | | |--------> | Subscriber | |___________| |__________| '--------> |____________| Figure 1: Publish/Subscribe over CoAP Data is forwarded from publishers to subscribers via "topics" at the broker. This way, both publishers and subscribers do not need to have any knowledge each other; they just have to share the topic they're publishing and subscribing to. Topics have to be created and configured before any data can be published. These are managed by the broker. Clients may propose new topics to be created; however, it is up to the broker to choose if and how a topic is created. The broker also decides the URI of each topic. The creation, configuration, and discovery of topics at a broker is specified in Section 3. 2.2. Publish and Subscribe A simple publish/subscribe mechanism can be implemented over CoAP by having publishers submit their data in PUT requests to a broker- managed resource and letting subscribers observe this resource [RFC7641]. However, the requirements may not always be simple: When observing a resource, notifications are sent on a best-effort basis. Subscribers are notified of as many state changes as possible; however, when a publisher is publishing faster than subscribers can be notified, subscribers might not see every publication. If this is not desired, an alternative mechanism needs to be be used. Such alternatives are supported, but their specification is out of this document's scope. The simple publish/subscribe mechanism is specified in Section 4. Hartke Expires November 10, 2020 [Page 4] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 2.3. Notational Conventions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. 3. Topics The configuration side of a "publish/subscribe broker" consists of a collection of topics. These topics as well as the collection itself are exposed by a CoAP server as resources (see Figure 2). ___ Topic / \ Collection \___/ \ \____________________ \___ \___ \___ Topics / \ / \ ... / \ \___/ \___/ \___/ Figure 2: Resources of a Publish-Subscribe Broker 3.1. Topic Creation and Configuration Each topic has a topic configuration. A CoAP client can create a new topic by submitting an initial configuration for the topic. It can also read and update the configuration of existing topics and delete them when they are no longer needed. The configuration of a topic itself consists of a set of properties. These fall into one of two categories: configuration properties and status properties. Configuration properties can be set by a client and describe the desired configuration of a topic. Status properties are read-only, managed by the server, and provide information about the actual status of a topic. When a client submits a configuration to create a new topic or update an existing topic, it can only submit configuration properties. When a server returns the configuration of a topic, it returns both the configuration properties and the status properties of the topic. Every property has a type and a value. The type takes the form of an IRI [RFC3987]. This IRI serves only as an identifier; it must not be dereferenced by clients. The value can be either a Boolean value, an Hartke Expires November 10, 2020 [Page 5] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 integer, a floating-point number, a date/time value, a byte string, a text string, or a resource reference in the form of a URI [RFC3986]. 3.1.1. Configuration Properties The following configuration properties are defined: TODO. 3.1.2. Status Properties The following status properties are defined: TODO. 3.1.3. Topic Configuration Representation A topic configuration is represented as a CoRAL document [I-D.ietf-core-coral] containing the configuration properties and status properties of the topic as top-level elements. Each property is represented as a link where the link relation type is the property type and the link target is the property value. 3.2. Topic Discovery Topics can be discovered by a client on the basis of configuration properties and status properties. For example, a client could fetch a list of all topics that have a property of type "foo" or that have a property of type "bar" with the value 42. Alternatively, topics can also be discovered simply by getting the full list of all topics. 3.2.1. Topic List Representation A list of topics is represented as a CoRAL document [I-D.ietf-core-coral] containing the topics in the list as top-level elements. Each topic is represented as a link where the link relation type is and the link target is the topic URI. This link can optionally contain (a subset of) the configuration properties and status properties of the topic as nested elements. Each property is represented in the same way as in Section 3.1.3. Hartke Expires November 10, 2020 [Page 6] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 3.2.2. Filter Query Representation TODO. 3.3. Interactions 3.3.1. Getting All Topics A client can list a collection of topics by making a GET request to the collection URI. On success, the server returns a 2.05 (Content) response with a representation of the list of all topics (see Section 3.2.1) in the collection. Example: => 0.01 GET Uri-Path: pubsub Uri-Path: topics <= 2.05 Content Content-Format: 65536 item item item 3.3.2. Getting Topics by Properties A client can filter a collection of topics by submitting the representation of a topic filter (see Section 3.2.2) in a FETCH request to the topic collection URI. On success, the server returns a 2.05 (Content) response with a representation of a list of topics in the collection (see Section 3.2.1) that match the filter. Example: => 0.05 FETCH Uri-Path: pubsub Uri-Path: topics Content-Format: TODO TODO <= 2.05 Content Hartke Expires November 10, 2020 [Page 7] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 Content-Format: 65536 item item item 3.3.3. Creating a Topic A client can add a new topic to a collection of topics by submitting a representation of the initial topic configuration (see Section 3.1.3) in a POST request to the topic collection URI. If client just wants all the default configuration properties, it can simply submit an empty CoRAL document. On success, the server returns a 2.01 (Created) response indicating the topic URI of the new topic. Example: => 0.02 POST Uri-Path: pubsub Uri-Path: topics Content-Format: 65536 foo "xyz" bar 42 <= 2.01 Created Location-Path: pubsub Location-Path: topics Location-Path: 1234 3.3.4. Reading the Configuration of a Topic A client can read the configuration of a topic by making a GET request to the topic URI. On success, the server returns a 2.05 (Content) response with a representation of the topic configuration (see Section 3.1.3). Example: => 0.01 GET Uri-Path: pubsub Uri-Path: topics Uri-Path: 1234 Hartke Expires November 10, 2020 [Page 8] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 <= 2.05 Content Content-Format: 65536 Max-Age: 300 foo "xyz" bar 42 3.3.5. Updating the Configuration of a Topic A client can update the configuration of a topic by submitting the representation of the updated topic configuration (see Section 3.1.3) in a PUT request to the topic URI. Any existing properties in the configuration are replaced by this update. On success, the server returns a 2.04 (Updated) response. Example: => 0.03 PUT Uri-Path: pubsub Uri-Path: topics Uri-Path: 1234 Content-Format: 65536 foo "abc" <= 2.04 Updated 3.3.6. Deleting a Topic A client can delete a topic by making a DELETE request on the topic URI. On success, the server returns a 2.02 (Deleted) response. Any subscribers to the topic are automatically unsubscribed. Example: => 0.04 DELETE Uri-Path: pubsub Uri-Path: topics Uri-Path: 1234 <= 2.02 Deleted Hartke Expires November 10, 2020 [Page 9] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 4. Publish/Subscribe Unless a topic is configured to use a different mechanism, publish/ subscribe is performed as follows: A publisher publishes to a topic by submitting the data in a PUT request to a broker-managed "topic data resource". This causes a change to the state of that resources. Any subscriber observing the resource [RFC7641] at that time receives a notification about the change to the resource state. ___ Topic / \ Collection \___/ \ \___________________________ \ \ \ \ ...... \ ...... \ ...... : \___ : : \___ : : \___ : Topic : / \ : : / \ : : / \ : Configuration : \___/ : : \___/ : : \___/ : : _|_ : : _|_ : ... : _|_ : Topic : / \ : : / \ : : / \ : Data : \___/ : : \___/ : : \___/ : :.......: :.......: :.......: Figure 3: Resources of a Publish-Subscribe Broker The topic data resource (which is different from the resource holding the topic configuration) does not exist until some initial data has been published to it. Before initial data has been published, the topic data resource yields a 4.04 (Not Found) response. If such a "half created" topic is undesired, the creator of the topic can simply immediately publish some initial placeholder data to make the topic "fully created". All URIs for configuration and data resources are broker-generated. There does not need to be any URI pattern dependence between the URI where the data exists and the URI of the topic configuration. Topic configuration and data resources might even be hosted on different servers. 4.1. Topic Lifecycle When a topic is newly created, it is first placed by the server into the HALF CREATED state (see Figure 4). In this state, a client can read and update the configuration of the topic and delete the topic. A publisher can publish to the topic data resource. However, a subscriber cannot yet observe the topic data resource. Hartke Expires November 10, 2020 [Page 10] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 HALF FULLY CREATED CREATED ___ ___ Publish/ -----------> | | ------------------> | | ------------. Create |___| Publish |___| <-----------' \ / Subscribe | ^ \ ___ / | ^ Read/ | | '--> | | <--' | | Read/ Update | | Delete |___| Delete | | Update '-' '-' DELETED Figure 4: Lifecycle of a Topic After a publisher publishes to the topic for the first time, the topic is placed into the FULLY CREATED state. In this state, a client can read and update the configuration of the topic and delete the topic; a publisher can publish to the topic data resource; and a subscriber can observe the topic data resource. When a client deletes a topic, the topic is placed into the DELETED state and shortly after removed from the server. In this state, all subscribers are removed from the list of observers of the topic data resource and no further interactions with the topic are possible. 4.2. Rate Limiting The server hosting a data resource may have to handle a potentially very large number of publishers and subscribers at the same time. This means the server can easily become overwhelmed if it receives too many publications in a short period of time. In this situation, if a client is sending publications too fast, the server can return a 4.29 (Too Many Requests) response [RFC8516]. As described in RFC 8516, the Max-Age option [RFC7252] in this response indicates the number of seconds after which the client may retry. When a client receives a 4.29 (Too Many Requests) response, it should not send any new publication requests to the same topic data resource before the time indicated by the Max-Age option has passed. 4.3. Interactions 4.3.1. Publishing to a Topic A client can publish data to a topic by submitting it in a PUT request to the topic data URI. The topic data URI is indicated by the status property of type in the Hartke Expires November 10, 2020 [Page 11] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 topic configuration. (Note: The topic data URI is not identical to the topic URI!) The data MUST be in the content format specified by the configuration property of type in the topic configuration. On success, the server returns a 2.04 (Updated) response. However, when data is published to the topic for the first time, the server may instead return a 2.01 (Created) response. If the request does not have an acceptable content format, the server returns a 4.15 (Unsupported Content Format) response. If the client is sending publications too fast, the server returns a 4.29 (Too Many Requests) response [RFC8516]. Example: => 0.03 PUT Uri-Path: pubsub Uri-Path: data Uri-Path: 6578616d706c65 Content-Format: 112 [...SenML data...] <= 2.04 Updated 4.3.2. Subscribing to a Topic A client can get the latest published data and subscribe to newly published data by observing the topic data URI with a GET request that includes the Observe option [RFC7641]. On success, the server returns 2.05 (Content) notifications with the data. Example: => 0.01 GET Uri-Path: pubsub Uri-Path: data Uri-Path: 6578616d706c65 Observe: 0 <= 2.05 Content Content-Format: 112 Hartke Expires November 10, 2020 [Page 12] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 Observe: 10001 Max-Age: 15 [...SenML data...] <= 2.05 Content Content-Format: 112 Observe: 10002 Max-Age: 15 [...SenML data...] <= 2.05 Content Content-Format: 112 Observe: 10003 Max-Age: 15 [...SenML data...] 4.3.3. Unsubscribing from a Topic A client can unsubscribe simply by cancelling the observation as described in Section 3.6 of RFC 7641. 4.3.4. Getting the Latest Published Data A client can just get the latest published data by making a GET request to the topic data URI. On success, the server returns 2.05 (Content) response with the data. Example: => 0.01 GET Uri-Path: pubsub Uri-Path: data Uri-Path: 6578616d706c65 <= 2.05 Content Content-Format: 112 Max-Age: 15 [...SenML data...] Hartke Expires November 10, 2020 [Page 13] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 5. Security Considerations TODO. 6. IANA Considerations TODO. 7. References 7.1. Normative References [I-D.ietf-core-coral] Hartke, K., "The Constrained RESTful Application Language (CoRAL)", draft-ietf-core-coral-03 (work in progress), March 2020. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform Resource Identifier (URI): Generic Syntax", STD 66, RFC 3986, DOI 10.17487/RFC3986, January 2005, . [RFC3987] Duerst, M. and M. Suignard, "Internationalized Resource Identifiers (IRIs)", RFC 3987, DOI 10.17487/RFC3987, January 2005, . [RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained Application Protocol (CoAP)", RFC 7252, DOI 10.17487/RFC7252, June 2014, . [RFC7641] Hartke, K., "Observing Resources in the Constrained Application Protocol (CoAP)", RFC 7641, DOI 10.17487/RFC7641, September 2015, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . Hartke Expires November 10, 2020 [Page 14] Internet-Draft Publish/Subscribe over CoAP using CoRAL May 2020 [RFC8516] Keranen, A., ""Too Many Requests" Response Code for the Constrained Application Protocol", RFC 8516, DOI 10.17487/RFC8516, January 2019, . 7.2. Informative References [I-D.ietf-core-coap-pubsub] Koster, M., Keranen, A., and J. Jimenez, "Publish- Subscribe Broker for the Constrained Application Protocol (CoAP)", draft-ietf-core-coap-pubsub-09 (work in progress), September 2019. [REST] Fielding, R., "Architectural Styles and the Design of Network-based Software Architectures", Ph.D. Dissertation, University of California, Irvine, 2000, . [RFC7228] Bormann, C., Ersue, M., and A. Keranen, "Terminology for Constrained-Node Networks", RFC 7228, DOI 10.17487/RFC7228, May 2014, . Acknowledgements This document is based on a proposal for pub/sub broker improvements presented at IETF 105 in Montreal by Michael Koster, Ari Keranen, and Klaus Hartke. It borrows text from [I-D.ietf-core-coap-pubsub] by Michael Koster, Ari Keranen, and Jaime Jimenez. Thanks to Christian Amsuess, Carsten Bormann, Rikard Hoeglund, Michael McCool, Francesca Palombini, Ivaylo Petrov, Jim Schaad, Peter van der Stok, and Marco Tiloca for helpful comments and discussions that have shaped the document. Author's Address Klaus Hartke Ericsson Torshamnsgatan 23 Stockholm SE-16483 Sweden Email: klaus.hartke@ericsson.com Hartke Expires November 10, 2020 [Page 15]