Network Working Group W. Ivancic
Internet-Draft W. M. Eddy
Intended status: Experimental MTI Systems
Expires: June 21, 2014 A. Hylton
D. Iannicca
NASA GRC
J. A. Ishac
NASA GRC
December 18, 2013

Store, Carry and Forward Problem Statement
draft-ivancic-scf-problem-statement-01

Abstract

This document provides a problem statement for non-realtime communication between systems that are generally disconnected, which require multiple network hops between source and destination, and which may never be fully connected end-to-end at any given time. This document describes a number of use cases that motivate having a standard method to communicate between such systems, as multi-organization and multi-vendor support and interoperability is highly desirable. These use cases include dismounted soldiers, sensorwebs, medical devices, animal tracking, low-Earth-orbiting satellites and data mule scenarios. To avoid confusion in terminology when trying to focus on the problem and requirements without bias towards particular technical solutions, at this time, we refer to the protocol instances that would support such communications as Store, Carry, and Forwarding (SCF) agents, and refer to their activity as SCF networking. The concepts involved in SCF networking are not entirely new and several facets of the problem have been solved in multiple incompatible ways in existing or historic systems. This document describes the core SCF problem and gives an assessment of the ability to use existing technologies as solutions.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on June 21, 2014.

Copyright Notice

Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

This document may not be modified, and derivative works of it may not be created, except to format it for publication as an RFC or to translate it into languages other than English.


Table of Contents

1. Terminology

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].

"What's in a Word. Words make a difference. They affect how we think about something. The terms chosen to describe a concept are a crucial part of any model. The right concepts with terms that give the wrong connotation can make a problem much more difficult. The right terms can make it much easier. Adopting the mindset of the terms may allow you to see things you might not otherwise see." - John Day [Patterns]

In developing this document, we have intentionally avoided some terminology used by other protocols - particularly other store-and-forward protocols - to avoid biases and confusion that may otherwise ensue.

2. Introduction and Background

Internet technology has become pervasive and is now present in many types of devices that end up being deployed in the field for use in scenarios where they do not have good (or any) actual Internet connectivity. The networking stacks are used to support data transfer during episodes of connectivity, and the applications and protocol configurations avoid reliance on many typical infrastructure services (e.g. DNS). For instance, these devices may be only intermittently connected to other devices, and are used to support data flows where the source and ultimate destination might never be fully connected to one another at any time. These applications operate highly asynchronously, with non-realtime constraints on their communication. Often, there are intermediate relaying nodes (or "agents") that must "carry" the data while waiting for connectivity to develop. The means for relaying data has been highly specialized in such systems (almost per-deployment), and varies widely, with little code-reuse or commonality in the supporting network design. This "problem statement" document describes several of these scenarios generically, motivates the development of a common solution, and describes shortcomings in existing technologies.

This problem statement is explicitly not trying to look at the situation where a smart phone or mobile computer is temporarily off or removed from the Internet, and then is reattached directly to the edge of a well-connected network. Such systems are well-suited to utilize standard Internet protocols and are able to support realtime communications when connected. The systems and applications that this document is concerned with are primarily operating with a much higher level of asynchrony between the data producers, individual relays, and eventual data consumers. We call these "Store, Carry, and Forward" (SCF) systems to distinguish them from typical Store-and-Forward (SF) systems, which generally operate over a better-connected infrastructure. This section clarifies the distinction between SCF and the better-understood SF concept, which is already implemented by a number of different networking technologies.

Because so many analogies exist between SF/SCF-based computer communications and offline non-computer-based systems, it is useful to understand (very) early communications. Relay systems have been present even in ancient civilizations, where rulers utilized intelligence gathering via courier services to convey and obtain information. Relays consisted of runners, messengers, and even pigeons conveying messages and documents. These types of relay agents had only intermittent connectivity with one another and needed to hold onto messages for possibly long amounts of time before delivering them. Later relay systems included the ancient Greeks using fires, mirrors, or colored flags for visually communicating over large distances, and the the 18th century French using a system of telescopes and semaphores. These involved relatively well-connected systems of relay infrastructure, compared to the earlier methods that involved physical carriage of the stored messages for some time in order to reach the next forwarding point. Telegraph and later systems had equally well-connected infrastructures.

In computer networking, numerous technologies that support SF message communications between systems have evolved, and some have incorporated pieces of what SCF systems require. We very roughly group these developments into "generations" in order to highlight a general progression of capabilities. This is not prescriptive, and though some detailed aspects of the classification may be debatable, the basic notions hold.

1st-generation Store and Forward systems consisted of Message switching, with buffering of messages at intermediate nodes in order to handle intermittent connectivity. There was little or no automation, intelligence, or capabilities for forming routing tables, security, network management, and handling anything but rather slow-scale dynamics. Examples include UUCP, FidoNet. "In FidoNet As all modem phone numbers are published in the nodelist, point-to-point transfers are always possible. But, as store-and-forward capabilities are specified in the basic standards, email tends to be routed through a world-wide hierarchic topology and enews via a world-wide ad hoc, but generally geographically hierarchic, acyclic graph."

2nd-generation SF consist of Internet email via SMTP + POP / IMAP + S/MIME and so on. Key features include separation of message transfer agents, user agents, and message submission/delivery agents. There are increased capabilities for security and management. There is some (weak) separation between message format and message transfer protocols. Email servers generally operate within a well-connected environment. There are major performance problems outside this well-connected environment, because there is little diversity in message transport between nodes, and little or no improvement in dealing with dynamics.

3rd-generation SF protocols are an advance over 2nd-generation concepts. These are used to implement messaging middleware and are applied as the basis of enterprise service bus systems. There is an increased separation between message format and message transfer protocols with increased message transform / mediation capability. There has been a proliferation of proprietary formats, APIs and systems, with great diversity in capabilities. Generally, there are still problems operating outside a well-connected environment, and the security mechanisms are not advanced beyond 2nd-generation ones. Examples include: JMS/OpenWire, AMQP, STOMP, XMPP, and MQTT.

4th-generation SF protocols are directed at systems with long delays and intermittent connectivity and are known as Delay Tolerant Networking (DTN) [RFC4838], [RFC5050]. There is very strong separation between format and transfer protocol as well as strong separation between format and addressing/routing. Architecturally, there are many alternatives available for transfer, addressing, and routing with heavy tailoring per each pocket of deployment. Security is possible for implementation in a limited subset of intended use cases. There are a number of experimental implementations including Interplanetary Overlay Network (ION), DTN2 and others including substantial profiles of features and capabilities. DTNRG.

5th-generation (conceptual) systems include significant amounts of longer-term storage that can be "carried" between episodes of connectivity as a main component (i.e. Store, Carry and Forward) There is an increased separation of message metadata (i.e. shipping labels) from message bodies (i.e. containers) beyond the 4th generation, enabling new approaches to security, forwarding, and possibilities for pull-based routing. Emphasis is on the "carry" function and need for strong automation in management of stored data, including support for implementing policy in QoS, security, and routing. 5th-generation systems that embody the SCF concept have not yet been widely deployed. The fielded applications that could benefit from such SCF capabilities are either using point solutions adapted from prior generations of SF technology, or are lacking the strength in automation, security, and other features that the SCF technology should provide. Later sections of this document describe a generic network architecture expected to be supported by SCF / 5th-generation systems, the envisioned scenarios and use cases for these systems, more detailed comparison/contrast with existing / prior-generation systems, and a summary of lessons learned from experience with earlier systems.

3. Generic Architecture

Figure 1 illustrates a generic SCF network architecture, with the SCF agents (labelled "SCF") frequently partitioned into time-varying disconnected subsets. Depending on specifics of an individual scenario, it may be likely that some SCF agents are permanently attached to a connected network in order to provide stable gateways to/from the other SCF agents. However, in general, the system should be considered to consist of a number of primarily disconnected SCF agents at any point in time. The importance of this consideration as it relates to design, implementation, and test of potential SCF protocols is emphasized later in this document, as its effects have plagued prior SF systems.



           _,.--SCF -.
      SCF '      /    `-SCF
                .'       /   DISCONNECTED
                /       /      NETWORK
               .'     SCF_
              SCF     /   `-..
                `.   .'       SCF
                  `. |
                  ,SCF.
                 /     \
                /       \
               /         \
              ; CONNECTED :
              |  NETWORK  |
              :           ;
               \         /
                \       /
                 \     /
                  ,SCF`.
       SCF----''''  |   `-.
       |            \      `.SCF
       /             |      .-'
      /              SCF_.-'
     SCF            .'
       `.         .'        DISCONNECTED
         `-.    .'            NETWORKS
            SCF
 

Store, Carry and Forward Network

Figure 1

When visualizing an SCF network, it may help to think more along the lines of a topologically dynamically-changing (mobile) relay system of agents that can periodically communicate with one another, and may be able to make at least rough predictions about their future contacts. The distribution of news and mail in the mid-1800s as described in Herman Melville's book, "Moby Dick," is a good analogy.

"For the long absent ship, the outward-bounder, perhaps, has letters on board; at any rate, she will be sure to let her have some papers of a date a year or two later than the last one on her blurred and thumb-worn files. And in return for that courtesy, the outward-bound ship would receive the latest whaling intelligence from the cruising-ground to which she may be destined, a thing of the utmost importance to her. And in degree, all this will hold true concerning whaling vessels crossing each other's track on the cruising-ground itself, even though they are equally long absent from home. For one of them may have received a transfer of letters from some third, and now far remote vessel; and some of those letters may be for the people of the ship she now meets.....

...Every whale-ship takes out a goodly number of letters for various ships, whose delivery to the persons to whom they may be addressed, depends upon the mere chance of encountering them in the four oceans. Thus, most letters never reach their mark; and many are only received after attaining an age of two or three years or more."

Another analogy that illustrates aggregation and deaggregation are the parcel post delivery companies. Here, individual packages (containers) are delivered from a source to destinations via numerous transport mechanisms, e.g. trucks, planes, trains and boats. Along the way, these packages are aggregated into larger and larger shipping containers as they move further from the source, and then and deaggregated into smaller and smaller containers as they move closer to the destination. Such aggregation and deaggregation enable scaling of the system. There is a strong parallel between this flow of packages and the data flows seen in some of the scenarios described later in this document.

4. Operational Considerations

Some of the key operational considerations for SCF are:

5. Use Cases and Deployment Scenarios

There are numerous deployment scenarios for SCF systems. The following section highlights a few more common scenarios.

5.1. Data Mule

The Data Mule scenario is a common generic scenario and shows up in many deployments. In the Data Mule scenario, SCF agents communicate with each other mainly via some type of circulating entity carrying data, called the "mule". This entity may be an unmanned (or manned) aircraft, a ship, a bus, or any type of vehicle that periodically moves over the same relative area. Connectivity is likely to consist of high periods of disruption followed by short periods of connectivity over relatively high-bandwidth, low-delay, and possibly symmetric, links.

In the Data Mule scenario, connectivity is generally of the episodic variety (opportunistic). There may be one or a larger number of mules; each of which runs its own SCF agent.

   .-------.
   | SCF   |                                .-------.
   |Agent 1|                                | SCF   |   .--o
   `-------'  /''\.                         |Agent 5|  /    `.
         `.  .'   '._    .-----.,--.        `-------' /      |
           \ |       `.._|Data |    ``.          \   /       `.
            /            |Mule |       `-.        \  '        |_
            |            `-----'          `.       ,/           |
           |            Movement            \.,.---             |
           '           <<=======                                |
          /                                                      |
         /                                                       |
         |                     Path Well     .-------.           |
         |                     Traveled      | SCF   |           |
        |                                    |Agent 3|           |
        /                              ...._ `-------'            \
       |                              |     `.   /                 `\
        \ _..                         |       `./                  .'
        `'   `---`...                 `.._       `-.              ,'
      .-------.  _,  ".                   ``.       \         _,,'
      | SCF   |,'      `.                    \       `--...,-'.
      |Agent 2|          `.                   `.              |
      `-------'            `. _                 \       .-------.
                             ' `'-...._          |      | SCF   |
                                       `"--------'      |Agent 4|
                                                        `-------'
		
		

Data Mule Scenario

Figure 2

Within this type of Data Mule scenario, the generic use case for SCF networking involves an application being able to push its data into containers on a SCF agent, who then interacts with the Data Mule SCF agents in order to deliver the containers to destination applications attaching to other SCF agents. In order to realize this use case, the SCF agents need to be identifiable to one another during periods of episodic connectivity, and the mule needs to somehow be able to express its expected future capability to relay containers towards the destination application.

The Data Mule is a common military scenario. It is often used to join partitioned connected networks such as groups of mobile ad-hoc networks (manets). In Figure 2, the SCF Agents could be concentrator points in a manet cluster that enables communication between disjoint manets on the battlefield, thus enabling communications between clusters on the far ends of the communication infrastructure, the edge networks.

5.2. Data Gathering

The generic Data Gathering scenario is also quite common and applicable to SCF networks. Specific use cases involving sensorwebs, medical monitoring and animal tracking would fit into this scenario. Figure 3 illustrates a sensorweb where some sensors wake up and forward data through other SCF agents until they reach an SCF connected to a more powerful radio system. A gathering agent may then come by from time to time (e.g. days, weeks, months) and collect the data.




                                        GATHERING AGENT

                                             //
                               /____        /X//  _
                               \          (\ //-'//
                                          / \)---'
                                         /X//




              ))'((                ))'((             ))'((
                |                    |                 |
               SCF                  SCF               SCF
           _,-' | `..              /   \           _,' | '-.
         ,'     |    `-.         ,'     `        ,'    |    `._
       SCF     SCF     SCF     SCF      SCF    SCF    SCF    SCF
      /   \                                          /   \
    ,'     `                                       ,'     `
  SCF      SCF                DATA SOURCES       SCF      SCF
	  
	

Data Gathering Scenario

Figure 3

Major challenges of the use cases in a Data Gathering scenario, which go beyond those of a Data Mule, are related to the increased level of complexity in the topology between SCF agents. There is potentially less predictability, potentially more heterogeneity (or hierarchy) in SCF agent capabilities, and potentially a higher risk of routing loops or wasted resources.

The use cases for Data Gathering do, however, involve data flows that are generally either all directed from sensors up through the Gathering Agents or down from the Gathering Agents, so these still represent a sort of core network that all containers eventually go through (similar to the Data Mules).

5.3. Traveling the Beaten Path

There are many instances where communications may occur between entities traveling a well-worn path. In this scenario, communication is more ad-hoc than the data-mule example. The probability of encountering other SCF agents is quite high. Such scenarios include: communication in mining operations, among hikers, among boats along well traveled waterways, within the fisheries industry (the Moby Dick example) and along trade routes.

Figure 4 illustrates Traveling the Beaten Path. Consider a nomadic trade route in a third-world country. Here, SCF-1 may travel from the one end of the path to the other in one direction, while SCF-5 moves in the other direction. SCF-1 and SCF-5 will encounter all other SCFs along the way. SCFs 2, 3 and 4 only move along portions of this trade route. Most likely, none of this information is known in advance, and the movements may or may not be predictably repeatable.


                             PATH
   ==-._                                _,,---.=-..__
  '`-`''`;--=...__           _....,,...'-''     `''`.\      ___
               `''':::.._,,,-'-''                     `---.:,-'''
                      '`'''

   --------- SCF-1 -------------------------------------------->

               <---- SCF-2 --->

                     <----- SCF-3 ------>

                  <--------------------------- SCF-4 --->

   <----------------------------------------------- SCF-5 -----

	

Traveling the Beaten Path

Figure 4

5.4. Rapid Disruption

Many wireless networks (particularly military ones), when connected, may be relatively well-connected to a large number of other nodes in near real-time. However, individual nodes or subsets of the network may be only episodically connected because of variable link conditions given terrain, foliage, weather, jamming, or desire to evade detection. Furthermore, even when basically connected, temporary radio signal power fades can cause rapid, short, periods of disconnection. All of these lower-layer hindrances may result in short periods of disruption witnessed by applications, as well as rapid changes in network topology and the set of reachable relays.

SCF provides some potential solutions for such networks. SCF routing may be capable of moving containers towards destinations via store-and-forward if the proper naming structures, addressing and routing algorithms can be developed.

5.5. Dismounted Soldier

On the battlefield, it often occurs that a group of soldiers is on a mission and arrives via a vehicle or group of vehicles, one of which may have very good connectivity to larger networks. Once dismounted, much of the communications may be via use of that vehicle communication system as a relay or anchor. Once the group moves a significant distance from this anchor and from one other, they may become disconnected for periods of time. At this point, SCF becomes necessary to improve communications and maintain situational awareness. SCF provides this communication in two ways. Near-synchronous communications might be maintained via multi-hops through other SCF agents, or in the worst case, asynchronous communications can be served sporadically when within radio contact of the anchor relay on the vehicle.

This scenario is also applicable to first responders during disaster situations where infrastructure may be severely damaged.

5.6. Low Earth Orbiting Sensor Satellite

The Low Earth Orbit (LEO) scenario described is for a sensor satellite communicating directly with ground terminals. One such network is described in reference [UK-DMC]. Note, in this scenario, no geostationary relay satellite is involved. Here, the contact times may be known in order to direct the satellite transmitter to turn on. Some type of automated hailing could also be used.

                              LEO
                             Sensor
                            Satellite
                      +-------+ _ +-------+
                      |       |:_:|       |
                      +-------+   +-------+
                            ,'      `.
          Space/Ground    ,'          `-.    Space/Ground
            Link 1     ,-'               `.    Link 2
                     ,'                    `._
                   ,'                         `.
                 ,'  Connect            Connect `._
              _,'      T1                 T2       `.
          .  '                                       `   /
  Ground  `.                                         ..,'  Ground
 Station  /\''    ___...------'''''''------....__      /\ Station
    1    /__\.,--'                               `'--./__\   2
      _,,-' ._                                     _,    `--.._
     '        `.  Ground/Ground    Ground/Ground ,'            '
                `-.  Link 1           Link 2  ,-'
                   `._                     ,-'
                      `.                _,'
                        ' .-----------.
                          |   Data    |
                          |Collection |
                          |  Center   |
                          `-----------'

            

Low Earth Orbit Sensor Satellite

Figure 5

Low Earth Orbit (LEO) is a low-propagation-delay environment of less than ten milliseconds delay to ground, with long periods of disconnection between passes over ground stations. Contact times consist of a few minutes per ground station for Earth satellites orbiting at a couple hundred kilometers altitude (~100 minutes per orbit). Thus, the more ground systems that are available, the greater the potential for contact. The ground stations are connected across the terrestrial Internet, or other backbones.

In this scenario, the SCF agent onboard the satellite does not need to perform forwarding of received containers. The satellite is a source for sensor data and may be a sink for command data.

The main reason to use SCF in this scenario is to provide a standardized relaying technique and to decouple the control loops between the space/ground link and the ground/ground link.

There are numerous companies and systems today that transfer extremely large sensor data sets from LEO to ground without a standardized SCF method. Those data sets are in the multi-gigabyte region and growing. However, they are using protocols and implementations that are not compatible with one another, and which require them to go through various levels of customization per-use.

6. Consideration of Existing Technologies

In this document, we characterized DTN-based systems as 4th-generation SF rather than SCF systems. Several aspects of DTN are highly desirable for SCF though, and DTN technology may represent a basis for developing SCF standards. DTN utilizes "bundle agents" that are similar to the "SCF agent". Several DTN routing protocols, that exist at varying levels of maturity, can work well for individual SCF scenarios that have been outlined. For instance, Contact Graph Routing is particularly useful in scenarios where future connectivity is predictable/known ahead of time.

The SCF container aggregation and deaggregation bears some surface similarity to bundle fragmentation operations in DTN, but there are major differences. SCF containers are intended to be aggregatable within the network, even if they are not portions of the same original container from the application. Additionally, some SCF applications (e.g. science data collection) may find (optional) partial reception of subsets of large containers that have been deaggregated into smaller containers, to still be useful, whereas DTN only delivers entire (reassembled) bundles. This does require the data formats used by such SCF applications to be self-synchronizing, so that they can be parsed, but this is an issue for the application.

SCF scenarios require some features that are not yet a part of the DTN specifications:

  • The ability to avoid DoS by propagating an application's permit/deny filters to SCF agents.
  • The ability to generate and prove ownership of globally-unique application identifiers.

DTN goes beyond SCF in one way, which is the targeting of operation over very high-latency data links. SCF does not explicitly attempt to operate over such links, though it may end up being possible. Since these links are mostly only applicable to deep-space scenarios with small numbers of nodes, SCF motivations do not include high-delay.

This document also identified JMS and MQ Telemetry Transport message-broker systems as 3rd-generation SF rather than SCF systems. JMS "messages" transferred between "brokers" and applications are similar to the containers transferred between SCF agents and applications. JMS offers both point-to-point (unicast) and publish-subscribe (multicast) models of communication. JMS uses named "queues" (in the point-to-point model) or "topics" (in the publish-subscribe model) in order to identify destinations. JMS brokers often implement a "durable" messaging service that allows messages (and queues) to persist even when the applications that created them (or need to receive them) are disconnected from the broker.

SCF scenarios require some features that are not yet really reflected within the JMS specifications:

  • Multi-hop relaying among brokers and secure propagation of information about the queues/topics present or acceptable is not standardized.
  • Communication of an application's desired permit/deny filters on queues that it owns is not standardized.

JMS is an API and not a protocol standard. This is the primary hurdle in using JMS to support SCF, as the wire-protocols and other mechanisms used in a particular JMS implementation are not necessarily compatible with others. SCF requires full vendor/platform interoperability in order to be cost-effective to pursue in preference to point-solutions. JMS is also significantly focused on transfer of Java objects rather than generic containers of bytes as SCF should be.

One of the biggest challenges to using existing systems (whether they be DTN implementations, JMS products, or some others) is that most have been designed to include a multitude of additional (optional) features and this results in, at best, limited compatibility between implementations. For instance, the DTN Bundle Protocol is an excellent platform for experimentation due to its flexibility and ease of defining new "blocks" to implement different functionalities. DTN has been used or demonstrated in a wide range of scenarios with differing needs, including simulated military exercises, connecting people in remote regions, moving data from LEO spacecraft, deep-space missions, mining operations, and others. However, individual implementations have grown up to support distinct subsets of the defined blocks, identifier schemes, and algorithms that suit the unique properties of their pet environments. Developing, and then maintaining, a baseline for compatibility has not been a primary concern. For an operational system, a baseline profile of the required functionality would need to exist, which could be present across the spectrum of vendors. For SCF, this type of profile is not present in the existing systems to a level that would enable the scenarios described in this document. Saving energy and running on very small devices (e.g. sensors and embedded medical devices) also motivates having such a profile that could aid in developing very small, yet fully compatible, implementations.

MQ Telemetry Transport is a lightweight protocol used for publish/subscribe messaging between devices. MQTT was designed for low-bandwidth, high latency networks while attempting to ensure reliability of delivery. A major design criteria was simplicity - must be simple to implement and must not add too many "bells and whistles" that would complicate implementation, while still providing a solid building block which can easily be integrated into other solutions. MQTT is designed to handle frequent short periods of network disruption using a technique called "Last Will and Testament". Although not part of the specification, MQTT has been modified to operate in multi-hop environments.

7. Characteristics of Information

Since information has to be transported and stored in an ACF use case, it is important to be aware of the key characteristics of the information being acted upon. All information has a source and one or more eventual destinations. All application information ready to be sent, has a size that may be very small or quite large (several bytes to multiple gigabytes). Size is important because storage is not unlimited in either the source application's system nor in the relays, and because transmission bandwidth and contact times limit the amount of data that can be sent during any given contact time.

Information may have security restrictions placed on it - sensitive or restricted (for your eyes only), and in some cases this may be handled at the application layer, as is done by securing email.

All information has a useful lifetime. It may be very short (seconds) or very long (days, weeks, years). Regardless, it is only the users of the information that know what the real useful lifetime is, and it is the application that would be required to set that lifetime. With the exception of specific cases, it is not at all clear that the application can generally make that decision.

When investigating the use of DTN bundle lifetimes in DTN deployments and implementations, we have found that the lifetimes have generally been set to match the duration of the experiment. There are instances where some finer granularity has been deployed such and in the Defense Advanced Research Project Agency's (DARPA) Wireless Network after Next (WNaN) where a small number of choices in lifetimes of minutes or hours are used depending on the nature of the data. The DTN bundle lifetime can be used for two purposes: expedited forwarding for end-applications and purging stale information from the relays. Thus, the real requirement we see for SCF should be the ability to expedite forwarding of priority containers and purge stale containers from the system, but not necessarily to specify time-sensitivity on a per-second basis. There may be other means to accommodate this requirement without having to burden the SCF agents with the management and synchronization of notions of "time" - particularly per container - which has been burdensome in DTN use.

Often data has a "freshness" characteristic. For a given application, data that is more recent (fresher) is often of greater value that data that is older (stale). In such cases, it may be more important to forward the most recent data, rather that the data that is near the end of its useful lifetime. One might even purge the system of stale data. One trival example that illustrates why data freshness is important would be reception of stock quotes. Obviously, one would not expect an SCF systems to be used for commodities trading due to disruption and ordering issues (assured timeliness). Rather, applications such as sensor data transmissions, software updates or distributed security-key databases are more amenable to SCF deployments.

8. Network Management

Network management is needed to keep the network running smoothly. It is required for system configuration and maintenance, and monitors the system to determine faults, performance, security issues and accounting. From the scenarios presented, it appears that network management is likely to be per-scenario, and may be effectively accomplished out-of-band. For example: in the Data Mule scenario, one may manage the data mule, but not the edge SCF systems. In the Data Gathering scenario, one is likely to preconfigure the remote sensor nodes and only manage the data-gathering SCF and perhaps the data concentrator SCFs, the ones with high-power radios.

This does not imply the network management could not or should not be performed in-band; only that it may not be required.

Since resources (e.g. bandwidth, transmit power, and storage) are a precious commodity in SCF networks, policy that manages those resources is expected to be a major component of system configuration. For example: a particular SCF agent may restrict particular information sources to limited storage space and limited storage time. Such policy may restrict all information to a limited storage time in order to purge stale containers. Also, particular sources may get preferential treatment per peering agreements.

9. Lessons Learned Summary

There are numerous lessons to be learned from previous deployments of manets and 4th-generation store and forward networks such as DTNs. Some of the more critical and important pieces of knowledge are listed below:

  • SCF systems are generally connected via radio networks. Some radio systems may take far less power to listen than to transmit, though this varies by individual link technology. Wasted transmission is wasted power on a wireless system and can quickly drain a battery. The problem is compounded for devices whose entire lifetime is determined by their battery (e.g. non-rechargeable sensor nodes). Thus, reducing wasted transmissions is high desirable.
    • The ability to reactively fragment large data sets en-route is highly desirable. This has been demonstrated in DTN experiments.
    • Routing loops in the SCF will not be caught by layers below. It is imperative that data dies naturally and quickly so as to not waste bandwidth or transmission power. Such loops have been encountered in early experiments with DTN overlays, and are correctable.
    • It is highly desirable for the sender to know early in a transmission whether or not the receiver will accept the data. This permits a savings in power and optimization of network capacity usage. For instance, in DTN experiments with large bundles, the entire large bundle may be sent, only to be discarded due to security, resource scarcity, or other issues.

  • Disconnected networks are difficult, if not impossible, to globally synchronize state across.
  • Managing fine-grained notions of time in a protocol is difficult and adds considerable complexity.
  • Having a relay protocol be time-dependent opens is up to security vulnerabilities.
  • Having a relay protocol be time-dependent complicates use of that protocol to synchronize the system - even for coarse synchronization.
  • It is highly desirable for a receiving agent to determine early within a transfer whether or not to accept the data. Large data sets utilize significant processing and storage resources for data that may end up being discarded due to security, resource constraints, or other policy issues.
  • It is highly desirable to keep forwarding tables small, and make forwarding decisions ahead of time for predicted contacts. Book-keeping type of processing while forwarding a large number of small containers can overload the processing system.
  • Testing should be thorough and include exercising both the storage and forwarding systems. Failure to do so will lead to erroneous results [I-D.ivancic-scf-testing-requirements]. Thus, any testing and validation should exercise both the storage and forwarding mechanisms of the implementation. To do otherwise may lead to misleading results.

10. Security Considerations

Applications need to authenticate to an SCF agent before they can send or receive containers.

Authentication of SCF agents to one another needs to be tackled before advertisements of forwarding capability can be acted upon.

Bandwidth, Storage, and Processing Power are precious resources in an SCF. In order to reduce DoS vulnerabilities and properly allocate resources, an SCF should be able to determine whether or not to act on a container based solely on the Shipping Label.

Applications should be able to limit DoS by expressing explicit desires to a serving SCF agent for/against certain traffic selectors. It may be beneficial for this information to propagate between SCF agents, though it should be recognized that any dynamics in these preferences causes a risk of data loss due to lack of synchronization of the filter rules.

While some aspects of Public-Key Infrastructure (PKI) may be applicable to SCF, PKI itself is not because, in general, PKI requires connectivity. Public-Keys with caching may be applicable; however, this would require at least some coarse network synchronization.

11. IANA Considerations

This document neither creates nor updates any registries or codepoints, so there are no IANA Considerations.

12. Acknowledgements

Much work builds on lessons learned from the work performed by the IRTF DTN Research Group.

Work on this document at NASA's Glenn Research Center was funded by the NASA Glenn Research Center Innovation Funds.

13. References

13.1. Normative References

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768, August 1980.

13.2. Informative References

, ", ", ", ", ", ", ", ", ", ", ", "
[RFC4838] Cerf, V., Burleigh, S., Hooke, A., Torgerson, L., Durst, R., Scott, K., Fall, K. and H. Weiss, "Delay-Tolerant Networking Architecture", RFC 4838, April 2007.
[RFC5050] Scott, K. and S. Burleigh, "Bundle Protocol Specification", RFC 5050, November 2007.
[I-D.ivancic-scf-testing-requirements] Ivancic, W., Eddy, W., Iannicca, D. and J. Ishac, "Store, Carry and Forward Testing Requirements", Internet-Draft draft-ivancic-scf-testing-requirements-00, July 2012.
[I-D.ivancic-scf-requirements-expectations] Ivancic, W., Eddy, W., Iannicca, D. and J. Ishac, "Store, Carry and Forward Requirements and Expectations", Internet-Draft draft-ivancic-scf-requirements-expectations-00, July 2012.
[Patterns] Day, J., "Patterns In Network Architectures - A Return to Fundamentals", Prentice Hall , 2008.
[UK-DMC] Wood, L., Ivancic, W., Eddy, W., Stewart, D., Jackson, C. and A. da Silva Curiel, "Use of the Delay-Tolerant Networking Bundle Protocol from Space", 59th International Astronautical Congress, Glasgow Paper IAC-08-B2.3.10 (also NASA-TM-2009-215582), September 2008.
[UPPC]Unix-to-Unix Copy", .
[FidoNet]FidoNet Home Page", .
[SMTP]Simple Mail Transfer Protocole", .
[POP]Post Office Protocol", .
[IMAP]Internet Message Access Protocol", .
[SMIME]Secure/Multipurpose Internet Mail Extensions", .
[JMS]JSM/OpenWire", .
[AMQP]Advanced Message Queuing Protocol", .
[STOMP]Streaming Text Oriented Messaging Protocol", .
[XMPP]Extensible Messaging and Presence Protocol", .
[DTNRG]DTN Research Group Wiki", .
[MQTT]Message Queuing Telemetry Transport", .

Authors' Addresses

William Ivancic NASA Glenn Research Center 21000 Brookpark Road Cleveland, Ohio 44135 United States Phone: +1-216-433-3494 EMail: william.d.ivancic@nasa.gov
Wesley M. Eddy MTI Systems EMail: wes@mti-systems.com
Alan G. Hylton NASA Glenn Research Center 21000 Brookpark Road Cleveland, Ohio 44135 United States Phone: +1-216-433-6045 EMail: alan.g.hylton@nasa.gov
Dennis C. Iannicca NASA Glenn Research Center 21000 Brookpark Road Cleveland, Ohio 44135 United States Phone: +1-216-433-6493 EMail: dennis.c.iannicca@nasa.gov
Joseph A. Ishac NASA Glenn Research Center 21000 Brookpark Road Cleveland, Ohio 44135 United States Phone: +1-216-433-6587 EMail: jishac@nasa.gov