Internet DRAFT - draft-irtf-icnrg-evaluation-methodology

draft-irtf-icnrg-evaluation-methodology



 



ICNRG                                                K. Pentikousis, Ed.
Internet-Draft                                                      EICT
Intended Status: Informational                                 B. Ohlman
Expires: October 14, 2016                                       Ericsson
                                                               E. Davies
                                                  Trinity College Dublin
                                                               S. Spirou
                                                        Intracom Telecom
                                                               G. Boggia
                                                     Politecnico di Bari
                                                          April 12, 2016


 Information-centric Networking: Evaluation and Security Considerations
               draft-irtf-icnrg-evaluation-methodology-05


Abstract

   This document presents a number of considerations regarding
   evaluating Information-centric Networking (ICN) and sheds some light
   on the impact of ICN on network security. It also surveys the
   evaluation tools currently available to researchers in the ICN area
   and provides suggestions regarding methodology and metrics. 


Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as
   Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html


 


Pentikousis, et al.     Expires October 14, 2016                [Page 1]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


Copyright and License Notice

   Copyright (c) 2015 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  2
   2.  Evaluation Considerations  . . . . . . . . . . . . . . . . . .  4
     2.1.  Topology Selection . . . . . . . . . . . . . . . . . . . .  4
     2.2.  Traffic Load . . . . . . . . . . . . . . . . . . . . . . .  5
     2.3.  Choosing Relevant Metrics  . . . . . . . . . . . . . . . . 10
       2.3.1.  Traffic Metrics  . . . . . . . . . . . . . . . . . . . 13
       2.3.2.  System Metrics . . . . . . . . . . . . . . . . . . . . 14
     2.4.  Resource Equivalence and Tradeoffs . . . . . . . . . . . . 15
   3.  ICN Security Aspects . . . . . . . . . . . . . . . . . . . . . 15
     3.1. Authentication  . . . . . . . . . . . . . . . . . . . . . . 16
     3.2. Authorization, Access Control and Logging . . . . . . . . . 18
     3.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . . 18
     3.4. Changes to the Network Security Threat Model  . . . . . . . 19
   4.  Evaluation Tools . . . . . . . . . . . . . . . . . . . . . . . 20
     4.1.  Open-source Implementations  . . . . . . . . . . . . . . . 20
     4.2.  Simulators and Emulators . . . . . . . . . . . . . . . . . 21
       4.2.1.  ndnSIM . . . . . . . . . . . . . . . . . . . . . . . . 22
       4.2.2.  ccnSIM . . . . . . . . . . . . . . . . . . . . . . . . 22
       4.2.3.  Icarus simulator . . . . . . . . . . . . . . . . . . . 22
     4.3.  Experimental Facilities  . . . . . . . . . . . . . . . . . 23
       4.3.1.  Open Network Lab [ONL] . . . . . . . . . . . . . . . . 23
       4.3.2.  POINT testbed  . . . . . . . . . . . . . . . . . . . . 24
       4.3.3.  CUTEi: Container-based ICN testbed . . . . . . . . . . 24
   5.  Security Considerations  . . . . . . . . . . . . . . . . . . . 25
   6.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 25
   7.  Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . 25
   8.  Informative References . . . . . . . . . . . . . . . . . . . . 25
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 34



1.  Introduction

 


Pentikousis, et al.     Expires October 14, 2016                [Page 2]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   Information-centric Networking (ICN) is a networking concept that
   arose from the desire to align the operation model of a network with
   the model of its typical use.  For TCP/IP networks, this implies
   changing the mechanisms of data access and transport from a host-to-
   host model to a user-to-information model.  The premise is that the
   effort invested in changing models will be offset, or even surpassed,
   by the potential of a "better" network.  However, such a claim can be
   validated only if it is quantified.

   Different ICN approaches are evaluated in the peer-reviewed
   literature using a mixture of theoretical analysis, simulation and
   emulation techniques, and empirical (testbed) measurements.  The
   specific methodology employed may depend on the experimentation goal,
   e.g., whether one wants to evaluate scalability, quantify resource
   utilization, or analyze economic incentives.  In addition, though, we
   observe that ease and convenience of setting up and running
   experiments can sometimes be a factor in published evaluations.  As
   discussed in [RFC7476], the development phase that ICN is going
   through and the plethora of approaches to tackle the hardest problems
   make this a very active and growing research area but, on the
   downside, it also makes it more difficult to compare different
   proposals on an equal footing.

   Performance evaluation using actual network deployments has the
   advantage of realistic workloads and reflects the environment where
   the service or protocol are to be deployed.  In the case of ICN,
   however, it is not currently clear what qualifies as a "realistic
   workload".  Trace-based analysis of ICN is in its infancy, and more
   work is needed towards defining characteristic workloads for ICN
   evaluation studies.  Accordingly, the experimental process and the
   evaluation methodology per se are actively being researched for
   different ICN architectures.   Numerous factors affect the
   experimental results, including the topology selected, the background
   traffic that an application is being subjected to, network conditions
   such as available link capacities, link delays, and loss-rate
   characteristics throughout the selected topology; failure and
   disruption patterns; node mobility and the diversity of devices used.

   The goal of this document is to summarize evaluation guidelines and
   tools alongside suggested data sets and high-level approaches.  We
   expect this to be of interest to the ICN community as a whole as it
   can assist researchers and practitioners alike to compare and
   contrast different ICN designs against each other, as well as against
   the state of the art in host-centric solutions, and identify the
   respective strengths and weaknesses.  We note that apart from the
   technical evaluation of the functionality of an ICN architecture, its
   future success will be largely driven by its deployability and
   economic viability.  Therefore, ICN evaluations should assess
 


Pentikousis, et al.     Expires October 14, 2016                [Page 3]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   incremental deployability in the existing network environment
   together with a view of how the technical functions will incentivize
   deployers to invest in the capabilities that allow the architecture
   to spread across the network.

   This document has been produced by the IRTF Information-centric
   Networking Research Group (ICNRG). The main objective of the ICNRG is
   to couple ongoing ICN research in the above areas with solutions that
   are relevant for evolving the Internet at large. The ICNRG produce
   documents that provides guidelines for experimental activities in the
   area of ICN so that different, alternative solutions can be compared
   consistently, and information sharing accomplished for experimental
   deployments. This document incorporates input from ICNRG participants
   and their corresponding text contributions; it has been reviewed by
   several ICNRG active participants (see section 7), and represents the
   consensus of the research group.  That said, note that this document
   does not constitute an IETF standard; see also [RFC5743].

   The remainder of this document is organized as follows. Section 2
   presents various techniques and considerations for evaluating
   different ICN architectures.  Section 3 discusses the impact of ICN
   on network security. Section 4 surveys the tools currently available
   to ICN researchers.



2.  Evaluation Considerations

   It is clear that the way we evaluate IP networks will not be directly
   applicable for evaluating ICN. In IP the focus is on the performance
   and characteristics of end-to-end connections between a source and a
   destination. In ICN the "source" responding to a request can be any
   ICN node in the network and may change from request to request. This
   makes it difficult to use concepts like delay and throughput in a
   traditional way. In addition evaluating resource usage in ICN is a
   more complicated task as memory used for caching affects delays and
   use of transmission resources, see the discussion on resource
   equivalents at the end of this section.

   There are two major types of evaluations of ICN that we see a need to
   make. One type is to compare ICN to traditional networking and one
   type is to compare different ICN implementations and approaches
   against each other.

   In this section we details some of the functional components needed 
   when evaluating different ICN implementations and approaches.

2.1.  Topology Selection
 


Pentikousis, et al.     Expires October 14, 2016                [Page 4]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   There's a wealth of earlier work on topology selection for simulation
   and performance evaluation of host-centric networks. While the
   classic dumbbell topology is regarded as inappropriate for ICN, most
   ICN studies so far have been based on that earlier work for host-
   centric networks [RFC7476]. However, there is no single topology that
   can be used to easily evaluate all aspects of ICN. Therefore, one
   should choose from a range of topologies depending on the focus of
   the evaluation.   

   For scalability and resilience studies, there is a wide range of
   synthetic topologies, such as the Barabasi-Albert model [BA] and the
   Watts-Strogatz small-world topology [WATTS]. These allow experiments
   to be performed whilst controlling various key parameters (e.g., node
   degree). These synthetic topologies are appropriate in the general
   case, as there are no practical assurances that a future information-
   centric network will share the same topology with today's networks.

   When studies look at cost (e.g., transit cost) or migration to ICN,
   realistic topologies should be used. These can be inferred from
   Internet traces, such as the CAIDA Macroscopic Internet Topology Data
   Kit (http://www.caida.org/data/active/internet-topology-data-kit) and
   Rocketfuel
   (http://www.cs.washington.edu/research/networking/rocketfuel). A
   problem is the large size of the topology (approx. 45K ASes, close to
   200K links), which may limit the scalability of the employed
   evaluation tool. Katsaros et al. [ICNScale] address this problem by
   using scaled down topologies created following the methodology
   described in [COMPLEX].

   Studies that focus on node or content mobility can benefit from
   topologies and their dynamic aspects as used in the DTN community. As
   mentioned in [RFC7476], DTN traces are available to be used in such
   ICN evaluations.

   As with host-centric topologies, defining just a node graph will not
   be enough for most ICN studies. The experimenter should also clearly
   define and list the respective matrices that correspond to the
   network, storage and computation capacities available at each node as
   well as the delay characteristics of each link [Montage]. Real values
   for such parameters can be taken from existing platforms such as
   iPlane (http://iplane.cs.washington.edu). Synthetic values could be
   produced with specific tools [DELAY].

2.2.  Traffic Load 

   In this subsection we provide a set of common guidelines, in the form
   of what we will refer to as a content catalog for different
   scenarios.   This catalog, which is based on previously published
 


Pentikousis, et al.     Expires October 14, 2016                [Page 5]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   work, could be used to evaluate different ICN proposals, for
   instance, on routing, congestion control, and performance, and can be
   considered as other kinds of ICN contributions emerge.  As we are
   still lacking ICN-specific traffic workloads we can currently only
   extrapolate from today's workloads.  A significant challenge then
   relates to the identification of the applications contributing to the
   observed traffic (e.g., Web or peer-to-peer), as well as to the exact
   amount of traffic they contribute to the overall traffic mixture. 
   Efforts in this direction can take heed from today's traffic mix
   comprising web, peer-to-peer file sharing, and User Generated Content
   (UGC) platforms (e.g., YouTube), as well as Video on Demand (VoD)
   services.  Publicly available traces for these include those
   available from web sites such as
   http://multiprobe.ewi.tudelft.nl/multiprobe.html,
   http://an.kaist.ac.kr/traces/IMC2007.html, and
   http://traces.cs.umass.edu/index.php/Network/Network.

   Taking a more systematic approach, and with the purpose of modeling
   the traffic load, we can resort to measurement studies that
   investigate the composition of Internet traffic, such as [1][2].  In
   [1] a large scale measurement study was performed, with the purpose
   of studying the traffic crossing inter-domain links.  The results
   indicate the dominance of Web traffic, amounting to 52% over all
   measured traffic. However, Deep Packet Inspection (DPI) techniques
   reveal that 25-40% of all HTTP traffic actually carries video
   traffic.  Results from DPI techniques also reveal the difficulty in
   correctly identifying the application type in the case of P2P
   traffic: mapping observed port numbers to well-known applications
   shows P2P traffic constituting only 0.85% of overall traffic, while
   DPI raises this percentage to 18.32% [1].  Relevant studies on a
   large ISP show the percentage of P2P traffic ranging from 17 to 19%
   of overall traffic [2].  Table I provides an overview of these
   figures. The "other" traffic type denotes traffic that cannot be
   classified in any of the first three application categories, and
   consists of unclassified traffic and traffic heavily fragmented into
   several applications (e.g., 0.17% DNS traffic). 












 


Pentikousis, et al.     Expires October 14, 2016                [Page 6]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   Table I. Traffic type ratios of total traffic [1, 2]

   Traffic Type | Ratio
   =====================
   Web          | 31-39%
   ---------------------
   P2P          | 17-19%
   ---------------------
   Video        | 13-21%
   ---------------------
   Other        | 29-31%
   =====================


   The content catalog for each type of traffic can be characterized by
   a specific set of parameters:

   a) The cardinality of the estimated content catalog.

   b) The size of the exchanged contents (either chunks or entire named
      information objects).

   c) The popularity of objects expressed in their request frequency.  


   In most application types, the popularity distribution follows some
   power law, indicating that a small number of information items
   trigger a large proportion of the entire set of requests.  The exact
   shape of the power law popularity distribution directly impacts the
   performance of the underlying protocols.  For instance, highly skewed
   popularity distributions (e.g., a Zipf-like distribution with a high
   slope value) favor the deployment of caching schemes, since caching a
   very small set of information items can dramatically increase the
   cache hit ratio.

   Several studies in the past few years have stated that Zipf's law is
   the discrete distribution that best represents the request frequency
   in a number of application scenarios, ranging from the Web to video
   on demand (VoD) services.  The key aspect of this distribution is
   that the frequency of a content request is inversely proportional to
   the rank of the content itself, i.e., the smaller the rank, the
   higher the request frequency.  If we denote with M the content
   catalog cardinality and with 1 <= i <= M the rank of the i-th most
   popular content, we can express the probability of requesting the
   content with rank "i" as:

   P(X=i) = ( 1/i^(alpha) ) / C, with C = SUM(1 / j^(alpha)), alpha > 0

 


Pentikousis, et al.     Expires October 14, 2016                [Page 7]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   where the sum is obtained considering all values of j, 1 <= j <= M.

   A recent analysis of HTTP traffic showed that content popularity is
   better reflected by a trimodal distribution model in which the head
   and tail of a Zipf distribution (with slope value 0.84) are replaced
   by two discrete Weibull distributions with shape parameter values 0.5
   and 0.24 respectively [IMB2014].

   A variation of the Zipf distribution, termed the Mandelbrot-Zipf
   distribution was suggested [P2PMod] to better model environments
   where nodes can locally store previously requested content.  For
   example, it was observed that peer-to-peer file sharing applications
   typically exhibited a 'fetch-at-most-once' style of behavior.  This
   is because peers tend to persistently store the files they download,
   a behavior that may also be prevalent in ICN.

   Popularity can also be characterized in terms of:

   a) The temporal dynamics of popularity, i.e., how requests are
      distributed in time.  The popularity distribution expresses the
      number of requests submitted for each information item
      participating into a certain workload.  However, they do not
      describe how these requests are distributed in time.  This aspect
      is of primary importance when considering the performance of
      caching schemes since the ordering of the requests obviously
      affects the contents of a cache.  For example, with a Least
      Frequently Used (LFU) cache replacement policy, if all requests
      for a certain item are submitted close in time, the item is
      unlikely to be evicted from the cache, even by a (globally) more
      popular item whose requests are more evenly distributed in time. 
      The temporal ordering of requests gains even more importance when
      considering workloads consisting of various applications, all
      competing for the same cache space.

   b) The spatial locality of popularity i.e., how requests are
      distributed throughout a network.  The importance of spatial
      locality relates to the ability to avoid redundant traffic in the
      network.  If requests are highly localized in some area of the
      entire network, then similar requests can be more efficiently
      served with mechanisms such as caching and/or multicast i.e., the
      concentration of similar requests in a limited area of the network
      allows increasing the perceived cache hit ratios at caches in the
      area and/or the traffic savings from the use of multicast. Table
      II provides an overview of distributions that can be used to model
      each of the identified traffic types i.e., Web, Video (based on
      YouTube measurements) and P2P (based on BitTorrent measurements). 
      These distributions are the outcome of a series of modeling
      efforts based on measurements of real traffic workloads
 


Pentikousis, et al.     Expires October 14, 2016                [Page 8]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


      [3][4][5][6][7][8][9][10][11][12][13].  A tool for the creation of
      synthetic workloads following these models, and also allowing the
      generation of different traffic mixes is described in [14].


   Table II. Overview of traffic types models

       |  Object Size   |  Temporal Locality   | Popularity Distribution
   =====================================================================
   Web | Concatenation  | Ordering via LRU     | Zipf: p(i)=K/i^a
       | of Lognormal   | stack model [5]      | i: popularity rank
       | (body) and     |                      | N: total items
       | Pareto (tail)  | Exact timing via     | K: 1/Sum(1/i^a)
       | [7,8]          | exponential          | a: distribution slope
       |                | distribution [6]     | values 0.64-0.84 [3][4]
   ---------------------------------------------------------------------
   VoD | Duration/size: | No analytical models | Weibull: k=0.513,
       | Concatenated   |                      | lambda=6010
       | normal, most   | Random distribution  |
       | videos         | across total         | Gamma: k=0.372,
       | ~330 kb/s [13] | duration             | theta=23910 [12]
   ---------------------------------------------------------------------
   P2P | Wide variation | Mean arrival rate of | Mandelbrot-Zipf [9]:
       | on torrent     | 0.9454 torrents/hour | p(i)=K/((i+q)/a)
       | sizes [9].     | Peers in a swarm     | q: plateau factor,
       | No analytical  | arrive as            | 5 to 100.
       | models exist:  | l(t)= l0*e^(-t/tau)  | Flatter head than in
       | Sample a real  | l0: initial arrival  | Zipf-like distribution
       | BitTorrent [11]| rate (87.74 average) | (where q=0)
       | distribution   | tau: object          |
       | or use fixed   | popularity           |
       | value          | (1.16 average)* [10] |
   =====================================================================

   * Random ordering of swarm births (first request).  For each swarm
     calculate a different tau.  Based on average tau and object
     popularity.  Exponential decay rule for subsequent requests.


   Table III summarizes the content catalog.  With this shared point of
   reference, the use of the same set of parameters (depending on the
   scenario of interest) among researchers will be eased, and different
   proposals could be compared on a common base.





 


Pentikousis, et al.     Expires October 14, 2016                [Page 9]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   Table III. Content catalog

   Traffic | Catalog |  Mean Object Size  |  Popularity Distribution
   Load    | Size    |  [L4][L5][L7][L8]  |  [L3][L5][L6][L11][L12]
           | [L1][L2]|  [L9][L10]         |
           | [L3][L5]|                    |
   ==================================================================
   Web     |  10^12  | Chunk: 1-10 kB     | Zipf with
           |         |                    | 0.64 <= alpha <= 0.83 
   ------------------------------------------------------------------
   File    | 5x10^6  | Chunk: 250-4096 kB | Zipf with
   sharing |         | Object: ~800 MB    | 0.75 <= alpha<= 0.82
   ------------------------------------------------------------------
   UGC     |  10^8   | Object: ~10 MB     | Zipf, alpha >= 2
   ------------------------------------------------------------------
   VoD     |  10^4   | Object: ~100 MB    | Zipf, 0.65 <= alpha <= 1
   (+HLS)  |         | 	~1 kB (*)   | 
   (+DASH) |         | 	~5.6 kB (*) | 
   ==================================================================

   UGC = User Generated Content VoD = Video on Demand 

   (*) Using adaptive video streaming (e.g., HLS and DASH), with an
   optimal segment length (2 s for DASh and 10 s for HLS) and a bitrate
   of 4500 kbps [W16][Led12]



2.3.  Choosing Relevant Metrics

   Quantification of network performance requires a set of standard
   metrics.  These metrics should be broad enough so they can be applied
   equally to host-centric and information-centric (or other) networks. 
   This will allow reasoning about a certain ICN approach in relation to
   an earlier version of the same approach, to another ICN approach or
   to the incumbent host-centric approach.  It will therefore be less
   difficult to gauge optimization and research direction.  On the other
   hand, the metrics should be targeted to network performance only and
   should avoid unnecessary expansion into the physical and application
   layers.  Similarly, at this point, it is more important to capture as
   metrics only the main figures of merit and to leave more esoteric and
   less frequent cases for the future.

   To arrive at a set of relevant metrics, it would be beneficial to
   look at the metrics used in existing ICN approaches, such as CCN
   [CCN] [VoCCN] [NDNP], NetInf [4WARD6.1] [4WARD6.3] [SAIL-B2] [SAIL-
   B3], PURSUIT [PRST4.5], COMET [CMT-D5.2] [CMT-D6.2], Connect [SHARE]
   [RealCCN], and CONVERGENCE [ICN-Web] [ICN-Scal] [ICN-Tran].  The
 


Pentikousis, et al.     Expires October 14, 2016               [Page 10]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   metrics used in these approaches fall into two categories: metrics
   for the approach as a whole, and metrics for individual components
   (name resolution, routing, and so on).  Metrics for the entire
   approach are further subdivided into traffic and system metrics.  It
   is important to note that the various approaches do not name or
   define metrics consistently.  This is a major problem when trying to
   find metrics that allow comparison between approaches.  For the
   purposes of exposition, we have tried to smooth over differences by
   classifying similarly defined metrics under the same name.  Also, due
   to space constraints, we have chosen to report here only the most
   common metrics between approaches.  For more details the reader
   should consult the references for each approach.

   Traffic metrics in existing ICN approaches are summarized in Table
   IV.  These are metrics for evaluating an approach mainly from the
   perspective of the end user, i.e., the consumer, provider, or owner
   of the content or service.  Depending on the level where these
   metrics are measured, we have made the distinction into user,
   application and network-level traffic metrics.  So for example,
   network-level metrics are mostly focused on packet characteristics,
   whereas user-level metrics can cover elements of human perception. 
   The approaches do not make this distinction explicitly, but we can
   see from the table that CCN and NetInf have used metrics from all
   levels, PURSUIT and COMET have focused on lower-level metrics, and
   Connect and CONVERGENCE opted for higher-level metrics.  Throughput
   and download time seem to be the most popular metrics altogether.

   Table IV. Traffic metrics used in ICN evaluations

                   User   |    Application    |        Network
               ======================================================
                 Download | Goodput | Startup | Throughput |  Packet
                   time   |         | latency |            |  delay
   ==================================================================
   CCN         |    x     |    x    |         |      x     |    x 
   ------------------------------------------------------------------
   NetInf      |    x     |         |    x    |      x     |    x
   ------------------------------------------------------------------
   PURSUIT     |          |         |    x    |      x     |    x
   ------------------------------------------------------------------
   COMET       |          |         |    x    |      x     |     
   ------------------------------------------------------------------
   Connect     |    x     |         |         |            |     
   ------------------------------------------------------------------
   CONVERGENCE |    x     |    x    |         |            |    
   ==================================================================


 


Pentikousis, et al.     Expires October 14, 2016               [Page 11]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   While traffic metrics are more important for the end user, the owner
   or operator of the networking infrastructure is normally more
   interested in system metrics, which can reveal the efficiency of an
   approach. The most common system metrics used are: protocol overhead,
   total traffic, transit traffic, cost savings, router cost, and router
   energy consumption.

   Besides the traffic and systems metrics that aim to evaluate an
   approach as a whole, all surveyed approaches also evaluate the
   performance of individual components.  Name resolution, request/data
   routing, and data caching are the most typical components, as
   summarized in Table V.  FIB size and path length, i.e., the routing
   component metrics, are almost ubiquitous among approaches, perhaps
   due to the networking background of the involved researchers.  That
   might be also the reason for the sometimes decreased focus on traffic
   and system metrics, in favor of component metrics.  It can certainly
   be argued that traffic and system metrics are affected by component
   metrics, however no approach has made the relationship clear.  With
   this in mind and taking into account that traffic and system metrics
   are readily useful to end users and network operators, we will
   restrict ourselves to those in the following sections.

   Table V. Component metrics in existing ICN approaches

                      Resolution      |    Routing    |    Cache
               ======================================================
                 Resolution | Request | FIB  |  Path  | Size |  Hit
                    time    |  rate   | size | length |      | ratio
   ==================================================================
   CCN         |     x      |         |  x   |   x    |   x  |   x 
   ------------------------------------------------------------------
   NetInf      |     x      |    x    |      |   x    |      |   x 
   ------------------------------------------------------------------
   PURSUIT     |            |         |  x   |   x    |      |     
   ------------------------------------------------------------------
   COMET       |     x      |    x    |  x   |   x    |      |   x 
   ------------------------------------------------------------------
   CONVERGENCE |            |    x    |  x   |        |   x  |     
   ==================================================================


   Before proceeding, we should note that we would like our metrics to
   be applicable to host-centric networks as well.  Standard metrics
   already exist for IP networks and it would certainly be beneficial to
   take them into account.  It is encouraging that many of the metrics
   used by existing ICN approaches can also be used on IP networks and
   that all of the approaches have tried on occasion to draw the
   parallels.
 


Pentikousis, et al.     Expires October 14, 2016               [Page 12]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


2.3.1.  Traffic Metrics

   The IETF has been working for more than a decade on devising metrics
   and methods for measuring the performance of IP networks. The work
   has been carried out largely within the IP performance metrics (IPPM)
   working group, guided by a relevant framework [RFC2330].  IPPM
   metrics include delay, delay variation, loss, reordering, and
   duplication.  While the IPPM work is certainly based on packet-
   switched IP networks, it is conceivable that it can be modified and
   extended to cover ICN networks as well.  However, more study is
   necessary to turn this claim into a certainty.  Many experts have
   toiled for a long time on devising and refining the IPPM metrics and
   methods, so it would be an advantage to use them for measuring ICN
   performance.  In addition, said metrics and methods work already for
   host-centric networks, so comparison with information-centric
   networks would entail only the ICN extension of the IPPM framework. 
   Finally, an important benefit of measuring the transport performance
   of a network at its output, using QoS metrics such as IPPM, is that
   it can be done mostly without any dependence to applications.

   Another option for measuring transport performance would be to use
   Quality of Service (QoS) metrics, not at the output of the network
   like with IPPM, but at the input to the application.  For live video
   streaming application the relevant metrics would be startup latency,
   playout lag and playout continuity.  The benefit of this approach is
   that it abstracts away all details of the underlying transport
   network, so it can be readily applied to compare between networks of
   different concepts (host-centric, information-centric, or other).  As
   implied earlier, the drawback of the approach is its dependence on
   the application, so it is likely that different types of applications
   will require different metrics.  It might be possible to identify
   standard metrics for each type of application, but the situation is
   not as clear as with IPPM metrics and further investigation is
   necessary.

   At a higher level of abstraction, we could measure the network's
   transport performance at the application output.  This entails
   measuring the quality of the transported and reconstructed
   information as perceived by the user during consumption.  In such an
   instance we would use Quality of Experience (QoE) metrics, which are
   by definition dependent on the application. For example, the
   standardized methods for obtaining a Mean Opinion Score (MOS) for
   VoIP (e.g., ITU-T P.800) is quite different from those for IPTV
   (e.g., PEVQ).  These methods are notoriously hard to implement, as
   they involve real users in a controlled environment.  Such
   constraints can be relaxed or dropped by using methods that model
   human perception under certain environments, but these methods are
   typically intrusive.  The most important drawback of measuring
 


Pentikousis, et al.     Expires October 14, 2016               [Page 13]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   network performance at the output of the application is that only one
   part of each measurement is related to network performance.  The rest
   is related to application performance, e.g., video coding, or even
   device capabilities, both of which are irrelevant to our purposes
   here and are generally hard to separate.  We therefore see the use of
   QoE metrics in measuring ICN performance as a poor choice at this
   stage.


2.3.2.  System Metrics

   Overall system metrics that need to be considered include
   reliability, scalability, energy efficiency, and delay/disconnection
   tolerance.  In deployments where ICN is addressing specific
   scenarios, relevant system metrics could be derived from current
   experience.  For example, in IoT scenarios, which were discussed
   earlier in [RFC7476], it is reasonable to consider the current
   generation of sensor nodes, sources of information, and even
   measurement gateways (e.g., for smart metering at homes) or
   smartphones.  In this case, ICN operation ought to be evaluated with
   respect not only to overall scalability and network efficiency, but
   also the impact on the nodes themselves.  Karnouskos et al.
   [SensReqs] provide a comprehensive set of sensor and IoT-related
   requirements, for example, which include aspects such as resource
   utilization, service life-cycle management and device management.

   Additionally, various specific metrics are also critical in
   constrained environments, such as processing requirements, signaling
   overhead, and memory allocation for caching procedures in addition to
   power consumption and battery lifetime.  For gateways, which
   typically act as a point of service to a large number of nodes and
   have to satisfy the information requests from remote entities we need
   to consider scalability-related metrics, such as frequency and
   processing of successfully satisfied information requests.

   Finally, given the in-network caching functionality of ICNs,
   efficiency and performance metrics of in-network caching have to be
   defined.  Such metrics will need to guide researchers and operators
   regarding the performance of in-network caching algorithms.  A first
   step on this direction has been made in [L9].  The paper proposes a
   formula that approximates the proportion of time that a content stays
   in a network cache.  The model takes as input the rate of requests
   for a given content (the Content of Interest) and the rate of
   requests for all other contents that go through the given network
   element (router) and move the CoI down in the (LRU) cache.  The
   formula takes also into account the size of the cache of this router.

   The output of the model essentially reflects the probability that the
 


Pentikousis, et al.     Expires October 14, 2016               [Page 14]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   CoI will be found in a given cache.  An initial study [L9] is applied
   to the CCN/NDN framework, where contents get cached at every node
   they traverse.  The formula according to which the probability or
   proportion is calculated is given by:

   pi = [mu/(mu+lambda)]^N,

   where lambda is the request rate for CoI, mu is the request rate for
   contents that move CoI down the cache and N is the size of the cache
   (in slots).

   The formula can be used to assess the caching performance of the
   system and can also potentially be used to identify the gain of the
   system due to caching. This can then be used to compare against gains
   by other factors, e.g., addition of extra bandwidth in the network.


2.4.  Resource Equivalence and Tradeoffs

   As we have seen above, every ICN network is built from a set of
   resources, which include link capacities, different types of memory
   structures and repositories used for storing named data objects and
   chunks temporarily (i.e., caching) or persistently, as well as name
   resolution and other lookup services.  Complexity and processing
   needs in terms of forwarding decisions, management (e.g., need for
   manual configuration, explicit garbage collection, and so on), and
   routing (i.e., amount of state needed, need for manual configuration
   of routing tables, support for mobility, etc.) set the stage for a
   range of engineering tradeoffs.

   In order to be able to compare different ICN approaches it would be
   beneficial to be able to define equivalence in terms of different
   resources which today are considered incomparable.  For example,
   would provisioning an additional 5 Mb/s link capacity lead to better
   performance than adding 100 GB of in-network storage?  Within this
   context one would consider resource equivalence (and the associated
   tradeoffs) for example for cache hit ratios per GB of cache,
   forwarding decision times, CPU cycles per forwarding decision, and so
   on.


3.  ICN Security Aspects

   The introduction of an information-centric networking architecture
   and the corresponding communication paradigm results in changes to
   many aspects of network security.  These will affect all scenarios
   described in [RFC7476].  Additional evaluation will be required to
   ensure relevant security requirements are appropriately met by the
 


Pentikousis, et al.     Expires October 14, 2016               [Page 15]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   implementation of the chosen architecture in the various scenarios.

   The ICN security aspects described in this document is complementing
   the ICN security challenges outlined in the "ICN Research Challenges"
   document [ICNCHA]. [***EDITORS NOTE: The referenced document is
   expected to have become an RFC by the time this document is
   published, this reference should thus be updated before publication
   of this document.***]

   The ICN architectures currently proposed have concentrated on
   authentication of delivered content to ensure the integrity of the
   content.  However the approaches are primarily applicable to freely
   accessible content that does not require access authorization,
   although they will generally support delivery of encrypted content.

   The introduction of widespread caching mechanisms may also provide
   additional attack surfaces.  The caching architecture to be used also
   needs to be evaluated to ensure that it meets the requirements of the
   usage scenarios.

   In practice, the work on security in the various ICN research
   projects has been heavily concentrated on authentication of content. 
   Work on authorization, access control, privacy and security threats
   due to the expanded role of in-network caches has been quite limited.
   For Example, a roadmap for improving the security model in NetInf can
   be found in [NETINFSC]. As secure communications on the Internet are
   becoming the norm,  major gaps in ICN security aspects are bound to
   undermine the adoption of ICN.  

   In the rest of this section we briefly consider the issues and
   provide pointers to the work that has been done on the security
   aspects of the architectures proposed.


3.1. Authentication

   For fully secure content distribution, content access requires that
   the receiver needs to be able to reliably assess:

      validity:   is it a complete, uncorrupted copy of what was
                  originally published;

      provenance: can the receiver identify the publisher, and, if so,
                  whether it and the source of any cached version of the
                  document can be adequately trusted; and

      relevance:  is the content an answer to the question that the
                  receiver asked.
 


Pentikousis, et al.     Expires October 14, 2016               [Page 16]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   All ICN architectures considered in this document primarily target
   the validity requirement using strong cryptographic means to tie the
   content request name to the content.  Provenance and relevance are
   directly targeted to varying extents:  There is a tussle or trade-off
   between simplicity and efficiency of access and level of assurance of
   all these traits.   For example, maintaining provenance information
   can become extremely costly, particularly when considering (historic)
   relationships between multiple objects.  Architectural decisions have
   therefore been taken in each case as to whether the assessment is
   carried out by the information-centric network or left to the
   application.

   An additional consideration for authentication is whether a name
   should be irrevocably and immutably tied to a static piece of
   preexisting content or whether the name can be used to refer to
   dynamically or subsequently generated content.  Schemes that only
   target immutable content can be less resource hungry as they can use
   digest functions rather than public key cryptography for generating
   and checking signatures.  However, this can increase the load on
   applications because they are required to manage many names, rather
   than using a single name for an item of evolving content that changes
   over time (e.g., a piece of data containing an age reference).

   DONA (Data Oriented Network Architecture) [DONA] and CCN [CCN]
   [SECCONT] integrate most of the data needed to verify provenance into
   all content retrievals but need to be able to retrieve additional
   information (typically a security certificate) in order to complete
   the provenance authentication.  Whether the application has any
   control of this extra retrieval will depend on the implementation. 
   CCN is explicitly designed to handle dynamic content allowing names
   to be pre-allocated and attached to subsequently generated content. 
   DONA offers variants for dynamic and immutable content.

   PURSUIT [PSTSEC] appears to allow implementers to choose the
   authentication mechanism so that it can, in theory, emulate the
   authentication strategy of any of the other architectures.  It is not
   clear whether different choices would lead to lack of
   interoperability.

   NetInf uses the Named Information (ni) URI scheme [RFC6920] to
   identify content.  This allows NetInf to assure validity without any
   additional information but gives no assurance on provenance or
   relevance.  A "search" request allows an application to identify
   relevant content and applications may choose to structure content to
   allow provenance assurance but this will typically require additional
   network access.  NetInf validity authentication is consequently
   efficient in a network environment with intermittent connectivity as
   it does not force additional network accesses and allows the
 


Pentikousis, et al.     Expires October 14, 2016               [Page 17]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   application to decide on provenance validation if required.  For
   dynamic content NetInf can use e.g. signed manifests. For more
   details on NetInf security see [SCNETINF]. 

3.2. Authorization, Access Control and Logging

   A potentially major concern for all ICN architectures considered here
   is that they do not provide any inbuilt support for an authorization
   framework or for logging.  Once content has been published and cached
   in servers, routers or end points not controlled by the publisher,
   the publisher has no way to enforce access control, determine which
   users have accessed the content or revoke its publication.  In fact,
   in some cases, it is even difficult for the publishers themselves to
   perform access control, where requests do not necessarily contain
   host/user identifier information.

   Access could be limited by encrypting the content but the necessity
   of distributing keys out-of-band appears to negate the advantages of
   in-network caching.  This also creates significant challenges when
   attempting to manage and restrict key access. An authorization
   delegation scheme has been proposed [ACDICN]. This scheme allows
   semi-trusted entities (such as caches, CDN nodes) to delegate access
   control decisions to 3rd party access control providers that are
   trusted by the content publisher. The former entities have no access
   to subscriber related information and should respect the decisions of
   the access control providers. 

   A recent proposal for an extra layer in the protocol stack [LIRA]
   gives control of the name resolution infrastructure to the publisher.
    This enables access logging as well some degree of active cache
   management, e.g., purging of stale content.

   One possible technique that could allow for providing access control
   to heterogeneous groups and still allow for a single encrypted object
   representation that remains cacheable is Attribute Based Encryption
   (ABE). A first proposal for this is presented in [ABE]. To support
   heterogeneous group and avoid having a single authority that has a
   master key multi authority ABE can be used [LEWKO].

   Evaluating the impact of the absence of these features will be
   essential for any scenario where an ICN architecture might be
   deployed.  It may have a seriously negative impact on the
   applicability of ICN in commercial environments unless a solution can
   be found.


3.3. Privacy

 


Pentikousis, et al.     Expires October 14, 2016               [Page 18]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   Another area where the architectures have not been significantly
   analyzed is privacy.  Caching implies a trade-off between network
   efficiency and privacy.  The activity of users is significantly more
   exposed to the scrutiny of cache owners with whom they may not have
   any relationship. However it should be noted that it is only the
   first hop router/cache that can see who request what, this as
   requests are aggregated and only the previous hop router is visible
   when a request is forwarded.

   Although in many ICN architectures the source of a request is not
   explicitly identified, an attacker may be able to obtain considerable
   information if s/he can monitor transactions on the cache and obtain
   details of the objects accessed, the topological direction of
   requests and information about the timing of transactions.   The
   persistence of data in the cache can make life easier for an attacker
   by giving a longer timescale for analysis. 

   The impact of CCN on privacy has been investigated in [CCNSEC] and
   the analysis is applicable to all ICN architectures because it is
   mostly focused on the common caching aspect.  The privacy risks of
   named data networking are also highlighted in [CCNPRIV].  Further
   work on privacy in ICNs can be found in [CONPRV]. Finally, Fotiou et
   al. define an ICN privacy evaluation framework in [PRIFRA].

3.4. Changes to the Network Security Threat Model

   The architectural differences of the various ICN models as compared
   to TCP/IP have consequences for network security.  There is limited
   consideration of the threat models and potential mitigation in the
   various documents describing the architectures. [CCNSEC] and [CONPRV]
   also consider the changed threat model.  Some of the key aspects are:

    o Caching implies a tradeoff between network efficiency and user
      privacy as discussed in Section 4.3.

    o More powerful routers upgraded to handle persistent caching
      increase the network's attack surface.  This is particularly the
      case in systems that may need to perform cryptographic checks on
      content that is being cached.  For example, not doing this could
      lead routers to disseminate invalid content.

    o ICNs makes it difficult to identify the origin of a request as
      mentioned in Section 4.3 slowing down the process of blocking
      requests and requiring alternative mechanisms to differentiate
      legitimate requests from inappropriate ones as access control
      lists (ACLs) will probably be of little value for ICN requests.

    o Denial-of-service (DoS) attacks may require more effort on ICN
 


Pentikousis, et al.     Expires October 14, 2016               [Page 19]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


      than on TCP/IP but they are still feasible.  One reason for this
      is that it is difficult for the attacker to force repeated
      requests for the same content onto a single node; ICNs naturally
      spread content so that after the initial few requests, subsequent
      requests will generally be satisfied by alternative sources,
      blunting the impact of a DoS attack.  That said, there are many
      ways around this, e.g., generating random suffix identifiers that
      always result in cache misses.

    o Per-request state in routers can be abused for DoS attacks.

    o Caches can be misused in the following ways:

       + Attackers can use caches as storage to make their own content
         available.

       + The efficiency of caches can be decreased by attackers with the
         goal of DoS attacks.

       + Content can be extracted by any attacker connected to the
         cache, putting users' privacy at risk.

   Appropriate mitigation of these threats will need to be considered in
   each scenario.

4.  Evaluation Tools

   Since ICN is an emerging area, the community is in the process of
   developing effective evaluation environments, including releasing
   open-source implementations, simulators, emulators, and testbeds.  To
   date, none of the available evaluation tools can be seen as the one
   and only community reference evaluation tool.  Furthermore, no single
   environment supports all well-known ICN approaches, as we describe
   below, hindering the direct comparison of the results obtained for
   different ICN approaches.  The rest of this subsection reviews the
   publicly available ICN implementations, simulators and experimental
   facilities currently available to the community.

   An updated version of the information in this section will be
   maintained at the ICNRG Wiki page:
   https://trac.tools.ietf.org/group/irtf/trac/wiki/IcnEvaluationAndTestbeds


4.1.  Open-source Implementations

   The Named Data Networking (NDN) project has open-sourced a software
   reference implementation of the architecture and protocol called NDN
   (http://http://named-data.net).  NDN is available for deployment on
 


Pentikousis, et al.     Expires October 14, 2016               [Page 20]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   various operating systems and includes C and Java libraries that can
   be used to build applications.  

   CCN-lite (http://www.ccn-lite.net) is a lightweight implementation of
   the CCN protocol that supports most of the key features of CCNx and
   is interoperable with CCNx.  CCN-lite implements the core CCN logic
   in about 1000 lines of code, so it is ideal for classroom work and
   course projects as well as for quickly experimenting with CCN
   extensions.  For example, Baccelli et al. use CCN-lite on top of the
   RIOT operating system to conduct experiments over an IoT (Internet of
   Things) testbed [NDNIOT].

   PARC is offering CCN source code under various licensing schemes,
   please see http://www.ccnx.org for details.

   The PURSUIT project (http://www.fp7-pursuit.eu) has open-sourced its
   Blackhawk publish-subscribe (Pub/Sub) implementation for Linux and
   Android; more details are available at https://github.com/fp7-
   pursuit/blackadder.  Blackadder uses the Click modular router for
   ease of development.  The code distribution features a set of tools,
   test applications and scripts.  The POINT project (http://www.point-
   h2020.eu/) is currently maintaining Blackadder.

   The 4WARD and SAIL projects have open-sourced software that
   implements different aspects of NetInf, e.g., NetInf URI format, HTTP
   and UDP convergence layer, using different programming languages. 
   The Java implementation provides a local caching proxy and client. 
   Further, an OpenNetInf prototype is available as well as a hybrid
   host-centric and information-centric network architecture called the
   Global Information Network (GIN), a browser plug-in and video
   streaming software.  See http://www.netinf.org/open-source for more
   details.


4.2.  Simulators and Emulators

   Simulators and emulators should be able to capture faithfully all
   features and operations of the respective ICN architecture(s) and any
   limitations should be openly documented.  It is essential that these
   tools and environments come with adequate logging facilities so that
   one can use them for in-depth analysis as well as debugging. 
   Additional requirements include the ability to support medium- to
   large-scale experiments, the ability to quickly and correctly set
   various configurations and parameters, as well as to support the
   playback of traffic traces captured on a real testbed or network. 
   Obviously, this does not even begin to touch upon the need for strong
   validation of any evaluated implementations.

 


Pentikousis, et al.     Expires October 14, 2016               [Page 21]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


4.2.1.  ndnSIM

   The Named Data Networking (NDN) project (http://named-data.net/) has
   developed ndnSIM [ndnSIM][ndnSIM2]; this is a module that can be
   plugged into the ns-3 simulator (https://www.nsnam.org/) and supports
   the core features of NDN.  One can use ndnSIM to experiment with
   various NDN applications and services as well as components developed
   for NDN such as routing protocols, caching and forwarding strategies
   among others.  The code for ns-3 and ndnSIM is openly available to
   the community and can be used as the basis for implementing ICN
   protocols or applications.  For more details see
   http://ndnsim.net/2.0/.

4.2.2.  ccnSIM

   ccnSim [ccnSim] is CCN-specific simulator that was specially designed
   to handle forwarding of a large number of CCN-chunks
   (http://www.infres.enst.fr/~drossi/index.php?n=Software.ccnSim). 
   ccnSim is written in C++ for the OMNeT++ simulation framework
   (https://omnetpp.org/).  Other CCN-specific simulators include CCN
   Packet Level Simulator [CCNPL] and CCN-Joker [CCNj].  CCN-Joker
   emulates in user-space all basic aspects of a CCN node (e.g.,
   handling of Interest and Data packets, cache sizing, replacement
   policies), including both flow and congestion control.  The code is
   open source and is suitable for both emulation-based analyses and
   real experiments.  Finally, Cabral et al. [MiniCCNx] use container-
   based emulation and resource isolation techniques to develop a
   prototyping and emulation tool.


4.2.3.  Icarus simulator

   The Icarus simulator focuses on caching in ICN and is agnostic with
   respect to any particular ICN implementation. The simulator is
   implemented in Python, uses the Fast Network Simulator Setup tool
   [FNSS], and is available at http://icarus-sim.github.io/. Icarus has
   several caching strategies implemented, including among others
   ProbCache [PROBCACH], node-centrality-based caching [CL4M] and hash-
   route-based caching [HASHROUT].

   ProbCache [PROBCACH] is taking a resource management view on caching
   decisions and approximates the available cache capacity along the
   path from source to destination. Based on this approximation and in
   order to reduce caching redundancy across the path, it caches content
   probabilistically. According to [CL4M] the node with the highest
   betweenness centrality along the path from source to destination is
   responsible for caching incoming content. Finally, [HASHROUT]
   calculates the hash-function of a content's name and assigns contents
 


Pentikousis, et al.     Expires October 14, 2016               [Page 22]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   to caches of a domain according to that. The hash-space is split
   according to the number of caches of the network. Then upon
   subsequent requests, and based again on the hash of the name included
   in the request, edge routers redirect requests to the cache assigned
   with the corresponding hash-space. [HASHROUT] is an off-path caching
   strategy, in contrast to [PROBCACH] and [CL4M], which however,
   requires minimum co-ordination and redirection overhead. In its
   latest update, Icarus also includes implementation of the "Satisfied
   Interest Table" (SIT) [SIT]. The SIT table is pointing towards the
   direction where content has been sent recently. Among other benefits,
   this enables information resilience in case of network fragmentation
   (i.e., content can still be found in neighbour caches or in users'
   devices) and inherently supports user-assisted caching (i.e., P2P-
   like content distribution).

   The simulator is constantly being updated and more strategies are
   added in each update. Tortelli et al. [ICNSIMS] provide a comparison
   of ndnSIM, ccnSim, and Icarus.

4.3.  Experimental Facilities

   An important consideration in the evaluation of any kind of future
   Internet mechanism lies in the characteristics of that evaluation
   itself. Central to the assessment of the features provided by a novel
   mechanism, lies the consideration of how it improves over already
   existing technologies, and by "how much."  With the disruptive nature
   of clean-slate approaches generating new and different technological
   requirements, it is complex to provide meaningful results for a
   network layer framework, in comparison with what is deployed in the
   current Internet.  Thus, despite the availability of ICN
   implementations and simulators, the need for large-scale environments
   supporting experimental evaluation of novel research is of prime
   importance to the advancement of ICN deployment.

   Different experimental facilities have different characteristics and
   capabilities, e.g. low cost of use, reproducible configuration, easy-
   to-use tools, sharable, available background traffic. 

4.3.1.  Open Network Lab [ONL]

   An example of an experimental facility that supports CCN is the Open
   Network Lab [ONL] that currently comprises 18 extensible gigabit
   routers and over a 100 computers representing clients and is freely
   available to the public for running CCN experiments.  Nodes in ONL
   are preloaded with CCNx software.  ONL provides a graphical user
   interface for easy configuration and testbed setup as per the
   experiment requirements, and also serves as a control mechanism,
   allowing access to various control variables and traffic counters. 
 


Pentikousis, et al.     Expires October 14, 2016               [Page 23]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   Further, it is also possible to run and evaluate CCN over popular
   testbeds [PLANET] [EMULAB] [DETERLAB] [OFELIA] by directly running,
   for example, the CCNx open-source code [CCNOFELI] [ICNGRID] [CCNPL]
   [CCNOSN]. Also, the Network Experimentation Programming Interface
   [NEPI] is a tool developed for controlling and managing large-scale
   network experiments.  NEPI can be used to control and manage large-
   scale CCNx experiments e.g., on PlanetLab [NEPIICN].

4.3.2.  POINT testbed

   The POINT project is maintaining a testbed with 40 machines across
   Europe, North America (MIT) and Japan (NICT) interconnected in a
   topology containing one Topology Manager and one Rendezvous node that
   handle all publish/subscribe and topology formation requests [IEICE].
    All machines run Blackadder. New nodes can join and experiments can
   be run on request.

4.3.3.  CUTEi: Container-based ICN testbed

   NICT has also developed a testbed used for ICN experiments [AFI]
   comprising multiple servers located in Asia and other locations. 
   Each testbed server (or VM) utilizes a Linux kernel-based container
   (LXC) for node virtualization. This testbed enables users to run
   applications and protocols for ICN in two experimentation modes using
   two different container designs: 

     1. application-level experimentation using a "common container" and

     2. network-level experimentation using a "user container."  

   A common container is shared by all testbed users, and a user
   container is assigned to one testbed user.  A common container has a
   global IP address to connect with other containers or external
   networks, whereas each user container uses a private IP address and a
   user space providing a closed networking environment.  A user can
   login to his/her user containers using SSH with his/her certificate,
   or access them from PCs connected to the Internet using SSH
   tunnelling.  

   This testbed also implements an "on-filesystem cache" to allocate
   caching data on a UNIX filesystem.  The on-filesystem cache system
   accommodates two kinds of caches: "individual cache" and "shared
   cache."  Individual cache is accessible for one dedicated router for
   the individual user, while shared cache is accessible for a set of
   routers in the same group to avoid duplicated caching in the
   neighborhood for cooperative caching.


 


Pentikousis, et al.     Expires October 14, 2016               [Page 24]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


5.  Security Considerations

   This document does not impact the security of the Internet.


6.  IANA Considerations

   This document presents no IANA considerations.


7.  Acknowledgments

   Konstantinos Katsaros contributed the updated text of Section 2.2
   along with an extensive set of references.

   Priya Mahadevan, Daniel Corujo and Gareth Tyson contributed to an
   earlier version of this document.

   This document has benefited from reviews, pointers to the growing ICN
   literature, suggestions, comments and proposed text provided by the
   following members of the IRTF Information-Centric Networking Research
   Group (ICNRG), listed in alphabetical order: Marica Amadeo, Hitoshi
   Asaeda, E. Baccelli, Claudia Campolo, Christian Esteve Rothenberg,
   Suyong Eum, Nikos Fotiou, Dorothy Gellert, Luigi Alfredo Grieco,
   Myeong-Wuk Jang, Ren Jing, Will Liu, Antonella Molinaro, Luca
   Muscariello, Ioannis Psaras, Dario Rossi, Stefano Salsano, Damien
   Saucez, Dirk Trossen, Jianping Wang, Yuanzhe Xuan, and Xinwen Zhang.

   The IRSG review was provided by Aaron Falk.


8.  Informative References


   [RFC2330]  Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
              "Framework for IP Performance Metrics", RFC 2330, May
              1998.

   [RFC5743]  Falk, A., "Definition of an Internet Research Task Force
              (IRTF) Document Stream", RFC 5743, December 2009.

   [RFC6920]  Farrell, S., Kutscher, D., Dannewitz, C., Ohlman, B.,
              Keranen, A., and P. Hallam-Baker, "Naming Things with
              Hashes", RFC 6920, April 2013.

   [RFC7476]  Pentikousis, K., Ohlman, B., Corujo, D., Boggia, G.,
              Tyson, G., Davies, E., Molinaro, A., and S. Eum,
              "Information-Centric Networking: Baseline Scenarios ", RFC
 


Pentikousis, et al.     Expires October 14, 2016               [Page 25]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


              7476, March 2015.

   [ICNCHA]   Internet draft, "ICN Research Challenges" draft-irtf-
              icnrg-challenges-04, [***EDITORS NOTE: This reference is
              expected to have become an RFC by the time this document
              is published, this reference should thus be updated before
              publication of this document.***]

   [ndnSIM]   Afanasyev, A. et al., "ndnSIM: NDN simulator for NS-3",
              NDN Technical Report NDN-0005, Revision 2, October 2012.

   [ndnSIM2]  Mastorakis, S. et al., "ndnSIM 2.0: A new version of the
              NDN simulator for NS-3", NDN Technical Report NDN-0028,
              Revision 1, January 2015.

   [ccnSim]   Rossini, G. and D. Rossi, "Large scale simulation of CCN
              networks", Proc. Algotel 2012 , La Grande Motte, France,
              May 2012.

   [CCNPL]    Muscariello, L., "Content centric networking packet level
              simulator", available online at
              http://perso.rd.francetelecom.fr/muscariello/sim.html

   [CCNj]     Cianci, I. et al. "CCN - Java Opensource Kit EmulatoR for
              Wireless Ad Hoc Networks", Proc. 7th ACM Int. Conf. on
              Future Internet Technologies, Seoul, Korea, Sept., 2012.

   [IEICE]    G. Parisis, D. Trossen, and H. Asaeda, "A Node Design and
              a Framework for Development and Experimentation for an
              Information-Centric Network", IEICE Trans. Commun., vol.
              E96-B, no. 7, pp.1650-1660, July 2013.

   [PROBCACH] I. Psaras, W. Chai, G. Pavlou, "Probabilistic In-Network
              Caching for Information-Centric Networks", Proc. SIGCOMM
              ICN Workshop. ACM, 2012.

   [CL4M]     Chai, W. K. et al., "Cache 'Less for More' in Information-
              centric Networks", Proc. Networking.  IFIP, 2012.

   [SIT]      Vasilis Sourlas, Leandros Tassiulas, Ioannis Psaras and
              George Pavlou, "Information Resilience through User-
              Assisted Caching in Disruptive Content-Centric Networks"
              14th IFIP NETWORKING, Toulouse, France, May 2015.

   [HASHROUT] L. Saino, I. Psaras, G. Pavlou, "Hash-routing Schemes for
              Information-Centric Networking", Proc. SIGCOMM ICN
              Workshop. ACM, 2013.

 


Pentikousis, et al.     Expires October 14, 2016               [Page 26]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   [ICARUS]   L. Saino, I. Psaras, G. Pavlou, "Icarus: a Caching
              Simulator for Information Centric Networking (ICN)", Proc.
              SIMUTOOLS. ICST, 2014.

   [FNSS]     L. Saino, C. Cocora and G. Pavlou, "A Toolchain for
              Simplifying Network Simulation Setup", Proc. SIMUTOOLS. 
              ACM, 2013.

   [BA]       Barabasi, A. and R. Albert, "Emergence of scaling in
              random networks", Science, vol. 286, no. 5439, pp. 509-
              512, 1999.

   [WATTS]    Watts, D. J. and S. H. Strogatz, "Collective dynamics of
              small-world networks", Nature, vol. 393, no. 6684, pp. 40-
              "10, 1998.

   [Montage]  Hussain, A. and J. Chen, "Montage Topology Manager: Tools
              for Constructing and Sharing Representative Internet
              Topologies", DETER Technical Report, ISI-TR-684, Aug.
              2012.

   [DELAY]    Kaune, S. et al., "Modelling the Internet Delay Space
              Based on Geographical Locations", Proc. Euromicro, Weimar,
              Germany, 2009.

   [IMB2014]  C. Imbrenda, L. Muscariello, and D. Rossi,"Analyzing
              Cacheable Traffic in ISP Access Networks for Micro CDN
              Applications via Content-Centric Networking," In Proc. of 
              ACM ICN, 2014.

   [L1]       http://googleblog.blogspot.it/2008/07/we-knew-web-was-
              big.html

   [L2]       Zhang, C., Dhungel, P., and K. Di Wu., "Unraveling the
              BitTorrent ecosystem", IEEE Transactions on Parallel and
              Distributed Systems, pp. 1164-1177, 2010.

   [L3]       Cha, M., Kwak, H., Rodriguez, P., Ahn, Y.-Y., and S. Moon,
              "I tube, you tube, everybody tubes: analyzing the world's
              largest user generated content video system", Proc. ACM
              SIGCOMM conference on Internet measurement (IMC), San
              Diego (CA), USA, Oct. 2007.

   [L4]       Zhou, J., Li,  Y., Adhikari, K., and Z.-L. Zhang,
              "Counting YouTube videos via random prefix sampling", In
              Proc. of IMC'11, Berlin, Germany, Nov. 2011.

   [L5]       Fricker, C., Robert,  P., Roberts, J. and N. Sbihim,
 


Pentikousis, et al.     Expires October 14, 2016               [Page 27]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


              "Impact of traffic mix on caching performance in a
              content-centric network", In Proc. of IEEE NOMEN 2012,
              Workshop on Emerging Design Choices in Name-Oriented
              Networking, Orlando, USA, Mar. 2012.

   [L6]       Yu, H., Zheng,  D., Zhao, B. Y. and W. Zheng,
              "Understanding user behavior in large-scale video-on-
              demand systems", In SIGOPS Oper. Syst. Rev., Vol. 40, pp.
              333-344, April 2006.

   [L7]       Marciniak, P., Liogkas, N., Legout, A. and E. Kohler,
              "Small is not always beautiful",  In Proc. of IPTPS,
              International Workshop of Peer-to-Peer Systems, Tampa Bay,
              Florida (FL), USA, Feb. 2008.

   [L8]       Bellissimo, A., Levine, B. and P. Shenoy, "Exploring the
              use of BitTorrent as the basis for a large trace
              repository", University of Massachusetts, Tech. Rep.,
              2004.

   [L9]       Psaras, I. et al., "Modelling and Evaluation of CCN-
              Caching Trees", In Proc. of the 10th international IFIP
              conference on Networking, Valencia, Spain, May 2011.

   [L10]      Carofiglio, G., Gallo, M., Muscariello, L., and D. Perino,
              "Modeling Data Transfer in Content-Centric Networking", In
              Proc. of  ITC, San Francisco, USA, Sep. 2011.

   [L11]      Breslau, L., Cao,  P., Fan, L., Phillips, G. and S.
              Shenker, "Web caching and zipf-like distributions:
              evidence and implications", In Proc. of INFOCOM '99, New
              York (NY), USA,  Mar. 1999.

   [L12]      Mahanti, A., Williamson, C., and D. Eager., "Traffic
              analysis of a web proxy caching hierarchy", IEEE Network, 
              Vol.14, No.3, pp.16-23, May/June 2000.

   [P2PMod]   Saleh, O., and M. Hefeeda, "Modeling and caching of peer-
              to-peer traffic", Proc. ICNP. IEEE, 2006.

   [W16]      C. Westphal, Ed., "Adaptive Video Streaming over ICN",
              ICNRG Internet Draft.

   [Led12]    S. Lederer, C. Muller, and C. Timmerer, "Dynamic adaptive
              streaming over HTTP dataset", in Proceedings of the ACM
              Multimedia Systems Conference 2012, 2012, pp. 89-94.

   [CCN]      Jacobson, V. et al., "Networking Named Content", Proc.
 


Pentikousis, et al.     Expires October 14, 2016               [Page 28]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


              CoNEXT.  ACM, 2009.

   [VoCCN]    Jacobson, V. et al., "VoCCN: Voice-over Content-Centric
              Networks", Proc. CoNEXT Re-Arch Workshop.  ACM, 2009.

   [NDNP]     Zhang, L. et al., "Named Data Networking (NDN) Project",
              NDN Technical Report NDN-0001, Oct. 2010.  Available:
              http://named-data.net/publications/techreports/

   [4WARD6.1] Ohlman, B. et al., "First NetInf Architecture
              Description", 4WARD Project Deliverable D-6.1, Apr. 2009.

   [4WARD6.3] Ahlgren, B. et al., "NetInf Evaluation", 4WARD Project
              Deliverable D-6.3, June 2010.

   [SAIL-B2]  SAIL, "NetInf Content Delivery and Operations", SAIL
              Project Deliverable D-B.2, May. 2012.

   [SAIL-B3]  Kutscher, D. (ed.) et al., "Final NetInf Architecture",
              SAIL Project Deliverable D-B.3 , Jan. 2013.  Available:
              http://www.sail-project.eu/deliverables/

   [PRST4.5]  Riihijarvi, J. et al., "Final Architecture Validation and
              Performance Evaluation Report", PURSUIT Project
              Deliverable D4.5, Jan. 2013.

   [CMT-D5.2] B ben, A. et al, "Scalability of COMET System", COMET
              Project Deliverable D5.2, Feb. 2013.

   [CMT-D6.2] Georgiades, M. et al., "Prototype Experimentation and
              Demonstration", COMET Project Deliverable D6.2, Feb. 2013.

   [SHARE]    Muscariello, L. et al., "Bandwidth and storage sharing
              performance in information centric networking", Proc.
              SIGCOMM ICN Workshop.  ACM, 2011.

   [RealCCN]  Perino, D. et al., "A Reality Check for Content Centric
              Networking", Proc. SIGCOMM ICN Workshop. ACM, 2011.

   [ICN-Web]  Detti, A. et al., "Supporting the Web with an Information
              Centric Network that Routes by Name", Elsevier Computer
              Networks, vol. 56, no. 17, Nov. 2012.

   [ICN-Scal] Blefari Melazzi, N. et al., "Scalability Measurements in
              an Information-Centric Network", Springer Lecture Notes in
              Computer Science (LNCS), vol. 7586, 2012.

   [ICN-Tran] Salsano, S. et al., "Transport-Layer Issues in Information
 


Pentikousis, et al.     Expires October 14, 2016               [Page 29]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


              Centric Networks ", Proc. SIGCOMM ICN Workshop. ACM, 2012.

   [SensReqs] Karnouskos, S.  et al., "Requirement considerations for
              ubiquitous integration of cooperating objects", Proc.
              NTMS. IFIP, 2011.

   [NETINFSC] Renault, E, Ahmad, A., and M. Abid, "Towards a Security
              Model for the Future Network of Information", Proc. Conf.
              Ubiquitous Information Technologies and Applications,
              IEEE, 2009.

   [DONA]     Koponen, T. et al., "A Data-Oriented (and Beyond) Network
              Architecture", Proc. SIGCOMM.  ACM, 2007.

   [SECCONT]  Smetters, D., and V. Jacobson, "Securing network content",
              Technical Report TR-2009-01, PARC, 2009.

   [PSTSEC]   Tagger, B., et al, "Update on the Architecture and Report
              on Security Analysis", Deliverable 2.4, PURSUIT EU FP7
              project, April 2012.

   [ABE]      Mihaela Ion, Jianqing Zhang, and Eve M. Schooler. 2013.
              Toward content-centric privacy in ICN: attribute-based
              encryption and routing. In Proceedings of the ACM SIGCOMM
              2013 conference on SIGCOMM (SIGCOMM '13). ACM, New York,
              NY, USA, 513-514. DOI=10.1145/2486001.2491717
              http://doi.acm.org/10.1145/2486001.2491717

   [LEWKO]    Lewko, A., Waters, B.: Decentralizing attribute-based
              encryption. In: Proc. of EUROCRYPT 11, LNCS, vol. 6632,
              pp. 568-588 (2011).

   [SCNETINF] C. Dannewitz, J. Golic, B. Ohlman, B. Ahlgren, "Secure
              Naming for A Network of Information," Proc. 13th IEEE
              Global Internet Symp. 2010, San Diego, CA, Mar. 2010.

   [ACDICN]   Fotiou, N. et al., "Access control enforcement delegation
              for information-centric networking architectures", Proc.
              SIGCOMM ICN Workshop. ACM, 2012.

   [CCNSEC]   Lauinger, T., "Security and Scalability of Content-Centric
              Networking", Masters Thesis, Technische Universitaet
              Darmstadt and Eurecom, Sep. 2010.

   [CCNPRIV]  Lauinger, Y., et al, "Privacy Risks in Named Data
              Networking: What is the Cost of Performance?", ACM SIGCOMM
              Computer Communication Review Editorial Note, vol. 42,
              iss. 5, 2012
 


Pentikousis, et al.     Expires October 14, 2016               [Page 30]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   [CONPRV]   Chaabane, A et al, "Privacy in Content-Oriented
              Networking: Threats and Countermeasures", arXiv:1211.5183,
              2012.

   [1]        C. Labovitz, S. Iekel-Johnson, D. McPherson, J. Oberheide,
              and F. Jahanian. 2010. Internet inter-domain traffic. In
              Proceedings of the ACM SIGCOMM 2010 conference (SIGCOMM
              '10). ACM, New York, NY, USA, 75-86.
              DOI=10.1145/1851182.1851194,
              http://doi.acm.org/10.1145/1851182.1851194

   [2]        G. Maier, A. Feldmann, V. Paxson, and M. Allman. 2009. On
              dominant characteristics of residential broadband internet
              traffic. In Proceedings of the 9th ACM SIGCOMM conference
              on Internet measurement conference (IMC '09). ACM, New
              York, NY, USA, 90-102. DOI=10.1145/1644893.1644904
              http://doi.acm.org/10.1145/1644893.1644904

   [3]        L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Shenker,
              "Web Caching and Zipf-like Distributions: Evidence and
              Implications," in IEEE INFOCOM, 1999, pp. 126-134.

   [4]        A. Mahanti, C. Williamson, and D. Eager, "Traffic analysis
              of a web proxy caching hierarchy," IEEE Network, vol. 14,
              no. 3, pp. 16 -23, 2000.

   [5]        M. Busari and C. Williamson, "ProWGen: a synthetic
              workload generation tool for simulation evaluation of web
              proxy caches," Computer Networks, vol. 38, no. 6, pp. 779-
              794, 2002.

   [6]        M. F. Arlitt and C. L. Williamson, "Internet web servers:
              workload characterization and performance implications,"
              IEEE/ACM Transactions on Networking, vol. 5, pp. 631-645,
              1997.

   [7]        P. Barford and M. Crovella, "Generating representative web
              workloads for network and server performance evaluation,"
              in ACM SIGMETRICS/PERFORMANCE, 1998, pp. 151-160.

   [8]        P. Barford, A. Bestavros, A. Bradley, and M. Crovella,
              "Changes in web client access patterns: Characteristics
              and caching implications," World Wide Web, vol. 2, pp. 15-
              28, 1999.

   [9]        M. Hefeeda and O. Saleh, "Traffic Modeling and
              Proportional Partial Caching for Peer-to-Peer Systems,"
              IEEE/ACM Transactions on Net- working, vol. 16, no. 6, pp.
 


Pentikousis, et al.     Expires October 14, 2016               [Page 31]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


              1447-1460, 2008.

   [10]       L. Guo, S. Chen, Z. Xiao, E. Tan, X. Ding, and X. Zhang,
              "A performance study of BitTorrent-like peer-to-peer
              systems," IEEE Journal on Selected Areas in Communication,
              vol. 25, no. 1, pp. 155-169, 2007.

   [11]       A. Bellissimo, B. N. Levine, and P. Shenoy, "Exploring the
              use of BitTorrent as the basis for a large trace
              repository," University of Massachusetts Amherst, Tech.
              Rep., 2004.

   [12]       X. Cheng, C. Dale, and J. Liu, "Statistics and social
              network of YouTube videos," in IEEE IWQoS. IEEE, 2008, pp.
              229-238.

   [13]       X. Cheng, C. Dale, and J. Liu, "Understanding the
              Characteristics of Internet Short Video Sharing: YouTube
              as a Case Study," CoRR, vol. abs/0707.3670, 2007.

   [14]       K. V. Katsaros, G. Xylomenos, and G. C. Polyzos,
              "GlobeTraff: a traffic workload generator for the
              performance evaluation of future Internet architectures,"
              Proc.  IEEE/IFIP NTMS, 2012.

   [MiniCCNx] C. Cabral, et al., "High fidelity content-centric
              experiments with Mini-CCNx", Proc. ISCC. IEEE, 2014.


   [ICNSIMS]  M. Tortelli, et al., "CCN Simulators: Analysis and Cross-
              Comparison", Proc. SIGCOMM ICN. ACM, 2014.

   [AFI]      H. Asaeda, R. Li, and N. Choi, "Container-Based Unified
              Testbed for Information-Centric Networking," IEEE Network,
              vol. 28, no. 6, pp. 60-66, 2014.

   [NDNIOT]   E. Baccelli, et al., "Information Centric Networking in
              the IoT: Experiments with NDN in the Wild," Proc. SIGCOMM
              ICN. ACM, 2014.

   [ONL]      J. DeHart, et al., "The open network laboratory: a
              resource for networking research and education", ACM
              SIGCOMM CCR, vol. 35, no. 5, pp. 75-78, 2005.

   [PLANET]   B. Chun, et al., "Planetlab: an overlay testbed for broad-
              coverage services", ACM SIGCOMM CCR, vol. 33, no. 3, pp.
              3-12, 2003.

 


Pentikousis, et al.     Expires October 14, 2016               [Page 32]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   [EMULAB]   E. Eide, et. al., "An Experimentation Workbench for
              Replayable Networking Research", Proc. NSDI. USENIX, 2007.

   [DETERLAB] T. Benzel, "The Science of Cyber-Security Experimentation:
              The DETER Project", Proc. Annual Computer Security
              Applications Conference (ACSAC), Dec. 2011.

   [OFELIA]   M. Sune, et al., "Design and implementation of the OFELIA
              FP7 facility: The European OpenFlow testbedDesign and
              implementation of the OFELIA FP7 facility: The European
              OpenFlow testbed", Computer Networks, vol. 61, pp. 132-
              150, March 2014.

   [NEPI]     A. Quereilhac, et al., "NEPI: An integration framework for
              Network Experimentation", Proc. SoftCOM. IEEE, 2011.

   [NEPIICN]  A. Quereilhac, et al., "Demonstrating a unified ICN
              development and evaluation framework", Proc. SIGCOMM ICN.
              ACM, 2014.

   [CCNOFELI] S. Salsano, et al., "Information Centric Networking over
              SDN and OpenFlow: Architectural Aspects and Experiments on
              the OFELIA Testbed", Computer Networks, vol. 57, no. 16,
              pp. 3207-3221, November 2013.

   [ICNGRID]  G. Carofiglio, et al., "Optimal multipath congestion
              control and request forwarding in Information-Centric
              Networks", Proc. ICNP. IEEE, 2013.

   [CCNPL]    S. Awiphan, et al., "Video streaming over content centric
              networking: Experimental studies on PlanetLab", Proc.
              ComComAp. IEEE, 2013

   [CCNOSN]   C. Bernardini, et al., "Socially-aware caching strategy
              for content centric networking", Proc. Networking. IFIP,
              2014.

   [PRIFRA]   N. Fotiou, et al. "A framework for privacy analysis of ICN
              architectures", Proc. APF. Springer, 2014.

   [ICNScale] K. V. Katsaros, et al., "On the Inter-domain Scalability
              of Route-by-Name Information-Centric Network
              Architectures", Proc. Networking. IFIP, 2015.

   [COMPLEX]  X. Dimitropoulos, et al., "Graph annotations in modeling
              complex network topologies", ACM Trans. Model. Comput.
              Simul., vol. 19, no. 4, Nov. 2009.

 


Pentikousis, et al.     Expires October 14, 2016               [Page 33]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   [LIRA]     I. Psaras, K. Katsaros, L. Saino, and G. Pavlou, "Lira: A
              location independent routing layer based on source-
              provided ephemeral names," Electronic and Electrical Eng.
              Dept., UCL, London, UK, Tech. Rep. 2014. [Online].
              Available: http://www.ee.ucl.ac.uk/comit-
              project/publications.html

Authors' Addresses


   Kostas Pentikousis (editor)
   EICT GmbH
   Torgauer Strasse 12-15
   10829 Berlin
   Germany

   Email: k.pentikousis@eict.de


   Borje Ohlman
   Ericsson Research
   S-16480 Stockholm
   Sweden

   Email: Borje.Ohlman@ericsson.com


   Elwyn Davies
   Trinity College Dublin/Folly Consulting Ltd
   Dublin, 2
   Ireland

   Email: davieseb@scss.tcd.ie


   Spiros Spirou
   Intracom Telecom
   19.7 km Markopoulou Avenue
   19002 Peania, Athens
   Greece

   Email: spis@intracom-telecom.com


   Gennaro Boggia
   Dep. of Electrical and Information Engineering
   Politecnico di Bari
   Via Orabona 4
 


Pentikousis, et al.     Expires October 14, 2016               [Page 34]

INTERNET DRAFT   Evaluation and Security Considerations   April 12, 2016


   70125 Bari
   Italy

   Email: g.boggia@poliba.it















































Pentikousis, et al.     Expires October 14, 2016               [Page 35]