Internet DRAFT - draft-lefaucheur-cdni-logging-delivery

draft-lefaucheur-cdni-logging-delivery






Network Working Group                                     F. Le Faucheur
Internet-Draft                                          M. Viveganandhan
Intended status: Informational                                  K. Leung
Expires: January 14, 2013                                          Cisco
                                                           July 13, 2012


  CDNI Logging Formats for HTTP and HTTP Adaptive Streaming Deliveries
               draft-lefaucheur-cdni-logging-delivery-01

Abstract

   The interconnection of Content Distribution Networks (CDNs) is
   motivated by several use cases.  CDN Interconnection can be achieved
   through four CDNI interfaces, one of which is the CDNI Logging
   interface.  This document discusses CDNI logging formats for content
   deliveries performed using HTTP or HTTP adaptive streaming.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on January 14, 2013.

Copyright Notice

   Copyright (c) 2012 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents



Le Faucheur, et al.     Expires January 14, 2013                [Page 1]

Internet-Draft       CDNI Logging for HTTP Delivery            July 2012


   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
     1.1.  Terminology  . . . . . . . . . . . . . . . . . . . . . . .  4
   2.  CDNI Logging for regular HTTP Delivery . . . . . . . . . . . .  4
     2.1.  Regular HTTP Log Triggers  . . . . . . . . . . . . . . . .  4
     2.2.  Regular HTTP Log Fields  . . . . . . . . . . . . . . . . .  4
   3.  CDNI Logging for HTTP Adaptive Streaming . . . . . . . . . . .  6
     3.1.  HAS Chunk-Based Log Triggers . . . . . . . . . . . . . . .  6
     3.2.  HAS Chunk-Based Log fields . . . . . . . . . . . . . . . .  6
   4.  Performance Monitoring . . . . . . . . . . . . . . . . . . . .  7
   5.  CDNI Log Encoding and Transport  . . . . . . . . . . . . . . .  7
     5.1.  CDNI Log Header Information  . . . . . . . . . . . . . . .  8
     5.2.  CDNI per-Log-Record Information  . . . . . . . . . . . . .  8
     5.3.  CDNI Log Footer Information  . . . . . . . . . . . . . . .  9
     5.4.  CDNI Customized Log Format . . . . . . . . . . . . . . . .  9
   6.  Impact on CDNI Metadata Requirements . . . . . . . . . . . . .  9
   7.  Impact on CDNI Footprint and Capabilities Advertisement  . . . 10
   8.  Generation of CDNI Logs  . . . . . . . . . . . . . . . . . . . 10
   9.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 10
   10. Security Considerations  . . . . . . . . . . . . . . . . . . . 11
   11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 11
   12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11
     12.1. Normative References . . . . . . . . . . . . . . . . . . . 11
     12.2. Informative References . . . . . . . . . . . . . . . . . . 11
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12
















Le Faucheur, et al.     Expires January 14, 2013                [Page 2]

Internet-Draft       CDNI Logging for HTTP Delivery            July 2012


1.  Introduction

   The interconnection of Content Distribution Networks (CDNs) is
   motivated by several use cases, such as those described in
   [I-D.ietf-cdni-use-cases].  The overall problem space for CDN
   Interconnection (CDNI) is described in
   [I-D.ietf-cdni-problem-statement].  The CDN Interconnection framework
   is defined in [I-D.ietf-cdni-framework] and the requirements for the
   CDN Interconnection solution are specified in
   [I-D.ietf-cdni-requirements].

   One of the CDNI interfaces defined in these documents is the CDNI
   Logging interface whose description is quoted here:

   "o CDNI Logging interface: This interface allows the Logging system
   in interconnected CDNs to communicate the relevant activity logs in
   order to allow log consuming applications to operate in a multi-CDN
   environments.  For example, an upstream CDN may collect delivery logs
   from a downstream CDN in order to perform consolidated charging of
   the CSP or for settlement purposes across CDNs.  Similarly, an
   upstream CDN may collect delivery logs from a downstream CDN in order
   to provide consolidated reporting and monitoring to the CSP."

   The logging interface is discussed in details in
   [I-D.bertrand-cdni-logging].  The present document identifies a
   number of additional considerations regarding logging for content
   delivery performed using HTTP.

   [I-D.brandenburg-cdni-has] discusses the interactions of HTTP
   Adaptive Streaming with CDNI Logging and provides recommendations on
   how to specifically perform CDNI Logging for delievry performed using
   HTTP Adaptive Streaming.  The present document discusses in more
   details how these recommendations woudl impact the CDNI Logging
   interface.

   Section 2 discusses the aspect of CDNI Logging that are specific to
   content delivery performed via regular HTTP.  Section 3 discusses the
   aspect of CDNI Logging that are specific to content delivery
   performed via HTTP adaptive streaming.  Section 5 discusses the
   aspect of CDNI Logging that are generic to all the CDNI Logs for
   delivery.

   CDNI Logging for other events than content delivery (e.g. failures,
   request routing, service monitoring,...) are not discussed in the
   present document.






Le Faucheur, et al.     Expires January 14, 2013                [Page 3]

Internet-Draft       CDNI Logging for HTTP Delivery            July 2012


1.1.  Terminology

   This document uses all terminology defined in
   [I-D.brandenburg-cdni-has].


2.  CDNI Logging for regular HTTP Delivery

   This section discusses the triggers on which a CDNI log is generated
   for delivery of content using regular HTTP delivery, as well as the
   fields contained in the corresponding log.

2.1.  Regular HTTP Log Triggers

   A CDNI Log is generated for a regular HTTP delivery on the following
   triggers:

   +-----------------+-------------------------------------------------+
   | Event           | Description                                     |
   +-----------------+-------------------------------------------------+
   | content Request | Reception and processing of a request for a     |
   |                 | content                                         |
   +-----------------+-------------------------------------------------+

2.2.  Regular HTTP Log Fields

   A CDNI Log for regular HTTP delivery contains the following fields
   (or a subset therefof as discused in Section 5.4):

   +------------------+------------+-----------------------------------+
   | Field            | Descriptio | Examples                          |
   |                  | n          |                                   |
   +------------------+------------+-----------------------------------+
   | Current-Time     | Time, in   | [20/Feb/2012:00:29.510+0200]      |
   |                  | millisecon |                                   |
   |                  | ds, at     |                                   |
   |                  |  which the |                                   |
   |                  |  request   |                                   |
   |                  |  was       |                                   |
   |                  |  received. |                                   |
   | Time-to-Serve    | Time, in   | 952195                            |
   |                  | microsecon |                                   |
   |                  | ds, taken  |                                   |
   |                  |  to        |                                   |
   |                  |  complete  |                                   |
   |                  |  the       |                                   |
   |                  |  request.  |                                   |




Le Faucheur, et al.     Expires January 14, 2013                [Page 4]

Internet-Draft       CDNI Logging for HTTP Delivery            July 2012


   | Client-IP        | IP address | 203.0.113.2                       |
   |                  | of the     |                                   |
   |                  | requesting |                                   |
   |                  | client.    |                                   |
   | Action           | Squid      | TCP_HIT, TCP_MISS, ...            |
   |                  | action     |                                   |
   |                  | describing |                                   |
   |                  | how the    |                                   |
   |                  | request    |                                   |
   |                  | was        |                                   |
   |                  | treated    |                                   |
   |                  | locally    |                                   |
   |                  | descriptio |                                   |
   |                  | n(e.g.,    |                                   |
   |                  |  cache     |                                   |
   |                  |  hit/miss) |                                   |
   |                  | .          |                                   |
   | Status-Returned  | HTTP       | 200, 404, ...                     |
   |                  | response   |                                   |
   |                  | code.      |                                   |
   | Bytes-Transferre | Bytes sent | 23567992                          |
   | d                | to the     |                                   |
   |                  | client,    |                                   |
   |                  | including  |                                   |
   |                  | the        |                                   |
   |                  | headers.   |                                   |
   | Method           | HTTP       | GET                               |
   |                  | request    |                                   |
   |                  | method.    |                                   |
   | URI              | URI of     | http://cache3.cdn1.com/movie/ice/ |
   |                  | requested  | icemovie.mpg                      |
   |                  | content.   |                                   |
   | Content-Type     | MIME-Type  | video/mpeg                        |
   |                  | from the   |                                   |
   |                  | reply      |                                   |
   |                  | header.    |                                   |
   | User-Agent (and  | content of | Mozilla/5.0 (X11; Linux x86_64;   |
   | possibly some    | the        | rv:12.0) Gecko/20100101           |
   | other HTTP       | User-Agent | Firefox/12.0                      |
   | headers)         | HTTP       |                                   |
   |                  | Header     |                                   |










Le Faucheur, et al.     Expires January 14, 2013                [Page 5]

Internet-Draft       CDNI Logging for HTTP Delivery            July 2012


   | URI-Signing-Vali | Flag       | 0/1                               |
   | dation           | indicating |                                   |
   |                  | whether    |                                   |
   |                  | URI        |                                   |
   |                  | signature  |                                   |
   |                  | validation |                                   |
   |                  | was        |                                   |
   |                  | perfomed   |                                   |
   +------------------+------------+-----------------------------------+


3.  CDNI Logging for HTTP Adaptive Streaming

   This section discusses the CDNI logs for delivery of content using
   HTTP adaptive streaming.  In line with the recommendations of
   [I-D.brandenburg-cdni-has], this document proposes that a log record
   be generated for delivery of each chunk or manifest file.

3.1.  HAS Chunk-Based Log Triggers

   A chunk-based Log record is generated for HAS delivery on the
   following triggers:

   +--------------------+----------------------------------------------+
   | Event              | Description                                  |
   +--------------------+----------------------------------------------+
   | Chunk/Manifest     | Reception of a request for a Segment (or     |
   | Request            | Manifest File)                               |
   +--------------------+----------------------------------------------+

3.2.  HAS Chunk-Based Log fields

   A chunk-based Log record for HTTP adaptive streaming may contain all
   the same fields as a CDNI Log for a regular HTTP request, as well as
   the following additional fields:

   +------------+-------------------+----------------------------------+
   | Field      | Description       | Examples                         |
   +------------+-------------------+----------------------------------+
   | Content    | Identifier for    | format and scope of unicity are  |
   | Collection | Content           | TBD                              |
   | ID         | Collection        |                                  |









Le Faucheur, et al.     Expires January 14, 2013                [Page 6]

Internet-Draft       CDNI Logging for HTTP Delivery            July 2012


   | Session-id | a string          | 6141F5795BE774691D234A0465B9667A |
   |            | generated by the  |                                  |
   |            | delivering CDN    |                                  |
   |            | and unique (to    |                                  |
   |            | the delivering    |                                  |
   |            | CDN) to identify  |                                  |
   |            | the Session. (*)  |                                  |
   +------------+-------------------+----------------------------------+

   (*) The Session-ID value to be included in a log record by the
   delivering CDN is such that:

   o  different per-chunk log records with the same Session-ID value
      must correspond to the same user session (.i.e delivery of same
      content to same enduser at a given point in time)

   o  log records for different chunks of the same user session (.i.e
      delivery of same content to same enduser at a given point in time)
      should be provided with the same session-ID value.  While
      undesirable, there may be situations where the delivering CDN uses
      more than one session-ID value for different per-chunk log records
      of a given session, for example in scenarios of fail-over or load-
      balancing across multiple Surrogates and where the delivering CDN
      does not implement mechanism to synchronize session-IDs across
      Surrogates.


4.  Performance Monitoring

   The CDNI Log fields listed in the previous sections allow monitoring
   of essential delivery performance indicators across the CDN Mesh.
   For example, for a regular HTTP delivery, these fields allow tracking
   of the time taken to serve the requested content, tracking of content
   delivery failures, tracking of partial deliveries and tracking of
   cache hit ratios.  As another example, for HTTP adaptive streaming,
   these fields allow tracking of the presentation (and its fluctuation
   over time) served to the End-User as well as the End-User random
   content access (e.g.  Play/Stop/Seek).

   Subsequent versions of this document may discuss potential additional
   log fields for enhanced performance monitoring.


5.  CDNI Log Encoding and Transport

   Details for CDNI Log encoding and transport will be specified in
   subsequent versions.  WE observe that this is expected to allow
   optional use of common compression techniques (e.g. gzip).  However,



Le Faucheur, et al.     Expires January 14, 2013                [Page 7]

Internet-Draft       CDNI Logging for HTTP Delivery            July 2012


   Section 5.1 identifies the information that is to be included in the
   header of CDNI Logs, Section 5.2 identifies information that is to be
   attached to every CDNI Log record and Section 5.3 identifies the
   information that is to be included in the footer of CDNI Logs.
   Finally, Section 5.4 discusses the notion of customized Logging.

5.1.  CDNI Log Header Information

   The header of CDNI Logs contains the following fields:

   +----------------+-------------------+------------------------------+
   | Field          | Description       | Examples                     |
   +----------------+-------------------+------------------------------+
   | Format-Version | Version of the    | v1.0                         |
   |                | CDNI Log format.  |                              |
   | Log-Field-List | The list of the   | time cs-method cs-uri        |
   |                | fields provided   |                              |
   |                | in the log        |                              |
   |                | records           |                              |
   | Log-ID         | Unique identifier |                              |
   |                | for the CDNI Log  |                              |
   |                | (facilitates      |                              |
   |                | detection of      |                              |
   |                | duplicate Logs    |                              |
   |                | and tracking in   |                              |
   |                | case of           |                              |
   |                | aggregation).     |                              |
   | Log-Timestamp  | Time, in          | [20/Feb/2012:00:29.510+0200] |
   |                | milliseconds, the |                              |
   |                | CDNI Log was      |                              |
   |                | generated.        |                              |
   +----------------+-------------------+------------------------------+

5.2.  CDNI per-Log-Record Information

   In addition to the log fields discussed in previous sections, each
   CDNI log record contains the following fields:

   +----------+---------------------------------------------+----------+
   | Field    | Description                                 | Examples |
   +----------+---------------------------------------------+----------+
   | Log      | Digest of the Log records (facilitates      |          |
   | Record   | recovery of uncorrupted Log records inside  |          |
   | Digest   | a corrupted CDNI Log)                       |          |
   +----------+---------------------------------------------+----------+






Le Faucheur, et al.     Expires January 14, 2013                [Page 8]

Internet-Draft       CDNI Logging for HTTP Delivery            July 2012


5.3.  CDNI Log Footer Information

   The footer of CDNI Logs contains the following fields:

   +---------+----------------------------------------------+----------+
   | Field   | Description                                  | Examples |
   +---------+----------------------------------------------+----------+
   | Log     | Digest of the complete Log (facilitates      |          |
   | Digest  | detection of Log corruption)                 |          |
   +---------+----------------------------------------------+----------+

5.4.  CDNI Customized Log Format

   This document proposes that customized logs be supported by CDNI in
   the following manner:

   o  the uCDN uses the CDNI Metadata interface to indicate to the dCDN
      which subset of the CDNI logging fields are to be provided in a
      log record for corresponding to a request for a given content

   o  the dCDN provides, via the CDNI Logging interface, log records
      containing the subset of CDNI logging fields requested by the
      uCDN.

   o  The dCDN explicitly lists in the CDNI Log Header the fields
      actually provided (as discussed in Section 5.1).


6.  Impact on CDNI Metadata Requirements

   We request that the following requirements be added in section 6 of
   [I-D.ietf-cdni-requirements]:

   "

   META-X [HIGH] The CDNI Metadata Distribution interface shall support
   an OPTIONAL mechanism allowing the Upstream CDN to indicate to the
   Downstream CDN which CDNI Log fields are to be provided for all, for
   specific sets of, or for specific content items delivered using HTTP.
   A CDNI implementation that does not support this optional CDNI
   Metadata Distribution Interface mechanism MUST ignore this log format
   indication and generate CDNI logging format for adaptive streaming
   using the default set of CDNI Logging fields.  [Editor's note: A
   default set of logging fields need to be defined]

   META-X [MID] The CDNI Metadata Distribution interface shall allow the
   uCDN to signal to the dCDN the Content Collection ID value for all,
   for specific sets of, or for specific content items delivered using



Le Faucheur, et al.     Expires January 14, 2013                [Page 9]

Internet-Draft       CDNI Logging for HTTP Delivery            July 2012


   HTTP.  Whenever the dCDN is instructed by the uCDN (using the
   customized logging mechanism described in Section 5.4) to report the
   Content Collection ID field in the log records, the dCDN is to use
   the value provided through the CDNI Metadata interface for the
   corresponding content.

   "


7.  Impact on CDNI Footprint and Capabilities Advertisement

   We request that the following requirement be added in section 5 of
   [I-D.ietf-cdni-requirements]:

   "

   REQ-X [MID] The CDNI Request Routing/Footprint and Advertisement
   Interface shall support advertisement of the following capabilities:

   o  support for customized CDNI Logging

   o  support of Content Collection ID logging

   o  support for Session-ID logging

   "


8.  Generation of CDNI Logs

   Like other CDNI interfaces, the CDNI Logging interface specifies
   operations across CDNs and not inside a CDN.  Therefore, the log
   formats specified in this document apply to CDNI logging information
   exchanged across CDNs and does not constrain the process for
   generating such inter-CDN logs within a given CDN.  The format of the
   logs generated by a given CDN Surrogate is beyond the scope of the
   present document.  We observe that a given CDN could elect to have
   its Surrogates natively generate logs in the same format as the one
   to be used for exchange with another CDN, or that the CDN could elect
   to have its Surrogates generate logs in any other format (as long as
   they include the necessary information) and have these logs then
   reformatted prior to exchange with another CDN.


9.  IANA Considerations

   [This will be specified in subsequent versions].




Le Faucheur, et al.     Expires January 14, 2013               [Page 10]

Internet-Draft       CDNI Logging for HTTP Delivery            July 2012


10.  Security Considerations

   CDNI Logs exchanged over the CDNI Logging interface can be consumed
   by very sensitive applications including inter-CDN accounting and
   billing.  The associated security concerns are discussed in
   [I-D.ietf-cdni-framework].  At this stage, we observe that the CDNI
   Logging interface can leverage the existing security mechanisms
   supported by the underlying transport protocol that will be selected
   for transport of CDNI Logs (e.g. to support authentication of the
   entities exchanging CDNI Logs through the CDNI Logging interface, to
   support privacy and protection against spoofing).  This will be
   further discussed in subsequent versions of this document.


11.  Acknowledgements

   The authors want to thank Gene Halbrooks for his input into this
   document.


12.  References

12.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

12.2.  Informative References

   [I-D.bertrand-cdni-logging]
              Bertrand, G. and S. Emile, "CDNI Logging Interface",
              draft-bertrand-cdni-logging-00 (work in progress),
              February 2012.

   [I-D.brandenburg-cdni-has]
              Brandenburg, R., Deventer, O., Faucheur, F., and K. Leung,
              "Models for adaptive-streaming-aware CDN Interconnection",
              draft-brandenburg-cdni-has-03 (work in progress),
              July 2012.

   [I-D.ietf-cdni-framework]
              Peterson, L. and B. Davie, "Framework for CDN
              Interconnection", draft-ietf-cdni-framework-00 (work in
              progress), April 2012.

   [I-D.ietf-cdni-problem-statement]
              Niven-Jenkins, B., Faucheur, F., and N. Bitar, "Content
              Distribution Network Interconnection (CDNI) Problem



Le Faucheur, et al.     Expires January 14, 2013               [Page 11]

Internet-Draft       CDNI Logging for HTTP Delivery            July 2012


              Statement", draft-ietf-cdni-problem-statement-08 (work in
              progress), June 2012.

   [I-D.ietf-cdni-requirements]
              Leung, K. and Y. Lee, "Content Distribution Network
              Interconnection (CDNI) Requirements",
              draft-ietf-cdni-requirements-03 (work in progress),
              June 2012.

   [I-D.ietf-cdni-use-cases]
              Bertrand, G., Emile, S., Burbridge, T., Eardley, P., Ma,
              K., and G. Watson, "Use Cases for Content Delivery Network
              Interconnection", draft-ietf-cdni-use-cases-09 (work in
              progress), July 2012.


Authors' Addresses

   Francois Le Faucheur
   Cisco Systems
   Greenside, 400 Avenue de Roumanille
   Sophia Antipolis  06410
   France

   Phone: +33 4 97 23 26 19
   Email: flefauch@cisco.com


   Mahesh Viveganandhan
   Cisco Systems
   375 East Tasman Drive
   San Jose  95134
   USA

   Email: mvittal@cisco.com


   Kent Leung
   Cisco Systems
   3625 Cisco Way
   San Jose  95134
   USA

   Phone: +1 408 526 5030
   Email: kleung@cisco.com






Le Faucheur, et al.     Expires January 14, 2013               [Page 12]