| Network Working Group | O. Finkelman |
| Internet-Draft | Qwilt |
| Intended status: Standards Track | S. Mishra |
| Expires: May 3, 2018 | Verizon |
| October 30, 2017 |
CDNI SVA Extensions
draft-finkelman-cdni-sva-extensions-00
The Open Caching working group of the Streaming Video Alliance is focused on the delegation of video delivery request from commercial CDNs to a caching layer at the ISP. In that aspect, Open Caching is a specific use case of CDNI, where the commercial CDN is the upstream CDN (uCDN) and the ISP caching layer is the downstream CDN (dCDN).
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on May 3, 2018.
Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
In this document, we describe the different use cases of Open Caching and the interface and functionality extensions they require, compared to the existing CDNI RFCs. For consistency, this document follows the CDNI notation of uCDN (the commercial CDN) and dCDN (the ISP caching layer). When using the term CP in this document we refer to a video content provider.
The CDNI Logging interface is described in [RFC7937].
The CDNI metadata interface is described in [RFC8006].
The CDNI footprint and capability interface is described in [RFC8008].
The CDNI control interface / triggers is described in [RFC8007].
This document reuses the terminology defined in [RFC6707], [RFC8006], [RFC8007], and [RFC8008].
Additionally, the following terms are used throughout this document and are defined as follows:
This section lists extensions required by request routing features.
Open Caching uses iterative request redirect as defined in [RFC7336]. In order for the uCDN to redirect to the dCDN it requires a request router address. CDNI RFCs do not specify how the request router address is advertised and suggests it may be passed via a bootstrap protocol / interface, which is currently not defined.
We propose to add the request router address as a capability under the Footprint and Capabilities interface.
Example FCI.RequestRouterAddress object:
{
"capabilities": [
{
"capability-type": "FCI.RequestRouterAddress",
"capability-value": {
"address": <endpoint object>
},
"footprints": [
<Footprint objects>
]
}
]
}
Open Caching requires that the uCDN should provide a fallback address to the dCDN to be used in cases where the dCDN cannot properly handle the request. To avoid redirect loops, the dCDN would redirect the request back to the uCDN but to a different location than the original uCDN address, the uCDN will not redirect requests coming to that other address.
Example MI.FallbackAddress object:
{
"generic-metadata-type": "MI.FallbackAddress",
"generic-metadata-value":
{
"sources": [
{
"endpoints": [
"fallback-a.service123.ucdn.example",
"fallback-b.service123.ucdn.example"
],
"protocol": "http/1.1"
},
{
"endpoints": ["origin.service123.example"],
"protocol": "http/1.1"
}
]
}
}
Open Caching uses the CDNI CI/T [RFC8007] as an interface for content management operations. The basic operations are the ones defined in the RFC (i.e. purge, invalidate, pre-position).
RFC8007 provides means to match on full content URL or patterns with wildcards. The Open Caching working group proposes to add two more match rule types.
Using regexp one can create more complex rules to match on objects for the cases of invalidation and purge.
Name: content.regexs
Description: Regexs of content the CI/T Trigger Command
applies to.
Value: A JSON array of Regexs represented as JSON strings.
Mandatory: No, but at least one of "metadata.*", "content.*"
or "playlist.urls" MUST be present and non-empty.
Using video playlist files, one can trigger an operation that will work on a collection of distinct media files in a representation that is natural for the content provider. A playlist may have several formats, specifically HLS *.m3u8 manifest [RFC8216], MSS *.ismc client manifest, and DASH XML MPD file [ISO/IEC 23009-1:2014].
Name: playlist.urls
Description: URLs of video playlist the CI/T Trigger Command
applies to.
Value: A JSON array of Regexs represented as JSON strings.
Mandatory: No, but at least one of "metadata.*", "content.*"
or "playlist.urls" MUST be present and non-empty.
A content operation may apply for a specific geographical region, or need to be excluded from a specific region. In this case, the trigger should be applied only to parts of the network that are included or not excluded by the geo limit. Note that the limit here is on the cache location rather than client location.
Example of trigger specification with a geo limit:
POST /triggers HTTP/1.1
User-Agent: example-user-agent/0.1
Host: dcdn.example.com
Accept: */*
Content-Type: application/cdni; ptype=ci-trigger-command
Content-Length: 352
{
"trigger": {
"type": "preposition",
"content.urls": [
"https://www.example.com/a/b/c/1",
"https://www.example.com/a/b/c/2"
]
},
"locations": [
{
"action": "allow" / "deny",
"footprints": [
{
"footprint-type": "countrycode",
"footprint-value": ["us"]
}
]
}
],
"cdn-path": [ "AS64496:1" ]
}
A uCDN may wish to perform content management operation on the dCDN with a defined local time schedule.
Example of trigger specification with a schedule limit:
POST /triggers HTTP/1.1
User-Agent: example-user-agent/0.1
Host: dcdn.example.com
Accept: */*
Content-Type: application/cdni; ptype=ci-trigger-command
Content-Length: 352
{
"trigger": {
"type": "preposition",
"content.urls": [
"https://www.example.com/a/b/c/1",
"https://www.example.com/a/b/c/2"
]
},
"time-windows": [
{
"time-type": "local" / "UTC",
"start": "<seconds since UNIX epoch>",
"end": "<seconds since UNIX epoch>"
}
],
"cdn-path": [ "AS64496:1" ]
}
There are cases in which some new data has to pass in the trigger which was not thought of in advance. We propose the add a mechanism to the trigger spec which will be similar to the MI generic metadata, allowing parties to easily add more information, that can later be standardized if required.
Example of trigger extension:
POST /triggers HTTP/1.1
User-Agent: example-user-agent/0.1
Host: dcdn.example.com
Accept: */*
Content-Type: application/cdni; ptype=ci-trigger-command
Content-Length: 352
{
"trigger": {
"type": "purge",
"content.patterns": [
"https://www.example.com/*"
]
},
"generic-trigger-spec-type": <type-name>,
"generic-trigger-spec-value":
{
<properties of this object>
}
}
The capabilities added to the triggers interface are not mandatory to support and are, therefore, best negotiated via the FCI.
Different CDNs and Content Providers apply different access control and authentication of user requests. It is not feasible for a dCDN, or ISP cache layer, to implement every scheme a uCDN may have thought of, and, unfortunately, it is not reasonable to expect that uCDNs and CPs will move from their current implementation to a new standard, any time soon. In some cases, existing implementation also include secrets under NDA; sharing them with a third party dCDN is unlikely to happen. Therefore, we aim to look for a solid, generic solution that keeps the access control, authentication and authorization logic in the origin/uCDN.
The following diagram gives a high level sequence view of the URI signing use case.
+------+ +------+ +------+ +-----+
|Client| |dCDN | |uCDN | | CP |
| | | | | | | |
+---+--+ +---+--+ +---+--+ +--+--+
| | | |
+----------------+ | | |
|Access video on | | | |
|CP web site | | | |
+-------+--------+ | | |
| Get master manifest location | |
+-----------------+----------------------+----------------->
| |Respond with signed URI to manifest |
<----------------------------------------+-----------------+
| Get manifest | | |
+----------------------------------------> |
| | | |
| | +-------+----------+ |
| | |Verify URI signing| |
| | +-------+----------+ |
| | Redirect to dCDN | |
<----------------------------------------+ |
| Get manifest | | |
+-----------------> | |
| |Authenticate URI | |
| +----------------------> |
| |Authentication success| |
| Master manifest <----------------------+ |
<-----------------+ | |
| Get sub manifest| | |
+-----------------> | |
| |Authenticate URI | |
| +----------------------> |
| |Authentication success| |
| <----------------------+ |
| | | |
| +------------------+ | |
| |Save authenticated| | |
| |session token | | |
| +--------+---------+ | |
| Sub manifest | | |
<-----------------+ | |
| Request chunk 1 | | |
+-----------------> | |
| | | |
| +---------------------+ | |
| | Use session state to| | |
| | authenticate client | | |
| | chunk requests | | |
| +----------+----------+ | |
| chunk 1 | | |
<-----------------+ | |
|-Request chunk 2-> | |
<------chunk 2----| | |
|-Request chunk 3-> | |
<------chunk 3----+ | |
| | | |
+ + + +
Figure 1
CORS (Cross Origin Resource Sharing) is a mechanism designed to allow a resource from domain A to access other resources in domain B, overriding the same-origin policy. When a uCDN delegate traffic to a dCDN (or ISP) the dCDN is required to comply with the same CORS server behavior the uCDN would have had. For example, if a resource from domain A is accessible for request coming from a resource domain B, but not accessible to requests coming from a resource of domain C, the same logic must be done by the dCDN.
Though CORS can possibly be handled by simply echoing the Origin header value, or *, back to the client, in some cases it is not sufficient, and it also breaks the concept of CORS as an access control mechanism. As proper CORS handling is not possible without a delegation scheme, the Open Caching working group sees it as an essential part of inter-CDN delegation, and therefore propose to adopt it under CDNI and draft it for CDNI RFC.
The following diagram gives a high level sequence view of CORS delegation from uCDN to dCDN using the CORS caching alternative.
+------+ +------+ +------+ +-----+
+-|Client| |dCDN | | uCDN | | CP |
| |1 | | | | B | | A |
|2+---+--+ +---+--+ +---+--+ +--+--+
+--+--|+ | | |
+-------+-------------+ | | |
|Access resource on CP| | | |
|www.example.com | | | |
+-------+-------------+ | | |
| | Get resource A from example.com | |
| +-----------------+----------------------+----------------->
| | CP resource A points to a resource B on uCDN cdn.com |
| <----------------------------------------+-----------------+
| | Get B from uCDN ucdn.com | |
| | Origin: example.com | |
| +----------------------------------------> |
| | | | |
| | | +----------+-----------+ |
| | | |uCDN Delegate to dCDN | |
| | | +----------+-----------+ |
| | | Redirect to dCDN | |
| <----------------------------------------+ |
| |Get B from dCDN | | |
| |Origin: example.com | |
| +-----------------> | |
| | | Request CORS for B | |
| | | Origin: example.com | |
| | +----------------------> |
| | | Provide CORS for B | |
| | | Origin: example.com | |
| | <----------------------+ |
| | +--------+-----------+ | |
| | + cache B CORS rules | | |
| | + Origin: example.com| | |
| | +--------+-----------+ | |
| | Provide B with | | |
| | CORS headers | | |
| <-----------------+ | |
| Get B from uCDN ucdn.com | |
| Origin: example.com | |
|-------------------------------------------> |
| | | | |
| | | +----------+-----------+ |
| | | |uCDN delegate to dCDN | |
| | | +----------+-----------+ |
| | | Redirect to dCDN | |
<-------------------------------------------+ |
| Get B from dCDN | | |
| Origin: example.com| | |
|--------------------> | |
| | +--------+-----------+ | |
| | + use B cached CORS | | |
| | + Origin: example.com| | |
| | +--------+-----------+ | |
| | Provide B with | | |
| | CORS headers | | |
<--------------------+ | |
+ + + + +
Figure 2
In the above simplified example, we depict the caching alternative for CORS solution.
Client 1 accesses resource A on CP domain example.com. Resource A, refers client 1 to resource B on uCDN ucdn.com. Without delegation, at this points uCDN has to resolve CORS and decide if a resource from example.com is allowed to access a resource at ucdn.com. However, once delegated to dCDN, it becomes the dCDNs duty to resolve it for the client request arrives at the dCDN cache. The dCDN sends a CORS request to the uCDN, for resource B with origin example.com, it then uses the response to respond to client 1, and caches the response. When client 2's request arrives at the dCDN, the required CORS information is already in cache and the dCDN can serve client 2 without reiterating to uCDN.
For simplicity, in this diagram, we have ignored some of the challenges of CORS delegation like preflight requests and "null" origin after HTTP redirect.
This section outlines creation of service delivery logs at the dCDN (ISP) and transmittal of the logs by the dCDN to the uCDN. The key motivation for logging outlined below as compared to CDNI Logging Interface [RFC7937] is the ability for dCDN and uCDN to negotiate and agree on a log transport mechanism.
The logging mechanism provides the flexibility for CDNs to leverage common transport mechanism in-use already. Second, the open caching working group has selected Squid based file format given its wide usage within the CDN environments for access and cache logs, result codes and error messages. As an example, the result codes in squid return both the status code returned by downstream as well as result code indicator such as HIT, MISS, REFRESH_HIT, etc. Between the two statuses, it is easier to discern the delivery status. As an example, if the request was forbidden by the origin, the status field will likely be MISS/403 or if it is a cache error response, it will be HIT/503. So, leveraging the Squid log already in use within the CDN environment and, equally important, the ability for CDNs to negotiate and agree on a file transport mechanisms, were the key motivations for open caching. These are therefore proposed as complementary extensions to the CDNI Logging Interface [RFC7937].
The sub-sections below explain extensions to the Footprint and Capabilities [RFC8008] and Metadata Interface [RFC8006]. The specific extension includes FCI announcement of supported log file transport types by dCDN and metadata response by uCDN to provision one or more log file types from the list sent by the dCDN.
The diagram below illustrates the use cases:
Delivery Service A (VOD) Delivery Service C RR Logs
Delivery Service B (Live) (Linear)
+----------------------+ +-----------------+ +-----------------+
| | | | | |
|Log Destination 1(VOD)| |Log Destination 2| |Log Destination 3|
| Logstash | | Kafka | | SFTP |
| | | | | |
+------------^---------+ +----------^------+ +----------^------+
| | |
| | |
+----------------------------------------------+
uCDN |
+----------------------------------------------------------------------+
dCDN |
Delivery Logs +----------+-------+
Service A/B/C | |
+-------+--------+-----> | RR Logs
| | | | dCDN Open Cache <-----------+
| | | | Controller | |
+----+-----+ | | | | |
| | | | +-------^-------+--+ |
| +--+-+----+ | | ^ +----+-----+
| | | | +---+----+ | | Request |
| | +--+---+--+ | | | | Router |
| | | | | +--+--+-+ | |
+-------+ | | | | | +----------+
+------+ Cache | | | Cache |
| | +-----+ |
+---------+ +-------+
Figure 3
Proposal
Delivery logs are created and then transferred from log producing entities at the dCDN premises (mainly caches and Request Router) to log destinations at the uCDN premises. The dCDN may offload logs from these entities to logging at the dCDN premises to facilitate log transfers, or, logs may be transferred directly from log producing entities to uCDN.
Various transport mechanisms may suit the use case of transferring log data, for example SFTP, HTTP upload, Kafka, Logstash or other methods as per the agreement between a dCDN and a uCDN.
In compliance with the CDNI Footprint and Capabilities Interface, and therefore, as per the above use cases, the dCDN is responsible to advertise supported Logging "record-types", as well as Logging "fields" which are marked as optional for the s pecified "record-types" as defined by the CDNI "Logging Capability Object".
The CDNI Logging Capability Object is extended to contain additional properties that hold information on record format, such as fields that should be obfuscated by the dCDN. Note that the uCDN can further control field obfuscation when configuring a logging integration.
During provisioning process the dCDN may reject configuration if a selected record format is not available for a selected Log Integration Type.
This is a proposal of a Logging Capability object that extends the CDNI "FCI.Logging" object.
The following shows an example of Logging Capability object serialization, for a dCDN that supports the optional fields "hostname" and "cache-key", for the "oc_http_request_v1" record type. The "client-address" field is hashed.
In this example, the logging integration types that are supported are named "kafka" and "logstash"
{
"capabilities": [
{
"capability-type": "FCI.Logging",
"capability-value": {
"transport-types": [
"kafka",
"logstash"
],
"record-type": "oc_http_request_v1",
"fields": [
"hostname",
"cache-key"
],
"hash-fields": [
"client-address"
]
},
"footprints": [
<footprint-objects>
]
}
]
}
This is a proposal of Logging Metadata and Transport Metadata objects that comply with the CDNI "Service Metadata" interface
The following shows an example of Logging Configuration MI.Logging Metadata object serialization, for a logging integration that includes the optional field "hostname" in the log record.
{
"metadata": [
{
"generic-metadata-type": "MI.Logging",
"generic-metadata-value": {
"include-fields": [
"hostname"
]
},
"footprints": [
<footprint-objects>
]
}
]
}
An initial set of logging transport types and their respective configuration objects should be defined. More types can be added in the future as needed. The following shows an example of Transport Configuration MI.LoggingTransport Metadata object serialization, for a "kafka" logging integration type.
{
"metadata": [
{
"generic-metadata-type": "MI.LoggingTransport",
"generic-metadata-value": {
"type": [
"kafka",
],
"config":
<kafka-integration-config-object>
]
},
"footprints": [
<footprint-objects>
]
}
]
}
This document requests the registration of the following CDNI Payload Types under the IANA CDNI Payload Type registry [RFC7736]:
| Payload Type | Specification |
|---|---|
| FCI.RequestRouterAddress | RFCthis |
| MI.FallbackAddress | RFCthis |
| MI.Logging | RFCthis |
| MI.LoggingTransport | RFCthis |
[RFC Editor: Please replace RFCthis with the published RFC number for this document.]
Purpose: The purpose of this payload type is to distinguish RequestRouterAddress FCI objects (and any associated capability advertisement)
Interface: FCI
Encoding: see Section 2.1
Purpose: The purpose of this payload type is to distinguish FallbackAddress MI objects (and any associated capability advertisement)
Interface: MI/FCI
Encoding: see Section 2.2
Purpose: The purpose of this payload type is to distinguish Logging MI objects (and any associated capability advertisement)
Interface: MI/FCI
Encoding: see Section 6.2.1
Purpose: The purpose of this payload type is to distinguish LoggingTransport MI objects (and any associated capability advertisement)
Interface: MI/FCI
Encoding: see Section 6.2.2
TBD.
The authors would like to thank Kevin J. Ma for his guidance and support.
The authors would like to thank all members of the SVA's Open Caching Working Group for their contribution in support of this document.
| [RFC7736] | Ma, K., "Content Delivery Network Interconnection (CDNI) Media Type Registration", RFC 7736, DOI 10.17487/RFC7736, December 2015. |