Internet DRAFT - draft-finkelman-httpbis-has-redirecting

draft-finkelman-httpbis-has-redirecting







Network Working Group                                       O. Finkelman
Internet-Draft                                                     Qwilt
Intended status: Informational                                  A. Begen
Expires: May 8, 2019                                     Networked Media
                                                        November 4, 2018


               HTTP Redirects in HTTP Adaptive Streaming
               draft-finkelman-httpbis-has-redirecting-00

Abstract

   This document motivates the need for clarifications of client
   behavior in cases of HTTP redirect to a content delivery network
   (CDN) when the redirected object contains relative references.  This
   document focuses on HTTP Adaptive Streaming (HAS) use cases, but it
   might be possible to generalize to other use cases.  The goal of this
   document is to present the current status of things and to explore
   potential solutions.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on May 8, 2019.

Copyright Notice

   Copyright (c) 2018 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must



Finkelman & Begen          Expires May 8, 2019                  [Page 1]

Internet-Draft  HTTP Redirects in HTTP Adaptive Streaming  November 2018


   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Glossary of Terms . . . . . . . . . . . . . . . . . . . . . .   3
   3.  Current State of Things . . . . . . . . . . . . . . . . . . .   3
   4.  Objectives  . . . . . . . . . . . . . . . . . . . . . . . . .   4
   5.  Potential Solutions . . . . . . . . . . . . . . . . . . . . .   4
   6.  Examples and Interpretation of the Current RFCs . . . . . . .   5
     6.1.  Example 1: Absolute URI to Manifest . . . . . . . . . . .   6
     6.2.  Example 2: Relative Reference in an Encapsulating Entity    7
     6.3.  Example 3: Relative Reference with Redirect . . . . . . .   8
   7.  Conclusion  . . . . . . . . . . . . . . . . . . . . . . . . .   8
   8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   9
   9.  Security Considerations . . . . . . . . . . . . . . . . . . .   9
   10. Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .   9
   11. Contributors  . . . . . . . . . . . . . . . . . . . . . . . .   9
   12. References  . . . . . . . . . . . . . . . . . . . . . . . . .   9
     12.1.  Normative References . . . . . . . . . . . . . . . . . .   9
     12.2.  Informative References . . . . . . . . . . . . . . . . .  10
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  10

1.  Introduction

   HTTP redirect, commonly using the "302 Found" response code, is
   widely used in Content Delivery Networks (CDN) for HTTP Adaptive
   Streaming (HAS), specifically for HTTP Live Streaming (HLS) [RFC8216]
   and Dynamic Adaptive Streaming over HTTP (DASH) [MPEG-DASH].  Using
   HTTP redirect rather than delegation by CNAME is a more powerful and
   flexible solution.

   At a high level, the required behavior from a HAS client is that
   after getting an HTTP redirect to a playlist address that takes the
   client to a CDN cache, if the playlist contains relative references
   to the media segments, the client should be 'sticky' and request the
   subsequent media segments directly from the cache without the need
   for repeated redirects, which naturally would add undesirable
   latency.  Unfortunately, in some cases, common misinterpretations of
   the RFC lead to wrong behavior of clients, while in other cases, this
   required behavior contradicts the HTTP RFC.  Thus, we are facing
   issues with streaming clients that behave differently under different
   use cases of HTTP redirects when relative references are in use
   within the playlist.





Finkelman & Begen          Expires May 8, 2019                  [Page 2]

Internet-Draft  HTTP Redirects in HTTP Adaptive Streaming  November 2018


   The Streaming Video Alliance (SVA) and DASH Industry Forum (DASH-IF)
   are attempting to resolve these issues and create clear guidelines
   describing how video players, browsers and streaming applications
   should behave in such HTTP redirect cases.

   A discussion of this use case can also be found in Section 2.2.1 of
   [RFC6983].

   The following sequence explains this step by step:

   1.  The client gets a URI to the playlist file from the origin
       website (content provider).  This URI may point directly at a CDN
       address or be CNAMEd to a CDN node.

   2.  The client requests the playlist file from the CDN.

   3.  The CDN responds with an HTTP redirect, or a series of redirects
       that eventually takes the client to the final cache.

   4.  The client requests the playlist from this final cache.

   5.  The client parses the relative URIs to media segments from the
       playlist.

   6.  The client uses the final cache address to construct the absolute
       URIs for the media segments that point to the final cache.

   7.  The client requests the media segments from the final cache.

   The redirection should still be temporary in the sense that if a
   client does a refresh, or repeatedly fails to get the playlist or the
   media files from the cache address, it can go back to the original
   playlist URI and request it again.

2.  Glossary of Terms

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

3.  Current State of Things

   Currently, there are inconsistencies in the behaviors of HAS clients
   when constructing the URI of a relative referenced object, read from
   another resource, which was redirected.  In some cases the client
   constructs these URI relative to the redirected address, while in



Finkelman & Begen          Expires May 8, 2019                  [Page 3]

Internet-Draft  HTTP Redirects in HTTP Adaptive Streaming  November 2018


   other cases the client constructs it relative to the original URI of
   the playlist before the redirection.

   The result of the latter behavior is that the client returns to the
   original host for each and every request, and then gets redirected
   again to the cache address.  This has a significant negative impact
   on the the user experience (due to increased risk of slower media
   segment downloads).  Further, it causes redundant work in the
   redirecting CDN host.  The reasons for this are, in some cases,
   misinterpretation of the standard and, in other cases, contradictions
   between this requirement and the existing standard.

4.  Objectives

   Our objectives are to document a clear understanding when a client
   SHOULD compute the absolute URI for relative reference taken from a
   playlist using the actual retrieval URI of the playlist, and have a
   specification that leads to a consistent behavior that meets the HAS
   requirement, regardless of how the retrieval URI of the playlist was
   resolved.

5.  Potential Solutions

   The below are high-level initial thoughts in order to get the
   feedback from the HTTPbis working group.  As we see it, the
   alternatives could be one of the following, depending on the
   understanding of the current state of things:

   1.  Have a single consistent interpretation of the existing RFCs
       showing in which cases the behavior should be as required above.

   2.  Draft an extension to Section 5 of [RFC3986] to handle the
       required cases.

   3.  Create a new HTTP 3xx response code that will be specified to
       serve the required use cases.

   4.  Handle it outside the HTTP standard, i.e., in the SVA and DASH-
       IF.

   A solution should take under consideration the more advanced use
   cases of:

   1.  A chain of redirects to get to the final cache.

   2.  Redirect back: What happens if there is a need to redirect the
       client back to the original URI?  How does the client behave




Finkelman & Begen          Expires May 8, 2019                  [Page 4]

Internet-Draft  HTTP Redirects in HTTP Adaptive Streaming  November 2018


       after such redirect of a specific media object?  Is there any
       difference between this case and another forward redirection?

   3.  An absolute URI appearing in the middle of a playlist, between
       relative references (for example, for an advertisement segment)
       should not break the sequence of things, it should be treated as
       an absolute URI and the subsequent segments that are relative
       references should be requested from the cache again.

   4.  Failover: If the cache does not answer, the client should go back
       and request the playlist from the original URI, and not request
       the relative reference object directly as it does not know how to
       construct an alternative URI for it.

   5.  The client MAY, on its own consideration, re-request the playlist
       again from the original URI, and may be redirected again to the
       same cache, or other cache.  In that sense the redirect is
       temporary.

   6.  The solution should handle both the use case in which the
       encapsulating entity was redirected and it includes relative
       references to the media segment, as in HLS (Section 5.1.3 of
       [RFC3986]), and the use case that encapsulating entity includes
       the base URL embedded in it, as in DASH (Section 5.1.1 of
       [RFC3986]), which may lead to the understanding that DASH
       services should avoid using an absolute URI as base URL in the
       Media Presentation Description (MPD), if they wish HTTP redirect
       to CDN to work properly.

6.  Examples and Interpretation of the Current RFCs

   This is our interpretation of the current state of things using HLS
   manifest examples, in each example the manifest URI is resolved
   differently and we observe the implications on the resolution of the
   relative references contained within the manifest file.

   In the first example the master manifest URI is given as an absolute
   URI, while in the second example the master manifest is given itself
   as a relative reference from the encapsulation that provided it, both
   lead to the required behavior.

   In the third example, similar to the second, the manifest is given as
   a relative reference but the request for the manifest is redirected.
   This leads to the undesired behavior of the client going back to the
   original location for every request, requiring redirect for every
   media chunk.





Finkelman & Begen          Expires May 8, 2019                  [Page 5]

Internet-Draft  HTTP Redirects in HTTP Adaptive Streaming  November 2018


6.1.  Example 1: Absolute URI to Manifest

   Let us take a HLS example and assume the player is passed an absolute
   URI for the HLS manifest (e.g., a user has entered the manifest
   address directly into the player):

      https://video.cdn.example.com/vod/movie/master.m3u8

      GET /vod/movie/master.m3u8

      Host: video.cdn.example.com

      HTTP/1.1 302 Found

      Location: https://edge1.dcdn.example.net/vod/movie/master.m3u8

   The client then requests the HLS manifest from the edge cache.

      GET /vod/movie/master.m3u8

      Host: edge1.dcdn.example.net

      HTTP/1.1 200 OK

   The client reads the per-bitrate manifest reference from the master
   manifest.

      #EXT-X-STREAM-INF:BANDWIDTH=2121091,RESOLUTION=640x360,CODECS="mp4
      a.40.2,avc1.4d001f"

      1800K/1800_complete.m3u8

   The player needs to resolve 1800K/1800_complete.m3u8.

   Following [RFC3986]: The m3u8 does not have a base URI embedded
   within it, so, Section 5.1.1 does not apply.  The relative reference
   (and therefore the representation it references) is enclosed in an
   encapsulating entity, so, Section 5.1.2 applies.  Section 5.1.2 says
   'Thus, the default base URI of a representation is the base URI of
   the entity in which the representation is encapsulated.'

   So what is the base URI of the encapsulating entity (/vod/movie/
   master.m3u8)?

   1.  If we follow [RFC3986] again: The master.m3u8 does not have a
       base URI embedded within it, so, Section 5.1.1 does not apply.





Finkelman & Begen          Expires May 8, 2019                  [Page 6]

Internet-Draft  HTTP Redirects in HTTP Adaptive Streaming  November 2018


   2.  The master.m3u8 is not encapsulated in any other entity, so,
       5.1.2 does not apply.  A URI was used to retrieve the m3u8, so,
       Section 5.1.3 applies.

   3.  Section 5.1.3 says 'if a URI was used to retrieve the
       representation, that URI shall be considered the base URI.  Note
       that if the retrieval was the result of a redirected request, the
       last URI used (i.e., the URI that resulted in the actual
       retrieval of the representation) is the base URI.'

   4.  As the retrieval was the result of a redirected request, the
       second sentence applies and the last URI is
       https://edge1.dcdn.example.net/vod/movie/master.m3u8 and so that
       is the base URI for the manifest (the encapsulating entity).

   5.  Therefore the base URI for the relative URI
       1800K/1800_complete.m3u8 is
       https://edge1.dcdn.example.net/vod/movie/master.m3u8 and the
       player should request https://edge1.dcdn.example.net/vod/
       movie/1800K/1800_complete.m3u8

6.2.  Example 2: Relative Reference in an Encapsulating Entity

   Let us take the same example, but instead of the player being passed
   an absolute URI, let us assume the URI is relative and embedded
   within another entity, i.e., a top-level HTML page
   https://video.cdn.example.com/index.html.  In that case /vod/movie/
   master.m3u8 (the encapsulating entity of 1800K/1800_complete.m3u8) is
   itself encapsulated (in /index.html) and Section 5.1.2 does apply
   this time, and the base URI of /vod/movie/master.m3u8 depends on the
   base URI of /index.html

   The base URI of /index.html (which is not encapsulated in another
   entity) follows Section 5.1.3, and depends on how /index.html was
   retrieved.

   1.  If /index.html was retrieved without redirection, the base URI is
       https://video.cdn.example.com/index.html and the base URI for the
       relative URI 1800K/1800_complete.m3u8 is
       https://video.cdn.example.com/vod/movie/master.m3u8 and the
       player should request https://video.cdn.example.com/vod/
       movie/1800K/1800_complete.m3u8

   2.  If /index.html was retrieved via a redirect (e.g., to
       https://edge1.dcdn.example.net/index.html), the base URI is
       https://edge1.dcdn.example.net/index.html, and the base URI for
       the relative URI 1800K/1800_complete.m3u8 is
       https://edge1.dcdn.example.net/vod/movie/master.m3u8 and the



Finkelman & Begen          Expires May 8, 2019                  [Page 7]

Internet-Draft  HTTP Redirects in HTTP Adaptive Streaming  November 2018


       player should request https://edge1.dcdn.example.net/vod/
       movie/1800K/1800_complete.m3u8

6.3.  Example 3: Relative Reference with Redirect

   In this example, as in Example 2 above, the manifest URI is relative
   and embedded within another entity within a top-level HTML page
   https://video.cdn.example.com/index.html.  The base URI of
   /index.html is an absolute URI and the request for /index.html is not
   redirected, but, in this case, the request for /vod/movie/master.m3u8
   is redirected.

   What would be resulting URI for 1800K/1800_complete.m3u8 in this case
   ?

   1.  According to Section 5.1.2 of [RFC3986] 1800K/1800_complete.m3u8
       will get its base URI from the base URI of the encapsulating
       entity which is /vod/movie/master.m3u8.

   2.  As /vod/movie/master.m3u8 is itself encapsulated within
       /index.html then according to Section 5.1.2 its base URI is the
       taken from the base URI of /index.html.  Since this base URI is
       https://video.cdn.example.com/index.html, the base URI of the
       master.m3u8 is resolved to
       https://video.cdn.example.com/vod/movie/master.m3u8, though it
       was redirected and its actual retrieval URI was
       https://edge1.dcdn.example.net/vod/movie/master.m3u8

   3.  The 1800K/1800_complete.m3u8, taking its base URI from the the
       encapsulating master.m3u8, is resolved to
       https://video.cdn.example.com/vod/movie/1800K/1800_complete.m3u8

   One can see that in this case, as Section 5.1.2 does not define what
   happens if the encapsulating entity was redirected, the
   straightforward interpretation leads to the undesired result of the
   1800_complete.m3u8 being requested from the original URI rather than
   the redirection URI.

7.  Conclusion

   The examples above show that the behavior under playlist redirect
   depends on how the playlist URI was resolved, and whether the
   playlist URI was an absolute URI or a relative one.  It also depends
   on how exactly Section 5.1.2 of [RFC3986] is interpreted in the
   absence of clear directions for the case that the encapsulating
   entity itself was a relative reference which was redirected.





Finkelman & Begen          Expires May 8, 2019                  [Page 8]

Internet-Draft  HTTP Redirects in HTTP Adaptive Streaming  November 2018


   As explained above, we would like to have a single clear
   specification defining a consistent client behavior that is 'sticky'
   after a playlist redirect, regardless of how the playlist URI was
   resolved.  We, therefore, hope that the members of HTTPbis working
   group could share their views regarding the preferred approach in
   order to resolve this issue.

8.  IANA Considerations

   There are no IANA considerations.

9.  Security Considerations

   TBC.

10.  Acknowledgements

   The authors would like to thank the members of the Open Caching
   working group of the Streaming Video Alliance (SVA) for their
   feedback and especially Ben Niven-Jenkins and Kevin J.  Ma for their
   help in understanding the current state of the standards for the
   provided examples.

11.  Contributors

   TBC.

12.  References

12.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC3986]  Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
              Resource Identifier (URI): Generic Syntax", STD 66,
              RFC 3986, DOI 10.17487/RFC3986, January 2005,
              <https://www.rfc-editor.org/info/rfc3986>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.







Finkelman & Begen          Expires May 8, 2019                  [Page 9]

Internet-Draft  HTTP Redirects in HTTP Adaptive Streaming  November 2018


12.2.  Informative References

   [MPEG-DASH]
              ISO/IEC 23009-1:2014, "Dynamic Adaptive Streaming over
              HTTP-- Part 1: Media Presentation Description and Segment
              Formats", 2014.

   [RFC6983]  van Brandenburg, R., van Deventer, O., Le Faucheur, F.,
              and K. Leung, "Models for HTTP-Adaptive-Streaming-Aware
              Content Distribution Network Interconnection (CDNI)",
              RFC 6983, DOI 10.17487/RFC6983, July 2013,
              <https://www.rfc-editor.org/info/rfc6983>.

   [RFC8216]  Pantos, R., Ed. and W. May, "HTTP Live Streaming",
              RFC 8216, DOI 10.17487/RFC8216, August 2017,
              <https://www.rfc-editor.org/info/rfc8216>.

Authors' Addresses

   Ori Finkelman
   Qwilt
   6, Ha'harash
   Hod HaSharon  4524079
   Israel

   Email: orif@qwilt.com


   Ali Begen
   Networked Media
   Konya
   Turkey

   Email: ali.begen@networked.media

















Finkelman & Begen          Expires May 8, 2019                 [Page 10]