Internet DRAFT - draft-johndoe-http2-large-header-blocks

draft-johndoe-http2-large-header-blocks






HTTPbis                                                           J. Doe
Internet-Draft                                          ACME Corporation
Intended status: Standards Track                           M. Mustermann
Expires: January 3, 2015                                       ACME GmbH
                                                            July 2, 2014


                 HTTP/2 Extension: Large Header Blocks
               draft-johndoe-http2-large-header-blocks-00

Abstract

   This document defines an extension to HTTP/2 for transmitting large
   header blocks.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on January 3, 2015.

Copyright Notice

   Copyright (c) 2014 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.





Doe & Mustermann         Expires January 3, 2015                [Page 1]

Internet-Draft                CONTINUATION                     July 2014


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Conventions and Terminology  . . . . . . . . . . . . . . . . .  3
   3.  Additions and Changes to HTTP/2  . . . . . . . . . . . . . . .  3
   4.  Header Compression and Decompression . . . . . . . . . . . . .  3
   5.  Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . .  4
     5.1.  CONTINUATION . . . . . . . . . . . . . . . . . . . . . . .  4
     5.2.  HEADERS  . . . . . . . . . . . . . . . . . . . . . . . . .  5
     5.3.  PUSH_PROMISE . . . . . . . . . . . . . . . . . . . . . . .  6
   6.  Settings . . . . . . . . . . . . . . . . . . . . . . . . . . .  7
   7.  HTTP Request/Response Exchange . . . . . . . . . . . . . . . .  7
     7.1.  Examples . . . . . . . . . . . . . . . . . . . . . . . . .  8
   8.  Additional Requirements/Considerations . . . . . . . . . . . . 10
   9.  Security Considerations  . . . . . . . . . . . . . . . . . . . 10
   10. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 11
     10.1. Frame Type Registry Update . . . . . . . . . . . . . . . . 11
     10.2. Settings Registry Update . . . . . . . . . . . . . . . . . 11
   11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 11
   12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11
     12.1. Normative References . . . . . . . . . . . . . . . . . . . 11
     12.2. Informative References . . . . . . . . . . . . . . . . . . 12
   Appendix A.  Change Log (to be removed by RFC Editor before
                publication)  . . . . . . . . . . . . . . . . . . . . 12
     A.1.  Since draft-johndoe-http2-large-header-blocks-00 . . . . . 12


























Doe & Mustermann         Expires January 3, 2015                [Page 2]

Internet-Draft                CONTINUATION                     July 2014


1.  Introduction

   The Hypertext Transfer Protocol version 2 (HTTP/2 [http2]) introduced
   a framing-based transport layer with multi-stream multiplexing.
   HTTP/2 also introduced header compression.  However, HTTP/2 header
   blocks are restricted to the size of the payload of a single frame
   (currently 16383 bytes).

   This document defines an extension to HTTP/2 for transmitting large
   header blocks.  A new HTTP/2 CONTINUATION frame type is introduced
   for virtually extending the length of a HEADERS or PUSH_PROMISE
   frame.

   Note that special care must be taken to implement this extension.
   CONTINUATION frames allow a header block larger than one frame to be
   fragmented into multiple frames, yet taken together, the frames are
   logically equivalent to a single frame.  Since header blocks can
   modify the shared compression context in HTTP/2, other frames (from
   any stream) MUST NOT occur between HEADERS frames and any
   CONTINUATION frames that might follow, or between PUSH_PROMISE frames
   and any CONTINUATION frames that might follow.

2.  Conventions and Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

3.  Additions and Changes to HTTP/2

   This document defines a new HTTP/2 CONTINUATION frame type.

   This document defines a new HTTP/2 SETTINGS_ENABLE_CONTINUATION
   setting.

   This document adds an END_HEADERS flag to the HEADERS and
   PUSH_PROMISE frame types.

4.  Header Compression and Decompression

   In HTTP/2, header sets are collections of zero or more header fields
   serialized into a header block using HTTP Header Compression
   [COMPRESSION].  However, blocks of headers are restricted to the size
   of the payload of a single frame (currently 16383 bytes).

   This document lifts the requirement for header blocks to fit within a
   single frame.  Instead, the serialized header block can be divided
   into one or more octet sequences, called header block fragments.  The



Doe & Mustermann         Expires January 3, 2015                [Page 3]

Internet-Draft                CONTINUATION                     July 2014


   fragments are transmitted within the payload of HEADERS
   (Section 5.2), PUSH_PROMISE (Section 5.3) or CONTINUATION
   (Section 5.1) frames.

   A complete header block consists of either:

   o  a single HEADERS or PUSH_PROMISE frame, with the END_HEADERS flag
      set, or

   o  a HEADERS or PUSH_PROMISE frame with the END_HEADERS flag cleared
      and one or more CONTINUATION frames, where the last CONTINUATION
      frame has the END_HEADERS flag set.

   The last frame in a header block MUST have the END_HEADERS flag set.
   This allows a header block which contains one or more CONTINUATION
   frames to be logically equivalent to a single frame.

   Since HTTP/2 header compression is stateful (using a single
   compression context for the entire connection), header blocks MUST be
   transmitted as a contiguous sequence of frames, with no interleaved
   frames of any other type or from any other stream.

   Header block fragments can only be sent as the payload of HEADERS,
   PUSH_PROMISE or CONTINUATION frames.  Since these frames carry data
   that can modify the compression context maintained by a receiver, an
   endpoint receiving HEADERS, PUSH_PROMISE or CONTINUATION frames MUST
   reassemble header blocks and perform decompression even if the frames
   are to be discarded.  A receiver MUST terminate the connection with a
   connection error of type COMPRESSION_ERROR if it does not decompress
   a header block.

   A receiving endpoint reassembles a header block by concatenating its
   fragments, then decompresses the block to reconstruct the header set.
   Each header block is processed as a discrete unit.

5.  Frames

   This specification introduces a new CONTINUATION and a new flag for
   the HEADERS and PUSH_PROMISE frame types.

5.1.  CONTINUATION

   The CONTINUATION frame (type=0x9) is used to continue a sequence of
   header block fragments (Section 4).  Any number of CONTINUATION
   frames can be sent on an existing stream, as long as the preceding
   frame is on the same stream and is a HEADERS, PUSH_PROMISE or
   CONTINUATION frame without the END_HEADERS flag set.




Doe & Mustermann         Expires January 3, 2015                [Page 4]

Internet-Draft                CONTINUATION                     July 2014


     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                   Header Block Fragment (*)                 ...
    +---------------------------------------------------------------+

                        CONTINUATION Frame Payload

   The CONTINUATION frame payload contains a header block fragment
   (Section 4).

   The CONTINUATION frame defines the following flag:

   END_HEADERS (0x4):  Bit 3 being set indicates that this frame ends a
      header block (Section 4).

      If the END_HEADERS bit is not set, this frame MUST be followed by
      another CONTINUATION frame.  A receiver MUST treat the receipt of
      any other type of frame or a frame on a different stream as a
      connection error of type PROTOCOL_ERROR.

   The CONTINUATION frame changes the connection state as defined in
   Section 4.

   CONTINUATION frames MUST be associated with a stream.  If a
   CONTINUATION frame is received whose stream identifier field is 0x0,
   the recipient MUST respond with a connection error of type
   PROTOCOL_ERROR.

   A CONTINUATION frame MUST be preceded by a HEADERS, PUSH_PROMISE or
   CONTINUATION frame without the END_HEADERS flag set.  A recipient
   that observes violation of this rule MUST respond with a connection
   error of type PROTOCOL_ERROR.

   If a CONTINUATION frame size exceeds any defined limit, or is too
   small to contain mandatory frame data, the receiver MUST terminate
   the connection with a connection error of type FRAME_SIZE_ERROR.

5.2.  HEADERS

   This document modifies the definition of a HEADERS frame such that it
   may contain a complete header block or a header block fragment.

   This document modifies the definition of the END_STREAM flag and
   defines a new END_HEADERS flag.  The other flags remain as defined.






Doe & Mustermann         Expires January 3, 2015                [Page 5]

Internet-Draft                CONTINUATION                     July 2014


   END_STREAM (0x1):  Bit 1 being set indicates that the header block
      (Section 4) is the last that the endpoint will send for the
      identified stream.  Setting this flag causes the stream to enter
      one of "half closed" states.

      A HEADERS frame that is followed by CONTINUATION frames carries
      the END_STREAM flag that signals the end of a stream.  A
      CONTINUATION frame cannot be used to terminate a stream.

   END_HEADERS (0x4):  Bit 3 being set indicates that this frame
      contains an entire header block (Section 4) and is not followed by
      any CONTINUATION frames.

      A HEADERS frame without the END_HEADERS flag set MUST be followed
      by a CONTINUATION frame for the same stream.  A receiver MUST
      treat the receipt of any other type of frame or a frame on a
      different stream as a connection error of type PROTOCOL_ERROR.

   The payload of a HEADERS frame contains a header block fragment
   (Section 4).  A header block that does not fit within a HEADERS frame
   is continued in a CONTINUATION frame (Section 5.1).

5.3.  PUSH_PROMISE

   This document modifies the definition of a PUSH_PROMISE frame such
   that it may contain a complete header block or a header block
   fragment.

   This document defines a new END_HEADERS flag.  The other existing
   flags remain as defined.

   END_HEADERS (0x4):  Bit 3 being set indicates that this frame
      contains an entire header block (Section 4) and is not followed by
      any CONTINUATION frames.

      A PUSH_PROMISE frame without the END_HEADERS flag set MUST be
      followed by a CONTINUATION frame for the same stream.  A receiver
      MUST treat the receipt of any other type of frame or a frame on a
      different stream as a connection error of type PROTOCOL_ERROR.

   The payload of a PUSH_PROMISE frame contains a header block fragment
   (Section 4).  A header block that does not fit within a PUSH_PROMISE
   frame is continued in a CONTINUATION frame (Section 5.1).

   The inclusion of a header block fragment (Section 4) in a
   PUSH_PROMISE frame potentially modifies the state maintained for
   header compression.




Doe & Mustermann         Expires January 3, 2015                [Page 6]

Internet-Draft                CONTINUATION                     July 2014


6.  Settings

   This document defines the following new setting:

   SETTINGS_ENABLE_CONTINUATION (TBD):  Indicates the sender's ability
      and willingness to accept CONTINUATION frames.  An endpoint MUST
      NOT send a CONTINUATION frame unless it receives this parameter
      set to a value of 1.

      An endpoint that has not set this parameter to 1, but receives a
      CONTINUATION frame MAY accept it.  Since CONTINUATION frames
      likely contain data that will alter the compression context,
      recipients should treat unwanted CONTINUATION frames as a
      connection error of type PROTOCOL_ERROR.

      The initial value is 0, which indicates an endpoint MUST NOT send
      CONTINUATION frames.  Any value other than 0 or 1 MUST be treated
      as a connection error of type PROTOCOL_ERROR.

7.  HTTP Request/Response Exchange

   The sequence of frames in an HTTP message (request or response) is
   updated as follows in consideration of CONTINUATION frames:

   1.  one HEADERS frame (followed by zero or more CONTINUATION frames)
       containing the message headers (see [RFC7230], Section 3.2), and

   2.  zero or more DATA frames containing the message payload (see
       [RFC7230], Section 3.3), and

   3.  optionally, one HEADERS frame, followed by zero or more
       CONTINUATION frames containing the trailer-part, if present (see
       [RFC7230], Section 4.1.2).

   The last frame in the sequence bears an END_STREAM flag, noting that
   a HEADERS frame bearing the END_STREAM flag can be followed by
   CONTINUATION frames that carry any remaining portions of the header
   block.

   Other frames (from any stream) MUST NOT occur between either HEADERS
   frame and any CONTINUATION frames that might follow.

   Otherwise, frames MAY be interspersed on the stream between these
   frames, but those frames do not carry HTTP semantics.  In particular,
   HEADERS frames (and any CONTINUATION frames that follow) other than
   the first and optional last frames in this sequence do not carry HTTP
   semantics.




Doe & Mustermann         Expires January 3, 2015                [Page 7]

Internet-Draft                CONTINUATION                     July 2014


   Trailing header fields are carried in a header block that also
   terminates the stream.  That is, a sequence starting with a HEADERS
   frame, followed by zero or more CONTINUATION frames, where the
   HEADERS frame bears an END_STREAM flag.  Header blocks after the
   first that do not terminate the stream are not part of an HTTP
   request or response.

7.1.  Examples

   This section shows HTTP/1.1 requests and responses, with
   illustrations of equivalent HTTP/2 requests and responses.  The
   examples are intentionally similar to the examples in [XXX], but have
   been modified to show the use of CONTINUATION frames and the
   END_HEADERS flag on HEADERS frames.

   An HTTP GET request includes request header fields and no body and is
   therefore transmitted as a single HEADERS frame, followed by zero or
   more CONTINUATION frames containing the serialized block of request
   header fields.  The HEADERS frame in the following has both the
   END_HEADERS and END_STREAM flags set; no CONTINUATION frames are
   sent:

     GET /resource HTTP/1.1           HEADERS
     Host: example.org          ==>     + END_STREAM
     Accept: image/jpeg                 + END_HEADERS
                                          :method = GET
                                          :scheme = https
                                          :path = /resource
                                          host = example.org
                                          accept = image/jpeg

   Similarly, a response that includes only response header fields is
   transmitted as a HEADERS frame (again, followed by zero or more
   CONTINUATION frames) containing the serialized block of response
   header fields.

     HTTP/1.1 304 Not Modified        HEADERS
     ETag: "xyzzy"              ==>     + END_STREAM
     Expires: Thu, 23 Jan ...           + END_HEADERS
                                          :status = 304
                                          etag = "xyzzy"
                                          expires = Thu, 23 Jan ...

   An HTTP POST request that includes request header fields and payload
   data is transmitted as one HEADERS frame, followed by zero or more
   CONTINUATION frames containing the request header fields, followed by
   one or more DATA frames, with the last CONTINUATION (or HEADERS)
   frame having the END_HEADERS flag set and the final DATA frame having



Doe & Mustermann         Expires January 3, 2015                [Page 8]

Internet-Draft                CONTINUATION                     July 2014


   the END_STREAM flag set:

     POST /resource HTTP/1.1          HEADERS
     Host: example.org          ==>     - END_STREAM
     Content-Type: image/jpeg           - END_HEADERS
     Content-Length: 123                  :method = POST
                                          :path = /resource
     {binary data}                        content-type = image/jpeg

                                      CONTINUATION
                                        + END_HEADERS
                                          host = example.org
                                          :scheme = https
                                          content-length = 123

                                      DATA
                                        + END_STREAM
                                      {binary data}

   Note that data contributing to any given header field could be spread
   between header block fragments.  The allocation of header fields to
   frames in this example is illustrative only.

   A response that includes header fields and payload data is
   transmitted as a HEADERS frame, followed by zero or more CONTINUATION
   frames, followed by one or more DATA frames, with the last DATA frame
   in the sequence having the END_STREAM flag set:

     HTTP/1.1 200 OK                  HEADERS
     Content-Type: image/jpeg   ==>     - END_STREAM
     Content-Length: 123                + END_HEADERS
                                          :status = 200
     {binary data}                        content-type = image/jpeg
                                          content-length = 123

                                      DATA
                                        + END_STREAM
                                      {binary data}

   Trailing header fields are sent as a header block after both the
   request or response header block and all the DATA frames have been
   sent.  The HEADERS frame starting the trailers header block has the
   END_STREAM flag set.








Doe & Mustermann         Expires January 3, 2015                [Page 9]

Internet-Draft                CONTINUATION                     July 2014


     HTTP/1.1 200 OK                  HEADERS
     Content-Type: image/jpeg   ==>     - END_STREAM
     Transfer-Encoding: chunked         + END_HEADERS
     Trailer: Foo                         :status = 200
                                          content-length = 123
     123                                  content-type = image/jpeg
     {binary data}                        trailer = Foo
     0
     Foo: bar                         DATA
                                        - END_STREAM
                                      {binary data}

                                      HEADERS
                                        + END_STREAM
                                        + END_HEADERS
                                          foo = bar

8.  Additional Requirements/Considerations

   After sending a GOAWAY frame, the sender can discard frames for
   streams with identifiers higher than the identified last stream.
   However, any frames that alter connection state cannot be completely
   ignored.  For instance, HEADERS, PUSH_PROMISE and CONTINUATION frames
   MUST be minimally processed to ensure the state maintained for header
   compression is consistent (see Section 4).

   CONTINUATION frames do not result in state transitions and are
   effectively part of the HEADERS or PUSH_PROMISE that they follow.

   PUSH_PROMISE frames can be sent by the server in response to any
   client-initiated stream, but the stream MUST be in either the "open"
   or "half closed (remote)" state with respect to the server.
   PUSH_PROMISE frames are interspersed with the frames that comprise a
   response, though they cannot be interspersed with HEADERS and
   CONTINUATION frames that comprise a single header block.

9.  Security Considerations

   A large header block (Section 4) can cause an implementation to
   commit a large amount of state.  In servers and intermediaries,
   header fields that are critical to routing, such as ":authority",
   ":path", and ":scheme" are not guaranteed to be present early in the
   header block.  In particular, values that are in the reference set
   cannot be emitted until the header block ends.

   This can prevent streaming of the header fields to their ultimate
   destination, and forces the endpoint to buffer the entire header
   block.  Since there is no hard limit to the size of a header block,



Doe & Mustermann         Expires January 3, 2015               [Page 10]

Internet-Draft                CONTINUATION                     July 2014


   an endpoint could be forced to exhaust available memory.

   A server that receives a larger header block than it is willing to
   handle can send an HTTP 431 (Request Header Fields Too Large) status
   code [RFC6585].  A client can discard responses that it cannot
   process.  The header block MUST be processed to ensure a consistent
   connection state, unless the connection is closed.

10.  IANA Considerations

   This document updates the registries for frame types and settings in
   the "Hypertext Transfer Protocol (HTTP) 2 Parameters" section.

10.1.  Frame Type Registry Update

   This document updates the "HTTP/2 Frame Type" registry.  The entries
   in the following table are registered by this document.

   +--------------+------+-------------+
   | Frame Type   | Code | Section     |
   +--------------+------+-------------+
   | CONTINUATION | 0x9  | Section 5.1 |
   +--------------+------+-------------+

10.2.  Settings Registry Update

   This document updates the "HTTP/2 Settings" registry.  The entries in
   the following table are registered by this document.

   +------------------------------+------+-------------+---------------+
   | Name                         | Code | Initial     | Specification |
   |                              |      | Value       |               |
   +------------------------------+------+-------------+---------------+
   | SETTINGS_ENABLE_CONTINUATION | TBD  | 0           | Paragraph 1   |
   +------------------------------+------+-------------+---------------+

11.  Acknowledgements

   This document was ported from the HTTP/2 specification [http2] by
   Keith Shearl Morgan.

12.  References

12.1.  Normative References

   [COMPRESSION]  Ruellan, H. and R. Peon, "HPACK - Header Compression
                  for HTTP/2", draft-ietf-httpbis-header-compression-08
                  (work in progress), June 2014.



Doe & Mustermann         Expires January 3, 2015               [Page 11]

Internet-Draft                CONTINUATION                     July 2014


   [RFC2119]      Bradner, S., "Key words for use in RFCs to Indicate
                  Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC7230]      Fielding, R., Ed. and J. Reschke, Ed., "Hypertext
                  Transfer Protocol (HTTP/1.1): Message Syntax and
                  Routing", RFC 7230, June 2014.

   [http2]        Belshe, M., Peon, R., and M. Thomson, "Hypertext
                  Transfer Protocol version 2",
                  draft-ietf-httpbis-http2-13 (work in progress),
                  June 2014.

12.2.  Informative References

   [RFC6585]      Nottingham, N. and R. Fielding, "Additional HTTP
                  Status Codes", RFC 6585, April 2012.

Appendix A.  Change Log (to be removed by RFC Editor before publication)

A.1.  Since draft-johndoe-http2-large-header-blocks-00

   None (yet).

Authors' Addresses

   John Doe
   ACME Corporation


   Max Mustermann
   ACME GmbH




















Doe & Mustermann         Expires January 3, 2015               [Page 12]