HTTPbis J. Doe Internet-Draft ACME Corporation Intended status: Standards Track M. Mustermann Expires: January 3, 2015 ACME GmbH July 2, 2014 HTTP/2 Extension: Large Header Blocks draft-johndoe-http2-large-header-blocks-00 Abstract This document defines an extension to HTTP/2 for transmitting large header blocks. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on January 3, 2015. Copyright Notice Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Doe & Mustermann Expires January 3, 2015 [Page 1] Internet-Draft CONTINUATION July 2014 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Conventions and Terminology . . . . . . . . . . . . . . . . . 3 3. Additions and Changes to HTTP/2 . . . . . . . . . . . . . . . 3 4. Header Compression and Decompression . . . . . . . . . . . . . 3 5. Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 5.1. CONTINUATION . . . . . . . . . . . . . . . . . . . . . . . 4 5.2. HEADERS . . . . . . . . . . . . . . . . . . . . . . . . . 5 5.3. PUSH_PROMISE . . . . . . . . . . . . . . . . . . . . . . . 6 6. Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 7. HTTP Request/Response Exchange . . . . . . . . . . . . . . . . 7 7.1. Examples . . . . . . . . . . . . . . . . . . . . . . . . . 8 8. Additional Requirements/Considerations . . . . . . . . . . . . 10 9. Security Considerations . . . . . . . . . . . . . . . . . . . 10 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 10.1. Frame Type Registry Update . . . . . . . . . . . . . . . . 11 10.2. Settings Registry Update . . . . . . . . . . . . . . . . . 11 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 11 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11 12.1. Normative References . . . . . . . . . . . . . . . . . . . 11 12.2. Informative References . . . . . . . . . . . . . . . . . . 12 Appendix A. Change Log (to be removed by RFC Editor before publication) . . . . . . . . . . . . . . . . . . . . 12 A.1. Since draft-johndoe-http2-large-header-blocks-00 . . . . . 12 Doe & Mustermann Expires January 3, 2015 [Page 2] Internet-Draft CONTINUATION July 2014 1. Introduction The Hypertext Transfer Protocol version 2 (HTTP/2 [http2]) introduced a framing-based transport layer with multi-stream multiplexing. HTTP/2 also introduced header compression. However, HTTP/2 header blocks are restricted to the size of the payload of a single frame (currently 16383 bytes). This document defines an extension to HTTP/2 for transmitting large header blocks. A new HTTP/2 CONTINUATION frame type is introduced for virtually extending the length of a HEADERS or PUSH_PROMISE frame. Note that special care must be taken to implement this extension. CONTINUATION frames allow a header block larger than one frame to be fragmented into multiple frames, yet taken together, the frames are logically equivalent to a single frame. Since header blocks can modify the shared compression context in HTTP/2, other frames (from any stream) MUST NOT occur between HEADERS frames and any CONTINUATION frames that might follow, or between PUSH_PROMISE frames and any CONTINUATION frames that might follow. 2. Conventions and Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 3. Additions and Changes to HTTP/2 This document defines a new HTTP/2 CONTINUATION frame type. This document defines a new HTTP/2 SETTINGS_ENABLE_CONTINUATION setting. This document adds an END_HEADERS flag to the HEADERS and PUSH_PROMISE frame types. 4. Header Compression and Decompression In HTTP/2, header sets are collections of zero or more header fields serialized into a header block using HTTP Header Compression [COMPRESSION]. However, blocks of headers are restricted to the size of the payload of a single frame (currently 16383 bytes). This document lifts the requirement for header blocks to fit within a single frame. Instead, the serialized header block can be divided into one or more octet sequences, called header block fragments. The Doe & Mustermann Expires January 3, 2015 [Page 3] Internet-Draft CONTINUATION July 2014 fragments are transmitted within the payload of HEADERS (Section 5.2), PUSH_PROMISE (Section 5.3) or CONTINUATION (Section 5.1) frames. A complete header block consists of either: o a single HEADERS or PUSH_PROMISE frame, with the END_HEADERS flag set, or o a HEADERS or PUSH_PROMISE frame with the END_HEADERS flag cleared and one or more CONTINUATION frames, where the last CONTINUATION frame has the END_HEADERS flag set. The last frame in a header block MUST have the END_HEADERS flag set. This allows a header block which contains one or more CONTINUATION frames to be logically equivalent to a single frame. Since HTTP/2 header compression is stateful (using a single compression context for the entire connection), header blocks MUST be transmitted as a contiguous sequence of frames, with no interleaved frames of any other type or from any other stream. Header block fragments can only be sent as the payload of HEADERS, PUSH_PROMISE or CONTINUATION frames. Since these frames carry data that can modify the compression context maintained by a receiver, an endpoint receiving HEADERS, PUSH_PROMISE or CONTINUATION frames MUST reassemble header blocks and perform decompression even if the frames are to be discarded. A receiver MUST terminate the connection with a connection error of type COMPRESSION_ERROR if it does not decompress a header block. A receiving endpoint reassembles a header block by concatenating its fragments, then decompresses the block to reconstruct the header set. Each header block is processed as a discrete unit. 5. Frames This specification introduces a new CONTINUATION and a new flag for the HEADERS and PUSH_PROMISE frame types. 5.1. CONTINUATION The CONTINUATION frame (type=0x9) is used to continue a sequence of header block fragments (Section 4). Any number of CONTINUATION frames can be sent on an existing stream, as long as the preceding frame is on the same stream and is a HEADERS, PUSH_PROMISE or CONTINUATION frame without the END_HEADERS flag set. Doe & Mustermann Expires January 3, 2015 [Page 4] Internet-Draft CONTINUATION July 2014 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Header Block Fragment (*) ... +---------------------------------------------------------------+ CONTINUATION Frame Payload The CONTINUATION frame payload contains a header block fragment (Section 4). The CONTINUATION frame defines the following flag: END_HEADERS (0x4): Bit 3 being set indicates that this frame ends a header block (Section 4). If the END_HEADERS bit is not set, this frame MUST be followed by another CONTINUATION frame. A receiver MUST treat the receipt of any other type of frame or a frame on a different stream as a connection error of type PROTOCOL_ERROR. The CONTINUATION frame changes the connection state as defined in Section 4. CONTINUATION frames MUST be associated with a stream. If a CONTINUATION frame is received whose stream identifier field is 0x0, the recipient MUST respond with a connection error of type PROTOCOL_ERROR. A CONTINUATION frame MUST be preceded by a HEADERS, PUSH_PROMISE or CONTINUATION frame without the END_HEADERS flag set. A recipient that observes violation of this rule MUST respond with a connection error of type PROTOCOL_ERROR. If a CONTINUATION frame size exceeds any defined limit, or is too small to contain mandatory frame data, the receiver MUST terminate the connection with a connection error of type FRAME_SIZE_ERROR. 5.2. HEADERS This document modifies the definition of a HEADERS frame such that it may contain a complete header block or a header block fragment. This document modifies the definition of the END_STREAM flag and defines a new END_HEADERS flag. The other flags remain as defined. Doe & Mustermann Expires January 3, 2015 [Page 5] Internet-Draft CONTINUATION July 2014 END_STREAM (0x1): Bit 1 being set indicates that the header block (Section 4) is the last that the endpoint will send for the identified stream. Setting this flag causes the stream to enter one of "half closed" states. A HEADERS frame that is followed by CONTINUATION frames carries the END_STREAM flag that signals the end of a stream. A CONTINUATION frame cannot be used to terminate a stream. END_HEADERS (0x4): Bit 3 being set indicates that this frame contains an entire header block (Section 4) and is not followed by any CONTINUATION frames. A HEADERS frame without the END_HEADERS flag set MUST be followed by a CONTINUATION frame for the same stream. A receiver MUST treat the receipt of any other type of frame or a frame on a different stream as a connection error of type PROTOCOL_ERROR. The payload of a HEADERS frame contains a header block fragment (Section 4). A header block that does not fit within a HEADERS frame is continued in a CONTINUATION frame (Section 5.1). 5.3. PUSH_PROMISE This document modifies the definition of a PUSH_PROMISE frame such that it may contain a complete header block or a header block fragment. This document defines a new END_HEADERS flag. The other existing flags remain as defined. END_HEADERS (0x4): Bit 3 being set indicates that this frame contains an entire header block (Section 4) and is not followed by any CONTINUATION frames. A PUSH_PROMISE frame without the END_HEADERS flag set MUST be followed by a CONTINUATION frame for the same stream. A receiver MUST treat the receipt of any other type of frame or a frame on a different stream as a connection error of type PROTOCOL_ERROR. The payload of a PUSH_PROMISE frame contains a header block fragment (Section 4). A header block that does not fit within a PUSH_PROMISE frame is continued in a CONTINUATION frame (Section 5.1). The inclusion of a header block fragment (Section 4) in a PUSH_PROMISE frame potentially modifies the state maintained for header compression. Doe & Mustermann Expires January 3, 2015 [Page 6] Internet-Draft CONTINUATION July 2014 6. Settings This document defines the following new setting: SETTINGS_ENABLE_CONTINUATION (TBD): Indicates the sender's ability and willingness to accept CONTINUATION frames. An endpoint MUST NOT send a CONTINUATION frame unless it receives this parameter set to a value of 1. An endpoint that has not set this parameter to 1, but receives a CONTINUATION frame MAY accept it. Since CONTINUATION frames likely contain data that will alter the compression context, recipients should treat unwanted CONTINUATION frames as a connection error of type PROTOCOL_ERROR. The initial value is 0, which indicates an endpoint MUST NOT send CONTINUATION frames. Any value other than 0 or 1 MUST be treated as a connection error of type PROTOCOL_ERROR. 7. HTTP Request/Response Exchange The sequence of frames in an HTTP message (request or response) is updated as follows in consideration of CONTINUATION frames: 1. one HEADERS frame (followed by zero or more CONTINUATION frames) containing the message headers (see [RFC7230], Section 3.2), and 2. zero or more DATA frames containing the message payload (see [RFC7230], Section 3.3), and 3. optionally, one HEADERS frame, followed by zero or more CONTINUATION frames containing the trailer-part, if present (see [RFC7230], Section 4.1.2). The last frame in the sequence bears an END_STREAM flag, noting that a HEADERS frame bearing the END_STREAM flag can be followed by CONTINUATION frames that carry any remaining portions of the header block. Other frames (from any stream) MUST NOT occur between either HEADERS frame and any CONTINUATION frames that might follow. Otherwise, frames MAY be interspersed on the stream between these frames, but those frames do not carry HTTP semantics. In particular, HEADERS frames (and any CONTINUATION frames that follow) other than the first and optional last frames in this sequence do not carry HTTP semantics. Doe & Mustermann Expires January 3, 2015 [Page 7] Internet-Draft CONTINUATION July 2014 Trailing header fields are carried in a header block that also terminates the stream. That is, a sequence starting with a HEADERS frame, followed by zero or more CONTINUATION frames, where the HEADERS frame bears an END_STREAM flag. Header blocks after the first that do not terminate the stream are not part of an HTTP request or response. 7.1. Examples This section shows HTTP/1.1 requests and responses, with illustrations of equivalent HTTP/2 requests and responses. The examples are intentionally similar to the examples in [XXX], but have been modified to show the use of CONTINUATION frames and the END_HEADERS flag on HEADERS frames. An HTTP GET request includes request header fields and no body and is therefore transmitted as a single HEADERS frame, followed by zero or more CONTINUATION frames containing the serialized block of request header fields. The HEADERS frame in the following has both the END_HEADERS and END_STREAM flags set; no CONTINUATION frames are sent: GET /resource HTTP/1.1 HEADERS Host: example.org ==> + END_STREAM Accept: image/jpeg + END_HEADERS :method = GET :scheme = https :path = /resource host = example.org accept = image/jpeg Similarly, a response that includes only response header fields is transmitted as a HEADERS frame (again, followed by zero or more CONTINUATION frames) containing the serialized block of response header fields. HTTP/1.1 304 Not Modified HEADERS ETag: "xyzzy" ==> + END_STREAM Expires: Thu, 23 Jan ... + END_HEADERS :status = 304 etag = "xyzzy" expires = Thu, 23 Jan ... An HTTP POST request that includes request header fields and payload data is transmitted as one HEADERS frame, followed by zero or more CONTINUATION frames containing the request header fields, followed by one or more DATA frames, with the last CONTINUATION (or HEADERS) frame having the END_HEADERS flag set and the final DATA frame having Doe & Mustermann Expires January 3, 2015 [Page 8] Internet-Draft CONTINUATION July 2014 the END_STREAM flag set: POST /resource HTTP/1.1 HEADERS Host: example.org ==> - END_STREAM Content-Type: image/jpeg - END_HEADERS Content-Length: 123 :method = POST :path = /resource {binary data} content-type = image/jpeg CONTINUATION + END_HEADERS host = example.org :scheme = https content-length = 123 DATA + END_STREAM {binary data} Note that data contributing to any given header field could be spread between header block fragments. The allocation of header fields to frames in this example is illustrative only. A response that includes header fields and payload data is transmitted as a HEADERS frame, followed by zero or more CONTINUATION frames, followed by one or more DATA frames, with the last DATA frame in the sequence having the END_STREAM flag set: HTTP/1.1 200 OK HEADERS Content-Type: image/jpeg ==> - END_STREAM Content-Length: 123 + END_HEADERS :status = 200 {binary data} content-type = image/jpeg content-length = 123 DATA + END_STREAM {binary data} Trailing header fields are sent as a header block after both the request or response header block and all the DATA frames have been sent. The HEADERS frame starting the trailers header block has the END_STREAM flag set. Doe & Mustermann Expires January 3, 2015 [Page 9] Internet-Draft CONTINUATION July 2014 HTTP/1.1 200 OK HEADERS Content-Type: image/jpeg ==> - END_STREAM Transfer-Encoding: chunked + END_HEADERS Trailer: Foo :status = 200 content-length = 123 123 content-type = image/jpeg {binary data} trailer = Foo 0 Foo: bar DATA - END_STREAM {binary data} HEADERS + END_STREAM + END_HEADERS foo = bar 8. Additional Requirements/Considerations After sending a GOAWAY frame, the sender can discard frames for streams with identifiers higher than the identified last stream. However, any frames that alter connection state cannot be completely ignored. For instance, HEADERS, PUSH_PROMISE and CONTINUATION frames MUST be minimally processed to ensure the state maintained for header compression is consistent (see Section 4). CONTINUATION frames do not result in state transitions and are effectively part of the HEADERS or PUSH_PROMISE that they follow. PUSH_PROMISE frames can be sent by the server in response to any client-initiated stream, but the stream MUST be in either the "open" or "half closed (remote)" state with respect to the server. PUSH_PROMISE frames are interspersed with the frames that comprise a response, though they cannot be interspersed with HEADERS and CONTINUATION frames that comprise a single header block. 9. Security Considerations A large header block (Section 4) can cause an implementation to commit a large amount of state. In servers and intermediaries, header fields that are critical to routing, such as ":authority", ":path", and ":scheme" are not guaranteed to be present early in the header block. In particular, values that are in the reference set cannot be emitted until the header block ends. This can prevent streaming of the header fields to their ultimate destination, and forces the endpoint to buffer the entire header block. Since there is no hard limit to the size of a header block, Doe & Mustermann Expires January 3, 2015 [Page 10] Internet-Draft CONTINUATION July 2014 an endpoint could be forced to exhaust available memory. A server that receives a larger header block than it is willing to handle can send an HTTP 431 (Request Header Fields Too Large) status code [RFC6585]. A client can discard responses that it cannot process. The header block MUST be processed to ensure a consistent connection state, unless the connection is closed. 10. IANA Considerations This document updates the registries for frame types and settings in the "Hypertext Transfer Protocol (HTTP) 2 Parameters" section. 10.1. Frame Type Registry Update This document updates the "HTTP/2 Frame Type" registry. The entries in the following table are registered by this document. +--------------+------+-------------+ | Frame Type | Code | Section | +--------------+------+-------------+ | CONTINUATION | 0x9 | Section 5.1 | +--------------+------+-------------+ 10.2. Settings Registry Update This document updates the "HTTP/2 Settings" registry. The entries in the following table are registered by this document. +------------------------------+------+-------------+---------------+ | Name | Code | Initial | Specification | | | | Value | | +------------------------------+------+-------------+---------------+ | SETTINGS_ENABLE_CONTINUATION | TBD | 0 | Paragraph 1 | +------------------------------+------+-------------+---------------+ 11. Acknowledgements This document was ported from the HTTP/2 specification [http2] by Keith Shearl Morgan. 12. References 12.1. Normative References [COMPRESSION] Ruellan, H. and R. Peon, "HPACK - Header Compression for HTTP/2", draft-ietf-httpbis-header-compression-08 (work in progress), June 2014. Doe & Mustermann Expires January 3, 2015 [Page 11] Internet-Draft CONTINUATION July 2014 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing", RFC 7230, June 2014. [http2] Belshe, M., Peon, R., and M. Thomson, "Hypertext Transfer Protocol version 2", draft-ietf-httpbis-http2-13 (work in progress), June 2014. 12.2. Informative References [RFC6585] Nottingham, N. and R. Fielding, "Additional HTTP Status Codes", RFC 6585, April 2012. Appendix A. Change Log (to be removed by RFC Editor before publication) A.1. Since draft-johndoe-http2-large-header-blocks-00 None (yet). Authors' Addresses John Doe ACME Corporation Max Mustermann ACME GmbH Doe & Mustermann Expires January 3, 2015 [Page 12]