Internet DRAFT - draft-schwartz-tls-lb

draft-schwartz-tls-lb







tls                                                          B. Schwartz
Internet-Draft                                                Google LLC
Intended status: Standards Track                        October 31, 2019
Expires: May 3, 2020


                    TLS Metadata for Load Balancers
                        draft-schwartz-tls-lb-02

Abstract

   A load balancer that does not terminate TLS may wish to provide some
   information to the backend server, in addition to forwarding TLS
   data.  This draft proposes a protocol between load balancers and
   backends that enables secure, efficient delivery of TLS with
   additional information.  The need for such a protocol has recently
   become apparent in the context of split mode ESNI.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on May 3, 2020.

Copyright Notice

   Copyright (c) 2019 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of




Schwartz                   Expires May 3, 2020                  [Page 1]

Internet-Draft                   TLS-LB                     October 2019


   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Conventions and Definitions . . . . . . . . . . . . . . . . .   2
   2.  Background  . . . . . . . . . . . . . . . . . . . . . . . . .   2
   3.  Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . .   3
   4.  Overview  . . . . . . . . . . . . . . . . . . . . . . . . . .   4
   5.  Encoding  . . . . . . . . . . . . . . . . . . . . . . . . . .   4
   6.  Defined ProxyExtensions . . . . . . . . . . . . . . . . . . .   6
     6.1.  padding . . . . . . . . . . . . . . . . . . . . . . . . .   6
     6.2.  client_address  . . . . . . . . . . . . . . . . . . . . .   6
     6.3.  destination_address . . . . . . . . . . . . . . . . . . .   6
     6.4.  esni_inner  . . . . . . . . . . . . . . . . . . . . . . .   6
     6.5.  certificate_padding . . . . . . . . . . . . . . . . . . .   7
     6.6.  overload  . . . . . . . . . . . . . . . . . . . . . . . .   7
     6.7.  ratchet . . . . . . . . . . . . . . . . . . . . . . . . .   8
   7.  Protocol wire format  . . . . . . . . . . . . . . . . . . . .   9
   8.  Security considerations . . . . . . . . . . . . . . . . . . .  10
     8.1.  Integrity . . . . . . . . . . . . . . . . . . . . . . . .  10
     8.2.  Confidentiality . . . . . . . . . . . . . . . . . . . . .  10
     8.3.  Fingerprinting  . . . . . . . . . . . . . . . . . . . . .  11
   9.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  11
   10. References  . . . . . . . . . . . . . . . . . . . . . . . . .  11
     10.1.  Normative References . . . . . . . . . . . . . . . . . .  11
     10.2.  Informative References . . . . . . . . . . . . . . . . .  12
   Appendix A.  Acknowledgements . . . . . . . . . . . . . . . . . .  12
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  12

1.  Conventions and Definitions

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

   Data encodings are expressed in the TLS 1.3 presentation language, as
   defined in Section 3 of [TLS13].

2.  Background

   A load balancer is a server or bank of servers that acts as an
   intermediary between the client and a range of backend servers.  As
   the name suggests, a load balancer's primary function is to ensure
   that client traffic is spread evenly across the available backend
   servers.  However load balancers also serve many other functions,



Schwartz                   Expires May 3, 2020                  [Page 2]

Internet-Draft                   TLS-LB                     October 2019


   such as identifying connections intended for different backends and
   forwarding them appropriately, or dropping connections that are
   deemed malicious.

   A load balancer operates at a specific point in the protocol stack,
   forwarding e.g.  IP packets, TCP streams, TLS contents, HTTP
   requests, etc.  Most relevant to this proposal are TCP and TLS load
   balancers.  TCP load balancers terminate the TCP connection with the
   client and establish a new TCP connection to the selected backend,
   bidirectionally copying the TCP contents between these two
   connections.  TLS load balancers additionally terminate the TLS
   connection, forwarding the plaintext to the backend server (typically
   inside a new TLS connection).  TLS load balancers must therefore hold
   the private keys for the domains they serve.

   When a TCP load balancer forwards a TLS stream, the load balancer has
   no way to incorporate additional information into the stream.
   Insertion of any additional data would cause the connection to fail.
   However, the load-balancer and backend can share additional
   information if they agree to speak a new protocol.  The most popular
   protocol used for this purpose is currently the PROXY protocol
   [PROXY], developed by HAPROXY.  This protocol prepends a plaintext
   collection of metadata (e.g. client IP address) onto the TCP socket.
   The backend can parse this metadata, then pass the remainder of the
   stream to its TLS library.

   The PROXY protocol is effective and widely used, but it offers no
   confidentiality or integrity protection, and therefore might not be
   suitable when the load balancer and backend communicate over the
   public internet.  It also does not offer a way for the backend to
   reply.

3.  Goals

   o  Enable TCP load balancers to forward metadata to the backend.

   o  Enable backends to reply.

   o  Reduce the need for TLS-terminating load balancers.

   o  Ensure confidentiality and integrity for all forwarded metadata.

   o  Enable split ESNI architectures.

   o  Prove to the backend that the load balancer intended to associate
      this metadata with this connection.

   o  Achieve good CPU and memory efficiency.



Schwartz                   Expires May 3, 2020                  [Page 3]

Internet-Draft                   TLS-LB                     October 2019


   o  Don't impose additional latency.

   o  Support backends that receive a mixture of direct and load-
      balanced TLS.

   o  Enable simple and safe implementation.

4.  Overview

   The proposed protocol supports a two-way exchange between a load
   balancer and a backend server.  It works by prepending information to
   the TLS handshake:

        +-----------+ +-----------+ +-----------+
        | Backend A | | Backend B | | Backend C |
        +-----------+ +-----------+ +-----------+
                          \/   /\
   4. EncryptedProxyData[ \/   /\  3. ClientHello (verbatim)
       got SNI info]      \/   /\  2. EncryptedProxyData[
   5. ServerHello, etc.   \/   /\       SNI="secret.b",
                          \/   /\       client=2, etc.]
                          \/   /\
                     +---------------+
                     | Load balancer |
                     +---------------+
                          \/   /\
   6. ServerHello, etc.   \/   /\  1. ClientHello[
      (verbatim)          \/   /\       ESNI=enc("secret.b")]
                          \/   /\
         +-----------+ +-----------+ +-----------+
         |  Client 1 | |  Client 2 | |  Client 3 |
         +-----------+ +-----------+ +-----------+

                        Figure 1: Data flow diagram

5.  Encoding

   A ProxyExtension is identical in form to a standard TLS Extension
   (Section 4.2 of [TLS13]), with a new identifier space for the
   extension types.

   struct {
     ProxyExtensionType extension_type;
     opaque extension_data<0..2^16-1>;
   } ProxyExtension;

   ProxyExtensions can be sent in an upstream (to the backend) or
   downstream (to the load balancer) direction



Schwartz                   Expires May 3, 2020                  [Page 4]

Internet-Draft                   TLS-LB                     October 2019


   enum {
     upstream(0),
     downstream(1),
     (255)
   } ProxyDataDirection;

   The ProxyData contains a set of ProxyExtensions.

   struct {
     ProxyDataDirection direction;
     ProxyExtension proxy_data<0..2^16-1>;
   } ProxyData;

   The EncryptedProxyData structure contains metadata associated with
   the original ClientHello (Section 4.1.2 of [TLS13]), encrypted with a
   pre-shared key that is configured out of band.

   struct {
     opaque psk_identity<1..2^16-1>;
     opaque nonce<8..2^16-1>
     opaque encrypted_proxy_data<1..2^16-1>;
   } EncryptedProxyData;

   o  "psk_identity": The identity of a PSK previously agreed upon by
      the load balancer and the backend.  Including the PSK identity
      allows for updating the PSK without disruption.

   o  "nonce": Non-repeating initializer for the AEAD.  This prevents an
      attacker from observing whether the same ClientHello is marked
      with different metadata over time.

   o  "encrypted_proxy_data": "AEAD-Encrypt(key, nonce, additional_data,
      plaintext=ProxyData)".  The key and AEAD function are agreed out
      of band and associated with "psk_identity".  The "additional_data"
      is context-dependent.

   When the load balancer receives a ClientHello, it serializes any
   relevant metadata into an upstream ProxyData, then encrypts it with
   the ClientHello as "additional_data" to produce the
   EncryptedProxyData.  The backend's reply is a downstream ProxyData
   struct, also transmitted as an EncryptedProxyData, using the upstream
   EncryptedProxyData as "additional_data".  Recipients in each case
   MUST verify that "ProxyData.direction" has the expected value, and
   discard the connection if it does not.

   The downstream ProxyData SHOULD NOT contain any ProxyExtensionType
   values that were not present in the upstream ProxyData.




Schwartz                   Expires May 3, 2020                  [Page 5]

Internet-Draft                   TLS-LB                     October 2019


6.  Defined ProxyExtensions

   Like a standard TLS Extension, a ProxyExtension is identified by a
   uint16 type number.  Load balancers MUST only include extensions that
   are registered for use in ProxyData.  Backends MUST ignore any
   extensions that they do not recognize.

   There are initially seven type numbers allocated:

   enum {
     padding(0),
     client_address(1),
     destination_address(2),
     esni_inner(3),
     certificate_padding(4),
     overload(5),
     ratchet(6),
     (65535)
   } ProxyExtensionType;

6.1.  padding

   The "padding" extension functions as described in [RFC7685].  It is
   used here to avoid leaking information about the other extensions.
   It can be used in upstream and downstream ProxyData.

6.2.  client_address

   The "client_address" extension functions as described in
   [I-D.kinnear-tls-client-net-address].  It conveys the client IP
   address observed by the load balancer.  Backends that make use of
   this extension SHOULD include an empty "client_address" extension in
   the downstream ProxyData.

6.3.  destination_address

   The "destination_address" extension is identical to the
   "client_address" extension, except that it contains the load
   balancer's server IP address that received this connection.

6.4.  esni_inner

   The "esni_inner" extension is only sent upstream, and can only be
   used if the ClientHello contains the encrypted_server_name extension
   [ESNI].  The "extension_data" is the ClientESNIInner (Section 5.1.1
   of [ESNI]), which contains the true SNI and nonce.  This is useful
   when the load balancer knows the ESNI private key and the backend
   does not, i.e. split mode ESNI.



Schwartz                   Expires May 3, 2020                  [Page 6]

Internet-Draft                   TLS-LB                     October 2019


6.5.  certificate_padding

   The "certificate_padding" extension always contains a single uint32
   value.  The upstream value conveys the padding granularity "G", and
   the downstream value indicates the unpadded size of the Certificate
   struct (Section 4.4.2 of [TLS13]).

   To pad the Handshake message (Section 4 of [TLS13]) containing the
   Certificate struct, the backend SHOULD select the smallest
   "length_of_padding" (Section 5.2 of [TLS13]) such that
   "Handshake.length + length_of_padding" is a multiple of "G".

   The load balancer SHOULD include this extension whenever it sends the
   "esni_inner" extension.

   Padding certificates from many backends to the same length is
   important to avoid revealing which backend is responding to a
   ClientHello.  Load balancer operators SHOULD ensure that no backend
   has a unique certificate size after padding, and MAY set "G" large
   enough to make all responses have equal size.

6.6.  overload

   In the upstream ProxyData, the "overload" extension contains a single
   uint16 indicating the approximate proportion of connections that are
   being routed to this server as a fraction of 65535.  If there is only
   one server, load balancers SHOULD set the value to 65535.

   In the downstream ProxyData, the value is an OverloadValue:

   enum {
     accepted(0),
     overloaded(1),
     rejected(2),
     (255)
   } OverloadState;
   struct {
     OverloadState state;
     uint16 load;
     uint32 ttl;
   } OverloadValue;

   When "OverloadValue.state" is "accepted", the backend is accepting
   connections normally.  The "overloaded" state indicates that the
   backend is accepting this connection, but would prefer not to receive
   additional connections.  A value of "rejected" indicates that the
   backend did not accept this connection.  When sending a "rejected"




Schwartz                   Expires May 3, 2020                  [Page 7]

Internet-Draft                   TLS-LB                     October 2019


   response, the backend SHOULD close the connection without sending a
   ServerHello.

   "OverloadValue.load" indicates the load fraction of the responding
   backend server, with 65535 indicating maximum load.

   The load balancer SHOULD treat this information as valid for
   "OverloadValue.ttl" seconds, or until it receives another
   OverloadValue from that server.

   Load balancers that have multiple available backends for an origin
   SHOULD avoid connecting to servers that are in the "overloaded" or
   "rejected" state.  When a connection is rejected, the load balancer
   MAY retry that connection by sending the ClientHello to a different
   backend server.  When multiple servers are in the "accepted" state,
   the load balancer MAY use "OverloadValue.load" to choose among them.

   When there is a server in an unknown state (i.e. a new server or one
   whose last TTL has expired), the load balancer SHOULD direct at least
   one connection to it, in order to refresh its OverloadState.

   If all servers are in the "overloaded" or "rejected" state, the load
   balancer SHOULD drop the connection.

6.7.  ratchet

   If the backend server is reachable without traversing the load
   balancer, and an adversary can observe packets on the link between
   the load balancer and the backend, then that adversary can execute a
   replay flooding attack, sending the backend server duplicate copies
   of observed EncryptedProxyData and ClientHello.  This attack can
   waste server resources on the Diffie-Hellman operations required to
   process the ClientHello, resulting in denial of service.

   The "ratchet" extension reduces the impact of such an attack on the
   backend server by allowing the backend to reject these duplicates
   after decrypting the ProxyData.  (This decryption uses only a
   symmetric cipher, so it is expected to be much faster than typical
   Diffie-Hellman operations.)  Its upstream payload consists of a
   RatchetValue:

   struct {
     uint64 index;
     uint64 floor;
   } RatchetValue;






Schwartz                   Expires May 3, 2020                  [Page 8]

Internet-Draft                   TLS-LB                     October 2019


   A RatchetValue is scoped to a single backend server and
   "psk_identity".  Within that scope, the load balancer initializes
   "index" to a random value, and executes the following procedure:

   1.  For each new forwarded connection (to the same server under the
       same "psk_identity"), increment "index".

   2.  Set "floor" to the "index" of the earliest connection that has
       not yet been connected or closed.

   The backend server initializes "floor" to the first
   "RatchetValue.floor" it receives (under a "psk_identity"), and then
   executes the following procedure for each incoming connection:

   1.  Define "a >= b" if the most significant bit of "a - b" is 0.

   2.  Let "newValue" be the RatchetValue in the ProxyData.

   3.  If "newValue.index < floor", ignore the connection.

   4.  If "newValue.floor >= floor", set "floor" to "newValue.floor".

   5.  OPTIONALLY, ignore the connection if "newValue.index" has been
       seen recently.  This can be implemented efficiently by keeping
       track of any "index" values greater than "floor" that appear to
       have been skipped.

   With these measures in place, replays can be rejected without
   processing the ClientHello.

   In principle, this replay protection fails after 2^64 connections
   when the "floor" value wraps.  On a backend server that averages 10^9
   new connections per second, this would occur after 584 years.  To
   avoid this replay attack, load balancers and backends SHOULD
   establish a new PSK at least this often.

   Backends that are making use of the "ratchet" extension SHOULD
   include an empty "ratchet" extension in their downstream ProxyData.

7.  Protocol wire format

   When forwarding a TLS stream over TCP, the load balancer SHOULD
   prepend a TLSPlaintext whose "content_type" is XX (proxy_header) and
   whose "fragment" is the EncryptedProxyData.

   Following this proxy header, the load balancer MUST send the full
   contents of the TCP stream, exactly as received from the client.  The
   backend will observe the proxy header, immediately followed by a



Schwartz                   Expires May 3, 2020                  [Page 9]

Internet-Draft                   TLS-LB                     October 2019


   TLSPlaintext containing the ClientHello.  The backend will decrypt
   the EncryptedProxyData using the ClientHello as associated data, and
   process the ClientHello and the remainder of the stream as standard
   TLS.

   Similarly, the backend SHOULD reply with the downstream
   EncryptedProxyData in a proxy header, followed by the normal TLS
   stream, beginning with a TLSPlaintext frame containing the
   ServerHello.  If the downstream ProxyHeader is not present, has an
   unrecognized version number, or produces an error, the load balancer
   SHOULD proxy the rest of the stream regardless.

8.  Security considerations

8.1.  Integrity

   This protocol is intended to provide both parties with a strong
   guarantee of integrity for the metadata they receive.  For example,
   an active attacker cannot take metadata intended for one stream and
   attach it to another, because each stream will have a unique
   ClientHello, and the metadata is bound to the ClientHello by AEAD.

   One exception to this protection is in the case of an attacker who
   deliberately reissues identical ClientHello messages.  An attacker
   who reuses a ClientHello can also reuse the metadata associated with
   it, if they can first observe the EncryptedProxyData transferred
   between the load balancer and the backend.  This could be used by an
   attacker to reissue data originally generated by a true client (e.g.
   as part of a 0-RTT replay attack), or it could be used by a group of
   adversaries who are willing to share a single set of client secrets
   while initiating different sessions, in order to reuse metadata that
   they find helpful.

   Backends that are sensitive to this attack SHOULD implement the
   "ratchet" mechanism in Section 6.7, including the optional defenses.

8.2.  Confidentiality

   This protocol is intended to maintain confidentiality of the metadata
   transferred between the load balancer and backend, especially the
   ESNI plaintext and the client IP address.  An observer between the
   client and the load balancer does not observe this protocol at all,
   and an observer between the load balancer and backend observes only
   ciphertext.

   However, an adversary who can monitor both of these links can easily
   observe that a connection from the client to the load balancer is
   shortly followed by a connection from the load balancer to a backend,



Schwartz                   Expires May 3, 2020                 [Page 10]

Internet-Draft                   TLS-LB                     October 2019


   with the same ClientHello.  This reveals which backend server the
   client intended to visit.  In many cases, the choice of backend
   server could be the sensitive information that ESNI is intended to
   protect.

8.3.  Fingerprinting

   Connections to different domains might be distinguishable by the
   cleartext contents of the ServerHello, such as "cipher_suite" and
   "server_share.group".  Load balancer operators with ESNI support
   SHOULD provide backend operators with a list of cipher suites and
   groups to support, and a preference order, to avoid different
   backends having distinctive behaviors.

9.  IANA Considerations

   IANA will be directed to add the following allocation to the TLS
   ContentType registry:

            +-------+--------------+---------+---------------+
            | Value | Description  | DTLS-OK | Reference     |
            +-------+--------------+---------+---------------+
            | XX    | proxy_header | N       | This document |
            +-------+--------------+---------+---------------+

   IANA will be directed to create a new "TLS ProxyExtensionType Values"
   registry on the TLS Extensions page.  Values less than 0x8000 will be
   subject to the "RFC Required" registration procedure, and the rest
   will be "First Come First Served".  To avoid codepoint exhaustion,
   proxy developers SHOULD pack all their nonstandard information into a
   single ProxyExtension.

10.  References

10.1.  Normative References

   [ESNI]     Rescorla, E., Oku, K., Sullivan, N., and C. Wood,
              "Encrypted Server Name Indication for TLS 1.3", draft-
              ietf-tls-esni-04 (work in progress), July 2019.

   [I-D.kinnear-tls-client-net-address]
              Kinnear, E., Pauly, T., and C. Wood, "TLS Client Network
              Address Extension", draft-kinnear-tls-client-net-
              address-00 (work in progress), March 2019.







Schwartz                   Expires May 3, 2020                 [Page 11]

Internet-Draft                   TLS-LB                     October 2019


   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC7685]  Langley, A., "A Transport Layer Security (TLS) ClientHello
              Padding Extension", RFC 7685, DOI 10.17487/RFC7685,
              October 2015, <https://www.rfc-editor.org/info/rfc7685>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

   [TLS13]    Rescorla, E., "The Transport Layer Security (TLS) Protocol
              Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018,
              <https://www.rfc-editor.org/info/rfc8446>.

10.2.  Informative References

   [PROXY]    Tarreau, W., "The PROXY protocol", March 2017,
              <https://www.haproxy.org/download/1.8/doc/proxy-
              protocol.txt>.

Appendix A.  Acknowledgements

   This is an elaboration of an idea proposed by Eric Rescorla during
   the development of ESNI.  Thanks to David Schinazi, David Benjamin,
   and Piotr Sikora for suggesting important improvements.

Author's Address

   Benjamin M. Schwartz
   Google LLC

   Email: bemasc@google.com
















Schwartz                   Expires May 3, 2020                 [Page 12]