Internet DRAFT - draft-fenter-tls-decryption

draft-fenter-tls-decryption



 



INTERNET-DRAFT                                                 S. Fenter
Intended Status: Informational          Enterprise Data Center Operators
Expires: September 6, 2018                                 March 5, 2018


            Why Enterprises Need Out-of-Band TLS Decryption 
                     draft-fenter-tls-decryption-00


Abstract

   Some enterprises are heavily TLS encrypted within their own
   enterprise network boundaries.  Many of these enterprises are also
   utilizing out-of-band TLS decryption in order to inspect their own
   traffic for purposes of troubleshooting, network security monitoring,
   and for other kinds of monitoring.  These monitoring functions are
   mission critical, and cannot just be done without when TLS 1.3
   (draft-ietf-tls-tls13-26) is released or when the RSA key exchange is
   someday deprecated from TLS 1.2 (RFC5246).  This draft will outline
   the use cases for out-of-band TLS decryption, as well as alternative
   suggestions for monitoring and troubleshooting and the limitations of
   those alternatives.

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as
   Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html





 


Fenter                 Expires September 6, 2018                [Page 1]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


Copyright and License Notice

   Copyright (c) 2018 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document. Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.


































 


Fenter                 Expires September 6, 2018                [Page 2]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  5
   2.  Out-of-Band Decryption Use Cases for Diagnostics and
       Troubleshooting  . . . . . . . . . . . . . . . . . . . . . . .  6
     2.1  Application Performance Monitoring  . . . . . . . . . . . .  6
     2.2  Network Diagnostics and Troubleshooting . . . . . . . . . .  6
       2.2.1  Network Packet Analysis . . . . . . . . . . . . . . . .  6
       2.2.2  Packet Analysis with Source Address Translation . . . .  7
       2.2.3  TCP Pipelining - Session Multiplexing . . . . . . . . .  7
       2.2.4  TLS Encrypted MQ to the mainframe . . . . . . . . . . .  8
       2.2.5  Application Layer Data  . . . . . . . . . . . . . . . .  8
       2.2.6  Customer Experience Monitoring  . . . . . . . . . . . .  9
       2.2.7  VoIP Analysis . . . . . . . . . . . . . . . . . . . . .  9
     2.3  Out-of-Band Decryption Use Cases for Network Security
          Monitoring  . . . . . . . . . . . . . . . . . . . . . . . . 10
       2.3.1  Layer 7 DDoS Attacks  . . . . . . . . . . . . . . . . . 10
       2.3.2  Fraud Monitoring  . . . . . . . . . . . . . . . . . . . 10
       2.3.3  Intrusion Detection System  . . . . . . . . . . . . . . 11
       2.3.4  Threat Detection and Incident Response  . . . . . . . . 11
       2.3.5  Regulatory Examples . . . . . . . . . . . . . . . . . . 11
         2.3.5.1 PCI (Payment Card Industry)  . . . . . . . . . . . . 11
           2.3.5.1.1 PCI and TLS Encryption . . . . . . . . . . . . . 11
           2.3.5.1.2 Intrusion Detection  . . . . . . . . . . . . . . 12
           2.3.5.1.3 TLS 1.2 and PCI  . . . . . . . . . . . . . . . . 12
         2.3.5.2 e-CFR (Electronic Code of Federal Regulations) . . . 12
           2.3.5.2.1 Insider Abuse  . . . . . . . . . . . . . . . . . 12
         2.3.5.3 Regulatory Requirements - Summary  . . . . . . . . . 13
   3.  Alternative Solutions Offered and Their Limitations  . . . . . 13
     3.1  Inline/MITM Decryption  . . . . . . . . . . . . . . . . . . 13
     3.2  Using TCP or UDP Extensions to Supply Extra Information . . 14
     3.3  Using IP and TCP Headers for Monitoring and
          Troubleshooting . . . . . . . . . . . . . . . . . . . . . . 15
     3.4  TLS 1.2 . . . . . . . . . . . . . . . . . . . . . . . . . . 15
     3.5  Logging . . . . . . . . . . . . . . . . . . . . . . . . . . 15
     3.6  Troubleshooting at the Endpoint . . . . . . . . . . . . . . 16
     3.7  Security Monitoring at the Endpoint . . . . . . . . . . . . 16
     3.8  Encrypted Traffic Inspection  . . . . . . . . . . . . . . . 17
     3.9  IPsec instead of TLS  . . . . . . . . . . . . . . . . . . . 17
   4.  An Examination of Arguments Against All Network Decryption . . 18
     4.1  Technical Arguments . . . . . . . . . . . . . . . . . . . . 18
       4.1.1  "I work for a large company and we don't have to 
              decrypt packets." . . . . . . . . . . . . . . . . . . . 18
     4.2  Privacy Arguments . . . . . . . . . . . . . . . . . . . . . 18
       4.2.1  "It's a violation of personal privacy to decrypt TLS 
              traffic anywhere except at the TLS endpoint." . . . . . 18
       4.2.2  "Pervasive Monitoring is an Attack" . . . . . . . . . . 19
     5.  Possible TLS 1.3 Decryption Solutions  . . . . . . . . . . . 19
 


Fenter                 Expires September 6, 2018                [Page 3]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


     5.1  Static Diffie Hellman . . . . . . . . . . . . . . . . . . . 19
     5.2  TLS 1.3 Option for Negotiation of Visibility in the
          Datacenter  . . . . . . . . . . . . . . . . . . . . . . . . 19
     5.3  Solution Summary  . . . . . . . . . . . . . . . . . . . . . 20
   6.  Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 20
   7.  Security Considerations  . . . . . . . . . . . . . . . . . . . 20
   8.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 20
   9.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 21
     9.1  Normative References  . . . . . . . . . . . . . . . . . . . 21
     9.2  Informative References  . . . . . . . . . . . . . . . . . . 21
   Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . . . 21
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 21




































 


Fenter                 Expires September 6, 2018                [Page 4]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


1.  Introduction

   Most enterprise networks originally transmitted packet data in the
   clear inside their internal networks.  Many still do today.  When
   certain enterprises started TLS encrypting their internal networks to
   protect against insider threat and/or for regulatory compliance
   reasons, they always had the option of using RSA key exchanges and
   using static RSA private keys for a small, privileged group to
   decrypt and inspect their traffic out-of-band.  Out-of-band
   decryption provides ubiquitous packet payload visibility inside the
   enterprise that cannot be replaced by inline/MITM decryption
   solutions.  Today there are enterprises with extensive packet broker
   networks who are doing out-of-band TLS decryption to feed network
   sniffers, intrusion detection devices, fraud detection, malware
   detection, application performance monitoring tools, customer
   experience monitoring tools, and other solutions.

   The capability to do out-of-band decryption has been available for
   twenty years, and for the first time in history it will be gone with
   the move to TLS1.3 [TLS13].  A large body of tools has grown up over
   the last twenty years that is dependent on out-of-band decryption. 
   These tools are performing mission critical functions for
   enterprises, and the loss of out-of-band decryption will create major
   operational problems for TLS encrypted enterprises if TLS 1.3 is
   implemented as-is inside the enterprise.  Ubiquitous packet capture
   and decryption are required for enterprise troubleshooting, and
   without this capability there will be high severity outages that
   cannot be solved in an acceptable time frame.  The outcome will be
   the same as extended Denial of Service attacks on enterprises
   worldwide. Without an out-of-band decryption solution, enterprises
   are left with the unattractive option of inline/MITM decryption at
   the data center edge and running traffic with legacy protocols or in
   the clear throughout the data center if they need packet payload
   visibility.  This opens certain enterprises up to significant
   regulatory and insider threat problems. There are reasons why other
   forms of troubleshooting and monitoring do not functionally replace
   the visibility lost from losing out-of-band TLS decryption.  These
   alternative suggestions are discussed in the sections below. 

   TLS 1.2 [RFC5246] is not a long term option for enterprises.  The RSA
   key exchange is gradually being removed by vendors as a TLS 1.2
   option. For example, mobile devices have been seen to send TLS 1.2
   Client Hello's with no RSA key exchange options.  There is also the
   risk that new vulnerabilities and weaknesses will be discovered with
   TLS 1.2 and/or RSA that will accelerate its removal by other vendors.



 


Fenter                 Expires September 6, 2018                [Page 5]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


2.  Out-of-Band Decryption Use Cases for Diagnostics and Troubleshooting

2.1  Application Performance Monitoring

   Network-based Application Performance Monitoring requires TLS
   decryption in order to track application response time by user and by
   URL, which is the information that the application owners and the
   lines of business need.  The end user IP address is obscured by TLS
   termination and by source address translation, both on the Internet
   and within the data center.  The identity of the user can only be
   determined by looking at the payload of the TLS packet.  URL
   identification allows the application support team to do granular,
   code level troubleshooting and performance monitoring at multiple
   tiers of an application.

2.2  Network Diagnostics and Troubleshooting

2.2.1  Network Packet Analysis

   The key to effective network packet troubleshooting is the ability to
   follow a transaction through multiple tiers of an application in
   order to isolate the fault domain.  The extensive use of both
   encryption and source address translation in some enterprises has
   made it difficult, or even impossible to follow a transaction without
   the ability to decrypt TLS and examine the payload of the packet.
   When the payload is available, the packet analyst can find unique
   identifiers (userids, session ids, etc.) as well as the URL the end
   user is accessing in order to identify the correct TCP conversation.

   The packet payload allows analysts to trace the slow or failing
   transaction above and below firewalls, above and below load
   balancers, above and below switches, etc., in order to isolate the
   fault domain.  This kind of analysis is not possible from the
   endpoint alone.  There are firewalls and load balancers that do not
   terminate TLS, and there can be a significant number of
   infrastructure devices in between the TLS endpoints, any one of which
   can be the cause of a problem.  This above/below analysis across all
   intermediate infrastructure devices often provides the only insights
   into where the root problem is introduced.

   It is noteworthy that adding more inline/MITM decryption solutions to
   a multi-tiered application environment would increase the need for
   above/below analysis to rule in or out the inline decryption solution
   itself as the cause of a problem.  The increased complexity of the
   environment would lead to more failures and longer problem resolution
   times.

   Packet payload visibility also allows an analyst to match up a
 


Fenter                 Expires September 6, 2018                [Page 6]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


   network packet trace with log entries in an application log that give
   an indication of the problem that occurred.  This kind of application
   level packet tracing could be between bare metal servers, or between
   two VM's on the same hypervisor.  Inline/MITM TLS decryption
   solutions do not scale for this kind of troubleshooting.  As an
   example, because a problem could hit anywhere, an inline/MITM
   solution would be needed between every possible pair of VM's in a
   virtual environment, which could scale into the thousands.  This
   large number of inline/MITM solutions would also create its own
   problems due to the complexity added to the environment.

2.2.2  Packet Analysis with Source Address Translation

   Content Delivery Networks on the Internet have multiple TLS
   termination points, and the user's source (client) IP is lost. When
   tracing a data center's Internet connection, it is not possible to
   even find the correct TCP conversation for an end user that is having
   a problem without TLS decryption.  The packets must be decrypted and
   the appropriate HTTP header fields examined in order to find the end
   user IP address. 

   Within the data center, the source IP for inbound TLS can again be
   changed multiple times.  If a load balancer does not terminate TLS it
   will NAT the source IP so that return packets will find their way
   back to the load balancer, and to the correct load balancer in a
   pair.  Alternatively, the load balancer may terminate TLS, and start
   an independent TLS session to the next layer below.  Again, the
   source IP becomes the load balancer's IP address, and return packets
   will find their way back to the correct load balancer.

   Reverse proxies, web servers, app servers, and middleware servers can
   all terminate TLS and start independent TLS calls to lower layers,
   each time altering the source (client side) IP of packets, and
   calling a completely different URL.  User sessions are also often
   sprayed randomly by load balancers to all these devices, and the
   network troubleshooter is left with no option except to trace all
   packets at a particular layer, decrypt them all, and look at the
   payload to find a user session.  Servers and infrastructure devices
   typically don't have the horsepower to trace and decrypt all packets
   like this, but an out-of-band packet broker/sniffer infrastructure is
   designed to handle this load, and also provides a centralized
   location for managing and securing capture files.

2.2.3  TCP Pipelining - Session Multiplexing

   When TCP Pipelining/Session Multiplexing is used, multiple end user
   sessions share the same TCP connection.  For the network
   troubleshooter, even if he/she could find the correct encrypted TCP
 


Fenter                 Expires September 6, 2018                [Page 7]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


   connection for an end user, there is no way to tell which packet
   belongs to which end user without decryption.

2.2.4  TLS Encrypted MQ to the mainframe

   MQ requests to the mainframe are farmed out to multiple processing
   nodes, and the MQ response comes back asynchronously from any one of
   those processing nodes.  There is no way to find the response to a
   particular request without looking at identifiers in the payload of
   the packet.  This requires TLS decryption.  MQ requests can also pass
   through many infrastructure devices (i.e. load balancers, firewalls,
   etc.) before reaching the mainframe.  Inline/MITM decryption
   solutions are not scalable for this environment, because visibility
   could be needed anywhere along the path.

2.2.5  Application Layer Data

   A decrypted TLS packet contains a wealth of critical troubleshooting
   information for HTTP (e.g. HTTP requests and return codes) as well as
   for a number of other protocols.  Without this level of information,
   network troubleshooters are blind, unless the problem is some kind of
   a basic network problem.  Even in the case of network problems, the
   application level detail is sometimes critical for isolating
   problems, for example, in the case of an intermittent network
   slowdown or failure.  When looking through millions of packets,
   transactional/application level detail can help the analyst zero in
   on the correct location in the trace where a network problem is
   occurring.

   It is not enough, though, to look only at the HTTP headers.
   Applications have been known, for example, to return an HTTP 200 OK,
   yet contain an error message in the payload of the HTTP response. 
   This can only be seen in the decrypted application layer payload of
   the packet.

   Applications also use XML or JSON structures in the payload of the
   packet to store interesting information like user ids and session
   IDs.  Oftentimes this is the level of information that the
   application support teams possess ("I sent out a request with this
   session ID and didn't get a response.").  The application team
   doesn't, however, know where their request went wrong among the many
   layers of infrastructure, network connections, etc., that their
   request passes through.  The network packet troubleshooters are able
   to follow the transaction through the many layers of infrastructure
   if they are able to access the packet payload and find a matching
   identifier.


 


Fenter                 Expires September 6, 2018                [Page 8]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


2.2.6  Customer Experience Monitoring

   Enterprises involved in online commerce have a business need to
   monitor customer behavior on their web sites.  This monitoring
   requires TLS decryption of the full packet payload, as any location
   on the web page can be of interest to the user and to those
   monitoring user behavior.

2.2.7  VoIP Analysis

   When attempting to monitor and/or troubleshoot user experience within
   voice and video communications, the ability to understand the
   signaling (session setup and teardown) is absolutely critical.  
   Session Initiation Protocol (SIP) is among the most commonly used
   control protocols in the VoIP environment, and increasingly it is
   being encrypted with TLS.

   SIP request and response codes, when visible, make it possible for
   even a novice user to understand the basics of what is being
   requested and what type of response is being provided (Request:
   INVITE, Response: 200 OK). The detail available in SIP messages
   enable a level of analysis difficult to mirror using any other
   method.  The SIP/RTP stream must be decrypted in order to allow
   analysis of these setup messages.  In fact, it is not possible to
   even find the UDP connections to analyze without decryption of the
   SIP header, because the IP/port pair is found in an SDP of the SIP
   signaling packets.

   Phone endpoints are typically not designed for detailed
   troubleshooting.  Many handsets do not have the ability to output SIP
   signaling information.  Endpoints are also not completely trustworthy
   in a troubleshooting scenario, and network analysis is needed to
   verify what is happening on the wire.  VoIP calls can also be
   affected by network conditions.  Tracing may be done in different
   locations to identify the effect the network is having on the VOIP
   call.  An inline/MITM solution doesn't scale for this use case.

   Session Border Controllers can trace SIP signaling, but tracing is
   often too resource intensive to run on these devices, as they are not
   designed to handle the extra load.  This means that VOIP analysts
   need out of band packet capture and decryption solutions which are
   designed for this purpose.

   Call quality on the audio RTP stream can be monitored with network
   based tools, if TLS on the audio stream can be decrypted.  This gives
   the VoIP analyst a view of the problem that they can't get from the
   endpoints.  The audio stream is peer-to-peer communication
   necessitating many visibility points.  Again, an inline/MITM
 


Fenter                 Expires September 6, 2018                [Page 9]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


   decryption strategy doesn't scale.

2.3  Out-of-Band Decryption Use Cases for Network Security Monitoring

2.3.1  Layer 7 DDoS Attacks

   Layer 7 DDoS attacks can involve multiple IPs and source-ports
   generating traffic very similar to that of genuine users. The only
   way to identify layer 7 attack traffic is via inspecting fields in
   the packet payload, which are invisible until the packet is
   decrypted.  

   Internet based DDoS protection services are not perfect.  If they are
   tuned too tightly they block some legitimate production traffic. If
   they are tuned too loosely, some attack traffic gets through. In
   reality, during a DDoS attack some attack traffic usually gets
   through, and enterprises have to be armed to protect themselves. One
   of the tools they need is the ability to decrypt Internet TLS traffic
   so they can block layer 7 DDoS attacks.

   This decryption could be done by an inline/MITM solution, although
   there is a possibility that an inline decryption solution could be
   overwhelmed by a volumetric DDoS attack or by an attack targeting
   session state, becoming a point of failure.  An out-of-band or
   transparent TLS decryption solution does not carry this risk of being
   overwhelmed and blocking all legitimate traffic.

2.3.2  Fraud Monitoring

   Fraud monitoring is the monitoring and detection of suspicious
   activities within, through, or perpetrated against a company. It must
   be reported to regulatory agencies as required by applicable laws and
   regulations.  Examples of fraud are unauthorized account access and
   identify theft.  Fraud monitoring is a mission critical function for
   financial institutions, and there are network-based tools performing
   this function with decrypted TLS packets. If fraud monitoring is
   down, then it is a severity one problem for critical applications.

   Fraud monitoring looks at network packets in many locations.  An
   inline/MITM solution in this environment does not scale.

   One of the major fraud monitoring applications consists of an array
   of servers, including a database, all talking to each other via TLS.
   Application errors for this fraud monitoring app need to be analyzed
   just like any other application, including network packet analysis,
   and TLS decryption is needed in order to match up log errors with
   network packets on the wire.

 


Fenter                 Expires September 6, 2018               [Page 10]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


2.3.3  Intrusion Detection System

   IDS inspection looks for known and custom malware signatures,
   potential attack patterns, and known observables associated to
   Indicators of Compromise in the payload of TLS packets when
   decryption is available.  IDS inspection for inbound and/or internal
   TLS sometimes depends on out-of-band TLS decryption, and its
   effectiveness is severely impacted if decryption is not available.

   IDS inspection is often a regulatory requirement, for example, for
   cardholder data environments, of which an enterprise may have many.
   An inline/MITM TLS decryption solution has scalability problems in
   this kind of environment.

2.3.4  Threat Detection and Incident Response

   IDS Alerts - Threat Detection teams receive IDS alerts and will
   analyze decrypted network packet traces in order to verify if the
   alert was valid or was a false alarm. 

   SQL Injection Attacks - This particular alert also needs manual
   analysis of packet traces in order to identify if the attack was
   successful, and if so, what data was returned.

   Endpoint Monitoring Alerts - These alerts often need to be verified
   with decrypted packet traces, including identification of the source
   of the attack on the endpoint.  Endpoints are less trustworthy than
   network monitoring tools, and network monitoring is also needed as a
   backstop for any failures of monitoring on the endpoint.

   Manual Hunting - Not all attacks are caught by automated monitoring.
   Threat Detection teams will do manual hunting for known
   vulnerabilities with decrypted packet traces.

   Ubiquitous packet payload visibility can be provided by out-of-band
   decryption for inbound or internal TLS sessions.  Traffic sources for
   malware can be anywhere within the enterprise or external to the
   enterprise.  An inline/MITM decryption solution doesn't scale.

2.3.5  Regulatory Examples

2.3.5.1 PCI (Payment Card Industry)

2.3.5.1.1 PCI and TLS Encryption

   The PCI Security Standards Council strongly recommends segmenting the
   cardholder data environment in order to protect the cardholder
   systems as well as to limit the scope of PCI assessment (PCI
 


Fenter                 Expires September 6, 2018               [Page 11]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


   Information Supplement: Guidance for PCI DSS Scoping and Network
   Segmentation, p. 6).  PCI has a concept of "connected to" systems,
   meaning that any system communicating with a system in the cardholder
   data environment is drawn into PCI requirements.  As a practical
   reality, large enterprises could not get a PCI assessment completed
   without segmenting the cardholder data environment with tools like
   firewalling and encryption of data in transit.  This creates the need
   for TLS decryption in order for those enterprises to do
   troubleshooting, network security monitoring, and other packet-based
   analysis in which the clear-text payload must be available.

2.3.5.1.2 Intrusion Detection

   The PCI DSS (Data Security Standard) requires IDS/IPS inspection at
   the perimeter of the cardholder data environment, as well as at
   critical points in the cardholder data environment (PCI DSS section
   11.4).  If an organization's monitoring and data loss prevention
   strategy includes payload inspection, TLS encrypted traffic in these
   environments must be decrypted.  PCI applications have grown up over
   time, and there may be many cardholder data environments in a data
   center.  Inline/MITM decryption solutions are not scalable for this
   environment due to cost, introduced latency, and production risk from
   the more complicated, inline/MITM environment.

2.3.5.1.3 TLS 1.2 and PCI

   When significant vulnerabilities were found in SSL and early TLS in
   late 2014 (including POODLE), it took the PCI Security Standards
   Council less than a year to require a migration plan away from these
   SSL/TLS versions (PCI Information Supplement: Migrating from SSL and
   Early TLS).  Enterprises are at risk that vulnerabilities could be
   found in TLS 1.2 or in the RSA key exchange, and that PCI will
   require upgrade to TLS 1.3.  There is no guarantee that TLS 1.2 will
   be available many years into the future.

2.3.5.2 e-CFR (Electronic Code of Federal Regulations)

2.3.5.2.1 Insider Abuse

   The United States e-CFR, Title 12, Chapter 1, Part 21 requires that
   national banks and federal branches and agencies of foreign banks
   monitor and report on insider abuse.  This monitoring looks for
   criminal behavior like employee fraud, and is a mission critical and
   legally required function for financial institutions operating in the
   United States.  If potentially illegal or fraudulent activity is
   detected, a Suspicious Activity Report must be filed with FinCEN (the
   Financial Crimes Enforcement Network of the US Department of the
   Treasury).
 


Fenter                 Expires September 6, 2018               [Page 12]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


   One of the tools for accomplishing this monitoring is a network based
   tool that uses out-of-band TLS decryption to inspect network packets
   and analyze user activity.  The user endpoint cannot be trusted for
   this monitoring, as it is controlled by the user being suspected of
   fraudulent or illegal activity.  The loss of out-of-band decryption
   would be crippling to this monitoring.  There are many monitoring
   points for this tool, and inline/MITM decryption is not a scalable
   option.  Also, these kinds of tools have no need to be inline, and
   forcing the solution inline adds unnecessary complexity and
   production risk into a mission critical environment.

2.3.5.3 Regulatory Requirements - Summary

   This section is by no means an exhaustive discussion of all
   regulatory requirements for all verticals in all countries worldwide.
   However, it does illustrate the kinds of issues that can come up when
   a long standing "feature" is eliminated from a network security
   protocol.

3.  Alternative Solutions Offered and Their Limitations

3.1  Inline/MITM Decryption

   This is a valid sounding option but presents scalability problems for
   large, diversified enterprises.

   TLS decryption for security monitoring is needed in more locations
   than just everything in and out of the Internet; for example, network
   security monitoring is done on Business to Business connections, the
   branch/MPLS head-end, the mainframe, cardholder data environments,
   wireless controllers, DNS servers, etc.

   Network troubleshooters need to be able to take traces and decrypt
   them anywhere in the enterprise network.  This can be hundreds or
   even thousands of locations, depending on the particular problem that
   hits, and can include the data center, branches, the virtual
   environment, and public or private clouds.  It's not scalable to try
   and put an inline/MITM decryption solution between any two servers in
   the enterprise who may be talking to each other via TLS.  This could
   include VM's talking to each other on the same hypervisor or
   containers talking to each other on the same VM.

   Cost - Bypass taps and TLS decryption appliances are expensive, and
   the cost adds up when adequate resiliency and failover is
   architected.  The cost for implementing an inline/MITM decryption
   strategy can quickly escalate into millions of dollars.

   Latency - TLS decryption appliances add 1-3 ms of latency per packet
 


Fenter                 Expires September 6, 2018               [Page 13]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


   in a hardware appliance.  Virtual decryption appliances may take a
   larger latency hit.  When multiple inline decryption locations are
   implemented, the latency becomes prohibitive.

   Production Risk - Bypass taps and TLS decryption appliances are
   complicated devices that can and do fail.  When an inline/MITM TLS
   decryption solution fails, production traffic is brought down. An
   inline/MITM solution is far more complex than putting in passive
   network taps, which are very simple and almost never fail.  

   Out-of-band TLS decryption is a better design for much of the
   enterprise.  It provides for ubiquitous packet payload visibility
   with lower cost, no latency, and almost non-existent production risk.

   Obtaining packets in the virtual and cloud environments is more
   complex, but an out-of-band TLS decryption solution is still more
   scalable than inline/MITM TLS decryption everywhere in the virtual or
   cloud environment.

3.2  Using TCP or UDP Extensions to Supply Extra Information

   This is also an argument that sounds good on the surface and looks
   like it helps preserve privacy.  However, there are a number of
   reasons why this idea doesn't work in an enterprise.

   There can be session identifiers in the payload of packets that are
   needed in order to match up packets (whose source IP may be changing)
   inside and outside of an infrastructure device, and also to match up
   application layer requests with application layer responses (the
   responses don't always ride on the same TCP conversation as the
   request).  These session identifiers are unique to each application,
   and it is not possible to anticipate, among thousands of
   applications, which fields in the payload are going to be important
   for a particular problem.  An individual enterprise can have
   thousands of unique applications.  The next enterprise down the road
   can have thousands more applications of their own, all unique and
   different from the first enterprise.

   Software bugs can leave telltale signs in the payload of a packet.
   These telltale signs are critical pieces of information for
   troubleshooting difficult problems.  It is not possible to
   anticipate, among thousands of applications, which parts of the
   payload are going to be important for finding a software bug.

   Indicators of Compromise can exist anywhere within the payload of a
   packet.  It is not possible to anticipate for every new attack which
   part of the payload will be important for threat detection and
   incident analysis.
 


Fenter                 Expires September 6, 2018               [Page 14]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


   Fields that can be used to block layer 7 DDoS attacks can be anywhere
   within the payload of the packet.  It's not possible to determine
   which field to block on for any particular DDoS attack until the full
   payload is decrypted and examined.

   Customer Experience Monitoring requires full packet payload.  A click
   anywhere on a web page can be of interest to those doing the
   monitoring.

3.3  Using IP and TCP Headers for Monitoring and Troubleshooting

   This approach has all the same problems as those outlined in section
   3.2 above.

3.4  TLS 1.2

   No enterprise wants to run an older, less secure version of a
   protocol for the long term.

   The RSA key exchange is already in the process of being deprecated
   from TLS 1.2 in some environments.  Examples of this have already
   been seen in the mobile device environment.

   Suggestions have been made on the TLS email list that we need to
   deprecate the RSA key exchange from TLS 1.2.  All we need is for
   another major RSA vulnerability be found, and this sentiment will
   gain traction.

3.5  Logging

   There are many enterprise outages and slowdowns where there is either
   no log message on the offending device, or there is a log message
   that indicates a problem but no clue as to the fault domain or the
   root cause.  Infrastructure devices do not understand layer 7 and so
   are unable to log meaningful information about a layer 7 transaction
   that had a problem, even if that particular infrastructure device was
   the cause of the problem. In many cases, network packet analysis with
   TLS decryption is required in order to identify the fault domain
   and/or get to the root cause.

   It is not possible for a code developer to anticipate every possible
   problem that is going to occur and put a log message in just the
   right place.  Also, the very nature of a software bug is that the
   developer doesn't know it's there, so there is not going to be any
   log message when a bug hits.

   It is not feasible to go through millions of lines of code in an
   enterprise environment and "improve" the logging on each device.
 


Fenter                 Expires September 6, 2018               [Page 15]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


   Between infrastructure and security devices, and application code,
   this would involve getting hundreds or thousands of vendors to invest
   and cooperate with this idea.  Vendors would have no idea what to log
   other than what is currently being logged in order to try and catch
   the "next" problem.

   Adding log messages after a problem hits is like playing enterprise
   "Whack-a-mole".  The next problem to hit is invariably something
   completely different.

3.6  Troubleshooting at the Endpoint

   The shortcomings of endpoint logging are covered in section 3.5
   above.   

   Endpoints in a typical enterprise don't have the robustness to run a
   full packet capture of all packets, decrypt them all, and keep the
   trace running all the time.  This kind of trace is necessary because
   analysts don't know which web or app server, for example, is going to
   be hit for a particular user session, and they don't know when an
   intermittent problem is going to hit.  Instead, enterprises have
   built up robust, out-of-band packet sniffing devices with TLS
   decryption capability fed by passive network taps and/or passive
   mirror ports.  

   Endpoint analysis also misses the crucial troubleshooting function of
   isolating the fault domain of a problem among many infrastructure
   devices between the TLS endpoints.

3.7  Security Monitoring at the Endpoint

   Network security monitoring is done by a number of purpose built
   network devices such as IDS/IPS and security analysis solutions.
   Network based fraud detection applications can include multiple
   servers and databases that all communicate with each other.  It's not
   feasible to put all this functionality into an endpoint and have its
   normal workload unaffected.

   Endpoints can be overwhelmed by too much security monitoring and
   their performance impacted.  Networks can also be overwhelmed by
   extensive security reporting from endpoints.  As a result, endpoint
   monitoring is often scaled back to a level that the endpoint and its
   network connection can handle.

   Endpoints cannot be completely trusted for network security
   monitoring.  Malware can delete logs and turn off future logging.
   It's also not always possible to secure data stored on an endpoint,
   for example, if the endpoint is a laptop and the user packs it up and
 


Fenter                 Expires September 6, 2018               [Page 16]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


   walks out of the enterprise.  The great variety of endpoint types
   also makes it difficult to implement a consistent monitoring strategy
   using endpoints alone.

   Network security monitoring is an important complement to endpoint
   security monitoring, and is part of a "defense in depth" strategy.

3.8  Encrypted Traffic Inspection 

   This technology, while interesting and applicable in some situations,
   does not fully satisfy the requirements of enterprise traffic
   inspection.

   From an application performance and availability perspective,
   encrypted traffic inspection will not figure out severity one
   slowdowns or outages, or any other level of problem that may hit an
   enterprise.  Large enterprises have thousands of unique applications
   that all behave differently at layer 7, and any one of these
   applications may need layer 7 analysis when a problem hits. This
   factor of troubleshooting alone is enough to make encrypted traffic
   inspection an unacceptable, or at least incomplete, solution for
   enterprise encryption problems.

   Encrypted traffic inspection does not address fraud detection for
   either internal or external fraud, both of which look at decrypted
   TLS packets.

   Encrypted packet inspection does not address Application Performance
   Monitoring, Customer Experience Monitoring, or the use of decrypted
   packets for regulatory compliance monitoring.

   From a security perspective, encrypted traffic inspection is not
   going to detect every zero day attack.  The parameters it is looking
   for in the TLS handshake can be varied by new malware. Encrypted
   traffic inspection, for some methodologies, may be less effective
   under TLS 1.3 when the handshake is encrypted.   

   Encrypted traffic inspection doesn't take into account the manual,
   deep packet inspection done by threat detection teams in order to
   analyze malware alerts, track down their source, and to identify if
   an attack succeeded or failed.

3.9  IPsec instead of TLS

   The enterprise rollout of internal TLS has been a multi-year project.
   Enterprises can't just flip a switch and start running IPsec.  Moving
   to IPsec would likely be a multi-year and expensive project.  There
   is extensive manual configuration that would need to be done.
 


Fenter                 Expires September 6, 2018               [Page 17]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


   There are a number of infrastructure services that don't support
   IPsec today.  For example, IPsec is not supported today by load
   balancers for data center load balancing services.  IPsec is also not
   supported by Internet proxies for outbound Internet services.

   A number of new IETF protocols are tied to TLS, including HTTP/2,
   DPRIVE, and QUIC.  Enterprises need a TLS decryption solution in
   order to support these protocols.

4.  An Examination of Arguments Against All Network Decryption

4.1  Technical Arguments

4.1.1  "I work for a large company and we don't have to decrypt
   packets."

   Large Internet companies being put forward as examples of promoting
   encryption are not necessarily encrypted through their entire private
   enterprise environment as some financial and health care institutions
   are.

   There are varying levels of data center depth and complexity between
   enterprises.  Some enterprises have a flatter data center structure,
   depending on the kinds of services they offer.  Other enterprises
   have many layers to their applications, with multiple layers of
   infrastructure like firewalls and load balancers, and many layers of
   middleware, authentication, fraud detection, mainframe, etc. There
   are also legacy applications that add complexity to the
   infrastructure, and add to the requirement for decrypted packet
   payload analysis.

   Network packet decryption, if it is happening, is likely not visible
   to all employees within an enterprise.  Typically, only select groups
   within an organization utilize or are aware of this level of detail.

   As more enterprises in different verticals add TLS decryption inside
   their data centers, they are going to realize that they also have a
   need for out-of-band TLS decryption.

4.2  Privacy Arguments

4.2.1  "It's a violation of personal privacy to decrypt TLS traffic
   anywhere except at the TLS endpoint."

   Enterprises have many legitimate business reasons for inspecting
   their own data, and the IETF should provide them with well studied
   and standardized options that meet these critical business needs.

 


Fenter                 Expires September 6, 2018               [Page 18]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


   All TLS data is already being decrypted multiple times in the
   enterprise data center, and customer data is already available to
   certain employees of that enterprise.  Network packet decryption is
   just one more decryption of its own data by the same enterprise.

4.2.2  "Pervasive Monitoring is an Attack"

   Pervasive monitoring inside the enterprise has many legitimate use
   cases, including troubleshooting and network security monitoring. As
   an example, some applications are too complicated to troubleshoot at
   the network packet level without advanced preparation of the packet
   level monitoring, meaning that it takes too much time during a
   critical outage to get traces set up in all the right places.  The
   answer to this is pervasive monitoring of that application or that
   environment so that network packet traces, including TLS decryption,
   are ready when a problem hits.

   RFC 7258 [RFC7258] should be modified to account for the many
   enterprise use cases where pervasive monitoring is not an attack.

5.  Possible TLS 1.3 Decryption Solutions

5.1  Static Diffie Hellman

   Static Diffie Hellman [draft-green] as described in draft-green-tls-
   static-dh-in-tls13 meets the enterprise need in a manner similar to
   running RSA key exchanges and using static RSA private keys. 
   Enterprises would be obligated to protect their static keys as they
   are today in the RSA environment.  Draft-green requires no changes to
   the TLS client, and no changes to the TLS 1.3 spec.  It has no impact
   on the CPU load of the TLS server.  Enterprises have the option of
   rotating their static Diffie-Hellman private keys as often as they
   see fit.

5.2  TLS 1.3 Option for Negotiation of Visibility in the Datacenter

   TLS 1.3 Option for Negotiation of Visibility in the Datacenter
   [draft-rhrd] as a solution has some alternative features in
   comparison to draft-green [draft-green].  It eliminates static
   Diffie-Hellman private keys from the TLS server as in the case of
   draft-green.  The key manager would only write static private keys
   from the SSWrapDH1 key pair to the decryption appliances in the
   protected enterprise network. Draft-rhrd provides for client opt-in
   and visibility on the wire that traffic payload inspection may be
   happening.  It also allows for decryption in the case of session
   reuse, which solves a large problem for enterprise monitoring and
   troubleshooting.  It will have some impact on the CPU load of the TLS
   server.
 


Fenter                 Expires September 6, 2018               [Page 19]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


5.3  Solution Summary

   For both of these solutions, a standard is needed so that all the
   related systems will interoperate.  Draft-green [draft-green] would
   need to be implemented by multiple TLS server vendors, multiple
   decryption appliance vendors, and multiple key management solutions.
   Draft-rhrd [draft-rhrd] would require all of these in addition to
   implementation by TLS clients.  For the above reasons, custom code is
   a highly unattractive, and possibly unworkable solution.

6.  Conclusion

   Out-of-band TLS decryption is used by a number of enterprise tools
   for mission critical functions, and it supplies ubiquitous packet
   payload visibility that can't be replaced by other methods.  Endpoint
   analysis is limited by its lack of robustness for analytic
   activities, by the fact that it can't be completely trusted, and by
   its blindness to issues in the intervening infrastructure. 
   Inline/MITM decryption adds cost, latency, and production risk at
   every point it is implemented, and it doesn't scale to meet the
   requirements of the use cases presented above.

7.  Security Considerations

   There are security tradeoffs that enterprises should be allowed to
   decide.  On the one side are the benefits of Forward Secrecy inside
   the enterprise.  On the other side are the benefits of ubiquitous
   packet payload visibility inside the enterprise. Enterprises are most
   qualified to make this business decision for themselves, and the TLS
   Working Group should provide them options rather than making the
   decision for them.

   Enterprises choosing to do out-of-band decryption need to continue to
   implement whatever security controls are appropriate for protection
   of this decryption environment, including protection of keys,
   controlling access to the decrypted data, etc.

8.  IANA Considerations

   There are no IANA considerations.








 


Fenter                 Expires September 6, 2018               [Page 20]

INTERNET DRAFT       draft-fenter-tls-decryption-00        March 5, 2018


9.  References

9.1  Normative References


   [draft-green] Green, M., Droms, R., Housley, R., Turner, P., and S.
   Fenter, "Data Center use of Static Diffie-Hellman in TLS 1.3", draft-
   green-tls-static-dh-in-tls13-01 (work in progress), July 2017.

   [draft-rhrd] Housley, R. and Droms, R., "TLS 1.3 Option for
   Negotiation of Visibility in the Datacenter", draft-rhrd-tls-tls13-
   visibility-01 (work in progress), March 2018.

   [RFC5246]  Dierks, T. and E. Rescorla, "The Transport Layer Security
   (TLS) Protocol Version 1.2", RFC 5246,  DOI 10.17487/RFC5246, August
   2008, <https://www.rfc-editor.org/info/rfc5246>.

   [RFC7258]  Farrell, S. and Tschofenig, H.,  "Pervasive Monitoring Is
   an Attack", BCP 188, RFC 7258, DOI 10.17487/RFC7258, May 2014,
   <https://www.rfc-editor.org/info/rfc7258>.

   [TLS13]  Rescorla, E., "The Transport Layer Security (TLS) Protocol
   Version 1.3", draft-ietf-tls-tls13-26 (work in progress),  March
   2018.


9.2  Informative References


Acknowledgments

   Nalini Elkins, Mike Ackermann, Darin Pettis, and Russ Housley
   contributed through discussion to the development of this document.


Authors' Addresses


   Steve Fenter
   Enterprise Data Center Operators, Inc.
   36A Upper Circle
   Carmel Valley, CA 93924

   EMail: info@e-dco.com







Fenter                 Expires September 6, 2018               [Page 21]