Individual D. Otis Internet-Draft D. Rand Intended status: Informational Trend Micro Expires: November 17, 2013 May 16, 2013 Macros Nixed by Major Providers draft-otis-spfbis-macros-nixed-00 Abstract SPF is a popular tool for managing email blocking issues. It has also helped reduce backscatter related with Non-Delivery Notifications as was intended. A practically unused feature is SPF macro capability that SPFbis still in part retains. This feature greatly diminishes SPF's utility and actually threatens email interchange, especially when used in conjunction with secondary policy layers while also being ignored by major providers. The macro mechanism is also unable to cope with internationalization, and is found in virtually none of the published resource records. It is counter productive to retain the largely unused and often unsupported macro feature, despite efforts expended to develop the macro language. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on November 17, 2013. Copyright Notice Otis & Rand Expires November 17, 2013 [Page 1] Internet-Draft Macros Nixed May 2013 Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. SMTP Internationalization . . . . . . . . . . . . . . . . . . 3 3. Mail From Parameter and SPF . . . . . . . . . . . . . . . . . 4 4. IPv6 and SPF with PTR or local-part Macros . . . . . . . . . . 5 4.1. Problems with Macros . . . . . . . . . . . . . . . . . . . 5 4.2. 'do not use' Advice Assumes Senders Cooperate . . . . . . 7 4.3. Retain SPF Type (99) . . . . . . . . . . . . . . . . . . . 7 5. Domains as a Basis for Managing Traffic . . . . . . . . . . . 7 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 7. Security Considerations . . . . . . . . . . . . . . . . . . . 8 8. References - Informative . . . . . . . . . . . . . . . . . . . 8 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 10 Otis & Rand Expires November 17, 2013 [Page 2] Internet-Draft Macros Nixed May 2013 1. Introduction Checking reverse DNS [RFC1034] entries per SMTP [RFC5321] connection requires queries against the entire IP address. SPF [I-D.ietf-spfbis-4408bis] macros may go a step further of searching for address matches with the SMTP client resolved from reverse DNS PTR record sets in a manner similar to address matches with the 'MX' mechanism hostname list. For IPv6, reverse DNS checks by SMTP servers may inadvertently induce DDoS effects against the DNS managed by the sender's network provider while at the same time consuming the receiving SMTP server's connection resources waiting for the subsequent DNS responses. Prior IPv4 mapping of reverse DNS is often used to avoid wasting of resources resulting from reverse DNS, but this strategy is not possible with IPv6. Use of reverse DNS is seldom practical simply due to network resource considerations. While use of SPF 'PTR' mechanism and the {%p} macro now offers 'do not use' advice, it is unfortunate similar considerations for uninvolved DNS servers that can be similarly burdened by local-part macros or various macro expansions of SMTP client addresses were not offered the same advice. At least reasonable limits were placed on the number of NX domain responses. Unfortunately even that strategy is undermined by a technique growing in popularity of using synthetic domains to act as an alternative for web cookies where the resulting responses actually increases the level of amplification. There is no justification to use macros that are able to impose queries constructed of odd elements of the email-address local-part or that of the SMTP client IP address. Windows, OS X, iOS, Linux employ IPv6 privacy extensions [RFC4941] where any problematic IP address is likely to be reassigned different addresses regularly and remain valid beyond the reassignment interval. Since a major portion of spam originates from compromised residential systems, any attempts to handle or cache this highly diverse information should not be considered practical. 2. SMTP Internationalization It should also be pointed out use of macros may not function when [RFC6531] permits international email addresses that includes both local-part and domain portions of the email address. Tests and comments by large email providers show an understandable reluctance to process SPF macros where often they simply do not implement the macro portion of SPF. The SPF review documented in [RFC6686] failed to provide a breakdown Otis & Rand Expires November 17, 2013 [Page 3] Internet-Draft Macros Nixed May 2013 on macro use. SPF's macros can prove hazardous, especially with IPv6. Rather than permitting receivers to decide their acceptance methods, an SPF resource record based on the Mail From parameter can induce 100 DNS transactions using labels constructed from random strings found in the message local-part. While recent changes made in the current version of SPF recommends against its use, to process the reverse namespace may place an extreme burden on DNS, its cache, and the SMTP server's connection resources. Too bad unrelated DNS providers where not afforded similar consideration. SPF macros have not anticipated the use of internationalized email addresses and lacks any negotiation with respect to this type of email. A fair amount of problematic repair regarding construction of DNS queries based on internationalized email-addresses can be avoided by simply acknowledging currently SPF macros are seldom used and often not implemented. 3. Mail From Parameter and SPF Access to an outbound SMTP servers may or may not restrict use of the Mail From Parameter. An alternative mitigation strategy may rely upon rate limits and denial of access subsequent to abuse reports. Reasons for publishing SPF records might be to mitigate backscatter based on records that offer Non-Delivery-Notifications (NDNs) authorization by the Mail From parameter where the loss of some of these notifications is considered acceptable. When used to authorize bulk senders, or provide results that are independent of the SMTP client, the process will not offer a status safely associated with any particular message. SPF failure rates of even a few percent are too high for it to offer a solid basis for either acceptance or rejection. There are some policy strategies for specific domains [I-D.kucherawy-dmarc-base] that attempt to combine SPF [I-D.ietf-spfbis-4408bis] and DKIM [RFC6376] where when either of their related domains manage to pass, only then is the message to be placed in the in-box. Domains that reference SPF resource records should not be considered authenticated because the SPF process authorized the SMTP client. Often an SMTP client is shared by many domains. As such, it would be incorrect to assume SPFv1 records that authorize a provider by way of the Mail From Parameters is only permitted by authenticated accounts from that domain. As a result, no domain information in SPF or DKIM, under current circumstances, can be used as a domain reference for either acceptance or reputation. Otis & Rand Expires November 17, 2013 [Page 4] Internet-Draft Macros Nixed May 2013 4. IPv6 and SPF with PTR or local-part Macros When IPv6 becomes more commonly used, some will attempt white-listing IP addresses as a practical albeit non-scalable method to deal with the highly increased address range confronted when dealing with IPv6. There are currently no practical solutions able to scale to the challenging range of IP addresses that might be controlled by malefactors. Unlike most DNS resources that segregate IPv4 and IPv6 datasets, SPF consolidates both IPv4 and IPv6 addresses into a common list comprised of a chain of DNS transactions. In addition to explicit chaining, SPF expects receivers to process the first SPF resource record which may contain 10 SPF mechanisms. Each mechanism may then require an additional 10 DNS transactions to resolve the mechanism's status. According to [spf-all.com] out of 112,311,175 domains, 24,073,182 (21.4 %) publish SPF records where only 12,768 (0.053%) employ macros. Even the 0.053% figure overstates use since many represent questionable server farm deployments. These are questionable since many major providers do not process SPF records that contain macros. Non-implementation of macros is very good from a security and reliability perspective, but problematic for email's integrity when used. 4.1. Problems with Macros 4.1.1. Use of SPF Macros Ignored When SPF was first introduced, AOL claimed they used SPF to create IP address white-lists based upon domains with whom they collaborate. White-listing would be less effective if macros were used in SPF records to modify results based upon variables such as the IP address of the client or the local-part of the email address. Ask a group of email providers about SPF, and you'll likely hear SPF is used to identify IP addresses used by a domain to send email. With this information, they can then check these addresses against popular reputation services to determine where problems are apparent. Silent discard or placement into spam folders reduces delivery status integrity and makes supporting unreliable delivery difficult. The information returned in the Authentication-Results header does not always clarify whether the SPF records were actually processed. In speaking with some of these providers about potential network amplification concerns along with resource consumption, they made assurances they do not process SPF macros and they have not had any complaints. It seems these providers are also aware of path registration failure rates associated with SPF, and do not reject email on the basis of SPF failure. Otis & Rand Expires November 17, 2013 [Page 5] Internet-Draft Macros Nixed May 2013 [I-D.kucherawy-dmarc-base] Changes the importance of SPF records being processed, as they will come into play during message acceptance and not be limited to just deciding whether a NDN is safe to send. As such, both 'do not use' and 'do not process' macros offers improved interchange and security. SPF's intent is to constrain the number of invalid NDNs being returned when someone spoofs their domain. Evidence of this intent might be denoted by SPF record's ending or providers overriding them with "?all". This approach better supports DMARC's efforts at improving delivery rates with improved policy enforcement. Because DMARC may actually convince a provider to refuse messages based upon SPF failures, it is important that it is not used for messages that are sent to mailing- lists for example. [I-D.ietf-spfbis-4408bis] purports to be documenting current use. It dropped the SPF resource record (99) due to sparse use and recommends against reverse PTR use. If a similar threshold were applied, all the SPF macro expansion aspects of this protocol would be removed as well. Several large providers have offered their assurance that their SPF libraries had these macros removed. Any effort hoping to define current use MUST carefully reconsider the inclusion of this ill-considered macro feature. Otherwise, it can not be suggested SPF-bis simply documents current use. Danger imposed by the use of the local-part macro is inherent in the query required to support both an IPv6 expansion, in conjunction with the expansion of local-part macros induced by possibly cached SPF records. The local-part, along with a range of IP addresses made readily possible by IPv6, can be manipulated to induce a flurry of large DNS transactions directed toward any otherwise uninvolved domain, all directed by cached DNS records that contain active content. It is never a good idea to allow free attacks enabled through foreign scripts. The SPF local-part macro would allow a cached DNS record to be repeatedly exploited by a spam campaign with an attacker consuming little of their resources beyond what they already use in the campaign. In this case, their recipients change the domains queried on behalf of the attackers based upon the same SPF resource record. High levels of amplification still apply if SPF specifications are not changed and resulting implementation are then assumed part of some greater security requirement and are no longer deemed experimental. The SPF macro DOS overview demonstrated in 2006 [otis-spf-dos-exploit] did not consider IPv6 use which offers higher gain, where the gain achieved sending a single 1KB message to a single recipient offered a 1:156 reflected gain over IPv4 that can never be filtered or blocked by using BCP38 [RFC2827]. When made against a wildcard record, even higher gains can be achieved. Otis & Rand Expires November 17, 2013 [Page 6] Internet-Draft Macros Nixed May 2013 Logically, local-part macros would not safely provide positive results without the query being combined with an IP address list of SMTP clients in some form. A list structure would need to be repeated for each user. Rcpt To using schemes like BATV provides an expedient way where the purported sender determines whether a NDN can be returned without the risk associated with active content placed within DNS resource records. Several parsers ignore local-part macro expansion since they rarely play any role in providing positive results. The [I-D.ietf-spfbis-4408bis] Addendum E indicates a possible use of SPF macros such as exists:{%l}._spf.{%d} or exists:{%i}_spf.{%d} can be used in "specialized" DNS servers able to understand encrypted local-parts or react to inordinate query rates which suggests use of resource records having extremely short TTLs. Such an approach can add a sizeable burden on recipients and targeted DNS servers. Such a scheme represents a poor alternative to providing authenticated domain identifiers. The general intent and purpose of SPF is to offer Non-Delivery Notification authorization where no macros are required and by most practical measure are not being used. 4.2. 'do not use' Advice Assumes Senders Cooperate Problems related to various types of network abuse made possible by SPF macro extensions are not prevented by assuming corrective behavior is obtained through advice given to senders. In general, it is the sender not to be trusted. Only by giving the advice 'do not use and do not implement' is both security and interchange ensured. 4.3. Retain SPF Type (99) Upon greater reflection regarding deprecation of the SPF type (99) record, this was a mistake. When overlaying the TXT resource record was originally proposed, many strongly objected. Studies then showed scant use of TXT records which convinced those in the then MARID WG they could adopt the TXT record at the domain apex as belonging to SPF as their means to support wildcard use. Although support of unknown RR types has improved, the vendor that made adoption of a different RR type problematic still persists with their related name service extensions imposing barriers. SPF Type (99) should be retained to better ensure extensions to DNS retain an ability to cope with unknown RR types. 5. Domains as a Basis for Managing Traffic A manageable basis for assessments can leverage a smaller number of related domains, compared to IPv6 or even IPv4 addresses. Although Otis & Rand Expires November 17, 2013 [Page 7] Internet-Draft Macros Nixed May 2013 technically the domain name space can be larger than the massively large IPv6 address space, in practice it is not. One hundred thousand domains control 90% of Internet traffic out of approximately 100 million domains active each month. The top 150 domains control 50% of the traffic, and the top 2,500 domains control 75%. This level of domain consolidation permits effective fast-path white- listing. Improvements achieved using domains to consolidate the threat landscape can easily justify added cryptographic authentication burdens. Even APL resource records [RFC3123] can authenticate EHLO using a single DNS transaction, but this would not allow IPv6 email to be more easily managed, which cryptographic technology can offer. 6. IANA Considerations This document requires no IANA consideration. 7. Security Considerations This draft intends to describe serious security and interchange concerns with use of SPF macros. The contained recommendations are expected to reduce these concerns. Greater caution is warranted before making assumptions about DNS activity. Due to high volumes of traffic, rarely is this traffic logged directly by the server. Some items might be triggered while passing through firewalls, but without being given the item to watch, discovering a well orchestrated attack is analogous to discovering a needle in the proverbial haystack. Many administrators overlook a serious problem made much worse by chatty protocols that impose processing delays. Examining server logs will not reveal any problem either, because the limited resource being consumed is the number of outstanding connections TCP is able to support. Reaching this limit will prevent new connections from being instantiated but this is not logged as an event. Over time administrators may hear complaints email is not being delivered or just see an ever growing percentage of spam. Add language clearly indicating BOTH the implementation or use of SPF macro features is hereby deprecated. Any such use provides a neutral result. In addition, restore SPF type 99 to better insure the future health of DNS. 8. References - Informative [I-D.ietf-spfbis-4408bis] Otis & Rand Expires November 17, 2013 [Page 8] Internet-Draft Macros Nixed May 2013 Kitterman, D., "Sender Policy Framework (SPF) for Authorizing Use of Domains in Email, Version 1", draft-ietf-spfbis-4408bis-14 (work in progress), April 2013. [I-D.kucherawy-dmarc-base] Kucherawy, M., "Domain-based Message Authentication, Reporting and Conformance (DMARC)", draft-kucherawy-dmarc-base-00 (work in progress), March 2013. [RFC1034] Mockapetris, P., "Domain names - concepts and facilities", STD 13, RFC 1034, November 1987. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", RFC 2460, December 1998. [RFC2782] Gulbrandsen, A., Vixie, P., and L. Esibov, "A DNS RR for specifying the location of services (DNS SRV)", RFC 2782, February 2000. [RFC2822] Resnick, P., "Internet Message Format", RFC 2822, April 2001. [RFC2827] Ferguson, P. and D. Senie, "Network Ingress Filtering: Defeating Denial of Service Attacks which employ IP Source Address Spoofing", BCP 38, RFC 2827, May 2000. [RFC3123] Koch, P., "A DNS RR Type for Lists of Address Prefixes (APL RR)", RFC 3123, June 2001. [RFC4033] Arends, R., Austein, R., Larson, M., Massey, D., and S. Rose, "DNS Security Introduction and Requirements", RFC 4033, March 2005. [RFC4034] Arends, R., Austein, R., Larson, M., Massey, D., and S. Rose, "Resource Records for the DNS Security Extensions", RFC 4034, March 2005. [RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing Architecture", RFC 4291, February 2006. [RFC4408] Wong, M. and W. Schlitt, "Sender Policy Framework (SPF) for Authorizing Use of Domains in E-Mail, Version 1", RFC 4408, April 2006. Otis & Rand Expires November 17, 2013 [Page 9] Internet-Draft Macros Nixed May 2013 [RFC4941] Narten, T., Draves, R., and S. Krishnan, "Privacy Extensions for Stateless Address Autoconfiguration in IPv6", RFC 4941, September 2007. [RFC4954] Siemborski, R. and A. Melnikov, "SMTP Service Extension for Authentication", RFC 4954, July 2007. [RFC5321] Klensin, J., "Simple Mail Transfer Protocol", RFC 5321, October 2008. [RFC5322] Resnick, P., Ed., "Internet Message Format", RFC 5322, October 2008. [RFC5451] Kucherawy, M., "Message Header Field for Indicating Message Authentication Status", RFC 5451, April 2009. [RFC6376] Crocker, D., Hansen, T., and M. Kucherawy, "DomainKeys Identified Mail (DKIM) Signatures", RFC 6376, September 2011. [RFC6530] Klensin, J. and Y. Ko, "Overview and Framework for Internationalized Email", RFC 6530, February 2012. [RFC6531] Yao, J. and W. Mao, "SMTP Extension for Internationalized Email", RFC 6531, February 2012. [RFC6532] Yang, A., Steele, S., and N. Freed, "Internationalized Email Headers", RFC 6532, February 2012. [RFC6577] Kucherawy, M., "Authentication-Results Registration Update for Sender Policy Framework (SPF) Results", RFC 6577, March 2012. [RFC6686] Kucherawy, M., "Resolution of the Sender Policy Framework (SPF) and Sender ID Experiments", RFC 6686, July 2012. [otis-spf-dos-exploit] http://tools.ietf.org/html/draft-otis-spf-dos-exploit-01, "Otis SPF DoS Exploitation", June 2006. [spf-all.com] http://spf-all.com/stats.html, "SPF-all.com Stats", May 2013. Otis & Rand Expires November 17, 2013 [Page 10] Internet-Draft Macros Nixed May 2013 Authors' Addresses Douglas Otis Trend Micro 10101 N. De Anza Blvd Cupertino, CA 95014 USA Phone: +1.408.257-1500 Email: doug_otis@trendmicro.com Dave Rand Trend Micro 10101 N. De Anza Blvd Cupertino, CA 95014 USA Phone: +1.408.257-1500 Email: dave_rand@trendmicro.com Otis & Rand Expires November 17, 2013 [Page 11]