Internet DRAFT - draft-otis-spfbis-macros-nixed
draft-otis-spfbis-macros-nixed
Individual D. Otis
Internet-Draft D. Rand
Intended status: Informational Trend Micro
Expires: November 29, 2013 May 28, 2013
Macros Nixed by Major Providers
draft-otis-spfbis-macros-nixed-01
Abstract
SPF is a popular tool for managing email blocking issues. It has
also helped reduce backscatter related with Non-Delivery
Notifications as was intended. A practically unused feature is the
SPF macro capability that SPFbis still in part retains. This feature
greatly diminishes SPF's utility and actually threatens email
interchange, especially when used in conjunction with secondary
policy layers when also ignored by major providers. The macro
mechanism is also unable to cope with internationalization, and is
found in virtually none of the published resource records. It is
counter productive to retain the largely unused and often unsupported
macro feature, despite efforts expended to develop the macro
language.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on November 29, 2013.
Copyright Notice
Otis & Rand Expires November 29, 2013 [Page 1]
Internet-Draft Macros Nixed May 2013
Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. SMTP Internationalization . . . . . . . . . . . . . . . . . . 3
3. Mail From Parameter and SPF . . . . . . . . . . . . . . . . . 4
4. IPv6 and SPF with PTR or local-part Macros . . . . . . . . . . 5
4.1. Problems with Macros . . . . . . . . . . . . . . . . . . . 5
4.2. 'do not use' Advice Assumes Senders Cooperate . . . . . . 8
4.3. Retain SPF Type (99) . . . . . . . . . . . . . . . . . . . 8
5. Domains as a Basis for Managing Traffic . . . . . . . . . . . 8
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9
7. Security Considerations . . . . . . . . . . . . . . . . . . . 9
8. References - Informative . . . . . . . . . . . . . . . . . . . 9
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12
Otis & Rand Expires November 29, 2013 [Page 2]
Internet-Draft Macros Nixed May 2013
1. Introduction
Checking reverse DNS [RFC1034] entries per SMTP [RFC5321] connection
requires queries against the entire IP address. SPF
[I-D.ietf-spfbis-4408bis] macros may go a step further of searching
for address matches with the SMTP client resolved from reverse DNS
PTR record sets in a manner similar to address matches with the 'MX'
mechanism hostname list. For IPv6, reverse DNS checks by SMTP
servers may inadvertently induce DDoS effects against the DNS managed
by the sender's network provider while at the same time consuming
receiving SMTP server's connection resources held waiting for
subsequent DNS responses. Prior IPv4 mapping of reverse DNS is often
used to avoid the wasting of resources resulting from reverse DNS,
but this strategy is not possible with IPv6. Use of reverse DNS is
seldom practical simply due to network resource considerations.
While use of the SPF 'PTR' mechanism and the {%p} macro now offers
'do not use' advice, it is unfortunate similar considerations for
uninvolved DNS servers that can be similarly burdened by local-part
macros or various macro expansions of SMTP client addresses were not
offered the same protective advice.
At least reasonable limits were placed on the number of NXDOMAIN
responses. Unfortunately even that strategy is undermined by a
technique growing in popularity of using synthetic domains to act as
an alternative for web cookies where the resulting responses actually
increases the level of amplification. There is no justification to
use macros that are able to impose queries constructed of odd
elements of the email-address local-part or that of the SMTP client
IP address.
Windows, OS X, iOS, Linux employ IPv6 privacy extensions [RFC4941]
where any problematic IP address is likely to be reassigned different
addresses regularly and remain valid beyond the reassignment
interval. Since a major portion of spam originates from compromised
residential systems, any attempt to handle or cache this highly
diverse information should not be considered practical.
2. SMTP Internationalization
It should also be pointed out use of macros may not function when
[RFC6531] permits international email addresses that includes both
local-part and domain portions of the email address. Tests and
comments by large email providers show an understandable reluctance
to process SPF macros where often they simply do not implement the
macro portion of SPF.
The SPF review documented in [RFC6686] failed to provide a breakdown
Otis & Rand Expires November 29, 2013 [Page 3]
Internet-Draft Macros Nixed May 2013
on macro use. SPF's macros can prove hazardous, especially with
IPv6. Rather than permitting receivers to decide their acceptance
methods, an SPF resource record based on the Mail From parameter can
induce 100 DNS transactions using labels constructed from random
strings such as those found in the message local-part or the
expansion of the SMTP client IP address. Recent changes made in the
current version of SPF recommends against reverse DNS use where
processing the reverse namespace may place an extreme burden on the
sender's network provider's DNS and the SMTP server's connection
resources. Too bad unrelated DNS providers where not afforded
similar consideration.
SPF macros have not anticipated the use of internationalized email
addresses and lack any negotiation with respect to this type of
email. A fair amount of problematic repair regarding construction of
DNS queries based on internationalized email-addresses can be avoided
by simply acknowledging that currently SPF macros are seldom used and
often not implemented by the larger providers.
[RFC5890] does not convert non-domain components permitted by
[RFC6531]. As such, this may inject UTF8 strings into code expecting
ASCII and potentially expose vulnerabilities that may permit SMTP
servers to be compromised. Once again, this concern is easily
avoided by deprecating implementation of macros, where such
deprecation should also be assumed by senders to ensure email
exchange.
3. Mail From Parameter and SPF
Access to an outbound SMTP servers may or may not restrict use of the
Mail From Parameter. An alternative mitigation strategy may rely
upon rate limits and denial of access subsequent to abuse reports.
Reasons for publishing SPF records might be to mitigate backscatter
based on records that offer Non-Delivery-Notifications (NDNs)
authorization by the Mail From parameter where the loss of some of
these notifications is considered acceptable. When used to authorize
bulk senders, or provide results that are independent of the SMTP
client, the process will not offer a status safely associated with
any particular message.
SPF failure rates of even a few percent are too high for it to offer
a solid basis for either acceptance or rejection. There are some
policy strategies for specific domains [I-D.kucherawy-dmarc-base]
that attempt to combine SPF [I-D.ietf-spfbis-4408bis] and DKIM
[RFC6376] where when either of their related domains manage to pass,
only then is the message to be placed in the in-box.
Otis & Rand Expires November 29, 2013 [Page 4]
Internet-Draft Macros Nixed May 2013
Domains that reference SPF resource records should not be considered
authenticated because the SPF process authorized the SMTP client.
Often an SMTP client is shared by many domains. As such, it would be
incorrect to assume SPFv1 records that authorize a provider by way of
the Mail From Parameters is only permitted by authenticated accounts
from that domain.
As a result, no domain information in SPF or DKIM, under current
circumstances, can be used as a domain reference for either
acceptance or reputation.
4. IPv6 and SPF with PTR or local-part Macros
When IPv6 becomes more commonly used, some will attempt white-listing
IP addresses as a practical albeit non-scalable method to deal with
the highly increased address range confronted when dealing with IPv6.
There are currently no practical solutions able to scale to the
challenging range of IP addresses that might be controlled by
malefactors.
Unlike most DNS resources that segregate IPv4 and IPv6 datasets, SPF
consolidates both IPv4 and IPv6 addresses into a common list
comprised of a chain of DNS transactions. In addition to explicit
chaining, SPF expects receivers to process the first SPF resource
record which may contain 10 SPF mechanisms. Each mechanism may then
require an additional 10 DNS transactions to resolve the mechanism's
status. According to [spf-all.com] out of 112,311,175 domains,
24,073,182 (21.4 %) publish SPF records where only 12,768 (0.053%)
employ macros. Even the 0.053% figure overstates use since many
represent questionable server farm deployments. These are
questionable since many major providers do not process SPF records
that contain macros. Non-implementation of macros is very good from
a security and reliability perspective, but problematic for email's
integrity when used.
4.1. Problems with Macros
4.1.1. Use of SPF Macros Ignored
When SPF was first introduced, AOL claimed they used SPF to create IP
address white-lists based upon domains with whom they collaborate.
White-listing would be less effective if macros were used in SPF
records to modify results based upon variables such as the IP address
of the client or the local-part of the email address. Ask a group of
email providers about SPF, and you'll likely hear SPF is used to
identify IP addresses used by a domain to send email. With this
information, they can then check these addresses against popular
Otis & Rand Expires November 29, 2013 [Page 5]
Internet-Draft Macros Nixed May 2013
reputation services to determine where problems are apparent. Silent
discard or placement into spam folders reduces delivery status
integrity and makes supporting unreliable delivery difficult.
The information returned in the Authentication-Results header does
not always clarify whether the SPF records were actually processed.
In speaking with some of these providers about potential network
amplification concerns along with resource consumption, they made
assurances they do not process SPF macros and they have not had any
complaints. It seems these providers are also aware of path
registration failure rates associated with SPF, and do not reject
email on the basis of SPF failure.
[I-D.kucherawy-dmarc-base] changes the importance of SPF records
being processed, as they will more likely come into play during
message acceptance and not be limited to just deciding whether a NDN
is safe to send. As such, both 'do not use' and 'do not process'
macros offers improved interchange and security. SPF's intent is to
constrain the number of invalid NDNs being returned when someone
spoofs their domain. Evidence of this intent might be denoted by SPF
record's ending or providers overriding them with "?all". This
approach better supports DMARC's efforts at improving delivery rates
with improved policy enforcement. Because DMARC may actually
convince a provider to refuse messages based upon SPF failures, it is
important that it is not used for messages that are sent to mailing-
lists for example.
[I-D.ietf-spfbis-4408bis] purports to be documenting current use. It
dropped the SPF resource record (99) due to sparse use and recommends
against reverse PTR use. If a similar threshold were applied, all
the SPF macro expansion aspects of this protocol would be removed as
well. Several large providers have offered their assurance that
their SPF libraries had these macros removed. Any effort hoping to
define current use MUST carefully reconsider the inclusion of this
ill-considered macro feature. Otherwise, it can not be suggested
SPF-bis simply documents current use.
Danger imposed by the use of the local-part macro is inherent in the
query required to support both an IPv6 expansion, in conjunction with
the expansion of local-part macros induced by possibly cached SPF
records. The local-part, along with a range of IP addresses made
readily possible by IPv6, can be manipulated to induce a flurry of
large DNS transactions directed toward any otherwise uninvolved
domain, all directed by cached DNS records that contain active
content.
It is never a good idea to allow free attacks enabled through foreign
scripts. The SPF local-part macro would allow a cached DNS record to
Otis & Rand Expires November 29, 2013 [Page 6]
Internet-Draft Macros Nixed May 2013
be repeatedly exploited by a spam campaign with an attacker consuming
little of their resources beyond what they already use in their
campaign. In this case, their recipients change the domains queried
on behalf of the attackers based upon the same SPF resource record.
High levels of amplification still apply if SPF specifications are
not changed and resulting implementation are then assumed part of
some greater security requirement and are no longer deemed
experimental. [dns-response-rate-limit] represents a DNS defense
strategy having limited benefit until most recursive DNS servers also
include Access Control Lists (ACLs). The adoption of the use of ACLs
is far more complex than blocking Directed Broadcasts in Routers
[RFC2644] which remains a problem 14 years later. See:
[smurf-attack]. In addition, even if [dns-response-rate-limit] was
fully implemented, SPF macros offer an effective bypass method for
rate limits imposed on common answers from IPv4 /24 or IPv6 /56
network blocks.
Even The SPF macro DOS overview demonstrated in 2006
[otis-spf-dos-exploit] did not consider IPv6 use which offers higher
gain, where the gain achieved sending a single 1KB message to a
single recipient offered a 1:156 reflected gain over IPv4 that can
never be filtered or blocked by using BCP38 [RFC2827]. When made
against a wildcard record, even higher gains can be achieved. It
should also be noted that [I-D.kucherawy-dmarc-base] section 4.3 "For
example, [DKIM] authenticates the domain that affixed a signature to
the message, while [SPF] authenticates either the domain that appears
in then RFC5321.MailFrom portion of [SMTP] or the RFC5321.EHLO/HELO
domain if the RFC5321.MailFrom is null (in the case of Delivery
Status Notifications). This conflicts with the assertion made by
[I-D.ietf-spfbis-4408bis] section 2.1 that states "If a conclusive
determination about the message can be made based on a check of HELO,
then the use of DNS resources to process the typically more complex
MAIL FROM can be avoided." Few bother to ensure verification of the
EHLO domain, and may even expect the EHLO will not be validated by
SPF. The result of this dichotomy in preferred results will likely
necessitate repeated SPF processing. No cached results can be used
whenever macros have been included and this can significantly
increase network amplifications and not constrain SPF to a single
process layer.
Logically, local-part macros would not safely provide positive
results without the query being combined with an IP address list of
SMTP clients in some form. A list structure would need to be
repeated for each user. Rcpt To using schemes like BATV provides an
expedient way where the purported sender determines whether a NDN can
be returned without the risk associated with active content placed
within DNS resource records. Several parsers ignore local-part macro
expansion since they rarely play any role in providing positive
Otis & Rand Expires November 29, 2013 [Page 7]
Internet-Draft Macros Nixed May 2013
results.
The [I-D.ietf-spfbis-4408bis] Addendum E indicates a possible use of
SPF macros such as exists:{%l}._spf.{%d} or exists:{%i}_spf.{%d} can
be used in "specialized" DNS servers able to understand encrypted
local-parts or react to inordinate query rates which suggests use of
resource records having extremely short TTLs. Such an approach can
add a sizeable burden on recipients and targeted DNS servers. Such a
scheme represents a poor alternative to providing authenticated
domain identifiers. The general intent and purpose of SPF is to
offer Non-Delivery Notification authorization where no macros are
required and by most practical measure are not being used.
4.2. 'do not use' Advice Assumes Senders Cooperate
Problems related to various types of network abuse made possible by
SPF macro extensions are not prevented by assuming corrective
behavior is obtained through advice given to senders. In general, it
is the sender not to be trusted. Only by giving the advice 'do not
use and do not implement' is both security and interchange ensured.
4.3. Retain SPF Type (99)
Upon greater reflection regarding deprecation of the SPF type (99)
record, this was a mistake. When overlaying the TXT resource record
was originally proposed, many strongly objected. Studies then showed
scant use of TXT records which convinced those in the then MARID WG
they could adopt the TXT record at the domain apex as belonging to
SPF as their means to support wildcard use. Although support of
unknown RR types has improved, the vendor that made adoption of a
different RR type problematic still persists with their related name
service extensions imposing barriers. SPF Type (99) should be
retained to better ensure extensions to DNS retain an ability to cope
with unknown RR types.
5. Domains as a Basis for Managing Traffic
A manageable basis for assessments can leverage a smaller number of
related domains, compared to IPv6 or even IPv4 addresses. Although
technically the domain name space can be larger than the massively
large IPv6 address space, in practice it is not. One hundred
thousand domains control 90% of Internet traffic out of approximately
100 million domains active each month. The top 150 domains control
50% of the traffic, and the top 2,500 domains control 75%. This
level of domain consolidation permits effective fast-path white-
listing. Improvements achieved using domains to consolidate the
threat landscape can easily justify added cryptographic
Otis & Rand Expires November 29, 2013 [Page 8]
Internet-Draft Macros Nixed May 2013
authentication burdens. Even APL resource records [RFC3123] can
authenticate EHLO using a single DNS transaction, but this would not
allow IPv6 email to be more easily managed when facing extensive use
of transitional technologies such as ISATAP, Teredo, 6to4, NAT64, and
DNS64, which cryptographic technology can offer.
Unfortunately, the complexity associated with SPF was never intended
to authenticate the SMTP client. SPF instead has become a mechanism
allowing bulk senders a means to shift accountability by offering an
authorization scheme to their customers. Unfortunately, this
authorization mechanism becomes unusable when navigating through
various IPv6 transitional technologies.
6. IANA Considerations
This document requires no IANA consideration.
7. Security Considerations
This draft intends to describe serious security and interchange
concerns with use of SPF macros. The contained recommendations are
expected to reduce these concerns. Greater caution is warranted
before making assumptions about DNS activity. Due to high volumes of
traffic, rarely is this traffic logged directly by the server. Some
items might be triggered while passing through firewalls, but without
being given the item to watch, discovering a well orchestrated attack
is analogous to discovering a needle in the proverbial haystack.
Many administrators overlook a serious problem made much worse by
chatty protocols that impose processing delays. Examining server
logs will not reveal any problem either, because the limited resource
being consumed is the number of outstanding connections TCP is able
to support. Reaching this limit will prevent new connections from
being instantiated but this is not logged as an event. Over time
administrators may hear complaints email is not being delivered or
just see an ever growing percentage of spam.
Add language clearly indicating BOTH the implementation or use of SPF
macro features is hereby deprecated. Any such use provides a neutral
result. In addition, restore SPF type 99 to better insure the future
health of DNS.
8. References - Informative
[I-D.ietf-spfbis-4408bis]
Otis & Rand Expires November 29, 2013 [Page 9]
Internet-Draft Macros Nixed May 2013
Kitterman, D., "Sender Policy Framework (SPF) for
Authorizing Use of Domains in Email, Version 1",
draft-ietf-spfbis-4408bis-15 (work in progress), May 2013.
[I-D.kucherawy-dmarc-base]
Kucherawy, M., "Domain-based Message Authentication,
Reporting and Conformance (DMARC)",
draft-kucherawy-dmarc-base-00 (work in progress),
March 2013.
[RFC1034] Mockapetris, P., "Domain names - concepts and facilities",
STD 13, RFC 1034, November 1987.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", RFC 2460, December 1998.
[RFC2644] Senie, D., "Changing the Default for Directed Broadcasts
in Routers", BCP 34, RFC 2644, August 1999.
[RFC2782] Gulbrandsen, A., Vixie, P., and L. Esibov, "A DNS RR for
specifying the location of services (DNS SRV)", RFC 2782,
February 2000.
[RFC2822] Resnick, P., "Internet Message Format", RFC 2822,
April 2001.
[RFC2827] Ferguson, P. and D. Senie, "Network Ingress Filtering:
Defeating Denial of Service Attacks which employ IP Source
Address Spoofing", BCP 38, RFC 2827, May 2000.
[RFC3123] Koch, P., "A DNS RR Type for Lists of Address Prefixes
(APL RR)", RFC 3123, June 2001.
[RFC4033] Arends, R., Austein, R., Larson, M., Massey, D., and S.
Rose, "DNS Security Introduction and Requirements",
RFC 4033, March 2005.
[RFC4034] Arends, R., Austein, R., Larson, M., Massey, D., and S.
Rose, "Resource Records for the DNS Security Extensions",
RFC 4034, March 2005.
[RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing
Architecture", RFC 4291, February 2006.
[RFC4408] Wong, M. and W. Schlitt, "Sender Policy Framework (SPF)
Otis & Rand Expires November 29, 2013 [Page 10]
Internet-Draft Macros Nixed May 2013
for Authorizing Use of Domains in E-Mail, Version 1",
RFC 4408, April 2006.
[RFC4941] Narten, T., Draves, R., and S. Krishnan, "Privacy
Extensions for Stateless Address Autoconfiguration in
IPv6", RFC 4941, September 2007.
[RFC4954] Siemborski, R. and A. Melnikov, "SMTP Service Extension
for Authentication", RFC 4954, July 2007.
[RFC5321] Klensin, J., "Simple Mail Transfer Protocol", RFC 5321,
October 2008.
[RFC5322] Resnick, P., Ed., "Internet Message Format", RFC 5322,
October 2008.
[RFC5451] Kucherawy, M., "Message Header Field for Indicating
Message Authentication Status", RFC 5451, April 2009.
[RFC5890] Klensin, J., "Internationalized Domain Names for
Applications (IDNA): Definitions and Document Framework",
RFC 5890, August 2010.
[RFC6376] Crocker, D., Hansen, T., and M. Kucherawy, "DomainKeys
Identified Mail (DKIM) Signatures", RFC 6376,
September 2011.
[RFC6530] Klensin, J. and Y. Ko, "Overview and Framework for
Internationalized Email", RFC 6530, February 2012.
[RFC6531] Yao, J. and W. Mao, "SMTP Extension for Internationalized
Email", RFC 6531, February 2012.
[RFC6532] Yang, A., Steele, S., and N. Freed, "Internationalized
Email Headers", RFC 6532, February 2012.
[RFC6577] Kucherawy, M., "Authentication-Results Registration Update
for Sender Policy Framework (SPF) Results", RFC 6577,
March 2012.
[RFC6686] Kucherawy, M., "Resolution of the Sender Policy Framework
(SPF) and Sender ID Experiments", RFC 6686, July 2012.
[dns-response-rate-limit]
http://ss.vix.com/~vixie/isc-tn-2012-1.txt, "DNS Response
Rate Limit", April 2012.
[otis-spf-dos-exploit]
Otis & Rand Expires November 29, 2013 [Page 11]
Internet-Draft Macros Nixed May 2013
http://tools.ietf.org/html/draft-otis-spf-dos-exploit-01,
"Otis SPF DoS Exploitation", June 2006.
[smurf-attack]
http://smurf.powertech.no//, "Smurf Amplifier Registry
(SAR)", May 2013.
[spf-all.com]
http://spf-all.com/stats.html, "SPF-all.com Stats",
May 2013.
Authors' Addresses
Douglas Otis
Trend Micro
10101 N. De Anza Blvd
Cupertino, CA 95014
USA
Phone: +1.408.257-1500
Email: doug_otis@trendmicro.com
Dave Rand
Trend Micro
10101 N. De Anza Blvd
Cupertino, CA 95014
USA
Phone: +1.408.257-1500
Email: dave_rand@trendmicro.com
Otis & Rand Expires November 29, 2013 [Page 12]
