IETF DNSOP Working Group S. Sato Internet-Draft T. Matsuura Intended status: Informational JPRS Expires: March 30, 2007 Y. Morishita PIE September 26, 2006 BGP Anycast Node Requirements for Authoritative Name Servers draft-sato-dnsop-anycast-node-requirements-00.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on March 30, 2007. Copyright Notice Copyright (C) The Internet Society (2006). Abstract This memo describes the details of requirements and preconditions for making "Global-Scope" IP anycast nodes for authoritative name servers, with the conformance of the practices in RFC 2182, 2870, 3258 and Anycast BCP. This memo is intended to be used for the DNS operators who are going to deploy IP Anycast nodes, especially for the TLD operators. Sato, et al. Expires March 30, 2007 [Page 1] Internet-Draft Anycast Node Requirements September 2006 1. Introduction IP anycast [1] is a technology to share one IP address for Internet services with multiple server nodes. It is now being deployed for improving service reliability, scalability, and stability. Especially, "Global-Scope" IP anycast is now being deployed for authoritative name servers, typically for root servers. By applying the IP anycast technology to DNS, name server operators can increase the number of authoritative name server nodes, and distribute them in topologically and geographically diverse locations, without violating the DNS protocol limitations [2] [3]. If IP anycast is appropreately operated for DNS server nodes, it improves the robustness against Denial-of-Service Attack and Distributed Denial-of-Service Attack and the reliability of DNS service. And it improves the DNS total response by decreasing RTT for authoritative name server, and distributes authoritative name servers' load, too. However, IP anycast needs more careful operation for achieving the original goals and improvements. In the IP anycast technology, IP address does not specify the individual (real) end-point for the Internet communications any longer. Which means that the real communication peer can not be specified by the destination IP address only. It means some new pitfalls and risks the DNS service, for example, monitoring the availability of the service becomes more difficult, and data of all DNS server nodes need syncing. For achieving the original goals of IP anycast, improving the robustness and the reliability of service. So by introduction of IP anycast, The reliability of the entire system must not decrease. Especially, DNS is one of an important infrastructures of the Internet, so, introducing "Global-Scope" IP anycast in critical DNS authoritative servers should be carefully. RFC 3258 [4] describes a set of practices to apply IP anycast technology for authoritative name servers. And "Operation of Anycast Services" Internet-Draft [5] (hereafter, called "Anycast BCP") describes a series of recommendations and considerations for distribution of services using IP anycast. On the other hand, operators of authoritative name servers can also refer to RFC 2182 [6] and 2870 [7] for general guidances on appropriate practices for authoritative name servers. This memo describes the details of requirements and preconditions for Sato, et al. Expires March 30, 2007 [Page 2] Internet-Draft Anycast Node Requirements September 2006 making "Global-Scope" IP anycast nodes for authoritative name servers, with the conformance of the practices in RFC 2182, 2870, 3258 and Anycast BCP. And this memo also describes our findings and experiences for making IP anycast nodes for us. Authors hope that it is useful for DNS operators that they will walk on the same way in the future, especially for TLD operators. In this memo, authors focus on "BGP anycast" for making "Global- scope" IP anycast nodes. It is the currently most popular technology for making IP anycast nodes and it is already used for making some root and TLD DNS server nodes. Authors think a part of the basic point of view can be applied to "Local-scope" IP anycast, too. 2. BGP Anycast and DNS Service BGP anycast is a part of IP anycasting technology. It uses a shared IP address block and a shared AS number for BGP anycast nodes, and their nodes are placed in the Internet. Reachability of each nodes are served by BGP routing protocol [8]. Each anycast nodes propagate the routing information of the shared IP address block and AS number by BGP. Each BGP routers in the Internet choose 'nearest' node by BGP's best route selection algorithm. That is, the accesses to the shared IP address block will be distributed to the each anycast nodes, depending on the clients' locations and network topologies. BGP anycast can control each anycast nodes by configuring as 'local node' or 'global node' using BGP's routing framework. Concretely, using the 'NO_EXPORT' [9] or 'NOPEER' [10] community, the 'local node' operators can limit distributing the routing information of anycast node only for the directly peering sites. Therefore, the 'local node' can localize the access to anycast node from directly peering sites. On the other hand, the 'global node' operators apply the normal BGP anycast for its node. In this memo, authors focus on 'global node' as main target, but authors believe it can be applied as 'local node' too. When one of BGP anycast nodes goes down, routing informations will be automatically recalculated. The datagrams to the anycast node are automatically rerouted to other anycast nodes. Thus, BGP anycast can provide redundancy for the Internet services. Current BGP anycast is hard to apply for longer-lived transaction Sato, et al. Expires March 30, 2007 [Page 3] Internet-Draft Anycast Node Requirements September 2006 service, because the instability of the dynamic routing protocol is harmful for them. But most of the DNS queries and answers are based on a single UDP packet, so, DNS is considered to be one of typical service in which BGP anycast can be applied. In this memo, authors focus the critical DNS infrastructure, especially root and TLD DNS servers. So, authors only focus a case of "a single IP address for DNS service covered by a single exported route". It means advertisement and withdrawal of a single covering prefix can be tightly coupled to the availability of the single associated service, as described "4.8.1. Multiple Covering Prefixes" of Anycast BCP. In this situation, DNS service providers need an exclusive IP address block which is a provider independent CIDR block and exclusive an AS number for making each anycast nodes. 3. Requirements and Preconditions for Critical DNS Server Nodes As described before, IP anycast is one of the effective ways for improving the robustness and the reliability of service. And in recent critical authoritative name servers, especially for large TLDs, ability to handle more data than before, more frequent data updating, and higher reliability are required. So, when BGP anycast technology is applied to their critical servers, carefulness must be necessary for their operators. Authors expect that the requirements and preconditions which described by this memo would be useful. In this section, this memo describes requirements and preconditions for making BGP anycast nodes for authoritave name servers in the following two points of view, the Internet access service, and data center. 3.1. Choosing the Internet Access Service For making BGP anycast nodes distributed in the wide area, it is important to make network environment with geographical and network topological diversity. In case of making such network environment, each anycast nodes should have Internet connectivity from different Internet access service provider (hereafter, called ISP) for ensuring network diversity. And in case of ensuring the BGP connectivity, the owner of the authoritative name servers must consider the following preconditions Sato, et al. Expires March 30, 2007 [Page 4] Internet-Draft Anycast Node Requirements September 2006 and requirements to choose the Internet access services. 3.1.1. Reliability of the Backbone Network When making a critical authoritative name server, higher reliability for ISP's network itself is needed. For implementing this, it is desirable for ISP itself to have owned and managed its backbone network. In general an ISP, which owns and manages the backbone network itself, is expected to have stronger responsibility for network stability. Then it is expectable that the stability of a network is higher. Of course, it is not absolute requirement, but it will surely be one of the important elements. 3.1.2. Connectivity of Outside Area In case of critical authoritative name servers, such as served for root and/or TLD zones, there are many accesses from not only its country and its local area but also outside area. Thus, connectivity for them must be needed. In the same reason of Section 3.1.1, it is desirable that an ISP which owns and manages the stable connectivity of all areas. 3.1.3. Peering When ensuring highly reliable Internet connectivity, it is an important element for ensuring the diversity of Internet routes including multiple alternative paths. Moreover, providing DNS service to multiple ISP networks efficiently, it is desirable for the ISP to have many BGP peers with other ISPs, and they are much stable. 3.1.4. Connectivity for Provider Independent CIDR Block and AS Number When making a set of BGP anycast node for critical authritative name servers, a provider independent CIDR block and an AS number must be prepare in advance, and they must be used for DNS service at each anycast nodes. It is also needed for making the multihomed connectivity. In this case, the ISP must support propagating CIDR block and AS number for anycasting service to the Internet widely, and the ISP must provide connectivity for them from the Internet. Concretely, the ISP must provide "transit" service. Sato, et al. Expires March 30, 2007 [Page 5] Internet-Draft Anycast Node Requirements September 2006 3.1.5. Connectivity for Administration and Data Synchronization As in RFC 3258, an Internet connectivity which is different for IP anycasting must be needed for anycast node administration. And as in Anycast BCP, the synchronisation of data between anycast nodes will involve transactions between non-anycast addresses. 3.1.6. Connectivity for IPv6 RFC 3513 [11] prohibited host-based anycast in IPv6. But new version of RFC 4291 [12] removes this limitation and it RFC 3513 is generally obsoleted. So, now there are no protocol limitations for using IP anycasting for IPv6 network by using the same technology as IPv4. But authors think that more experiences are needed for deployment of IPv6 anycasting. That is, the anycast node owner should ensure IPv6 connectivity. 3.2. Choosing the Location For choosing the BGP anycast node locations for critical authoritative name servers, RFC 2182 and 2870 can be refered for useful guidance on appropriate practices for them. By referencing them, when choosing the location for BGP anycast node for critical authoritative name servers, the owner of them must consider the following preconditions and requirements. 3.2.1. Providing Higher Security Level To realize higher defense performance to physical destruction and/or the intrusion from the outside, the location must provide higher security level. 3.2.2. Providing Higher Redundancy of Electrical Power Supply DNS service requires high continuity and stability, the location must provide higher redundancy of electrical power supply and urgent power supply equipment for emergency. 3.2.3. Providing Higher Tolerance Against Disasters For the same reason of Section 3.2.2, the location must provide higher tolerance against disasters, for example fire, earthquake and other disasters. Sato, et al. Expires March 30, 2007 [Page 6] Internet-Draft Anycast Node Requirements September 2006 3.2.4. Providing the Diversity of Locations For ensuring tolerance and redundancy, the diversity of locations is needed. Concretely, even if a fatal disaster occurred at one location, the continuity of critical DNS service must be ensured. 4. Cost and Operational Issue In the technical point of view, BGP anycast nodes can be made in numbers of locations. But it is not realistic to prepare them more than necessity. In general, to satisfy the preconditions and requirements which is previously described, BGP anycast node needs high cost, including financial, human and operational resources. In the current condition, this cost is mandatory for making BGP anycast node. Especially, to guarantee the quality of service, for example SLA (Service Level Agreement), needs higher cost than normal Internet connectivity. This is one of big burden for operating BGP anycast node. The authors believe that this is one of in the future issue for deploying IP anycasting. Furthermore, for administrating remote anycast nodes smoothly, many human recources are needed, including local and remote technical staffs and operators. When making BGP anycast node for critical DNS service, the owner of authoritative name servers must consider about this issue. 5. Measurement Issue To evaluate the practical effect of IP anycast, for example, to verify whether the selections of the IP anycast nodes are appropriate or not, objective measurement is very important. When making BGP anycast mesh in the wide area, the measurement must also be carried out in the wide area. In such case, there is an effective guideline defined by ICANN, called "CNNP test" [13]. This guideline is useful for making critical DNS server nodes. But the cost of the measurement is very high, and it is hard to make and maintain for many measurement points/probes. The continuity is one of an important points for measurement. And operators should verify that the continuity of DNS service is ensured by measurement. RIPE NCC's DNSMON service [14] is one of typical notable project. Sato, et al. Expires March 30, 2007 [Page 7] Internet-Draft Anycast Node Requirements September 2006 6. Our Findings through Making Oversea Anycast nodes In this section, this memo describes our findings for making oversea IP anycast nodes for our DNS server. At this point, these server nodes are still for reserach and development, but when they were made, we did some useful experiences and got some useful findings. Authors hope that it is useful for DNS operators that they will walk on the same way in the future, especially for TLD operators. 6.1. Running Cost is More Dominant Running cost is more dominant than initial cost. Human resources and traveling expense for troubleshooting and trouble recovery. Especially for oversea site, sometimes higher the wall of language exists. For instance, a simply replacement sometimes may reduce the total cost from fixing by using remote hands. 6.2. Difference of Custom Practices Difference of custom of each country is sometimes imporatant issue for practical server node operation. Some data center requires damage insurance contract. But it is hard to have contract with foreign customer. As a result, we had to contact and negotiate a lot of insurance companies. 6.3. Others to Remind During our operations, we encountered some unexpected trouble. In this section, this memo describes the term of them. 6.3.1. Thermal Overheat Thermal overheat is occurred due to cabinet placement and ventilation. So, we had to order rack rotation work and additional cabinet fans. 6.3.2. Broken Hardware Replacement Due to our contract of agent, broken hardware replacement must be "parts-based", not a whole of server hardware. So we had to need additional work for fixing server at oversea place. We cannot order the remote hands because the work is so complex and difficult. 7. Other Considerations and Issues In this section, this memo describes other related considerations and issues for making BGP anycast nodes for critical authoritative name Sato, et al. Expires March 30, 2007 [Page 8] Internet-Draft Anycast Node Requirements September 2006 servers. Authors will describe requirements and preconditions for these issues, but not yet issued. o Selection of Node Locations o Selection of Server Hardware o Selection of Server Software o Selection of Remote Maintenance Tools o Effective Remote Maintenance o Effective Measurement 8. Security Considerations IP anycast mitigates Denial-of-Service attack effect and constrains Distributed Denial-of-Service attack in their local network. It is one of most important goals of IP anycast. To keeping higher security level of each DNS server nodes is one of most important points of managing critical authrotative name servers. Of couese, it is the same as non-anycasted DNS service, but in IP anycasting environment, all of IP anycast nodes have the same IP address for authoritative DNS service, so, it means it is much important than single server because all of nodes should be applied the same higher security policy. In IP anycasting environment, number of nodes increases compared with non-anycasting environment. It means the place where the safety of data must be guaranteed also increases. And practical secure data synchronization method between nodes must be required, typically data encryption. 9. IANA Considerations This document requests no action from IANA. 10. Acknowledgements Paul Vixie and Bill Manning reviewed a previous version of this document. Joe Abley reviewed a previous version of this document and provided detailed and useful comments. Yoshiki Ishida, Tomoya Yoshida, George Michaelson and Peter Koch provided some useful comments and suggestions. The authors acknowledge many helpful suggestions from the members of JPRS Research and Development Department and System administration Sato, et al. Expires March 30, 2007 [Page 9] Internet-Draft Anycast Node Requirements September 2006 Department. This memo is included in the results of the research activities funded by National Institute of Information and Communications Technology (NICT). 11. References [1] Partridge, C., Mendez, T., and W. Milliken, "Host Anycasting Service", RFC 1546, November 1993. [2] Mockapetris, P., "DOMAIN NAMES - CONCEPTS AND FACILITIES", RFC 1034, November 1987. [3] Mockapetris, P., "DOMAIN NAMES - IMPLEMENTATION AND SPECIFICATION", RFC 1035, November 1987. [4] Hardie, T., "Distributing Authoritative Name Servers via Shared Unicast Addresses", RFC 3258, April 2002. [5] Abley, J. and K. Lindqvist, "Operation of Anycast Services", draft-ietf-grow-anycast-04.txt (work in progress), January 2006. [6] Elz, R., Bradner, S., and M. Patton, "Selection and Operation of Secondary DNS Servers", RFC 2182, July 1997. [7] Bush, R., Karrenberg, D., Kosters, M., and R. Plzak, "Root Name Server Operational Requirements", RFC 2870, June 2000. [8] Rekhter, Y. and T. Li, "A Border Gateway Protocol 4 (BGP-4)", RFC 1771, March 1995. [9] Chen, E. and J. Stewart, "A Framework for Inter-Domain Route Aggregation", RFC 2519, February 1999. [10] Huston, G., "NOPEER Community for Border Gateway Protocol (BGP) Route Scope Control", RFC 3765, April 2004. [11] Hinden, R. and S. Deering, "Internet Protocol Version 6 (IPv6) Addressing Architecture", RFC 3513, April 2003. [12] Hinden, R. and S. Deering, "IP Version 6 Addressing Architecture", RFC 4291, February 2006. [13] "ICANN Unsponsored TLD Agreement: Appendix D (.info)", May 2001. Sato, et al. Expires March 30, 2007 [Page 10] Internet-Draft Anycast Node Requirements September 2006 [14] "RIPE-NCC/DNS Server Monitoring", . Authors' Addresses Shinta Sato Technical Services Dept., Japan Registry Services Co., Ltd. Chiyoda First Bldg. East 13F, 3-8-1 Nishi-Kanda Chiyoda-ku, Tokyo 101-0065 Japan Email: shinta@jprs.co.jp URI: http://jprs.jp/ Takayasu Matsuura Technical Services Dept., Japan Registry Services Co., Ltd. Chiyoda First Bldg. East 13F, 3-8-1 Nishi-Kanda Chiyoda-ku, Tokyo 101-0065 Japan Email: matuura@jprs.co.jp URI: http://jprs.jp/ Yasuhiro Orange Morishita Pacific Internet Exchange LLC. M-200 200 Paul Ave. San Francisco, CA 94124 USA Email: orange (at) pie.us URI: http://PIE.us/ Sato, et al. Expires March 30, 2007 [Page 11] Internet-Draft Anycast Node Requirements September 2006 Full Copyright Statement Copyright (C) The Internet Society (2006). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Acknowledgment Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). Sato, et al. Expires March 30, 2007 [Page 12]