IETF DNSOP Working Group S. Sato Internet-Draft T. Matsuura Expires: September 6, 2007 Y. Morishita JPRS March 5, 2007 BGP Anycast Node Requirements for Authoritative Name Servers draft-sato-dnsop-anycast-node-requirements-01.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on September 6, 2007. Copyright Notice Copyright (C) The IETF Trust (2007). Abstract This memo describes the details of requirements and preconditions for making "Global-Scope" IP anycast nodes for authoritative name servers, with the conformance of the practices in RFC 2182, 2870, 3258 and 4786. This memo is intended to be used for the DNS operators who are going to deploy IP Anycast nodes, especially for the TLD operators. Sato, et al. Expires September 6, 2007 [Page 1] Internet-Draft Anycast Node Requirements March 2007 1. Introduction IP anycast [1] is a technology to share one IP address for the Internet services with multiple server nodes. It is now being deployed for improving service reliability, scalability, and stability. Especially, "BGP anycast" is now being deployed for authoritative name servers, typically for root servers. By applying the IP anycast technology to DNS, name server operators can increase the number of authoritative name server nodes, and distribute them in topologically and geographically diverse locations, without violating the DNS protocol limitations [2] [3]. If IP anycast is appropreately operated for DNS server nodes, it improves the robustness against Denial-of-Service (DoS) Attacks and Distributed Denial-of-Service (DDoS) Attacks and improves the reliability of DNS service. And it improves the DNS total response by decreasing RTT from the clients, and distributes authoritative name servers' load, too. However, IP anycast needs more careful operations for achieving the original goals and improvements. In the IP anycast technology, IP address does not specify the individual (real) end-point for the Internet communications any longer. Which means that the real communication peer can not be specified by the destination IP address only. It means some new risks for the DNS service can be found, due to the increase of physical servers. For example, difficulty of monitoring the availability of the service, and an extra effort for syncing the date of the all DNS server nodes. The original goals of IP anycast are to improve the robustness and the reliability of the services. By introducing IP anycast, the reliability of the entire system must not decrease. Especially, DNS is one of an important infrastructures of the Internet, so, introducing IP anycast in critical DNS authoritative servers should be done carefully. RFC 3258 [4] describes a set of practices to apply IP anycast technology for authoritative name servers. And "Operation of Anycast Services" RFC 4786 [5] describes a series of recommendations and considerations for distribution of services using IP anycast. On the other hand, operators of authoritative name servers can also refer to RFC 2182 [6] and 2870 [7] for general guidances on appropriate practices for authoritative name servers. This memo describes the details of requirements and preconditions for making "Global-Scope" IP anycast nodes for authoritative name servers, with the conformance of the practices in RFC 2182, 2870, Sato, et al. Expires September 6, 2007 [Page 2] Internet-Draft Anycast Node Requirements March 2007 3258 and 4786. And this memo also describes our findings and experiences for making IP anycast nodes for ourself. Authors hope that it is useful for DNS operators who will walk on the same way in the future, especially for TLD operators. Authors focus on "BGP anycast" for making "Global-scope" IP anycast nodes. It is the currently most popular technology for making IP anycast nodes and it is already used for making some root servers and some TLD DNS servers. Authors think a part of the basic point of view can be also applied to "Local-scope" IP anycast. 2. BGP Anycast and DNS Service BGP anycast is a part of IP anycasting technology. It uses a shared IP address block and a shared AS number for BGP anycast nodes, and their nodes are placed in the Internet. Reachability of each nodes are served by BGP routing protocol [8]. Each anycast nodes propagate the routing information of the shared IP address block and AS number by BGP. Each BGP routers in the Internet choose "nearest" node by BGP's best route selection algorithm. That is, the accesses to the shared IP address block will be distributed to the each anycast nodes, depending on the clients' locations and network topologies. BGP anycast can control each anycast nodes by configuring as "local node" or "global node" using BGP's routing framework. Using the "NO_EXPORT" [9] or "NOPEER" [10] community, the "local node" operators can limit distributing the routing information of anycast node only for the directly peering sites. Therefore, the "local node" can localize the access to anycast node from directly peering sites. On the other hand, the "global node" operators apply the normal BGP anycast for its node. In this memo, authors focus on "global node" as main target. When one of BGP anycast nodes goes down, routing informations will be automatically recalculated. The datagrams to the anycast node are automatically rerouted to other anycast nodes. Thus, BGP anycast can provide redundancy for the Internet services. Current BGP anycast is hard to apply for longer-lived transaction service, because the instability of the dynamic routing protocol can carry the packets to different anycast nodes. But most of the DNS queries and answers are based on a single UDP packet, so, DNS is considered to be one of the typical services which BGP anycast can be Sato, et al. Expires September 6, 2007 [Page 3] Internet-Draft Anycast Node Requirements March 2007 applied to. In this memo, authors focus the critical DNS infrastructure, especially root and TLD DNS servers. So, authors only focus a case of "a single IP address for DNS service covered by a single exported route". It means advertisement and withdrawal of a single covering prefix can be tightly coupled to the availability of the single associated service, as described "4.8.1. Multiple Covering Prefixes" of Anycast BCP. In this situation, DNS service providers need an exclusive IP address block which is a provider independent CIDR block and an exclusive AS number for making each anycast nodes. 3. Requirements and Preconditions for IP Anycast DNS Server Nodes As described above, IP anycast is one of the effective ways for improving the robustness and the reliability of the services. In the recent critical authoritative name servers, especially for large TLDs, ability to handle more data, more frequent data updating, and higher reliability than before are required. So, when BGP anycast technology is applied to the critical servers, the operators have to be very careful about their systems. Authors expect that the requirements and preconditions which described by this memo would be useful. In this section, this memo describes requirements and preconditions for making BGP anycast nodes for authoritave name servers in the following two points of view, the Internet access service, and data center. 3.1. Choosing the Internet Access Service For making BGP anycast nodes distributed in the wide area, it is important to make network environment with geographical and network topological diversity. In case of making such network environment, each anycast nodes should have Internet connectivity from different Internet access service provider (hereafter, called ISP) for ensuring network diversity. And in case of ensuring the BGP connectivity, the owner of the authoritative name servers must consider the following preconditions and requirements to choose the Internet access services. Sato, et al. Expires September 6, 2007 [Page 4] Internet-Draft Anycast Node Requirements March 2007 3.1.1. Reliability of the Backbone Network When making a critical authoritative name server, higher reliability for ISP's network itself is needed. For implementing this, it is desirable for ISP itself to have owned and managed its backbone network. In general, an ISP which owns and manages the backbone network itself is expected to have stronger responsibility for network stability. Then it is expectable that the stability of a network is higher. Of course, it is not absolute requirement, but it will surely be one of the important elements. 3.1.2. Connectivity to the Global Internet In case of critical authoritative name servers, such as the servers for root and/or TLD zones, there are many accesses from not only its country and its local area but also foreign area. Thus, connectivity for them must be needed. In the same reason of Section 3.1.1, it is desirable that an ISP which owns and manages the stable connectivity to the global Interenet by itself. 3.1.3. Number of the Peerings When ensuring highly reliable Internet connectivity, the number of the peering is an important element for ensuring the diversity of Internet routes, including multiple alternative paths. Moreover, providing DNS service to multiple ISP networks efficiently, it is desirable for the ISP to have many BGP peers with other ISPs to have much stable connectivity. 3.1.4. Connectivity for Provider Independent CIDR Block and AS Number When making a set of BGP anycast node for critical authritative name servers, a provider independent CIDR block and an AS number must be prepared in advance. The same CIDR block and the AS number must be used for the DNS service at all anycast nodes. It is similar to make a multihomed connectivity. In this case, the ISP must support propagating CIDR block and AS number for anycasting service to the Internet widely, and the ISP must provide connectivity for them from the Internet. Concretely, the ISP must provide "transit" service. As in the case of the multihomed network, each anycast node should improve its network reliability by its own local connectivities. Sato, et al. Expires September 6, 2007 [Page 5] Internet-Draft Anycast Node Requirements March 2007 3.1.5. Connectivity for Operations and Data Synchronization As in RFC 3258, an Internet connectivity which is different from IP anycasting must be needed for anycast node operations. And as in RFC 4786, the synchronization of data between anycast nodes involves transactions between non-anycast addresses. This connectivity should be separated from the connectivies for the DNS service. The assured operations are required at any time, even when the DNS service is not available. When using the same connectivity for both the operations and the DNS service, QoS may work well. 3.1.6. Connectivity for IPv6 RFC 3513 [11] prohibited host-based anycast in IPv6. But RFC 4291 [12] removed this limitation, and RFC 3513 is generally obsoleted. So, there are no protocol limitations for using IP anycasting for IPv6 network by using the same technology as IPv4. But currently, there are less experiences for the IPv6 anycast services. That is, the anycast node owner should take extra care for the IPv6 connectivity. 3.2. Choosing the Location RFC 2182 and 2870 can be refered for useful guidance on appropriate practices for choosing the BGP anycast node locations of the critical authoritative name servers, By referencing them, the owner of the anycast nodes should consider the following preconditions and requirements. 3.2.1. Providing Higher Security Level To realize higher defense performance to physical destruction and/or the intrusion from the outside, the location must provide higher security level. The IP anycast technology may reduce the impact of the failure at single node, but it does not mean the security requirements for the each node are relaxed. Malicious attacks against the physical servers can bring more serious problems, such as data falsifications. 3.2.2. Providing Higher Tolerance Against Disasters DNS service requires high continuity and stability, the location must provide high tolerance against disasters, for example fire, earthquake and other disasters. The IP anycast technology may reduce the impact of the failure at a Sato, et al. Expires September 6, 2007 [Page 6] Internet-Draft Anycast Node Requirements March 2007 single node or location, but the each locations must have high tolerance against disasters by its own. 3.2.3. Providing the Diversity of Locations For ensuring tolerance and redundancy, the diversity of locations is important. Concretely, even if a fatal disaster occurred at one location, other locations should not be damaged by the same factor. The continuity of the critical DNS service must be ensured. To select the locations, the choice of the place can be found from all around the world, because the DNS servers are accessed by the resolvers in the whole Internet. The basis for the selections may come from the measurements. The source addresses of the DNS queries shows the distributions of the clients. RIPE DNSMON can show the RTT from many regions. These measurements are the good hints for the selections of the locations. 4. Cost and Operational Issue In the technical point of view, BGP anycast nodes can be made in numbers of locations. To make the high tolerance to the DDoS attacks, the number of locations should be high. But it is not realistic to prepare them more than necessity. In general, to satisfy the preconditions and requirements which is previously described, BGP anycast node needs high cost, including financial, human and operational resources. In the current condition, this cost is mandatory for making BGP anycast node. Especially, to guarantee the quality of service, for example SLA (Service Level Agreement), needs higher cost than normal Internet connectivity. This is one of big burden for operating BGP anycast node. The authors believe that this is one of in the future issue for deploying IP anycast. Furthermore, for administrating remote anycast nodes smoothly, many human recources are needed, including local and remote technical staffs and operators. When making BGP anycast node for critical DNS service, the owner of authoritative name servers must consider about this issue. 5. Measurement Issue To evaluate the practical effect of IP anycast, for example, to verify whether the selections of the IP anycast nodes are appropriate Sato, et al. Expires September 6, 2007 [Page 7] Internet-Draft Anycast Node Requirements March 2007 or not, objective measurement is very important. When making BGP anycast mesh in the wide area, the measurement must also be carried out in the wide area. In such case, there is an effective guideline defined by ICANN, called "CNNP test" [13]. This guideline is useful for making critical DNS server nodes. The service continuity is another important point for the DNS service. The operators should ensure the continuity of DNS service by measurement. RIPE NCC's DNSMON service [14] is one of typical notable project. However, the cost of the measurement is very high, and it is hard to make and maintain many measurement points/probes. 6. Data Synchronization In this section, this memo describes the effective monitoring of the data synchronization among the anycast nodes. The anycast nodes tend to be widely distributed around the world, and the number of the servers increases as the anycast nodes increse. Thus, the data synchronization among all DNS servers and the monitoring of the synchronization come up as an important issue. 6.1. Synchronization Method The network connectivities between the master DNS server which provides zone data and the slave DNS servers in each anycast nodes may not be so stable. These can be happen when the advantages of the selection of the locations exceed the disadvantage of the network connectivity selections. To support these cases, data transaction should be designed to decrease the amount of the traffic. In particular, zone transfers should be performed in incremental zone transfer (IXFR) method to decrease the traffic and shorten the transaction time. However, from the operational point of view, not only IXFR but AXFR method should be provided to ensure the data. AXFR should be enforced in the case of finding the broken data consistency. All data synchronization transactions must be performed by the unicast addresses. And the the zone transfer source should be duplicated in more than two diverse locations. It is useful for providing the diversity to the network connectivity for the zone transfers. Sato, et al. Expires September 6, 2007 [Page 8] Internet-Draft Anycast Node Requirements March 2007 6.2. Monitoring the Synchcronization To check the synchronization of the all anycast nodes independently, the monitoring of the SOA serial number should be performed to the unicast addresses used for the zone transfers. The checks must be performed frequently, according to the zone update frequency. However, the continuous checks of the SOA serials of all anycast nodes are effective. To have a process only for querying and recording the SOA data, and another process to check the synchroneity of the data will make a effective monitoring system. The monitoring system should be seperated from the DNS servers itself to ensure the objective monitoring of the systems. 7. Our Findings through Making Overseas Anycast nodes In this section, this memo describes our findings for making oversea IP anycast nodes for our DNS server. At this point, these server nodes are still for reserach and development, but when they were made, we did some useful experiences and got some useful findings. Authors hope that it is useful for DNS operators that they will walk on the same way in the future, especially for TLD operators. 7.1. Importance of the Running Cost Running cost is more dominant than initial cost. Human resources and traveling expense are needed for troubleshooting and trouble recovery. Especially for oversea sites, sometimes the linquistic barriers apper as a high cost. For instance, a simple replacement may reduce the total cost from finding and fixing the minimum failure by using remote hands. 7.2. Difference of Business Practices Differences of business parctices among the countries are imporatant issue for practical server node operations. Some data center requires damage insurance contract. But it is hard to have contract with foreign customer. 7.3. Broken Hardware Replacement In many cases, both the server and network hardwares which have a lot of experiences in one country do not have any maintenance availability in other countries. It is important for the gears to Sato, et al. Expires September 6, 2007 [Page 9] Internet-Draft Anycast Node Requirements March 2007 have ability for the maintenance in many countries, especially in the location of the nodes. The gears of the global companies may be better to choose. 7.4. Operation and Maintenance Tools To ensure the operations and maintenances of the system, each anycast nodes should prepare the means to access each gears as many as possible. Those are remote KVM switches, serial console switches, remote management systems of the servers and others. The troubles cannot be all predicted, thus the means of operations should be prepared widely. The remote connectivity without using the Internet, such as dial-up network directly to the local network or the console switches of the anycast node, is also suggested. 7.5. Time Synchronization To make the operation, measurement and trouble shooting easily and effectively, all the anycast nodes must be using the synchronized clocks from the same source. In addition, time zones used for the gears should be uniformed. 8. Security Considerations IP anycast nodes mitigate DoS attack effect and constrain DDoS attack in their local network. It is one of the most important goals of IP anycast. To keep higher security level of each DNS server nodes is one of most important points of managing critical authrotative name servers. Of couese, it is the same as non-anycasted DNS service, but in IP anycasting environment, all of IP anycast nodes have the same IP address for authoritative DNS service, so, it means it is much important than single server because all of nodes should be applied the same higher security policy. In IP anycast environment, number of nodes increases compared with non-anycasting environment. It means the number of the places where the safety of the data must be guaranteed, also increases. And practical secure data synchronization method between nodes must be required, typically data encryption. Sato, et al. Expires September 6, 2007 [Page 10] Internet-Draft Anycast Node Requirements March 2007 9. IANA Considerations This document requests no action from IANA. 10. Acknowledgements Paul Vixie and Bill Manning reviewed a previous version of this document. Joe Abley reviewed a previous version of this document and provided detailed and useful comments. Yoshiki Ishida, Tomoya Yoshida, George Michaelson and Peter Koch provided some useful comments and suggestions. The authors acknowledge many helpful suggestions from the members of JPRS Research and Development Department and System administration Department. This memo is included in the results of the research activities funded by National Institute of Information and Communications Technology (NICT). 11. References [1] Partridge, C., Mendez, T., and W. Milliken, "Host Anycasting Service", RFC 1546, November 1993. [2] Mockapetris, P., "DOMAIN NAMES - CONCEPTS AND FACILITIES", RFC 1034, November 1987. [3] Mockapetris, P., "DOMAIN NAMES - IMPLEMENTATION AND SPECIFICATION", RFC 1035, November 1987. [4] Hardie, T., "Distributing Authoritative Name Servers via Shared Unicast Addresses", RFC 3258, April 2002. [5] Abley, J. and K. Lindqvist, "Operation of Anycast Services", RFC 4786, December 2006. [6] Elz, R., Bradner, S., and M. Patton, "Selection and Operation of Secondary DNS Servers", RFC 2182, July 1997. [7] Bush, R., Karrenberg, D., Kosters, M., and R. Plzak, "Root Name Server Operational Requirements", RFC 2870, June 2000. [8] Rekhter, Y. and T. Li, "A Border Gateway Protocol 4 (BGP-4)", RFC 1771, March 1995. Sato, et al. Expires September 6, 2007 [Page 11] Internet-Draft Anycast Node Requirements March 2007 [9] Chen, E. and J. Stewart, "A Framework for Inter-Domain Route Aggregation", RFC 2519, February 1999. [10] Huston, G., "NOPEER Community for Border Gateway Protocol (BGP) Route Scope Control", RFC 3765, April 2004. [11] Hinden, R. and S. Deering, "Internet Protocol Version 6 (IPv6) Addressing Architecture", RFC 3513, April 2003. [12] Hinden, R. and S. Deering, "IP Version 6 Addressing Architecture", RFC 4291, February 2006. [13] "ICANN Unsponsored TLD Agreement: Appendix D (.info)", May 2001. [14] "RIPE-NCC/DNS Server Monitoring", . Authors' Addresses Shinta Sato Japan Registry Services Co., Ltd. Chiyoda First Bldg. East 13F, 3-8-1 Nishi-Kanda Chiyoda-ku, Tokyo 101-0065 Japan Email: shinta@jprs.co.jp URI: http://jprs.jp/ Takayasu Matsuura Japan Registry Services Co., Ltd. Chiyoda First Bldg. East 13F, 3-8-1 Nishi-Kanda Chiyoda-ku, Tokyo 101-0065 Japan Email: matuura@jprs.co.jp URI: http://jprs.jp/ Sato, et al. Expires September 6, 2007 [Page 12] Internet-Draft Anycast Node Requirements March 2007 Yasuhiro Orange Morishita Japan Registry Services Co., Ltd. Chiyoda First Bldg. East 13F, 3-8-1 Nishi-Kanda Chiyoda-ku, Tokyo 101-0065 Japan Email: yasuhiro@jprs.co.jp URI: http://jprs.jp/ Sato, et al. Expires September 6, 2007 [Page 13] Internet-Draft Anycast Node Requirements March 2007 Full Copyright Statement Copyright (C) The IETF Trust (2007). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Acknowledgment Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). Sato, et al. Expires September 6, 2007 [Page 14]