Internet Engineering Task Force B. Khasnabish Internet-Draft ZTE USA, Inc. Intended status: Informational J. Chu Expires: December 30, 2012 S. Ma Y. Meng ZTE N. So Tata Communications P. Unbehagen Avaya M. Morrow Cisco Systems [Switzerland] GmbH M. Hasan Cisco Systems Y. Demchenko University of Amsterdam June 28, 2012 Cloud Reference Framework draft-khasnabish-cloud-reference-framework-03.txt Abstract This document presents a cloud reference framework. In general, a cloud-based system utilizes virtualized computing/communications/ storage resources and applications. In the emerging cloud-based systems, virtualized infrastructures and services are provisioned on on-demand basis, and configured for specific customer needs or tasks. The reference framework is based on the survey of the SDOs and WGs that are focusing on cloud-based systems and services (Cloud SDO, I-D.Khasnabish-cloud-sdo-survey) and other research and developments in the cloud technology area. Both intra-cloud and inter-cloud reference frameworks are presented and the requirements of each layer are discussed. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any Khasnabish, et al. Expires December 30, 2012 [Page 1] Internet-Draft Cloud Reference Framework June 2012 time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on December 30, 2012. Copyright Notice Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English. Khasnabish, et al. Expires December 30, 2012 [Page 2] Internet-Draft Cloud Reference Framework June 2012 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 7 3. Reference Cloud Services Model . . . . . . . . . . . . . . . . 8 3.1. HORIZONTAL/CSM LAYERS . . . . . . . . . . . . . . . . . . 9 3.1.1. Application/Service Layer . . . . . . . . . . . . . . 9 3.1.2. Resources Control Layer . . . . . . . . . . . . . . . 10 3.1.3. Resources Abstraction and Virtualization Layer . . . . 11 3.1.4. Physical Resources Layer . . . . . . . . . . . . . . . 12 3.2. VERTICAL LAYERS . . . . . . . . . . . . . . . . . . . . . 13 3.2.1. Cloud Management Layer -> Cloud Management Plane . . . 13 4. Inter-Cloud Framework . . . . . . . . . . . . . . . . . . . . 20 4.1. Inter-Cloud Requirements . . . . . . . . . . . . . . . . . 20 4.2. Intercloud Control and Management Plane (ICCMP) . . . . . 22 4.3. Intercloud Federation Framework (ICFF) . . . . . . . . . . 23 4.4. Intercloud Operation Framework (ICOF) . . . . . . . . . . 24 5. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 26 5.1. Virtual Network Management . . . . . . . . . . . . . . . . 26 5.2. Telecom Network Virtualization . . . . . . . . . . . . . . 26 5.3. Virtual Data Center . . . . . . . . . . . . . . . . . . . 28 5.4. Security Framework for VDCS . . . . . . . . . . . . . . . 29 6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 31 7. Security Considerations . . . . . . . . . . . . . . . . . . . 32 8. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 33 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 10. Normative references . . . . . . . . . . . . . . . . . . . . . 35 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 38 Khasnabish, et al. Expires December 30, 2012 [Page 3] Internet-Draft Cloud Reference Framework June 2012 1. Introduction Clouds are emerging as a common way of provisioning virtualised infrastructure services that are provisioned on demand (and configured for specific customer needs or tasks). Current development of the cloud technologies demonstrate movement to developing Intercloud models, architectures and integration tools that could allow integrating cloud based infrastructure services into existing enterprise and campus infrastructures, on one hand, and provide common/interoperable environment for moving existing infrastructures and infrastructure services to virtualised cloud environment. More complex and enterprise oriented use of cloud infrastructure services will require developing new service provisioning and security models that could allow creating complex project and group oriented infrastructures provisioned on-demand and across multiple providers. This document presents a general cloud reference framework that includes the proposed multilayer Cloud Services Model (CSM) and defines the Intercloud framework to address interoperability and integration issues in provisioning multi-domain heterogeneous cloud based infrastructures and services. The proposed CSM defines the basic functional layers to support the cloud based infrastructure services and applications virtualization, composition, delivery and operation. The CSM provides a basis for building Cloud Service Provider (CSP) internal infrastructure (or datacenter) that could also address inter-provider interoperability and cloud based services integration. The proposed Intercloud (Architecture) Framework (ICF/ICAF) addresses a number of issues in ensuring interoperability of cloud infrastructures built using different cloud software stacks/platform and provided by multiple providers. It should address Intercloud service control and management (under one cloud operator or integrator), Intercloud federation that should allow interoperability and integration of administratively independent cloud domains, and general cloud services provisioning and operation. The proposed in this document Intercloud Framework intends to provide a basis for inter-cloud infrastructures and services integration and interoperability that could span multiple providers, multiple management domains and include mutli-platform and multi-technology components/domains. The definition of the Intercloud Framework is based on previous experience of (of Internet community in) building large scale interoperable telecommunication and Internet systems and uses best practices and recommendations from the wide range of industry standards by IETF, OGF, ITU-T, TMF, DMTF and other Khasnabish, et al. Expires December 30, 2012 [Page 4] Internet-Draft Cloud Reference Framework June 2012 standardisation bodies specifically oriented on cloud technologies. The presented Cloud reference framework can be used to a) define requirements to the main cloud infrastructure functional elements and other components of the general Internet infrastructure to consistently support cloud services/infrastructure, b) define the interfaces between the functional elements, and c) propose further standardisation. It is important to mention that correct/consistent defintion of the general Cloud Services Model will provide a basis for seamless/smooth inter-cloud integration and operation. We develop a general cloud reference framework. This reference framework describes basic functions in different layers to support the requirements of virtualized applications and services. This reference framework can be used to standardize a) features of functional elements and b) the interfaces between the functions. Basically, the cloud reference framework includes o Five horizontal layers * Data/Content Layer(DCL) * Application/Service Layer(ASL) * Resource Control Layer(RCL) * Resource Abstract and Virtualization Layer(RAVL) * Physical Resource Layer(PRL) o One stacked vertical layer to support * Configuration management, registry, logging and auditing, security management, and service level agreement (SLA) management The Cloud Services Model includes another important components the security services (infrastructure) which task is to ensure normal operation of cloud services, protect user data, and enforce security, access control (authentication and authorisation) and operational policies at all layers of the cloud services model. In clouds, security services also need to be provisioned together with the provisioned on-demand main cloud services; in this respect cloud security infrastructure should support consistent security context and security sessions management during the whole lifecycle of the Khasnabish, et al. Expires December 30, 2012 [Page 5] Internet-Draft Cloud Reference Framework June 2012 provisioned cloud services. Khasnabish, et al. Expires December 30, 2012 [Page 6] Internet-Draft Cloud Reference Framework June 2012 2. Terminology Clouds Discussion Archive: http://www.ietf.org/mail-archive/web/clouds/current/maillist.html IETF Wiki Website for slides from Clouds bar BoFs: http://trac.tools.ietf.org/area/app/trac/wiki/Clouds Khasnabish, et al. Expires December 30, 2012 [Page 7] Internet-Draft Cloud Reference Framework June 2012 3. Reference Cloud Services Model +-----------------------+ +-------------------+ | Data/Content Layer | | Cloud Portal | | +------+ +---------+ | |(Public & Private) | | | Data | | Content | | +-----------------+ +-------------------+ | +------+ +---------+ | | | | +-----------------------+ | | | | | | | | | | +----------------------------------------------------------------+ | | | Application/Service Layer | | Cloud | | +-----------+ +----+ +---------------------------------------+ | | Management | | | | | | | SaaS(Applications) | | | | | | | | | | +------------+ +--------------------+ | | | | | | | | | | |BusinessApps| | ConsumerApps | | | | | | | | | | | |(M-payment) | |(Mobile Data backup)| | | | | | | +-------+ | | | | +------------+ +--------------------+ | | | | | | |Desktop| | | | | +------------+ +--------------------+ | | | | | | +-------+ | | | | |NetworkApps | | CommunicationApps | | | | | | | | | | | |(Hosted PBX)| |(VoIP, Video Serv.) | | | | +-------------+ | | | | | | | +------------+ +--------------------+ | | | |Configuration| | | | | | | +---------------------------------------+ | | | Management | | | | | | +-----------------------------------------+ | | +-------------+ | | | +-------+ | | PaaS(Software Environment) | | | | | | |Server | | | +-----------+ +-----------+ | |<-->| +-------------+ | | | +-------+ | | |Development| |Test | | | | | Registry & | | | | | | |Environment| |Environment| | | | | Repository | | | | | | +-----------+ +-----------+ | | | +-------------+ | | | | +----------------------------------------------+ | | | | | +------------------------------------------------+ | | +-------------+ | | | IaaS(Infrastructure) +----------+ +--------+ | | | | Audit & | | | | | Database | |Security| | | | | Logging | | | | +----------+ +--------+ | | | +-------------+ | | | +----------+ +--------+ | | | | | | |MiddleWare| | VLAN | | | | +-------------+ | | | +----------+ +--------+ | | | | SLA | | | +------------------------------------------------------------+ | | +-------------+ | +----------------------------------------------------------------+ | | | | | | +-------------+ | | | | | | Security | | +----------------------------------------------------------------+ | +-------------+ | | Resource Control Layer | | | | +---------+ +--------+ +--------+ +------------+ +-----------+ | | | | |Resource | |Resource| |Resource| |Resource | |Inter-Cloud| | | | | |Admission| |Authen. | |Schedule| |Availability| |Resource | |<-->| | | |Control | |&Author.| |Control | |Control | |Control | | | | Khasnabish, et al. Expires December 30, 2012 [Page 8] Internet-Draft Cloud Reference Framework June 2012 | | | |Control | | | | | | | | | | | +---------+ +--------+ +--------+ +------------+ +-----------+ | | | +----------------------------------------------------------------+ | | | | | | | | | | | | +----------------------------------------------------------------+ | | | Resource Abstract&Virtualization Layer | | | | +------------------------------------------------------------+ | | | | | Virtualized Resource | | | | | | +---------+ +--------+ +------+ +------------+ +-----+ | | | | | | | V- | | V- | | V- | | V- | | VPN | | | | | | | |Computing| |Storage | |Switch| |Netwk Intf. | +-----+ | | | | | | +---------+ +--------+ +------+ +------------+ | | | | | | +---------+ +--------+ +------+ +------------+ | | | | | | | V- | | V- | | V- | | V- | +-----+ | | | | | | |Database | |FireWall| |Router| |Network Link| |Other| | | | | | | +---------+ +--------+ +------+ +------------+ +-----+ | | | | | +------------------------------------------------------------+ |<-->| | |----------------------------------------------------------------| | | | +----+ +----+ +----+ +----+ +----+ | | | | | VM | | VM | | VM | | VM | | VM | | | | | +------------------------------------------------------------+ | | | | | Hypervisor | | | | | +------------------------------------------------------------+ | | | +----------------------------------------------------------------+ | | | | | | | | | | | | +----------------------------------------------------------------+ | | | Physical Resource Layer | | | | +----------+ +-------------+ +-------------------------------+ | | | | | SERVER | | STORAGE | | NETWORK | |<-->| | | | +------+ | | +---------+ | | +------+ +--------+ +------+ | | | | | | | CPU | | | |Hard Disk| | | |Router| |FireWall| |Switch| | | | | | | +------+ | | +---------+ | | +------+ +--------+ +------+ | | | | | | +------+ | | | | +-----------+ +------------+ | | | | | | |MEMORY| | | | | |Netwk Intf.| |Network Link| | | | | | | +------+ | | | | +-----------+ +------------+ | | | | | +----------+ +-------------+ +-------------------------------+ | | | +----------------------------------------------------------------+ +-----------------+ 3.1. HORIZONTAL/CSM LAYERS 3.1.1. Application/Service Layer Application/Service Layer defines the requirements of the basic functional entities based on the virtual resources needed to perform any tasks. The tasks are classified according to the 3 services Khasnabish, et al. Expires December 30, 2012 [Page 9] Internet-Draft Cloud Reference Framework June 2012 models IaaS, PaaS & SaaS. Some cloud services are illustrated as an example of applications like: o Server, desktop, database and VLAN for IaaS, o Development environment and test environment for PaaS, o Business, consumer, network and communication applications for SaaS. The requirements the basic functional entities provided include the following characteristics and parameters of the virtual resources: o Type of resources: CPU, memory, hard disk space, bandwidth, latency, jitter, and so on o Amount of resources o Nature of the resources: dedicated vs. shared, transport media exclusions, and so on o Timing of the resources: scheduled vs. on-demand o Duration of the Resources 3.1.2. Resources Control Layer Resources Control Layer manages the virtual resources, ensuring that the resources are efficient, secure and reliable. With the interface of virtual resources, the layer integrates the resources as a whole supplied to upper layer. The layer has the following responsibilities: o Resource security management. Resources must be accessed and owned by the right user, there are several function modules to fulfill this responsibility, include resource admission control, resource authentication and authorization control; o Resource schedule control. The layer manages resources in form of resource pool. In a resource pool, the layer balances the virtual resources on a set of physical equipments to achieve higher hardware utilization. Virtual resources can be migrated between physical equipments if necessary, and also can be allocated according to user's priority grade. o Inter-cloud resource control. Resources in a cloud can be shared with another cloud in some circumstances, so a cloud must control resources in other cloud, and supply cloud service to end users. Khasnabish, et al. Expires December 30, 2012 [Page 10] Internet-Draft Cloud Reference Framework June 2012 End users have no need to know where the resources are from. o Resource availability control. The layer supports fault-tolerance on resources. It can allocate another copy of resources as a backup, and switch over when some faults raised. o Services Lifecycle Management. This functional component is needed to support resources provisioning process/staes for an instant cloud infrastructure or service. 3.1.3. Resources Abstraction and Virtualization Layer Physical resources at the lowest level are the most complex to share among multiple users. There are several hardware details that don't need to visible to users, so we need a level of abstraction. In fact, these physical resources are abstracted first, next composed by the cloud management software (at composition and abstraction layer) and finally deployed as virtual resources on the virtualised physical resources. The function of resources abstraction and virtualization layer is to convert physical resources to virtual resources. Virtual resources are contained in resource pool. Resources can be allocated to users from the resource pool, and released to resource pool when it's not needed. Virtual resources are isolated from physical equipments, and have the features: o Have all features as physical resources, resource users can't distinguish the difference between them; o Can be allocated and released on demand; o Support heterogeneous physical equipments, and supply a consistency view of resources to users; o Support resource mobility, virtual resource can move from one physical equipment to another seamlessly; There are several types of resources, such as computing resource, storage resource, database, bandwidth and network. According to the type of resource, there are different methods to realize virtualization. The variety function modules for virtualization are contained in Resources abstraction and virtualization layer. The layer has the following responsibilities: o Through the interface of physical equipment to manage physical resource, mapping the virtual resources to physical resource; Khasnabish, et al. Expires December 30, 2012 [Page 11] Internet-Draft Cloud Reference Framework June 2012 o Supply the interface to upper layer to manage and access virtual resources; o Hide the details of physical equipments, mask the difference between physical equipments. 3.1.3.1. Networking (Resources) Layer Networking (Resources) layer converts and communicates network (LAN/ MAN/WAN) capabilities and capacities(such as Bandwidth, ports, Latency matrices, Jitter matrices, Availability, Restoration capabilities, etc) into a set of resource pools that can be understood and used by the above layers. The resource pools include o Virtual Switch o Virtual Router o Virtual Firewall o Virtual Network Interface o Virtual Network Link o VPN 3.1.4. Physical Resources Layer Physical Resources Layer include o CPU o Memory o Hard Disk o Network Interface Card o Network Link * Ports * Bandwidth Khasnabish, et al. Expires December 30, 2012 [Page 12] Internet-Draft Cloud Reference Framework June 2012 3.2. VERTICAL LAYERS 3.2.1. Cloud Management Layer -> Cloud Management Plane Cloud Management Layer (CML) provides monitoring and administration of the cloud network platform to keep the whole cloud operating normally. Key features of the Virtual System Management Layer include: o Automatically deploying the cloud system based on the configuration data and policy o Real-time monitoring and alerting of cloud status, resource usage and performance of cloud o Reporting and charting of historical events and performance metrics o Flexible IT management and operational status displays o Authenticating/Authorizing the published cloud service registry o Auditing the cloud environment to check whether its running smoothly o Controlling the SLA implemented in the cloud system o Maintenance concerned with performing repairs, upgrades and new nodes join into the Cloud o Providing Security mechanism for the Cloud Basically CML includes four Functions: o Cloud Configuration Management o Cloud Service Registry and Audit Management o Cloud SLA Management o Cloud Service Security Management 3.2.1.1. Cloud Configuration Management Cloud Configuration Management (CCM) is responsible for establishing and maintaining the consistent performance of the Clouds system or product and its functional and physical attributes throughout its Khasnabish, et al. Expires December 30, 2012 [Page 13] Internet-Draft Cloud Reference Framework June 2012 life-cycle. It mainly focuses on configuring the cloud system and retrieving the configuration information automatically. Requirements on Configuration Management are as follows: o Provide efficient and reliable means to provision large amounts of configuration data. Current versions of provision configuration data are CLI, SNMP and NETCONF. o Provide secure means to provision configuration data. The system must provide support for access control, authentication, integrity-checking, replay- protection and/or privacy security services. o Provide means to send feedback information to the management system. Feedback information include configuration data confirmation, network status and monitoring information, specific events, etc. o Provide expiration time and effective time capabilities to configuration data. It is required that some configuration data items be set to expire, and other items be set to never expire. o Provide facilities to help in tracing back configuration changes o Be flexible and extensible to accommodate future needs. Configuration management data models are not fixed for all time and are subject to evolution like any other management data model. o Leverage knowledge of the existing SNMP management infrastructure, such as the knowledge of and experience with MIBs and SMI. o Basically, the CM includes CM database, CM policy, system change management and version management. o Related protocol: CLI, SNMP, NETCONF 3.2.1.2. Cloud Service Registry/Repository Service Registry/Repository provides management and governance capabilities that enable the published cloud service to be authenticated in the cloud system and accessed by service client. It facilitates storing, accessing and managing service information, called service metadata, so that the cloud service can be easily published, selected, invoked, enriched, governed and reused. Requirements on Service Registry/Repository are as follows: Khasnabish, et al. Expires December 30, 2012 [Page 14] Internet-Draft Cloud Reference Framework June 2012 o Authentication & Authorization. Once a service is published by the service provider to the Cloud system, it should be authenticated to check the authority of the provider and the support capability of the Cloud. If the check is passed, the service is authorized and put into the repository, and the services and related metadata are classified into groups. o Publication & Discovery. The authorized service is published in the Cloud system, and you can keep an accurate record of the deployed services in your repository platform. The user can find the service from the repository platform using the service discovery engine. Cloud Service Registry/Repository is capable of a powerful query mechanism allows you to search and find the services that best fit the requirements of a given process. o Service Access Control. The service repository enables dynamic and efficient access to services information by enabling selection of services based on service metadata. o Optimize service usage. Service manage capability enables management of service metadata, as well as service interactions, dependencies and redundancies. You can classify services based on business objectives, manage policies for service usage and monitor how services are changed and versioned. This capability helps you optimize the use of services in cloud system by exchanging service metadata with runtime monitoring tools and operational data stores. o Impact analysis. By maintaining relationships, Cloud Service Registry/Repository has extensive support for analyzing the impact of service introduction, deletion or alteration. o Service life cycle. By creating user-definable entities and customizing the service life cycle, you can configure Cloud Service Registry/Repository precisely according to your business needs. You can easily implement best practices for service life- cycle management with the ability to promote services and life- cycle validations. o Policy support. You can publish policies that apply to services stored in Cloud Service Registry/Repository. These policies help you institute best practices in your Cloud deployment. o Governance profile. To help you get started easily and quickly, Cloud Service Registry/Repository provides a welldefined service model that includes templates, associated life cycles, governance policies, a classification system, roles and perspectives. Khasnabish, et al. Expires December 30, 2012 [Page 15] Internet-Draft Cloud Reference Framework June 2012 3.2.1.3. Cloud Audit Management Cloud Audit Management (CAM) is to provide an agent through which cloud providers and authorized consumers automate the Audit, Assertion, Assessment, and Assurance of the cloud infrastructure (IaaS), platform (PaaS), and application (SaaS) environments to reduce the risk. A common interface and namespace can be used by the CAM to facilitate these audit functions. Requirements on CAM are as follows: o A well-defined objective and scope tied to quality compliance and risk management processes o Establish clear policies, procedures, and metrics. Audit management should incorporate defined policies, procedures, and metrics as performance benchmarks. These elements should be reviewed periodically for continuous improvement. o Integrate essential quality management processes. An effective audit management system should automate the entire audit process and include integration of the following processes: * Corrective and preventive actions * Change control * Non-conformance tracking and management * Regulatory document/content management * Custom reporting, analysis and analytics * Training * Compliance intelligence dashboard 3.2.1.4. Cloud SLA Management (CSM) SLA is a part of a service contract where the level of service is formally defined between Cloud service providers and Cloud customers. Within the terms of their contracts, the SLA will have a technical definition, typical terms as MTTF (Mean Time To Failures), MTTR (Mean Time To Repair), ABA (Abandonment Rate), ASA (Average Speed to Answer), TSF (Time Service Factor), FCR (First Call Resolution), TAT (Turn Around Time), Uptime Agreements, various data rates, etc. SM is to control the usage and receipt of resources from and by third Khasnabish, et al. Expires December 30, 2012 [Page 16] Internet-Draft Cloud Reference Framework June 2012 parties. The strategy of CSM includes the negotiation of the contract and the monitoring of its realization in real-time. Thus, CSM encompasses the SLA contract definition (basic schema within QoS parameters), the SLA negotiation, the SLA monitoring, and the SLA enforcement. SM also need define rate reductions and discounts that are applied when a service provider fails to meet the desired service parameters or does not fulfill an agreement. Requirements on CSM are as follows: o SLA template specification. when service provider publishes a new service, an SLA template which describes the contract type that goes with the resource usage will be specified. Such a template may be hard to define we propose to develop a skeleton of a template with the corresponding write-up procedure. o Negotiation. Service client and service providers have to agree on the terms of the SLA binding them and also with the consequences to violations. o Resource Optimization. When the SLA processes the service access request from the service client, it also has to keep in mind the optimization of the usage of resources, and the QoS guaranteed in the SLA. o Monitoring. Once the cloud system has started providing access to its resources, it should monitor the operating resources. The monitored information is then used to prove the QoS agreed within the SLA being satisfied. o Re-negotiation. Some party of the contract may wish to change the resource usage policy while the system is running, in order to comply with a change in external conditions. In order to keep the behaviour of the process continuous, the agreed SLA need adjust to assure the process vitality after migration and resource shortage. o Evaluation. Besides the running information is interested by the managers and users, other data like contract violations or global statistics are also needed in order to verify the SLA. Evaluation is the process of analyzing the previously monitored information. An evaluation daemon may be proposed, based on the monitoring tools developed. o Accounting. The use of a resource generates an accounting sheet which describes the resources used and aligns them with the billing rules agreed in the SLA. This is a base to draft the real Khasnabish, et al. Expires December 30, 2012 [Page 17] Internet-Draft Cloud Reference Framework June 2012 financial exchange, which can be in disfavour of the provider in case of failure to comply with the compromised QoS. This subject is very sensible, and the development of tools for such themes should not be taken lightly. Related Language: WSLA, ITIL 3.2.1.5. Cloud Service Security Management Cloud Service Security (CSS) provides a set of security services and mechanisms (e.g. IP address filtering, firewall, message integrity & confidentiality, private key encryption, dynamic session key encryption, user authentication and Service certification) to protect Cloud Services and their operating environment from not authorised use, policy/operation violation and intrusion.. Requirements on CSS are as follows o Licensing. It is likely that your service is made up of many different components, most of which have some type of licensing agreement associated with them. You will need to review each of those agreements to determine if, or how, those licenses will be affected by deployment in a cloud. If your service uses a component that is licensed by CPU and you deploy it in a cloud environment designed to launch new instances and request more resources as load increases, for example, you could easily exceed your CPU license limit. You will need to understand how your licenses affect your ability to scale. o Processing requirements and memory locks. If dynamic scalability is your main reason for looking to the cloud, then your application should be designed to take advantage of a parallel architecture. If the application is designed with multi-threaded code that allows processing to be split into small chunks, it's well-suited for use within the cloud. An application that is designed around single monolithic thread processing, on the other hand, will find it difficult to take advantage of the cloud's distributed nature. o Communication protocol. The cloud is based on the Internet Protocol (IP), so for an service to be considered, it must use IP as its communication mechanism. While there are many protocols that can be run over IP, the IP layer can provide security mechanism to protect the security of the transmitted data. o Data security. The service needs to provide security at the data storage, processing and transmission stages. Data at rest must be protected by the service, that is the service must provide a Khasnabish, et al. Expires December 30, 2012 [Page 18] Internet-Draft Cloud Reference Framework June 2012 mechanism to protect the data stored in the cloud. Data in transit needs to be protected either at the service or the transmission level. Most services choose the transmission level for protection and the Secure Sockets Layer (SSL)/Transport Layer Security (TLS) protocols are often used. Server to server communications need to ensure the security from one cloud instance to another cloud instance. Khasnabish, et al. Expires December 30, 2012 [Page 19] Internet-Draft Cloud Reference Framework June 2012 4. Inter-Cloud Framework The inter-cloud interoperability and integration is motivated by a number of uses to provide multi-provider cloud services and infrastructures, multi-platform cloud services integration and traditional/legacy infrastructure services migration to clouds, and IT infrastructure services recovery or migration/move to a new location, platform, or service model. A Cloud Service Provider (CSP) can offer services using one or more date centers (DCs). These DCs can provide virtualized compute, storage, and networking resources on on-demand basis to the Cloud Service Consumers (CSC). Therefore, the DC infrastructure does not necessarily need to be a static entity as in a traditional DC. The infrastructure resources can span multiple CSPs and the entity that is offered to the consumer can be referred to as the Infrastructure as a Service (IaaS). With the IaaS, a CSC can acquire and release resources on on-demand basis. We therefore define an Inter-Cloud as a interconnction of clouds where two or more cloud service providers (any combination of Service-Provider-owned, private, public, etc.) can collaborate. The objective of the collaboration is to dynamically distribute the workloads based on mutually agreed upon service level agreement (SLA). Inter-Cloud Interface | +---------------------------+ ICI-1 +-------------------------+ | Cloud 1 |<-----|------>| Cloud 2 | | +-----------+ | | | +-----------+ | | |Inter-Cloud| | ICI-2 | |Inter-Cloud| | | |Resource | |<-----|------>| |Resource | | | |Control | | | | |Control | | | +-----------+ | ICI-3 | +-----------+ | | |<-----|------>| | +---------------------------+ | +-------------------------+ 4.1. Inter-Cloud Requirements o ICF should support communication between cloud applications and services belonging to different service layers (vertical integration), between cloud domains and heterogeneous platforms (horizontal integration). Khasnabish, et al. Expires December 30, 2012 [Page 20] Internet-Draft Cloud Reference Framework June 2012 o ICA should provide a possibility that applications could control infrastructure and related supporting services at different service layers to achieve run-time optimization and required Quality of Service (QoS) (typically related to Intercloud control and management functions). o ICA should support cloud services/infrastructures provisioning on- demand and their lifecycle management, including composition, deployment, operation, and monitoring, involving resources and services from multiple providers (this is typically related to service management and operations support functions). o Provide a framework for heterogeneous inter-cloud federation o Facilitate interoperable and measurable intra-provider infrastructures o Explicit/Guaranteed intra- and inter-Cloud network infrastructure provisioning (as NaaS service model) o Support existing Cloud Provider operational and business models and provide a basis for new forms of infrastructure services provisioning and operation More specific Inter-cloud funtional requirements may be articulated as follows: o Provide a mechanism for resource search and discovery, to determine which serving cloud might have certain resources available (including a match making mechanism). o Provide a mechanism to authenticate participating entities belonging to different cloud domains. o Provide a mechanism for requesting, controlling, and releasing resources between two clouds. o Provide a secure transport channel between the interconnecting entities. o Provide end-to-end isolation to support multitenancy. o Provide a mechanism for monitoring, assuring, and troubleshooting across the interconnection. o Provide a mechanism for defining the monitoring metrics such as Delay-Jitter-Loss. This may be useful for monitoring a flow such as TCP/UDP between IP prefix and a destination address across the Khasnabish, et al. Expires December 30, 2012 [Page 21] Internet-Draft Cloud Reference Framework June 2012 interconnection. Following the above requirements, we define the following complimentary components of the proposed Intercloud Architecture: (1) Intercloud Control and Management Plane (ICCMP) for Intercloud applications/infrastructure control and management, including inter- applications signaling, synchronization and session management, configuration, monitoring, run time infrastructure optimization including VM migration, resources scaling, and jobs/objects routing; (2) Intercloud Federation Framework (ICFF) to allow independent clouds and related infrastructure components federation of independently managed cloud based infrastructure components belonging to different cloud providers and/or administrative domains; this should support federation at the level of services, business applications, semantics, and namespaces, assuming necessary gateway or federation services; (3) Intercloud Operation Framework (ICOF) which includes functionalities to support multi-provider infrastructure operation including business workflow, SLA management, accounting. ICOF defines the basic roles, actors and their relations in sense of resources operation, management and ownership. ICOF requires support from and interacts with both ICCMP and ICFF. The following sections provides in details descriptions of the proposed ICF components definition and suggestions about required interfaces and supporting protocols. 4.2. Intercloud Control and Management Plane (ICCMP) The ICCMP defines functionality and functional components for Intercloud applications/infrastructure control and management, including inter-applications signaling, synchronization and session management, configuration, monitoring, runtime infrastructure optimization. ICCMP should support also more complex operations such as VM migration, resources scaling, and jobs/objects/data routing The ICCMP definition/development attempts to leverage the general Internet technologies such as provided by CDN [CNDI, I-D], XMPP [XMPP, RFC] and the Generalized Multi-Protocol Label Switching (GMPLS) [GMPLS, RFC]. Main functional components include o Cloud Resource Manager Khasnabish, et al. Expires December 30, 2012 [Page 22] Internet-Draft Cloud Reference Framework June 2012 o Network Infrastructure Manager Possible ICCMP Interfaces include o Signaling o Control o Monitoring o Management o Location o Message routing 4.3. Intercloud Federation Framework (ICFF) ICFF is defined to allow independent clouds and related infrastructure components federation of independently managed cloud based infrastructure components belonging to different cloud providers and/or administrative domains; this should support federation at the level of services, business applications, semantics, and namespaces, assuming necessary gateway or federation services, and also supporting federated security infrastructure including federated identity and trust management; The ICFF is built upon and extends current cloud federation concept [CloudFed] and leverage existing platforms for federated network access and federated identity management widely used for multi-domain and multi-provider infrastructure integration [22, 23]. One of the main components of the federated Intercloud architecture is the Intercloud gateway that provides translation and forwarding of the requests, protocols, data formats between cloud domains that may use different semantics, protocols, trust relations. The main goal of ICFF is to allow heterogeneous clouds integration at service and business level. Main functional components include o Trust and service broker o Identity provider o Service Registry Khasnabish, et al. Expires December 30, 2012 [Page 23] Internet-Draft Cloud Reference Framework June 2012 o Service discovery o Trust manager/router o Attribute/namespace resolver o Intercloud gateway and/or attribute/namespace translator Possible ICFF Interfaces o Naming, Addressing and Translation (if/as needed) o Publishing o Discovery o Attributes management o Trust/key management 4.4. Intercloud Operation Framework (ICOF) ICOF includes functionalities to support multi-provider infrastructure operation including business workflow, SLA management, accounting, and operational security. ICOF defines the basic roles, actors and their relations in sense of resources operation, management and ownership. ICOF requires support from and interacts with both ICCMP and ICFF. ICOF defines the main roles and actors based on the RORA model: Resource, Ownership, Role, Action. This should provide a basis for business processes definition, SLA management and access control policy definition and also Broker and Federation operation. The ICOF definition will include analysis and adoption of the TeleManagement Forum (TMF) documents related to eTOM and Operational Support Systems [TMF], Service Delivery Framework (SDF) [TMF SDF], and SLA Management [TMF SLAM]. Main functional components include o Service Broker o Service Registry o Cloud Service Provider, Cloud Operator, Cloud (physical) Resource provider, Cloud Carrier Khasnabish, et al. Expires December 30, 2012 [Page 24] Internet-Draft Cloud Reference Framework June 2012 Possible ICOF Interfaces o Provisioning, Deployment, Decommissioning/Termination o SLA management and negotiation o Services Lifecycle management o Services deployment Khasnabish, et al. Expires December 30, 2012 [Page 25] Internet-Draft Cloud Reference Framework June 2012 5. Use Cases 5.1. Virtual Network Management Configuration Management in VSML is responsible for creating and managing virtual network through the interface between the Configuration Manager and the Resources Abstraction&Virtualization Layer or Physical Resource Layer. This section is based on the information available in the following draft: draft-Okita-Clouds-VNM- model-for-PaaS-00, Okita-Clouds-VNM-model-for-PaaS-Sept10.pdf +------------------------------------------------+ +-----------------+ | Application/Service Layer |<-->| Cloud | +------------------------------------------------+ | Management | | | | | | | | | | | +------------------------------------------------+ | | | Resource Control Layer |<-->| | +------------------------------------------------+ | | | | | | | | | | | | +------------------------------------------------+ | | | Resource Abstraction&Virtualization Layer| | +-------------+ | | +---------+ +---------+ +---------+ | | |Configuration| | | | V- | | V- | | V- | |<-->| | Management | | | | Switch | |Interface| | Link | | | +-------------+ | | +---------+ +---------+ +---------+ | | | +------------------------------------------------+ | | | | | | | | | | | | +------------------------------------------------+ | | | Physical Resource Layer | | | | +-------+ | | | | |Network| |<-->| | | +-------+ | | | +------------------------------------------------+ +-----------------+ 5.2. Telecom Network Virtualization Telecom Network virtualization is the technology that enables the creation of logically isolated network partitions over shared physical network infrastructures so that multiple virtual telecom networks can simultaneously coexist over the shared infrastructures. The objectives of telecom network virtualization is to Khasnabish, et al. Expires December 30, 2012 [Page 26] Internet-Draft Cloud Reference Framework June 2012 o scale telecom services on demand o improve reliability and availability o efficiently use infrastructure In order to facilitate the deployment of telecom network virtualization, Manager Node provides control procedures such as creating Functional (Service) Entity operating on Execution Node, monitoring the status of Functional (Service) Entity and Execution Node, measuring the performance, retrieving deployment data from Infromation Server, and so on. This section is based on the information available in the following draft: draft-Yokota-Clouds-Telecom-Net-Virtualization-00, Yokota- Clouds-Telecom-Net-Virtualization-Sept10.pdf +---------------------------+ | Information Server |\ +---------------------------+ \ | \ +------------------------------------------------+ +-------------+ | Application/Service Layer |<-->| Cloud | +------------------------------------------------+ | Management | | | | | | | | | | | +------------------------------------------------+ | +---------+ | | Resource Control Layer |<-->| | Manager | | +------------------------------------------------+ | | Node | | | | | | +---------+ | | | | | | +------------------------------------------------+ | | | Resource Abstraction&Virtualization Layer| | | | +---------+ +---------+ +---------+ | | | | |Execution|----|Execution|----|Execution| |<-->| | | | Node | | Node | | Node | | | | | +---------+ +---------+ +---------+ | | | +------------------------------------------------+ | | | | | | | | | | | | +------------------------------------------------+ | | | Physical Resource Layer |<-->| | +------------------------------------------------+ +-------------+ Manager Node manages the Execution Node and communicates with Information Server to get configuration data. Khasnabish, et al. Expires December 30, 2012 [Page 27] Internet-Draft Cloud Reference Framework June 2012 Execution Node is a physical or virtual machines on which target telecomm functions (software) are running. For example, in IMS, CSCF and HSS are candidates of functions. Information Server (optional) is used for discovery and assignment of Execution Node for a session (e.g., P-CSCF at a UE's registration). 5.3. Virtual Data Center Virtual Data Center (VDC) can be constructed base on the virtualized resources in cloud environment. This section is based on the information available in the following draft: draft-bitar-datacenter-vpn-applicability-01.txt, draft-armd-datacenter-reference-arch-01.txt. ,---------. ,' `. ( IP/MPLS ) `. ,' `-+------+' +-----+ +-----+ | GW |<->| GW | +-----+ +-----+ / \ +--------+ +---------+ | Core | | Core | | SW/Rtr | | SW/Rtr | +--------+ +---------+ / \ / \ / \ / \ +-------+ +-------+ +------+ | ToR | | ToR | | ToR | +-------+ +-------+ +------+ / \ / \ / \ / \ / \ / \ +-----+ +-----+ +-----+ +-----+ | VSw | | VSw | | VSw | | VSw | +-----+ +-----+ +-----+ +-----+ The following network components are present in a DC: o VSw or virtual switch - software based Ethernet switch running inside the server blades. The individual VMs appear to a VSw as IP hosts connected via logical interfaces. The VSw may evolve to Khasnabish, et al. Expires December 30, 2012 [Page 28] Internet-Draft Cloud Reference Framework June 2012 support IP routing functionality. o ToR or Top of Rack - software-based or hardware-based Ethernet switch aggregating all Ethernet links from the server blades in a rack representing the entry point in the physical DC network for the hosts. ToRs may also perform routing functionality. o Core SW (switch) - high capacity core node aggregating multiple ToRs. This is usually a cost effective Ethernet switch. Core switches can also support routing capabilities. o DC GW - gateway to the outside world providing DC Interconnect and connectivity to Internet and VPN customers. In the current DC network model, this may be a Router with Virtual Routing capabilities and/or an IPVPN/L2VPN PE. 5.4. Security Framework for VDCS Virtualized Data Center Services (VDCS) Security Framework is a reference framework to build secure and interoperable services on top of a virtualized infrastructure. A security framework and the associated requirements for Protocols, Profiles, Network Interfaces, Operations and Management, and Application Interfaces(APIs) need to be proposed in an environment where virtualized resources are shared among a variety of public and private subscribers/clients seamlessly. The various applications and interworking protocols developed by the IETF MAY need to be extended or profiled to support the security requirements of virtualized services and infrastructure environment. o Applications & Services: The most widely used protocol that is in use today for application & services development areas like HTTP have been considered for the applications in the virtualized environment. The protocol may have to be profiled or extended with significant changes to be ready to handle the security requirements in a virtualized environment. o Infrastructure Operations & Management: The various security parameters related to operations and management of virtualized network resources in multiple administrative domains may need to be defined. The results of monitoring may need to be exchanged periodically to support the private and public virtualized domains and infrastructure in order to maintain the expected end-to-end security. The above protocol extension and operations & management requirements can be implemented in current cloud reference framework (CRF) based on the security functionality provided by cloud management layer, Khasnabish, et al. Expires December 30, 2012 [Page 29] Internet-Draft Cloud Reference Framework June 2012 resource authentication and authorization mechanism, and services/ users admission control. This section is based on the information available in the following draft: draft-karavettil-vdcs-security-framework-00.txt Khasnabish, et al. Expires December 30, 2012 [Page 30] Internet-Draft Cloud Reference Framework June 2012 6. Conclusion We presented a high-level cloud reference framework. A few examples on utilization of the reference framework are also discussed. Khasnabish, et al. Expires December 30, 2012 [Page 31] Internet-Draft Cloud Reference Framework June 2012 7. Security Considerations Contents in this section will be added based on discussion and contributions. Khasnabish, et al. Expires December 30, 2012 [Page 32] Internet-Draft Cloud Reference Framework June 2012 8. Acknowledgement We thank T. Sridhar (thsridhar@gmail.com), Simon Leinen (simon.leinen@switch.ch) for comments on an earlier version of this document. The Intercloud Framework definition is an outcome of the ongoing research and developments in the FP7 EU funded project, "Generalised Architecture for Dynamic Infrastructure Services" (GEYSERS, FP7-ICT- 248657, http://www.geysers.eu/) which provides implementation of the main components of ICCMP and ICOF. The presented work is also supported by the research on Cloud achitecture research at the System and Network Engineering group of the University of Amsterdam [UVA2011, UVA2012]. Khasnabish, et al. Expires December 30, 2012 [Page 33] Internet-Draft Cloud Reference Framework June 2012 9. IANA Considerations This document has no actions for IANA. Khasnabish, et al. Expires December 30, 2012 [Page 34] Internet-Draft Cloud Reference Framework June 2012 10. Normative references [CDN, I-D] IETF, "Leung, K. and Lee, Y. (2011). Content Distribution Network Interconnection (CDNI) Requirements", March 2012. [Cloud SDO] Khasnabish, B., "draft-khasnabish-cloud-sdo-survey-02", June 2012. [Cloud ServiceMobility] Yokota, H., "draft-yokota-cloud-service-mobility-01", March 2011. [CloudFed] Blog post by Krishnan Subramanian, "Defining Federated Cloud Ecosystems.", October 6 2011. [DSP0004] DMTF, "Common Information Model (CIM) Infrastructure", May 2009. [DSP1041] DMTF, "Resource Allocation Profile", June 2009. [DSP1042] DMTF, "System Virtualization Profile", April 2010. [DSP1057] DMTF, "Virtual System Profile", October 2009. [DSP1059] DMTF, "Generic Device Resource Virtualization Profile", July 2009. [FedNetwork] GEANT, "Federated Network Architectures. GEANT3 Project.", March 2012. [GMPLS] IETF, "RFC 3945. Generalized Multi-Protocol Label Switching (GMPLS) Architecture.", October 2004. [ITU-T FGCC] FGCC, "cloud-o-0046-funct_ref_arch", April 2011. [ITU-T Y.2011] ITU SG13, "Y.2011_General principles and general reference model for NGN", October 2004. [Industry WorkItem] Khasnabish, B., "draft-khasnabish-cloud-industry-workitems-survey-02", June 2012. Khasnabish, et al. Expires December 30, 2012 [Page 35] Internet-Draft Cloud Reference Framework June 2012 [NIST CCRA] NIST, "NIST SP 500-292, Cloud Computing Reference Architecture, v1.0.", October 2011. [NIST Cloud] NIST, "NIST SP 800-145, A NIST definition of cloud computing", October 2011. [OASIS IDCloud] OASIS IDCloud TC, "OASIS Identity in the Cloud", May 2012. [RFC2119] IETF, "Key words for use in RFCs to Indicate Requirement Levels", March 1997. [RFC4741] IETF, "NETCONF Configuration Protocol", December 2006. [TMF] TMF, "TM Forum Frameworx", March 2012. [TMF SLAM] TMF, "TMF SLA Management", November 2011. [TMF-SDF] TMF, "TR139, Service Delivery Framework (SDF) Overview, Release 2.0.", October 2010. [UML] OMG, "Unified Modeling Language", September 2002. [UVA2011] University of Amsterdam, "Generic Architecture for Cloud Infrastructure as a Service (IaaS) Provisioning Model, Release 1. SNE Techn. Report SNE-UVA-2011-03, 15 April 2011. [Online] http://staff.science.uva.nl/~demch/ worksin progress/ sne2011-techreport-2011-03-clouds-iaas-architecture- release1.pdf", 15 April 2011. [UVA2012] University of Amsterdam, "Intercloud Architecture for Interoperability and Integration, Release 1, Draft Version 0.4. SNE Techn. Report SNE-UVA-2012-03-02, 19 June 2012. [Online] http://staff.science.uva.nl/~demch/ worksinprogress/ sne2012-techreport-12-05-intercloud-architecture- draft04.pdf", 19 June 2012. [VDCS Security] Karavettil, S., "draft-karavettil-vdcs-security-framework-00.txt", June 2012. [VNet Model] Khasnabish, et al. Expires December 30, 2012 [Page 36] Internet-Draft Cloud Reference Framework June 2012 Okita, H., "draft-okita-ops-vnetmodel-03", April 2011. Khasnabish, et al. Expires December 30, 2012 [Page 37] Internet-Draft Cloud Reference Framework June 2012 Authors' Addresses Bhumip Khasnabish ZTE USA, Inc. 55 Madison Avenue, Suite 160 Morristown, NJ 07960 USA Phone: +1-781-752-8003 Email: vumip1@gmail.com, bhumip.khasnabish@zteusa.com Chu JunSheng ZTE No.50 Ruanjian Dadao Road, Yuhuatai District Nanjing China Phone: +86-25-8801-4630 Email: chu.junsheng@zte.com.cn Ma SuAn ZTE No.68 Zijinghua Rd,Yuhuatai District Nanjing China Phone: +86-25-5287-8189 Email: ma.suan@zte.com.cn Meng Yu ZTE No.50 Ruanjian Dadao Road, Yuhuatai District Nanjing China Phone: +86-25-8801-4631 Email: meng.yu@zte.com.cn Khasnabish, et al. Expires December 30, 2012 [Page 38] Internet-Draft Cloud Reference Framework June 2012 Ning So Tata Communications 2613 Fairbourne Cir. Plano, TX 75082 USA Phone: +1-972-955-0914 Email: ning.so@tatacommunications.com Paul Unbehagen Avaya USA Phone: +1-919-606-8845 Email: paul@unbehagen.net Monique Morrow Cisco Systems [Switzerland] GmbH Richistrasse 7 CH-8304 Wallisellen Switzerland Phone: Email: mmorrow@cisco.com Masum Hasan Cisco Systems 3675 Cisco Way San Jose, California 95134 USA Phone: Email: masum@cisco.com Yuri Demchenko University of Amsterdam Science Park 904 Amsterdam, 1098 XH The Netherlands Phone: Email: y.demchenko@uva.nl Khasnabish, et al. Expires December 30, 2012 [Page 39]