cloud federations patrizio dazzi (isti-cnr) [overall presentation] [email protected]...
TRANSCRIPT
Cloud FederationsPatrizio Dazzi (ISTI-CNR) [Overall Presentation]
Gaetano Anastasi (ISTI-CNR) [Hands-On][email protected]
contrail is co-funded by the EC 7th Framework Programme under Grant Agreement nr.
257438
http://contrail-project.eu
Presentation Outline
• Cloud Federations
• Contrail Federations
• Contrail Federation Architecture
• Federations and Resources
• A few details about the current status of Contrail Cloud Federations
• Introduction to the hands-on session
Cloud Federations
What is Cloud Computing ?
• Computing as a public utility– Clouds are the computing “power plants”
• Do not manage resources, just use them
• Choose the interaction model: SaaS, PaaS, IaaS
4
Principal Advantages of Cloud Comuting
• Advantages for Tenants:– Pay just what you get– Pay only when you get– Reduce costs for maintenance
• Advantages for Providers– maximizing the effectiveness of the shared
resources. – dynamically re-allocation depending on actual
usage
5
Anyway…
• Providers have:– A limited amount of resources– A limited range of resource types– Resources placed only in same countries
• Providers want:– to avoid that their customers leave them– To make (more) money
• Users want– To have all the resources they need (possibly more)– To pay less….
• How can we manage this ???6
Cloud Federations
• A Federation of Clouds is (maybe) the answer– By federating, providers can offer more
resources– More resources means…
• To be able to run bigger applications• To be more elastic• To provide (potentially more) different resources
– A federation (may) allow users to• Choose among a wider range of providers• Seamlessly use more providers for the same
application• Migrate from a provide to another
7
Federation of IaaS providers
• Infrastructure as a Service– provide computation, storage and network
• IaaS Providers– distinct Clouds may have different interfaces,
access rules, capabilities, prices, …
• Special Providers– Peculiar computational supports– Specific Storage capabilities– High performance networking
8
How Federation of IaaS impacts on PaaS and SaaS
• The background:– Platform and Software as a service offer an
even more complex landscape– PaaS and SaaS provider may be distinct from
IaaS ones
• IaaS Federation can– Allow properly instrumented PaaS and SaaS to
run on top of a huge amount of resources– Need to translate PaaS and SaaS requirements
• To address heterogeneity of providers• Enforce QoS guarantees
9
Unified Identity and Billing management
• Federations should be almost invisible from the point of view of users but:– User’s identity should be automatically
managed• In a cross-cloud fashion• Creating and managing proper identities in each
cloud (if needed)• Map by keeping proper capabilities and permissions
– Users have to be properly billed• Unified monitoring• Unified accounting
10
Security Mechanisms
• Authentication– Allow the access of users to Clouds– Exploit existing identity infrastructures
• E.g. OpenID
– Enact an identity federation
• Authorization– Access Control– Usage Control
• Particularly useful for long lasting applications
11
Cross-Cloud management of Service Level Agreements
• Cloud Applications have QoS requirements– Performance, reliability, security– Typically expressed as SLAs
• SLA management is a complex activity also in a single Cloud– Violations may occur and need to be managed
• Even more complex in a Federation of Clouds– Not all providers provide the same degree of
• guarantees• strictness• penalties• Compensation
– An overall coordination activity shall be performed
12
Last but not Least: Brokering
• Cloud Federations also behave as brokers– To find for each user’s application the best
Cloud(s) depending on• Application structure and requirements• Cloud reputation• Cost
– Splitting the user’s application in order to• Choose the best Cloud for each part of the
application• Allow a better exploitation of Specialized Clouds
13
14
Contrail Federations
Contrail Iaas Federation
A Contrail Federation integrates in a common platform multiple Clouds, both public and private ones.
To perform this task it provides:
• A common support for authentication, authorization and billing
• Mechanisms for policy definition, monitoring, and enforcing for all the QoS-related aspects
• An automated selection of providers depending on the user applications
• A support for resource provisioning to applications able to deal with heterogeneous sets of resources
Contrail development methodology
• Do not rebuild and reinvent– Contrail exploits Standards
• OVF• OCCI• OpenID
– Exploit existing platforms and functionalities• Open Nebula• SLA@SOI• Etc.
16
Develop a Federation support that integrates and actively coordinates SLA management provided by single Cloud providers
• Do not disrupt provider’s business model
• Allow exploiting a Federation seamlessly
• Federation Support must be scalable
Exploit Providers’ SLA support
Main Contrail Federation Innovations (1)
Federation = more than a simple broker, or portal, or decentralized cloud-bursting
– Interoperability
– Heterogeneous providers– Dynamically choosing best providers– At deploy time and at runtime– Allow to combine providers– Migration, elasticity– Security and privacy framework
19
Main Contrail Federation Innovations (2)
• Sophisticated SLA/QoS
– QoS via SLA– Via provider selection and integration– Enforcement mechanisms– Federation as a mediator and a 3rd party– Federation also acts as a coordinator
20
Contrail Federation Architecture
Simplified Blocks Architecture
Final Users ConPaaS
Cloud Federation
Cloud Provide
r 1
Cloud Provide
r 2…
Cloud Provide
r N
21
Distributed Federation Access Points
• Several Federation access points (FAPs)• FAP act as brokers, but share a common
view– Security, status of resources, users and
providers reputation
F
F
F
F
• Hierarchical structure• Common Policies• Detailed resource
allocation is on providers• AP may be hosted by
providers
Federation Interfaces
SLAAuth
AuthZFederation Core
Coarse-grain view of Contrail Federation Architecture
• Functionalities which extend horizontally in the platform
Provisioning Manager
Concrete Architecture
In the next slide
• Module View of a single Access point
• Interactions, some modules not shown
• Interfaces sit on top of this core
• Auth/AutZ mechanisms included
24
Contrail federation
25
Provisioning Manager
Provisioning Manager
26
Federations and Resources
Basic concepts
• What’s an application for IaaS?– A set of software entities which need to be
deployed on a suitable set of resources for execution
• The parties involved– User
• Who submit applications
– Provider• Any entity managing physical resources which may be
used to run applications
– Federation• The union of many providers under a common set of
APIs and functionalities, which can be exploited as a single Cloud
Application Description
• The requirements so far can be expressed as a Task Interaction Graph– An undirected graph G(N,E) can model a
distributed application– Each Node in N is a task (needs a resource)– Edges imply relations between tasks (need links)– Heavily labeled graph
• Nodes state all resource constraints and SLA measures
• Edges labeled with communication needs
– Cloud applications can gethard to describe
Types of resources
• Computation resources– Available VMs slots on top of physical hardware
• Storage resources– Shared filesystems– Shared Databases
• Networking resources– Virtual networks connect machine within an
application– Behaviour of the joining points between the
internet and the federation is important
How resources are measured ?
• Static specification by analogy with the physical counterparts– Memory size, CPU type and speed (peak)– Size, FS of storage– Nominal bandwidth and latency of networks
• Dynamic specification– Available computing power, memory– Actual (average, peak, used) storage speed– Observed in-Cloud bandwidth and latency– Observed bw/lat to the outside– Application-specific metrics
Provider’s View on resources
• Providers know exactly the layout and state of all physical (virtual) resources– Dedicated link bandwidths, physical memory…
• They can greatly optimize resource management– Turn off unused resources, exploit cheapest– They have their own goal (revenues)
QoS properties of resources
• Extend the set of characteristics to be measured on the platform
• Protection– Type of security mechanisms which are in place
• Auth. Protocols, Encryption mechanisms, Isolation
• Privacy– Guarantees offered by storage holder, network
infrastructure
• Geo-localization – Can have deep legal implications
• Power consumption– Overall power, power efficiency
QoS expressed via SLAs
• As the SLA is signed, the user should be able to trust resources from the platform– But not all Providers may offer the same
reliability
• How reliability can be measured ?– failures, SLA violations with different providers– lengthy task with poor reward for single users
Provider Reputation
• Management information– Available resources per kind– Features granted– Amount of users/apps ongoing– State of SLAs controlled by the federation– Static level of “trust” given from federation to
the provider
• Past History– History of SLA violation per user / type of app– Average level of satisfaction of SLA
Contrail approach to SLA
– Reuse SLA@SOI framework as a starting point
• Integration with Contrail internal interfaces and components
• Integration with domain-specific reasoning/monitoring plugins
– Extend SLA@SOI with:• Federation support• Integration with external providers (and their SLA
management systems)• Reputation model for providers• Cost-based QoS enforcement
Federation-level SLA Enforcing
• Federation will act as a SLA coordinator over providers– As much as possible the single providers is in charge of
the local SLA• Reduce reaction delay
– Federation evaluates the provider and the app
– Extend the SLA@SOI monitoring infrastructure to the federation
• Keep track of main application parameters
– Receive SLA violations from providers
– Reallocate some resources on a provider basis• May require a new negotiation between federation and provider
What if… a SLA is violated in a single provider scenario ?
The application is deployed on a single provider, which may still violate SLA
• Actions: – The provider resize the set of allocated
resources– If the application is no longer violating SLAs
• The application will be up and running and that’s it
– Otherwise• A renegotiation phase is conducted and if will not be
successfully the application will be terminated
…and what if the violation occurs in a multi-provider scenario ?
The application has been sent by the federation to a provider for the execution
and a SLA violation occurs
• Actions:– Previous slide shows what happens when the
provider is able to manage the violation by itself
– If it is not able, the federation can still migrate (part of) the application to a different provider
38Contrail S. School 2012 – P. Dazzi- Cloud Federations
Applications Running in Multiple Providers
Some applications could be also decomposed in parts and each deployed in a different provider• Actions
– resize part(s)• managed by providers
– migrate part(s)• may need to stop the application
– violate constraints• By leveraging the whole federation we push away the limit
where a violation is unavoidable
– rebalance constraints• By renegotiating with more providers, overall SLO may be
achieved• Increase resource availability where they are cheaper/more
available at the moment
Anyway SLA splitting is still an open issue
• Necessary to leverage SLA management at single providers
• How to derive a combination of SLA for separate parts of the application which allow to manage the application overall
• Not yet addressed in literature• Not hard if providers are specialized
– Compute, network, storage = SLA aspects
• As the user expresses SLA terms on parts of the application (task groups) this will become easier
Contrail Federation, the 3rd Player
• A Contrail Federation sits in the middle• Aims
– Serving the users– Exploit efficiently the providers
• If economic gain is sought, it comes form intermediation
• Can efficiently gather provider information– Gathers from all users and all providers– Can make better informed choices than users– Can afford to launch directed tests in doubt– Can trigger corrective actions impossible to
single providers
42
A few details about the current status of Contrail Cloud Federations
How it is used
• Federation Interfaces use REST• The Web portal and command line tools access
the REST layer• Different roles
– federation coordinator– cloud administrator– end user
• Main steps– account creation– SLA formation– Application upload (VMs and descriptor)– SLA negotiation and agreement– Deploy
43
What is implemented today
• User management– Registration, mapping are working– some integration activities are still ongoing
• SLA Management– Negotiation with providers counterpart is
working– A complete integration is expected in the next
release
• Application management– Applications can be submitted and executed– So far only one provider at a time can be used
for a single application
44
Aligning with standard formats
• OVF (Open Virtualization Format)– Open format for describing application– Allows to describe structured applications– Directly expresses only HW constraint and
deployment information• Partially overlaps with full SLA specification
– OVF is extendable– As of v1.1.0 it does not target Application
Management
• SLA@SOI provides a general formalization– Includes monitoring hierarchy and negotiation
• Exploit a combination of OVF and SLA@SOI mechanisms
“Securing” the implementation
• The current implementation of federation authentication– Support for OpenID– Additional details in Jens’s keynote
• The current implementation of federation authorization– Advanced support to access control– Ongoing work to finalize the support to usage
control
46
Ongoing work about advanced features
• Multi-criteria mapping algorithms– Genetic algorithms for application mapping
• Evolving mapping plans to achieve better allocations
• Application splitting– Interplay with SLA splitting
• Monitoring data aggregation and filtering
47
Introduction to the hands-on session
Contrail S. School 2012 – P. Dazzi- Cloud Federations 48
Sample Application
• OVF Representation• 2 Appliances
49Contrail S. School 2012 - M. Coppola - Cloud Federations
<VirtualSystem ovf:id="VirtualSystem1"> <VirtualHardwareSection> <Item>
<rasd:Description>Number of virtual CPUs</rasd:Description>
<rasd:ElementName>1 virtual CPU</rasd:ElementName>
<rasd:InstanceID>1</rasd:InstanceID><rasd:ResourceType>3</
rasd:ResourceType><rasd:VirtualQuantity>1</
rasd:VirtualQuantity> </Item>
<Item> <rasd:AllocationUnits>byte *
2^20</rasd:AllocationUnits><rasd:Description>Memory
Size</rasd:Description><rasd:ElementName>512 MB of
memory</rasd:ElementName><rasd:InstanceID>2</rasd:InstanceID>
<rasd:ResourceType>4</rasd:ResourceType>
<rasd:VirtualQuantity>512</rasd:VirtualQuantity> </Item>
Simplified Deployment Chain
• Users submit an OVF• Federation mapping phase:
– Which provider(s) for the application?
• Federation Application Submission • Provider Deployment
– Provisioning Manager: receives requests coming from the federation
– VEP: manages provider resources for deployment
• For simplicity do not consider SLAs
50Contrail S. School 2012 - M. Coppola - Cloud Federations
Thanks for you attention!
51
Questions?
Funded under: FP7 (Seventh Framework Programme)
Area: Internet of Services, Software & Virtualization (ICT-
2009.1.2)
Project reference: FP7-IST-257438
Total cost: 11,29 million euro
EU contribution: 8,3 million euro
Execution: From 2010-10-01 till 2013-09-30
Duration: 36 months
Contract type: Collaborative project (generic)
contrail is co-funded by the EC 7th Framework Programme