market oriented cloud
DESCRIPTION
Market-Oriented Cloud Computing (as part of cloud symposium of ACM Compute 2009) Srikumar Venugopal Grid Computing and Distributed Systems (GRIDS) Laboratory Dept. of Computer Science and Software Engineering The University of Melbourne, AustraliaTRANSCRIPT
Market-Oriented Cloud Computing
Srikumar Venugopal
Grid Computing and Distributed Systems (GRIDS) LaboratoryDept. of Computer Science and Software EngineeringThe University of Melbourne, AustraliaE: [email protected]
2
Agenda
Motivation for Market-based Computing A vision for Market-based Clouds Current research in GRIDS Lab
Aneka: A resource provider Brokering of resources and negotiation
Conclusion and Open Questions
3
Salient Features of Cloud Computing
Abstracted Infrastructure Using Resources without reference to their location
Fully Virtualized Servers are virtual instances
Dynamic Can add, delete new instances dynamically
Pay by Consumption No fixed long-term contracts
Configurable Any application or OS can be provided
Forrester Research, “Is Cloud Computing Ready For The Enterprise?”,March 2008
4
Yet..
Cloud SLAs are still in their infancy Limited options for higher Quality of Service
Flat Pricing Model Amazon
Cloud provisioning is not a core product Amazon is an e-commerce company
Google is a search company
Need innovative business models
A larger marketplace
5
The Gridbus Project @ Melbourne:Enable Leasing of ICT Services on Demand
WWG
Pushes Grid computing intomainstream computing
Gridbus
6
The Gridbus Project @ GRIDS Lab, The University of Melbourne:Toolkit for Creating and Deploying e-* Applications on Utility Grids
The Gridbus Project @ GRIDS Lab, The University of Melbourne:The Gridbus Project @ GRIDS Lab, The University of Melbourne:
Toolkit for Creating and DeployingToolkit for Creating and Deploying ee--** Applications on Utility GridsApplications on Utility Grids
Gridbus
Distributed Data
http://www.gridbus.org
• Gridbus is a “open source” Grid R&D
project with focus on Grid Economy, Utility
Grids and Service Oriented Computing.
• Gridbus Middleware components include:
– Aneka: .NET-based Enterprise Grid
– Grid Market Directory and Web Services
– Grid Bank: Accounting and TransactionManagement
– Visual Tools for Creation of DistributedApplications
– Grid Service Broker and Scheduling
– Workflow Management Engine
– Libra: SLA-based Resource Allocation
– GridSim Toolkit
7
8
Participants, Goals, Requirements
Consumers: - minimize expenses, meet QoS How do I express QoS requirements ? How do I trade between timeframe & cost ? How do I discover services and map jobs to meet my QoS needs? How do I manage Grid dynamics and get my work done? …
Providers:– maximise ROI and profit How do I decide service pricing models ? How do I specify them ? How do I translate them into resource allocations ? How do I enforce them ? How do I advertise & attract consumers ? How do I do accounting and handle payments? …
They need mechanisms, tools and technologies that help them in valueexpression, value translation, and value enforcement. Service Level Agreements (SLAs)
9
Market Mechanisms for Clouds: Avision
StorageCloud
ComputeCloud
StorageCloud
ComputeCloud
Directory
Bank
Auctioneer
MARKET
EnterpriseResourceManager(Proxy)
Broker1
BrokerN
RequestResources
Negotiate/Bid
..
..
CloudConsumer
10
Aneka: a resource provider for paralleland distributed applications
Message Handler / Dispatcher
Communication Layer
Container
Security
Pe
rsisten
ce
Allocation Manager
ThreadModel
TaskModel
DataflowModel
MPIModel
MapReduce
OtherModels
Applications
SLANegotiation
11
Advance Reservations
Commitment of a guaranteed share of aresource ahead of usage time Resources : Nodes, Bandwidth, Storage
Advantages: Lowers risk for user
Easier capacity planning for provider
Assured income
Applications : workflow, multimedia applications, etc.
Are a form of SLA
12
Aneka’s SLA-View for Resource Allocation
Master Node
User/Broker
Enterprise Grid
Reservation
Store
Task
Store
Node
SelectionPolicy
PricingPolicy
Membership
Store
Execution Node
TaskStore
ReservationStore
Time SlotSelection
Policy
ExecutionNode
ExecutionNode
ExecutionService
MembershipService
SchedulingService
Reservation Service
Allocation Service
Negotiation Protocol Engine
Ack: C.S. Yeo
13
Pricing of Reservations
Dynamic pricing based on utilization level
Where p is the unit price,
x is the static component (base price), and
y = load factor * z, is the dynamic component
a and b are the relative weights
b can be set higher when resource availability is low and viceversa
Serves as a method of admission control
takes advantage of market conditions
p ax by
C.S.Yeo, S. Venugopal, X. Chu, and R. Buyya, Autonomic Metered Pricing for a UtilityService, Technical Report, GRIDS-TR-2008-16, GRIDS Laboratory.
14
Revenue vs Strategy
15
Cloud Provider Architecture
Dispatcher
VMMonitor
Service RequestMonitor
Pricing Accounting
Service Request Examiner andAdmission Control
- Customer-driven Service Management- Computational Risk Management
- Autonomic Resource Management
Users/
Brokers
SLAResource
Allocator
Virtual
Machines(VMs)
PhysicalMachines
16
Gridbus Broker: Abstracting ResourceAccess
Aneka
Gateway
UnicoreData Store
Access Technology
Grid FTPSRB
-PBS-Condor-SGE
Globus
Job manager
fork() batch()
Gridbusagent
Data Catalog
-PBS-Condor-SGE-XGrid
SSH
fork()
batch()
Gridbusagent
Credential RepositoryMyProxy
Home Node/Portal
GridbusBroker
fork()
batch() -PBS-Condor-SGE
Po
rtle
ts
17
Broker-Provider Negotiation
NegotiationModule
NegotiationModule
SchedulerResource Allocation
Manager
Broker Provider
AdvanceReservation
Job Submissionand Monitoring
Broker acts as a user agent Broker translates user requirements to resource requirements However, the negotiation process is invisible to the end user.
18
The Negotiation Protocol
19
Effect of deadline urgency
S. Venugopal, X. Chu, and R. Buyya, “A Negotiation Mechanism for Advance ResourceReservation using the Alternate Offers Protocol”, IWQoS 2008.
20
MetaCDN: Brokering Cloud StorageProviders
Dr. James Broberg, University of Melbourne, http://www.metacdn.org
21
Open Questions
How to commoditise cloud services ?
What would be the structure of the Cloudservices market ?
What are the accounting and paymentmechanisms available ?
How to monitor and enforce the SLAs arrived atby negotiation ? Who arbitrates the process ?
Thank You
Questions?
http://www.gridbus.org