utility-oriented cloud & grid computing: a vision, hype, and reality dr. rajkumar buyya grid...
TRANSCRIPT
Utility-Oriented Cloud & Grid Computing: A Vision, Hype, and Reality
Dr. Rajkumar BuyyaGrid Computing and Distributed Systems (GRIDS) LabDept. of Computer Science and Software EngineeringThe University of Melbourne, Australiawww.gridbus.orgwww.buyya.comwww.manjrasoft.com
Gridbus Sponsors
ManjrasoftDr R ajkumar B uyya
C hief E xecutive O fficer
Manjrasoft P ty L tdR oom 5.31, IC T B uilding, 111, B arry S treet, C arlton,
Melbourne, VIC 3053, AustraliaP : +61-3-8344 1344 | F : +61-3-9348 1184
E : raj@ manjrasoft.comhttp://www.manjrasoft.com
Manjrasoft
2
The GRIDS Lab @ Melbourne
Youngest and one of the rapidly growing research labs in our School/University:
Founded in 2002 Houses 20+ researchers consisting of:
Research Fellows/PostDocs Software Engineers PhD candidates Honours/Masters students
Funding National and International organizations Australian Research Council & DEST Many industries (Sun, StorageTek, Microsoft,
IBM, Microsoft) University-wide collaboration:
Faculties of Science, Engineering, and Medicine
Many national and international collaborations.
Academics Industries
Software: Widely in academic and industrial users.
Publication: My research team produces over 20% of our
Dept’s research output.
EducationR & D
+ Community Services: e.g., IEEE TC for Scalable Computing
Manjrasoft
3
Agenda
Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services
Global Grids and Challenges Security, resource management, pricing models, …
Service-Oriented Grid Architecture and Gridbus Solutions
Market-based Management, GMD, Grid Bank, Aneka Grid Service Broker
Architecture, Design and Implementation Performance Evaluation: Experiments in Creation
and Deployment of Applications on Global Grids A Case Study in High Energy Physics
Summary and Conclusion
4
“Computer Utilities” Vision: Implications of the Internet
1969 – Leonard Kleinrock, ARPANET project “As of now, computer networks are still in their infancy,
but as they grow up and become sophisticated, we will probably see the spread of ‘computer utilities’, which, like present electric and telephone utilities, will service individual homes and offices across the country”
Computers Redefined 1984 – John Gage, Sun Microsystems
“The network is the computer” 2008 – David Patterson, U. C. Berkeley
“The data center is the computer. There are dramatic differences between of developing software for millions to use as a service versus distributing software for millions to run their PCs”
2008 – “Cloud is the computer” – Buyya!
5
Computing Paradigms and Attributes: Realizing the ‘Computer Utilities’
Vision Web Data Centres Utility Computing Service Computing Grid Computing P2P Computing Market-Oriented
Computing Cloud Computing …
-Ubiquitous access
-Reliability-Scalability-Autonomic-Dynamic discovery
- Composability-QoS-SLA- …
} +
Paradigms
Attributes/Capabilities
?-Trillion $ business- Who will own it?
* Since Grids have been around for sometime (early 2000), do we have a unified vision of what Grids can do?
* And did we make sufficient advances to turn vision of “computer utilities” into a
reality?
-- Let us take a look at views of -“industrial” practitioners & “academics”
7
“Industrial” vision of Grid computing
IBM On Demand Computing
Microsoft .NET
Oracle 10g
Sun N1 – Sun Grid Engine
HP Adaptive Enterprise
Amazon Elastic Compute Cloud Services
Manjrasoft Aneka for building enterprise Grids and Clouds.
8
Most academics view: Cyberinfrastructure for conducting
collaborative (e-)Science
Distributed instruments
Distributed computation
Distributed data
Peers sharing ideas and collaborative interpretation of data/results
2100210021002100
2100210021002100
Remote Visualization
Data & Compute Service
Cyberinfrastructure
E-Scientist
9
How do Grids look like?A Bird Eye View of a Global Grid
Grid Resource Broker
Resource Broker
Application
Grid Information Service
Grid Resource Broker
databaseR2R3
RN
R1
R4
R5
R6
Grid Information Service
10
How do Grids look like?A Bird Eye View of a Global Grid
Grid Resource Broker
Resource Broker
Application
Grid Information Service
Grid Resource Broker
databaseR2R3
RN
R1
R4
R5
R6
Grid Information Service
11
How Are Grids Used?
High-performance computing
Collaborative data-sharingCollaborative design
Drug discovery
Financial modeling
Data center automation
High-energy physics
Life sciences
E-Business
E-ScienceNatural language processing
Utility computing
Business Intelligence(Data Mining)
12
Agenda
Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services
Global Grids and Challenges Security, resource management, pricing models, …
Service-Oriented Grid Architecture and Gridbus Solutions
Market-based Management, GMD, Grid Bank, Aneka Grid Service Broker
Architecture, Design and Implementation Performance Evaluation: Experiments in Creation
and Deployment of Applications on Global Grids A Case Study in High Energy Physics
Summary and Conclusion
13
Grid Challenges
Security
Resource Allocation & Scheduling
Data locality
Network Management
System Management
Resource Discovery
Uniform Access
Computational Economy
Application Construction
14
Some Grid Initiatives Worldwide
Australia Nimrod-G Gridbus DISCWorld GrangeNet. APACGrid ARC eResearch
Brazil OurGrid, EasyGrid LNCC-Grid + many others
China ChinaGrid – Education CNGrid - application
Europe UK eScience EU Grids.. and many more...
India Garuda
Japan NAREGI
Korea...N*Grid
SingaporeNGP
USA Globus TeraGrid Cyberinfrasture AutoMate and many more...
Industry Initiatives IBM On Demand Computing HP Adaptive Computing Sun N1 Microsoft - .NET Oracle 10g Amzon – Elastic Compute Cloud Infosys – Enterprise Grid Satyam – Business Grid Manjrasoft – enterprise Clouds
and Grids and many more
Public Forums Open Grid Forum Conferences:
CCGrid Grid HPDC E-Science
http://www.gridcomputing.com
1.3 billion – 3 yrs
1 billion – 5 yrs
450million – 5 yrs
486million – 5 yrs
1.3 billion (Rs)
27 million
2? billion
120million – 5 yrs
15
Open-Source Grid Middleware Projects
Slide by Hiro
16
Driving Theme:Community vs. Utility Grids
Type
Feature
Community Grids Utility Grids (Now Clouds)
User QoS Best effort Contract/SLA
Service Pricing
Not considered /
free access
Usage, QoS level, Market supply and demand
Example Middleware
Globus, Condor, OMII, Unicore
Nimrod-G, Gridbus, & many inspired efforts (IBM Business Grid, Sun Grid Market)
.. Amazon EC2..
17
The Gridbus Project @ Melbourne:Enable Leasing of ICT Services on Demand
WWG
Pushes Grid computing into mainstream
computing
Gridbus
18
The Gridbus Project @ GRIDS Lab, The University of Melbourne: Toolkit for Creating and Deploying e-* Applications on Utility Grids
The Gridbus Project @ GRIDS Lab, The University of Melbourne: The Gridbus Project @ GRIDS Lab, The University of Melbourne: Toolkit for Creating and Deploying Toolkit for Creating and Deploying ee--** Applications on Utility GridsApplications on Utility Grids
Gridbus
Distributed Data
http://www.gridbus.org
• Gridbus is a “open source” Grid R&D project with focus on Grid Economy, Utility Grids and Service Oriented Computing.
• Gridbus Middleware components include:– Aneka: .NET-based Enterprise Grid
– Tools for Rapid Application Development
– Grid Service Broker and Scheduling
– Workflow Management Engine
– Grid Market Directory & Grid Bank:
– Libra: SLA-based Resource Allocation
– GridSim Toolkit + CloudSim
– Sensor Web
– Cloud-Oriented Technologies
Manjrasoft
19
Agenda
Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services
Global Grids and Challenges Security, resource management, pricing models, …
Service-Oriented Grid Architecture and Gridbus Solutions
Market-based Management, GMD, Grid Bank, Aneka Grid Service Broker
Architecture, Design and Implementation Performance Evaluation: Experiments in Creation
and Deployment of Applications on Global Grids A Case Study in High Energy Physics
Summary and Conclusion
20
What do Grid players want & require?
Grid Service Consumers (GSCs): - minimize expenses, meet QoS How do I express QoS requirements ? How do I trade between timeframe & cost ? How do I discover services and map jobs to meet my QoS needs? How do I manage Grid dynamics and get my work done? …
Grid Service Providers (GSPs):– maximise ROI How do I decide service pricing models ? How do I specify them ? How do I translate them into resource allocations ? How do I enforce them ? How do I advertise & attract consumers ? How do I do accounting and handle payments? …
They need mechanisms, tools and technologies that help them in value expression, value translation, and value enforcement.
21
Grid Node N
Service-Oriented Grid Architecture
Grid Service Consumer
Pro
gra
mm
ing
En
viro
nm
ents
Grid Resource Broker
Grid Explorer
Schedule Advisor
Trade Manager
Job ControlAgent
Deployment Agent
Information Service
Pricing Algorithms
Grid Node1
…
Core Middleware Services
…
…
HealthMonitor
Grid Market Services
JobExec
Info ?
Secure
Trading
QoS
Storage
Sign-on
Grid Bank
Ap
pli
cati
on
s
Data Catalogue
Grid Service Providers
Trade Server
Resource Allocation
ResourceReservation
R1
Misc. services
R2 Rm…
Accounting
22
Market-Oriented Grid Software: A union of Gridbus and other
technologies
AIXSolarisWindows Linux
.NETGridFabricSoftware
GridApplications
Core GridMiddleware
User-LevelMiddleware
GridBank
Grid Exchange & Federation
JVM
Grid Scheduling:
Task, Parametric, and Components Programming
Gridbus Resource Broker
MPI
Condor SGE TomcatPBS
Aneka Cloud (WS-based access + SLA
Workflow APIs
IRIX OSF1 Mac
Libra
Globus Unicore ……Grid
MarketDirectory
PDB
Worldwide Grid
GridFabricHardware
Grid PortalsScience Commerce Engineering ……Collaboratories
……
Grid Storage Economy
Gri
d E
con
om
y
NorduGrid XGrid
ExcellGrid
Grid Workflow Engine
APIs/Tools:
CDB
23
On Demand Assembly of Services in Market-Oriented Grid Environments
ASP Catalogue
Grid Info Service
Grid Market Directory
GSP(Accounting Service)
GridbusGridBank
GSP(e.g., Microsoft)
PEGSP
(e.g., Amazon)
PE
GSP(e.g., IBM)
CPUorPE
Grid Service (GS)
(Globus)
Aneka
EC2
GTS
Resource Allocation
Job
8
GridResource Broker
2
Visual Application Composer
Application CodeExplore
data1
36
45
Resu
lts9 7
Results+
Cost Info
10
11
Bill
12Data Catalogue
24
ASP Catalogue
Grid Info Service
Grid Market DirectoryASP Catalogue
Grid Info Service
Grid Market Directory
GSP(Accounting Service)
GridbusGridBank
GSP(Accounting Service)
GridbusGridBank
GSP(e.g., Microsoft)
PEGSP
(e.g., Amazon)
PE
GSP(e.g., IBM)
CPUorPE
Grid Service (GS)(Globus) Aneka
EC2
GTS
Resource Allocation
GSP(e.g., Microsoft)
PEGSP
(e.g., Amazon)
PE
GSP(e.g., IBM)
CPUorPE
Grid Service (GS)(Globus) Aneka
EC2
GTS
Resource Allocation
Job
8
Job
Job
Job
8
GridResource Broker
2
GridResource Broker
22
Visual Application Composer
Application CodeExplore
data1
Visual Application Composer
Application CodeExplore
data1
3366
4455
Res
ults
9
Res
ults
Res
ults
9 77
Results+
Cost Info
10
Results+
Cost Info
10
1111
Bill
12
BillBill
12Data CatalogueData Catalogue
On Demand Assembly of Services in Market-Oriented Grid Environments
25
Cloud Services
Infrastructure as a Services (IaaS) CPU, Storage: Amazon.com et. al
Platform as a Services (PaaS) Google App Engine, Microsoft Azure, and
Manjrasoft Aneka Software as a Service (SaaS)
SalesForce.Com
Clouds
Enterprise/Private Clouds
Public/Internet Clouds
26
Layered view of services within a Cloud stack
Infrastructure as a Service (IaaS) e.g., Amazon, Nirvanix
Software as a Service (SaaS) e.g., ..SalesForce.com
Platform as a Service (PaaS) e.g., ..Aneka
Aneka
A Software Platform for Building and Managing “Enterprise” Grids and
Clouds
Manjrasoft
28
Aneka: A 3rd Gen enterprise Grid Technology Cloud model
Generation Properties Technologies
First(-2001)
Application-specific platforms
distributed.net,SETI@Home
Second(2002-2006)
Single programming model, rigid architecture, no QoS
UD, XtremeWeb, Alchemi, Digipede, DataSynapse, BOINC
Third(2007-2012?)
SOA, extensible architecture, multiple programming models, multi-tenancy, enterprise QoS, SLAs, market-based resource allocation
Aneka
29
ANEKA – Product Overview (Alpha)
.NET based service-oriented platform for grid / cloud computing
Development and Run Time Environment
Includes Development and Management Tools
Suitable for Development of Enterprise
Grid / Cloud Applications Grid / Cloud enabling legacy
applications Ideal for Corporate
Developers, Software, SaaS, Hosting Vendors and Application / System Integrators
ANEKA Product Architecture
Message Handler / Dispatcher
Communication Layer
Container
Security
Persistence
Allocation Manager
ThreadModel
Task Model
Dataflow Model
MPIModel
Map Reduce
OtherModels
Applications
SLANegotiation
30
Aneka Deployment Models
Enterprise/Private Harness LAN connected
resources Application Development,
Testing, Execution Teaching and Learning Sensitive applications
Public Hosted by a 3rd party service
provider owning a large Data Center (1000s of servers)
Offers subscription-based services to their shared infrastructure on “pay-as go” model.to many users from different organisations.
Amazon.com, Microsoft Azure Aneka SDK + Execution Manger
Aneka
Enterprise/Private Clouds
Public Clouds
31
Aneka: components
public DumbTask: ITask { … public void Execute() { …… }}
for(int i=0; i<n; i++){ … DumbTask task = new DumbTask(); app.SubmitExecution(task);}
Executor
Scheduler
Executor
Executor Executor
ClientAgent
work units
internet
internet
Aneka enterprise Cloud
ClientAgent
work units
Aneka Users
Aneka Worker ServiceAneka Manager
Programming / Deployment Model
32
How does it solve the problem?
An Illustratioin
Application
Manager
Executor
Manager / Executor
GThreads/Tasks
Divide the problem in to multiple small tasks and distribute them run in parallel on multiple computers within a Cloud.
33
User scenario: GoFront(unit of China Southern Railway Group)
Aneka utilizes idle desktops (30) to decrease task time
from days to hours
Time (in hrs)
Single Server
Aneka Cloud
Raw Locomotive Design Files(Using AutoDesk Maya) Using Maya
Graphical Mode Directly
Case 1: Single Server
4 cores server
Aneka Maya Renderer
Use private Aneka Cloud
GoFront Private Aneka Cloud
LAN network (Running Maya Batch Mode on
demand)
Case 2: Aneka Enterprise Cloud Manjrasoft
Application: Locomotive design CAD rendering
34
Aneka: How can get it?
Available to Download: Software: www.manjrasoft.com Manual: Setting up Cloud using your LAN-network computers
Teaching material parallel and distributed computing and programming, List of possible assignments for students Possible Projects for Final year students..
Price – highly affordable = Fee you charge to 1 student (each year) and all
students/teachers in entire college/university can use it! Applications
Other Departments (Physics, Chemistry, Biology, Finance, Engineering) can use it for their applications.
35
ASP Catalogue
Grid Info Service
Grid Market DirectoryASP Catalogue
Grid Info Service
Grid Market Directory
GSP(Accounting Service)
GridbusGridBank
GSP(Accounting Service)
GridbusGridBank
GSP(e.g., Microsoft)
PEGSP
(e.g., Amazon)
PE
GSP(e.g., IBM)
CPUorPE
Grid Service (GS)(Globus) Aneka
EC2
GTS
Resource Allocation
GSP(e.g., Microsoft)
PEGSP
(e.g., Amazon)
PE
GSP(e.g., IBM)
CPUorPE
Grid Service (GS)(Globus) Aneka
EC2
GTS
Resource Allocation
Job
8
Job
Job
Job
8
GridResource Broker
2
GridResource Broker
22
Visual Application Composer
Application CodeExplore
data1
Visual Application Composer
Application CodeExplore
data1
3366
4455
Res
ults
9
Res
ults
Res
ults
9 77
Results+
Cost Info
10
Results+
Cost Info
10
1111
Bill
12
BillBill
12Data CatalogueData Catalogue
On Demand Assembly of Services in Market-Oriented Grid Environments
36
Agenda
Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services
Global Grids and Challenges Security, resource management, pricing models, …
Service-Oriented Grid Architecture and Gridbus Solutions
Market-based Management, GMD, Grid Bank, Aneka Grid Service Broker
Architecture, Design and Implementation Performance Evaluation: Experiments in Creation
and Deployment of Applications on Global Grids A Case Study in High Energy Physics
Summary and Conclusion
37
A resource broker for scheduling task farming data Grid applications with static or dynamic parameter sweeps on global Grids.
It uses computational economy paradigm for optimal selection of computational and data services depending on their quality, cost, and availability, and users’ QoS requirements (deadline, budget, & T/C optimisation)
Key Features A single window to manage & control experiment Programmable Task Farming Engine Resource Discovery and Resource Trading Optimal Data Source Discovery Scheduling & Predications Generic Dispatcher & Grid Agents Transportation of data & sharing of results Accounting
Grid Service Broker (GSB)
38
Core Middleware
Gridbus User Console/Portal/Application Interface
Grid Info Server
Schedule Advisor
Trading Manager
Gridbus Farming Engine
RecordKeeper
Grid Explorer
GE GIS, NWSTM TS
RM & TS
Grid Dispatcher
G
G
CU
Globus enabled node.
AL
DataCatalog
DataNode
Amazon EC2/S3 Cloud.
$
$
$
App, T, $, Optimization Preference
workload
Gridbus Broker
39
Gridbus Broker: Separating “applications” from “different” remote service access
enablers and schedulers
Aneka
AMI
Amazon EC2Data Store
Access Technology
Grid FTPSRB
-PBS-Condor-SGE
Globus
Job manager
fork() batch()
Gridbusagent
Data Catalog
-PBS-Condor-SGE-XGrid
SSH
fork()
batch()
Gridbusagent
Single-sign on securityHome Node/Portal
GridbusBroker
fork()
batch() -PBS-Condor-SGE-Aneka-XGrid
Application Development Interface
Sch
ed
ulin
gIn
terfa
ces
Alogorithm1
AlogorithmN
Plugin Actuators
40
Gridbus Services for eScience applications
Application Development Environment: XML-based language for composition of task farming (legacy)
applications as parameter sweep applications. Task Farming APIs for new applications. Web APIs (e.g., Portlets) for Grid portal development. Threads-based Programming Interface Workflow interface and Gridbus-enabled workflow engine. … Grid Superscalar – in cooperation with BSC/UPC
Resource Allocation and Scheduling Dynamic discovery of optional computational and data nodes that
meet user QoS requirements. Hide Low-Level Grid Middleware interfaces
Globus (v2, v4), SRB, Aneka, Unicore, and ssh-based access to local/remote resources managed by XGrid, PBS, Condor, SGE.
41
Drug DesignMade Easy!
Click Here for Demo
42
s
A Sample List of Gridbus Broker UsersA Sample List of Gridbus Broker UsersA Sample List of Gridbus Broker Users
http://www.gridbus.org
Molecular docking for drug design on Australian National Grid
Molecular docking for drug design on Australian National Grid
High Energy Physics: Particle Discovery
High Energy Physics: Particle Discovery
Melbourne University
NeuroScience: Brain Activity Analysis
NeuroScience: Brain Activity Analysis
EU Data Mining GridEU Data Mining Grid
DaimlerChrysler, Technion, U. Ljubljana, U. Ulster
Kidney/Human Physiome Modelling
Kidney/Human Physiome Modelling
Melbourne Medical Faculty, Université d'Evry, France
Finance /Investment Risk Studies: Spanish Stock Market
Finance /Investment Risk Studies: Spanish Stock Market
Universidad Complutense de Madrid, Spain
43
Agenda
Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services
Global Grids and Challenges Security, resource management, pricing models, …
Service-Oriented Grid Architecture and Gridbus Solutions
Market-based Management, GMD, Grid Bank, Aneka Grid Service Broker
Architecture, Design and Implementation Performance Evaluation: Experiments in Creation
and Deployment of Applications on Global Grids A Case Study in High Energy Physics
Summary and Conclusion
44
Case Study: High Energy Physics and Data Grid
The Belle Experiment KEK B-Factory, Japan Investigating fundamental violation
of symmetry in nature (Charge Parity) which may help explain “why do we have more antimatter in the universe OR imbalance of matter and antimatter in the universe?”.
Collaboration 1000 people, 50 institutes
100’s TB data currently
45
Case Study: Event Simulation and Analysis
B0->D*+D*-Ks
• Simulation and Analysis Package - Belle Analysis Software Framework (BASF)• Experiment in 2 parts – Generation of Simulated Data and Analysis of the distributed data
Analyzed 100 data files (30MB each) that were distributed among the five nodes within Australian Belle DataGrid platform.
46
Australian Belle Data Grid Testbed
Grid Service Broker
Replica Catalog
AARNET
NWS NameServer
VirtualOrganization
Analysis Request
Analysis Results
CertificateAuthority
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
GRIDS Lab, University of Melbourne
Dept. of Physics,University of Sydney
ANU, Canberra
Dept. of Computer Science, University of Adelaide
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Intel Pentium 2.0 Ghz, 512 MB RAM
Dept. of Physics,University of Melbourne
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
VPACMelbourne
47
Belle Data Grid (GSP CPU Service Price: G$/sec)
Grid Service Broker
Replica Catalog
AARNET
NWS NameServer
VirtualOrganization
Analysis Request
Analysis Results
CertificateAuthority
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
GRIDS Lab, University of Melbourne
Dept. of Physics,University of Sydney
ANU, Canberra
Dept. of Computer Science, University of Adelaide
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Intel Pentium 2.0 Ghz, 512 MB RAM
Dept. of Physics,University of Melbourne
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NA
G$4
G$4
Datanode
G$6VPAC
MelbourneG$2
48
Belle Data Grid (Bandwidth Price: G$/MB)
Grid Service Broker
Replica Catalog
AARNET
NWS NameServer
VirtualOrganization
Analysis Request
Analysis Results
CertificateAuthority
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
GRIDS Lab, University of Melbourne
Dept. of Physics,University of Sydney
ANU, Canberra
Dept. of Computer Science, University of Adelaide
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Intel Pentium 2.0 Ghz, 512 MB RAM
Dept. of Physics,University of Melbourne
NWSSensor
GridFTPGRIS
GlobusGatekeeper
Dual Intel Xeon 2.8 Ghz, 2 GB RAM
NA
G$4
G$4
Datanode
G$6VPAC
MelbourneG$2
34
31
38
31
30
3336
32
49
Deploying Application Scenario
A data grid scenario with 100 jobs and each accessing remote data of ~30MB
Deadline: 3hrs. Budget: G$ 60K Scheduling Optimisation Scenario:
Minimise Time Minimise Cost
Results:
SUMMARY OF EVALUATION RESULTS
Scheduling strategy Total Time Taken (mins.)
Compute Cost (G$)
Data Cost (G$)
Total Cost (G$)
Cost Minimization 71.07 26865 7560 34425 Time Minimization 48.5 50938 7452 58390
50
Time Minimization in Data Grids
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
Time (in mins.)
Nu
mb
er
of
job
s c
om
ple
ted
fleagle.ph.unimelb.edu.au belle.anu.edu.au belle.physics.usyd.edu.au brecca-2.vpac.org
51
Results : Cost Minimization in Data Grids
0
10
20
30
40
50
60
70
80
90
100
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63
Time(in mins.)
Nu
mb
er o
f jo
bs
com
ple
ted
fleagle.ph.unimelb.edu.au belle.anu.edu.au belle.physics.usyd.edu.au brecca-2.vpac.org
52
SUMMARY OF EVALUATION RESULTS
Scheduling strategy Total Time Taken (mins.)
Compute Cost (G$)
Data Cost (G$)
Total Cost (G$)
Cost Minimization 71.07 26865 7560 34425 Time Minimization 48.5 50938 7452 58390
Observation
Organization
Node details Cost (in G$/CPU-sec) Total Jobs Executed
Time Cost
CS,UniMelb belle.cs.mu.oz.au4 CPU, 2GB RAM, 40 GB HD, Linux
N.A. (Not used as a compute resource)
-- --
Physics, UniMelb fleagle.ph.unimelb.edu.au1 CPU, 512 MB RAM, 40 GB HD, Linux
2 3 94
CS, University of Adelaide
belle.cs.adelaide.edu.au4 CPU (only 1 available) , 2GB RAM, 40 GB HD, Linux
N.A. (Not used as a compute resource)
-- --
ANU, Canberra belle.anu.edu.au4 CPU, 2GB RAM, 40 GB HD, Linux
4 2 2
Dept of Physics, USyd
belle.physics.usyd.edu.au4 CPU (only 1 available), 2GB RAM, 40 GB HD, Linux
4 72 2
VPAC, Melbourne brecca-2.vpac.org180 node cluster (only head node used), Linux
6 23 2
53
Agenda
Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services
Global Grids and Challenges Security, resource management, pricing models, …
Service-Oriented Grid Architecture and Gridbus Solutions
Market-based Management, GMD, Grid Bank, Aneka Grid Service Broker
Architecture, Design and Implementation Performance Evaluation: Experiments in Creation
and Deployment of Applications on Global Grids A Case Study in High Energy Physics
Summary and Conclusion
54
Summary and Conclusion
Grids exploit synergies that result from cooperation of autonomous entities:
Resource sharing, dynamic provisioning, and aggregation at global level Great Science and Great Business!
Grids have emerged as enabler for Cyberinfrastructure that powers e-Science and e-Business applications.
SOA + Market-based Grid Management = Utility Grids Grids allow users to dynamically lease Grid services
at runtime based on their quality, cost, availability, and users QoS requirements.
Delivering ICT services as computing utilities. Clouds are rapidly emerging, but more work is
required Federation of Clouds, Cloud Exchange, and Application Scaling
55
Convergence of Competing Paradigms/Communities Needed
Web Data Centres Utility Computing Service Computing Grid Computing P2P Computing Cloud Computing Market-Oriented
Computing …
•Ubiquitous access•Reliability•Scalability•Autonomic•Dynamic discovery•Composability•QoS•SLA•…
} +
Paradigms
Attributes/Capabilities
?-Trillion $ business- Who will own it?
Manjrasoft
56
Thanks for your attention!
Are there any Questions? Comments/ Suggestions
We Welcome Cooperation in R&D and Business! http:/www.gridbus.org | www.Manjrasoft.com
[email protected] | [email protected]
Manjrasoft