design and evaluation of a resource selection framework for grid applications
DESCRIPTION
Design and Evaluation of a Resource Selection Framework for Grid Applications. University of Chicago. Problem. Co-selection of resources Resource Selection Resource configuration Mapping workload to resources. Outline. Problem > Related work Set Match Resource Selection Service - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/1.jpg)
Design and Evaluationof a Resource Selection Framework
for Grid Applications
University of Chicago
![Page 2: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/2.jpg)
Problem• Co-selection of resources
– Resource Selection
– Resource configuration
– Mapping workload to resources
![Page 3: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/3.jpg)
Outline
• Problem• > Related work• Set Match• Resource Selection Service• Experiments• Summary• Future work
![Page 4: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/4.jpg)
Related work• Networked Batch Queue System
– PBS, LSF, NQE
• Application-specific scheduler– AppLeS, etc.
• Condor
![Page 5: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/5.jpg)
Outline
• Problem• Related work• > Set Match• Resource Selection Service• Experiments• Summary• Future work
![Page 6: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/6.jpg)
Set Match• Resource Set:
– Several resources with similar characteristics that may be located in different sites.
• Set Match– A Set Match occurs between a resource request and a
resource set. If the resource set fulfill the resource request and the resource request fulfill resource’s management policy, they match each other.
![Page 7: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/7.jpg)
Set Match• Matchmaker
Set-ExtendedClassAd
Resource
MatchMaker
Resource Set
Yes/NoRank
Request
![Page 8: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/8.jpg)
Set Match
• ClassAds Language
The classads language is a language for expressing and evaluating attributes. An attribute is a named expression. A classad is a set of named expressions. Expression similar to those found in C/C++.
B=[ name=”foo”; type="machine"; cpuspeed=800M; memory=512M]
A=[ Requirements = other.type = = "machine" && other.cpuspeed > 500M; Rank = other.cpuspeed ]
![Page 9: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/9.jpg)
Set Match
• Set-Extended ClassAds language– Aggregation functions
• Max(), Min(), Sum(), SizeOfSet()
– Other functions• suffix(S, L), Prefix(S, L)
![Page 10: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/10.jpg)
Set Match
An Example of the Set Extended ClassAds [ Type=”Set”; Domainlist={ucsd.edu, cs.uiuc.edu}; requirements = Sum(other.memory) > 1G && other.cpuspeed > 200M && suffix(other.Hostname, Domainlist); rank = Min(other.cpuspeed)*SizeOfSet() ]
![Page 11: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/11.jpg)
Resource Selection mechanism
Get all available resources and their status
Construct a Resource Set
Match request with the Resource Set andcalculate the quality of the match
Any other candidateResource Set ?
No
Yes
Output the Resource Set with highest quality
![Page 12: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/12.jpg)
Resource Selection mechanism
Get all available resources and their status
Filter Resources based on the resource request
Construct a Resource Set
Match request with the Resource Set andcalculate the quality of the match
Any other candidateResource Set ?
No
Yes
Output the Resource Set with highest quality
![Page 13: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/13.jpg)
Set BestSet and CandidateSet as empty
Choose X from Resource Pool such that for any Y inResource Pool
Rank(CandiateSet + X) > Rank(CandidateSet + Y)
Move X from Resource Pool to CandidateSet
Yes
No Match(CandidateSet, Request)
Resource Pool is Empty?Yes
No
Rank(CandidateSet) > Rank(BestSet)
Replace BestSet with CandidateSet
No
Yes
OUTPUT
![Page 14: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/14.jpg)
Outline
• Problem• Related work• Set Match• > Resource Selection Service• Experiments• Summary• Future work
![Page 15: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/15.jpg)
Resource Selection Service
• Help applications to choose a good Resource Set in Grid environment.
– Synchronous and asynchronous service
• Mapping application workload to resources if needed.
![Page 16: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/16.jpg)
Resource Selection Service
• Framework
NWS
ResourceMonitor
SetMatcher
Mapper
RSS
App
ResourceRequest
Result
GRISes
GIIS
MDSResourceInformation
![Page 17: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/17.jpg)
Resource Request• Owner: The sender of this request. • Type of Service: Synchronous or asynchronous.• Job description: The characteristics of the job to
be run, for example, problem size, problem specification and the performance model.
• Job resource requirements: User resource requirements, for example, memory capability, type of operating system, software packages installed, etc.
• Mapper: The kind of mapper algorithm to be used.• Rank: The criteria to rank the matched resources.
![Page 18: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/18.jpg)
Resource Request: An example1. [ Service = "InstantService";
2. iter=100; alpha=100; x=100; y=100; z=100;
3. computetime = x*y*alpha/other.cpuspeed*370;
4. comtime= ( other.RLatency+ y*x*254/other.RBandwidth
+other.LLatency+y*x*254/other.Lbandwidth);
5. exectime=(computetime+comtime)*iter+startup;
6. Mapper = [type ="dll";
libraryname="cactus"; function="mapper"];
7. requirements = Sum(other.MemorySize) >= (1.757 + 0.0000138*z*x*y) && suffix(other.machine, domains);
8. domains={ cs.utk.edu, ucsd.edu};
9. rank=Min(1/exectime) ]
Job description
Resource Requirements
Type of Service
Mapper
Rank
![Page 19: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/19.jpg)
Resource Selection Result
<virtualMachine>
<result statusCode="200" statusMessage="OK"/>
<machineList>
<machine dns="torc2.cs.utk.edu" processor= 2 x= 20>
<machine dns="torc3.cs.utk.edu" processor= 2 x= 15>
<machine dns="torc6.cs.utk.edu" processor= 2 x= 15>
</machineList>
</virtualMachine>
![Page 20: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/20.jpg)
Outline
• Problem• Related work• Set Match• Resource Selection Service• > Experiments• Summary• Future work
![Page 21: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/21.jpg)
Cactus application
• Application performance model– ExecTime= (Communication time + Computation time) * slowdown
• Application mapping algorithm– 1 Dimension mapping
R 1
![Page 22: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/22.jpg)
Cactus application
• Application performance model– ExecTime= (Communication time + Computation time) * slowdown
• Application mapping algorithm– 1 Dimension mapping
R 1 R 2
![Page 23: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/23.jpg)
Cactus application
• Application performance model– ExecTime= (Communication time + Computation time) * slowdown
• Application mapping algorithm– 1 Dimension mapping
R 1 R 2 R 3
![Page 24: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/24.jpg)
Cactus application
• Application performance model– ExecTime= (Communication time + Computation time) * slowdown
• Application mapping algorithm– 1 Dimension mapping
R 1 R 2 R 3
![Page 25: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/25.jpg)
Execution Time Prediction
0
100
200
300
400
500
600
700
100*100*100 140*140*280 180*180*360 220*220*420
Predicted time Execution time
Problem Size
Time (S)
torc1Torc1Torc5
Torc1Torc5
Torc1Torc5 Torc1
,3, 5
Torc1,3, 5
Torc1,3, 5, o.ucsd.edu
![Page 26: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/26.jpg)
Resource Selection Algorithm Test
• 1. o.ucsd.edu, mystere.ucsd.edu, saltimbanco.ucsd.edu
• 2: mystere.ucsd.edu, o.ucsd.edu
• 3: o.ucsd.edu, Saltimbanco.ucsd.edu 4. o.ucsd.edu
• 5: saltimbanco.ucsd.edu 6: mystere.ucsd.edu
0
50
100
150
200
250
300
1 2 3 4 5 6 7
Execution time
![Page 27: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/27.jpg)
Resource Selection Algorithm Test
• 1: torc6.cs.utk.edu 2: o.ucsd.edu 3: Saltimbanco.ucsd.edu
• 4. Torc6.cs.utk.edu + o.ucsd.edu 5: o.ucsd.edu + saltimbanco.ucsd.edu
• 6. o.ucsd.edu, mystere.ucsd.edu, torc6.cs.utk.edu
0
50
100
150
200
250
300
350
1 2 3 4 5 6 7
Execution time
![Page 28: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/28.jpg)
Outline
• Problem• Related work• Set Match• Resource Selection Service• Evaluation of the implementation• > Summary• Future work
![Page 29: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/29.jpg)
Summary
• Extended the ClassAds language to describe the requirement for a Resource Set
• Implement a Set Matchmaker• Create a general framework for the Resource
Selection Service• Validate our implementation by the Cactus
application
![Page 30: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/30.jpg)
Future Work
• Extend the resource selection algorithm to support other applications.– Gang match
– Resource with particular topology
• Provide fault tolerance to incomplete or error information
• A general matchmaking mechanism
![Page 31: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/31.jpg)
Welcome comments and suggestions
Thank you!
![Page 32: Design and Evaluation of a Resource Selection Framework for Grid Applications](https://reader030.vdocuments.mx/reader030/viewer/2022032606/56812d71550346895d928789/html5/thumbnails/32.jpg)
Problem
• Co-selection of resources– Dynamic nature of Grid resources
– Heterogeneity of resources
– The autonomy of resources
– The quality of resources depends on the application’s run characteristics